Bill Agee's bloghttp://blog.likewise.org/2017-02-19T19:00:00-08:00Debugging a JavaScript WebdriverIO project in Visual Studio Code2017-02-19T19:00:00-08:002017-02-19T19:00:00-08:00Bill Ageetag:blog.likewise.org,2017-02-19:2017/02/debugging-a-javascript-webdriverio-project-in-vscode/<p><img alt="vscode loves webdriverio" height="50%" src="http://blog.likewise.org/2017/02/debugging-a-javascript-webdriverio-project-in-vscode/images/vscode-heart-webdriverio.png" width="50%" /></p>
<p>When working on Selenium tests, do you appreciate a traditional IDE-based approach to debugging, with a GUI that lets you set breakpoints, step through your code line-by-line, inspect variables, and evaluate expressions on the fly?</p>
<p>Here's a .gif to illustrate what I mean:</p>
<p><img alt="vscode-debug-gif" src="http://blog.likewise.org/2017/02/debugging-a-javascript-webdriverio-project-in-vscode/images/video/out.gif" /></p>
<p>Have you also tried <a target="_blank" href="http://webdriver.io/">WebdriverIO</a> (the popular library for developing Selenium WebDriver test scenarios using Node.JS), but not yet come across a way to enjoy that type of debugging experience?</p>
<p><a target="_blank" href="https://code.visualstudio.com/">Visual Studio Code</a> is a useful, lightweight, open-source IDE with support for debugging Node.JS apps.</p>
<p>And it allows you to interactively debug WebdriverIO code. So, in this post I'll document how to configure VS Code as your one-stop-shop for interactive WebdriverIO debugging!</p>
<p>We'll start from the beginning, installing each piece of software you need - and once that's complete, we'll build and debug a WebdriverIO test in the VS Code IDE.</p>
<p>Let's begin!</p>
<h2>Installing Java</h2>
<p>Things will go more smoothly later if you first ensure you have <code>java</code> in your shell's PATH. WebdriverIO will expect it to be present so it can launch the Selenium standalone server (which is a Java app) for you.</p>
<p>So, install a JRE for your platform before going further (if you don't already have one).</p>
<p>The version I used when writing this tutorial was:</p>
<div class="highlight"><pre><span></span>$ java -version
java version <span class="s2">"1.8.0_05"</span>
Java<span class="o">(</span>TM<span class="o">)</span> SE Runtime Environment <span class="o">(</span>build <span class="m">1</span>.8.0_05-b13<span class="o">)</span>
Java HotSpot<span class="o">(</span>TM<span class="o">)</span> <span class="m">64</span>-Bit Server VM <span class="o">(</span>build <span class="m">25</span>.5-b02, mixed mode<span class="o">)</span>
</pre></div>
<h2>Installing Node.JS</h2>
<p>Both WebdriverIO and VS Code will expect you to provide your own installation of Node.JS. For that, you have a few choices:</p>
<ul>
<li>
<p>You can visit <a target="_blank" href="https://nodejs.org/">https://nodejs.org/</a> and download an installer.</p>
</li>
<li>
<p>Mac folks can use Homebrew or MacPorts.</p>
</li>
<li>
<p>Or, <a target="_blank" href="https://github.com/creationix/nvm">nvm</a> works great as well (it allows you to install multiple node versions).</p>
</li>
</ul>
<p>For this tutorial, I'll be using homebrew to install node. (Visit <a target="_blank" href="https://brew.sh/">https://brew.sh/</a> for more info on installing homebrew.)</p>
<p>To install node, I ran:</p>
<div class="highlight"><pre><span></span><span class="err">$</span> <span class="n">brew</span> <span class="n">install</span> <span class="n">node</span><span class="mi">@6</span>
<span class="err">$</span> <span class="n">echo</span> <span class="err">'</span><span class="n">export</span> <span class="n">PATH</span><span class="o">=</span><span class="s">"/usr/local/opt/node@6/bin:$PATH"</span><span class="err">'</span> <span class="o">>></span> <span class="o">~/</span><span class="p">.</span><span class="n">bash_profile</span>
<span class="err">$</span> <span class="n">source</span> <span class="o">~/</span><span class="p">.</span><span class="n">bash_profile</span>
</pre></div>
<p>Check the node and npm versions you now have active - my install ended up with:</p>
<div class="highlight"><pre><span></span>$ node -v <span class="o">&&</span> npm -v
v6.12.3
<span class="m">5</span>.6.0
</pre></div>
<p>That environment will be what I use for the rest of this tutorial.</p>
<h2>Creating a WebdriverIO Project</h2>
<p>These steps are based on the developer guide at <a target="_blank" href="http://webdriver.io/guide.html">http://webdriver.io/guide.html</a>, with a few small changes.</p>
<div class="highlight"><pre><span></span># Create a new dir for your project, with a boilerplate package.json
mkdir webdriverio-test && \
cd webdriverio-test && \
npm init -y
# Install WebdriverIO, and tell npm to add it as a dependency in package.json
npm install webdriverio --save-dev
</pre></div>
<p>Now, run the WebdriverIO config helper:</p>
<div class="highlight"><pre><span></span>./node_modules/.bin/wdio config
</pre></div>
<p>It'll prompt you for a series of questions - I accepted the defaults for each, EXCEPT the <code>Do you want to add a service to your test setup?</code> question.</p>
<p>For that one, select <code>selenium-standalone</code></p>
<p>Here's what my final settings resembled:</p>
<div class="highlight"><pre><span></span>? Where do you want to execute your tests? On my local machine
? Which framework do you want to use? mocha
? Shall I install the framework adapter for you? Yes
? Where are your test specs located? ./test/specs/**/*.js
? Which reporter do you want to use?
? Do you want to add a service to your test setup? selenium-standalone - https://github.com/webdriverio/wdio-selenium-standalone-service
? Shall I install the services for you? Yes
? Level of logging verbosity silent
? In which directory should screenshots gets saved if a command fails? ./errorShots/
? What is the base url? http://localhost
</pre></div>
<p>That will create <code>wdio.conf.js</code> in your current dir.</p>
<h3>Creating a test spec file</h3>
<p>Now create the spec dir that the config utility asked about:</p>
<div class="highlight"><pre><span></span>mkdir -p test/specs
</pre></div>
<p>Then create the file <code>test/specs/foo.spec.js</code> with these contents:</p>
<div class="highlight"><pre><span></span><span class="nt">var</span> <span class="nt">assert</span> <span class="o">=</span> <span class="nt">require</span><span class="o">(</span><span class="s1">'assert'</span><span class="o">);</span>
<span class="nt">describe</span><span class="o">(</span><span class="s1">'webdriver.io page'</span><span class="o">,</span> <span class="nt">function</span><span class="o">()</span> <span class="p">{</span>
<span class="err">it('should</span> <span class="err">have</span> <span class="err">the</span> <span class="err">right</span> <span class="err">title</span> <span class="err">-</span> <span class="err">the</span> <span class="err">fancy</span> <span class="err">generator</span> <span class="err">way',</span> <span class="err">function</span> <span class="err">()</span> <span class="err">{</span>
<span class="err">//</span> <span class="err">Tell</span> <span class="err">mocha</span> <span class="err">this</span> <span class="err">test</span> <span class="err">is</span> <span class="err">allowed</span> <span class="err">to</span> <span class="err">run</span> <span class="err">for</span> <span class="err">10</span> <span class="err">minutes,</span> <span class="err">so</span> <span class="err">we</span> <span class="err">have</span>
<span class="err">//</span> <span class="err">sufficient</span> <span class="err">time</span> <span class="err">for</span> <span class="err">debugging.</span>
<span class="err">//</span>
<span class="err">//</span> <span class="err">Note</span> <span class="err">you</span> <span class="err">can</span> <span class="err">also</span> <span class="err">set</span> <span class="err">this</span> <span class="err">globally</span> <span class="err">in</span> <span class="err">mochaOpts</span> <span class="err">in</span> <span class="err">wdio.conf.</span><span class="n">js</span><span class="p">:</span>
<span class="n">this</span><span class="o">.</span><span class="nf">timeout</span><span class="p">(</span><span class="mi">10</span> <span class="o">*</span> <span class="mi">60</span> <span class="o">*</span> <span class="mi">1000</span><span class="p">);</span>
<span class="err">browser.url('</span><span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">webdriver</span><span class="o">.</span><span class="n">io</span><span class="s1">');</span>
<span class="s1"> var title = browser.getTitle();</span>
<span class="s1"> assert.equal(title, '</span><span class="n">WebdriverIO</span> <span class="o">-</span> <span class="n">WebDriver</span> <span class="n">bindings</span> <span class="n">for</span> <span class="n">Node</span><span class="o">.</span><span class="n">js</span><span class="err">'</span><span class="p">);</span>
<span class="p">}</span><span class="o">);</span>
<span class="err">}</span><span class="o">);</span>
</pre></div>
<p>To see if all is well, run the following command to launch your test - you should see a browser launch, and eventually a passing result will be printed:</p>
<div class="highlight"><pre><span></span>$ ./node_modules/.bin/wdio
․
<span class="m">1</span> passing <span class="o">(</span><span class="m">10</span>.00s<span class="o">)</span>
</pre></div>
<h2>Debugging with VS Code</h2>
<p>If you haven't installed <a target="_blank" href="https://code.visualstudio.com/">VS Code</a> yet, please do so now :)</p>
<p>(The version of VS Code I used here was 1.20.0.)</p>
<p>Use the <code>File > Open</code> menu in VS Code to open the <code>webdriverio-test</code> directory you created earlier.</p>
<h3>launch.json configuration</h3>
<p>Now, let's create a launch.json file and add a configuration for debugging your test spec.</p>
<p>To do that, click <code>View > Debug</code> or the debugging icon in the left of the IDE:</p>
<p><img alt="vscode-debug-button" src="http://blog.likewise.org/2017/02/debugging-a-javascript-webdriverio-project-in-vscode/images/vscode/debug-button.png" /></p>
<p>Then click the small gear in the debug menu to open launch.json:</p>
<p><img alt="vscode-config-gear" src="http://blog.likewise.org/2017/02/debugging-a-javascript-webdriverio-project-in-vscode/images/vscode/config-gear.png" /></p>
<p>Click Node.js in the menu that appears - this will open a boilerplate launch.json.</p>
<p>Delete the contents of launch.json, and insert the following text:</p>
<div class="highlight"><pre><span></span>{
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"protocol": "legacy",
"address": "localhost",
"port": 5859,
"timeout": 20000,
"name": "WebdriverIO",
"runtimeExecutable": "<span class="cp">${</span><span class="n">workspaceRoot</span><span class="cp">}</span>/node_modules/.bin/wdio",
"windows": {
"runtimeExecutable": "<span class="cp">${</span><span class="n">workspaceRoot</span><span class="cp">}</span>/node_modules/.bin/wdio.cmd"
},
"cwd": "<span class="cp">${</span><span class="n">workspaceRoot</span><span class="cp">}</span>",
"console": "integratedTerminal",
// This args config runs only the file that's open and displayed
// (e.g., a file in test/spec/):
"args":[
"--spec", "<span class="cp">${</span><span class="n">relativeFile</span><span class="cp">}</span>"
// To run a specific file, you can use:
//"--spec", "test/specs/foo.spec.js"
]
}
]
}
</pre></div>
<h3>Enable debugging in wdio.conf.js</h3>
<p>Open the <code>wdio.conf.js</code> in the editor. (It should be in the root of your project.)</p>
<p>Just under the <code>exports.config = {</code> line, add these lines:</p>
<div class="highlight"><pre><span></span> debug: true,
execArgv: ['--debug=127.0.0.1:5859'],
</pre></div>
<p>This allows VS Code to connect to the wdio runner for debugging.</p>
<p>If you forget this step, the first time you try to debug the IDE will likely show you the error message: <code>Cannot connect to runtime process</code></p>
<h3>First run</h3>
<p>Now it's time to run a test! Open the <code>test/specs/foo.spec.js</code> file in the IDE, and make sure it's the active tab in the editor (so that the file name will be what the IDE plugs into <code>${relativeFile}</code> in your debug config in <code>launch.json</code>).</p>
<p>Now click the green play button next to DEBUG in the IDE. You should see your test run to completion, since no breakpoints are set.</p>
<p><img alt="vscode-debug-play-button" src="http://blog.likewise.org/2017/02/debugging-a-javascript-webdriverio-project-in-vscode/images/vscode/debug-play-button.png" /></p>
<h3>Setting a breakpoint</h3>
<p>Now place a breakpoint in foo.spec.js, at line 11, by clicking the gutter to the left of the line number (you'll see a red dot appear).</p>
<p>Then press the debug button again and watch the IDE stop execution at line 11, as shown here:</p>
<p><img alt="vscode-breakpoint-hit" src="http://blog.likewise.org/2017/02/debugging-a-javascript-webdriverio-project-in-vscode/images/vscode/breakpoint-hit.png" /></p>
<p>Click the <code>DEBUG CONSOLE</code> menu, click the small prompt box at its bottom, and experiment with adding JS statements - for example:</p>
<div class="highlight"><pre><span></span>browser.getTitle()
</pre></div>
<p>That should print the browser window's title as shown here:</p>
<p><img alt="vscode-debug-console" src="http://blog.likewise.org/2017/02/debugging-a-javascript-webdriverio-project-in-vscode/images/vscode/debug-console.png" /></p>
<p>Now try inspecting elements interactively with the <code>$</code> and <code>$$</code> WebdriverIO helpers - for example:</p>
<div class="highlight"><pre><span></span>$$("div").length
$$("div")[3].getText()
</pre></div>
<p>Note that code completion works here! It comes in handy when guessing at the first few characters of functions you suspect might exist:</p>
<p><img alt="vscode-debug-console-code-completion" src="http://blog.likewise.org/2017/02/debugging-a-javascript-webdriverio-project-in-vscode/images/vscode/debug-console-code-completion.png" /></p>
<p>That's all for now!</p>Setting up XScreenSaver on Ubuntu 16.042017-01-22T19:00:00-08:002017-01-22T19:00:00-08:00Bill Ageetag:blog.likewise.org,2017-01-22:2017/01/setting-up-xscreensaver-on-ubuntu-1604/<p><img alt="xscreensaver-config screenshot" height="60%" src="http://blog.likewise.org/2017/01/setting-up-xscreensaver-on-ubuntu-1604/images/xscreensaver-config.png" width="60%" /></p>
<p>Do you have a burning desire (or a mandatory policy) to make your Ubuntu machine lock its display when you step away?</p>
<p>Or do you have a fondness (maybe a <em>burn-in</em> desire?) for a time when screensavers had other uses?</p>
<p>If so, this post describes how I installed XScreenSaver (and optional-but-essential add-ons like the <a target="_blank" href="https://www.youtube.com/watch?v=Q54NVuxhGso">CompanionCube screensaver</a>) on Ubuntu 16.04.</p>
<p>This script takes care of everything you need - just paste this into a shell script and run it, or paste each command one at a time.</p>
<p>These steps are based on the Unity setup notes in <a target="_blank" href="https://www.jwz.org/xscreensaver/man1.html">the xscreensaver man page</a>:</p>
<div class="highlight"><pre><span></span><span class="cp"># Install xscreensaver and addons:</span>
<span class="n">sudo</span> <span class="n">apt</span><span class="o">-</span><span class="n">get</span> <span class="n">install</span> \
<span class="n">xscreensaver</span> <span class="n">xscreensaver</span><span class="o">-</span><span class="n">data</span><span class="o">-</span><span class="n">extra</span> \
<span class="n">xscreensaver</span><span class="o">-</span><span class="n">gl</span> <span class="n">xscreensaver</span><span class="o">-</span><span class="n">gl</span><span class="o">-</span><span class="n">extra</span>
<span class="cp"># Uninstall the gnome-screensaver package:</span>
<span class="n">sudo</span> <span class="n">apt</span><span class="o">-</span><span class="n">get</span> <span class="n">remove</span> <span class="n">gnome</span><span class="o">-</span><span class="n">screensaver</span>
<span class="cp"># Make GNOME's "Lock Screen" use xscreensaver:</span>
<span class="n">sudo</span> <span class="n">ln</span> <span class="o">-</span><span class="n">sf</span> <span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">xscreensaver</span><span class="o">-</span><span class="n">command</span> \
<span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">gnome</span><span class="o">-</span><span class="n">screensaver</span><span class="o">-</span><span class="n">command</span>
<span class="cp"># Turn off Unity's built-in blanking.</span>
<span class="cp"># NOTE: For more options see 'gsettings list-keys org.gnome.desktop.screensaver'</span>
<span class="cp"># These two are equivalent to going to "System Settings / Brightness & Lock" and:</span>
<span class="cp"># * Setting "Turn screen off when inactive for" to "Never"</span>
<span class="cp"># * Switching the "Lock" toggle button to OFF</span>
<span class="n">gsettings</span> <span class="n">set</span> <span class="n">org</span><span class="p">.</span><span class="n">gnome</span><span class="p">.</span><span class="n">desktop</span><span class="p">.</span><span class="n">session</span> <span class="n">idle</span><span class="o">-</span><span class="n">delay</span> <span class="mi">0</span>
<span class="n">gsettings</span> <span class="n">set</span> <span class="n">org</span><span class="p">.</span><span class="n">gnome</span><span class="p">.</span><span class="n">desktop</span><span class="p">.</span><span class="n">screensaver</span> <span class="n">lock</span><span class="o">-</span><span class="n">enabled</span> <span class="nb">false</span>
<span class="cp"># Configure xscreensaver as a startup application:</span>
<span class="n">mkdir</span> <span class="o">-</span><span class="n">p</span> <span class="o">~/</span><span class="p">.</span><span class="n">config</span><span class="o">/</span><span class="n">autostart</span>
<span class="n">echo</span> <span class="s">"[Desktop Entry]</span>
<span class="n">Type</span><span class="o">=</span><span class="n">Application</span>
<span class="n">Exec</span><span class="o">=</span><span class="n">xscreensaver</span> <span class="o">-</span><span class="n">nosplash</span>
<span class="n">Hidden</span><span class="o">=</span><span class="nb">false</span>
<span class="n">NoDisplay</span><span class="o">=</span><span class="nb">false</span>
<span class="n">X</span><span class="o">-</span><span class="n">GNOME</span><span class="o">-</span><span class="n">Autostart</span><span class="o">-</span><span class="n">enabled</span><span class="o">=</span><span class="nb">true</span>
<span class="n">Name</span><span class="p">[</span><span class="n">en_US</span><span class="p">]</span><span class="o">=</span><span class="n">xscreensaver</span>
<span class="n">Name</span><span class="o">=</span><span class="n">xscreensaver</span>
<span class="n">Comment</span><span class="p">[</span><span class="n">en_US</span><span class="p">]</span><span class="o">=</span>
<span class="n">Comment</span><span class="o">=</span>
<span class="s">" > ~/.config/autostart/xscreensaver.desktop</span>
</pre></div>
<p>Once the install is complete, launch the <code>xscreensaver-demo</code> config util to pick the list of screensavers to run (among other settings).</p>
<p>Useful settings here are the "Blank After" and "Lock Screen After" values. You can change those if you need to lock your screen after a shorter idle time than the default values:</p>
<div class="highlight"><pre><span></span>xscreensaver-demo
</pre></div>
<p>Mostly for my own reference, these are the screensavers I currently have enabled in my config file (<code>/home/bill/.xscreensaver</code>):</p>
<div class="highlight"><pre><span></span>bill@foo:~$ cat /home/bill/.xscreensaver | grep "^ GL"
GL: lament -root -fps \n\
GL: sonar -root \n\
GL: stairs -root \n\
GL: sierpinski3d -root \n\
GL: molecule -root \n\
GL: glmatrix -root \n\
GL: polyhedra -root \n\
GL: glhanoi -root \n\
GL: tangram -root \n\
GL: topblock -root \n\
GL: companioncube -root -count 9 \n\
GL: kaleidocycle -root \n\
GL: unknownpleasures -root \n\
GL: splitflap -root \n\
</pre></div>Markdown ordered lists with paragraphs2017-01-03T17:26:00-08:002017-01-03T17:26:00-08:00Bill Ageetag:blog.likewise.org,2017-01-03:2017/01/markdown-ordered-lists-with-paragraphs/<p>Keeping a Markdown ordered list in proper working order has always been tricky for me. I always seem to end up up with indentation characters that reset the next item back to the number 1.</p>
<p>Here's how to make a list containing paragraphs without the list item numbers resetting.</p>
<p>The key appears to be to indent each line of content by an equal number of spaces - and indent both blank lines and those with text.</p>
<h2>Example markdown source:</h2>
<div class="highlight"><pre><span></span> 1. This is an example ordered list
Look, a paragraph!
And another!
1. This is the second item.
1. Here's another list with a code block.
Here's the first paragraph.
And here's some code:
:::bash
echo "Hello world!"
And another block of code:
:::bash
echo "Bye world!"
All done!
1. This is the fourth list item.
</pre></div>
<h2>Here's what the above looks like after rendering with pelican:</h2>
<ol>
<li>
<p>This is an example ordered list</p>
<p>Look, a paragraph!</p>
<p>And another!</p>
</li>
<li>
<p>This is the second item.</p>
</li>
<li>
<p>Here's another list with a code block.</p>
<p>Here's the first paragraph.</p>
<p>And here's some code:</p>
<div class="highlight"><pre><span></span><span class="nb">echo</span> <span class="s2">"Hello world!"</span>
</pre></div>
<p>And another block of code:</p>
<div class="highlight"><pre><span></span><span class="nb">echo</span> <span class="s2">"Bye world!"</span>
</pre></div>
<p>All done!</p>
</li>
<li>
<p>This is the fourth list item.</p>
</li>
</ol>Switching from Octopress to Pelican2016-08-23T07:00:00-07:002016-08-23T07:00:00-07:00Bill Ageetag:blog.likewise.org,2016-08-23:2016/08/switching-from-octopress-to-pelican/<h3>Or: What we blog about when we blog about blogs</h3>
<h3>🐙 ➡️ 🐍</h3>
<p>This post is the story of how I switched the underlying framework for this blog from <a target="_blank" href="http://octopress.org/">Octopress</a> to <a target="_blank" href="http://blog.getpelican.com/">Pelican</a>.</p>
<p>And I added a side quest: Like many Octopress users, I grew fond of the default Octopress theme, and wanted to keep it after moving to Pelican.</p>
<p>Keeping the Octopress theme turned out to be easy; the pelican-octopress-theme github repo was what I needed:</p>
<ul>
<li><a target="_blank" href="https://github.com/duilio/pelican-octopress-theme">https://github.com/duilio/pelican-octopress-theme</a></li>
</ul>
<p>And for other tips, Google revealed blog posts from other folks who made the Octopress-to-Pelican switch over the years. Trip reports I referred to were:</p>
<ul>
<li><a target="_blank" href="https://jakevdp.github.io/blog/2013/05/07/migrating-from-octopress-to-pelican/">https://jakevdp.github.io/blog/2013/05/07/migrating-from-octopress-to-pelican/</a></li>
<li><a target="_blank" href="http://themodernscientist.com/posts/2013/2013-06-02-my_octopelican_python_blog/">http://themodernscientist.com/posts/2013/2013-06-02-my_octopelican_python_blog/</a></li>
<li><a target="_blank" href="http://jhshi.me/2015/10/11/migrating-from-octopress-to-pelican/">http://jhshi.me/2015/10/11/migrating-from-octopress-to-pelican/</a></li>
</ul>
<h2>The plan</h2>
<p>First off, after <a target="_blank" href="http://docs.getpelican.com/">reading the docs</a>, I wrote down the steps I assumed would be involved with the move; here they are:</p>
<ol>
<li>Create and activate a virtualenv, then use pip to install pelican</li>
<li>Use the <code>pelican-quickstart</code> util to set up a barebones Pelican blog, as a starting point for the move</li>
<li>Get used to launching a dev server to preview changes</li>
<li>Apply the pelican-octopress-theme to the barebones blog</li>
<li>Bring my old blog's CSS and sidebar customizations into pelican-octopress-theme</li>
<li>Create a <a target="_blank" href="https://github.com/billagee/blog.likewise.org">public github repo</a> containing the work so far</li>
<li>Copy the old Markdown post files and images from my Octopress dir to my pelican repo</li>
<li>Edit posts as needed to get them to display properly</li>
<li>Back up the contents of my existing blog's s3 bucket (for easy rollback if the first deploy doesn't go smoothly)</li>
<li>Publish the new files to s3!</li>
</ol>
<h2>The reality</h2>
<p>Based on the outline above, here's what actually happened when I started carrying out the plan:</p>
<ol>
<li>
<h4>Installing pelican</h4>
<p>For this step I set up a new virtualenv and installed the pelican and markdown packages:</p>
<div class="highlight"><pre><span></span>mkdir blog.likewise.org
<span class="nb">cd</span> blog.likewise.org/
virtualenv env
. env/bin/activate
pip install pelican markdown
</pre></div>
<p>For the record, I was using OS X, Python 2.7.12 from Macports, and Pelican 3.6.3.</p>
</li>
<li>
<h4>Running <code>pelican-quickstart</code></h4>
<p>This part of the process went smoothly. The quickstart util didn't bring up any surprises, and produced the files that became the blog you're reading now.</p>
<p>Here's a log of the input I gave <code>pelican-quickstart</code> to bring up a site with a dev server script, Makefile, and s3 publishing support:</p>
<div class="highlight"><pre><span></span>pelican-quickstart
> Where <span class="k">do</span> you want to create your new web site? <span class="o">[</span>.<span class="o">]</span>
> What will be the title of this web site? Bill Agee<span class="err">'</span>s blog
> Who will be the author of this web site? Bill Agee
> What will be the default language of this web site? <span class="o">[</span>en<span class="o">]</span>
> Do you want to specify a URL prefix? e.g., http://example.com <span class="o">(</span>Y/n<span class="o">)</span> n
> Do you want to <span class="nb">enable</span> article pagination? <span class="o">(</span>Y/n<span class="o">)</span> n
> What is your <span class="nb">time</span> zone? <span class="o">[</span>Europe/Paris<span class="o">]</span> America/Los_Angeles
> Do you want to generate a Fabfile/Makefile to automate generation and publishing? <span class="o">(</span>Y/n<span class="o">)</span> Y
> Do you want an auto-reload <span class="p">&</span> simpleHTTP script to assist with theme and site development? <span class="o">(</span>Y/n<span class="o">)</span>
> Do you want to upload your website using FTP? <span class="o">(</span>y/N<span class="o">)</span>
> Do you want to upload your website using SSH? <span class="o">(</span>y/N<span class="o">)</span>
> Do you want to upload your website using Dropbox? <span class="o">(</span>y/N<span class="o">)</span>
> Do you want to upload your website using S3? <span class="o">(</span>y/N<span class="o">)</span> y
> What is the name of your S3 bucket? <span class="o">[</span>my_s3_bucket<span class="o">]</span> blog.likewise.org
> Do you want to upload your website using Rackspace Cloud Files? <span class="o">(</span>y/N<span class="o">)</span>
> Do you want to upload your website using GitHub Pages? <span class="o">(</span>y/N<span class="o">)</span>
Done. Your new project is available at /Users/bill/github/billagee/blog.likewise.org
</pre></div>
</li>
<li>
<h4>Bringing up a dev server to preview changes</h4>
<p>This was super easy. Assuming you answered yes to the auto-reload & simpleHTTP question in <code>pelican-quickstart</code>, just run <code>make devserver</code>.</p>
<p>Then point your web browser at <code>localhost:8000</code> to browse your site.</p>
<p>To stop your dev server, run <code>make stopserver</code>.</p>
</li>
<li>
<h4>Apply pelican-octopress-theme to the demo blog</h4>
<p>At this point I paused to see what the docs had to say about <a target="_blank" href="http://docs.getpelican.com/en/3.6.3/settings.html#themes">how to use themes in Pelican</a>.</p>
<p>To try out the stock pelican-octopress-theme, first clone its repo:</p>
<div class="highlight"><pre><span></span>git clone https://github.com/duilio/pelican-octopress-theme.git <span class="se">\</span>
~/github/duilio/pelican-octopress-theme
</pre></div>
<p>Then install it in your site with the <code>pelican-themes</code> util:</p>
<div class="highlight"><pre><span></span>pelican-themes --install ~/github/duilio/pelican-octopress-theme
</pre></div>
<p>There's one more step - to make sure the dev server can find the theme, edit the <code>develop_server.sh</code> file in your pelican dir, and place the <code>--theme-path</code> option and value in the <code>PELICANOPTS</code> variable:</p>
<div class="highlight"><pre><span></span><span class="c1"># In develop_server.sh - replace the empty PELICANOPTS= var if it's present:</span>
<span class="nv">PELICANOPTS</span><span class="o">=</span><span class="s2">"--theme-path ~/github/duilio/pelican-octopress-theme"</span>
</pre></div>
<p>This is a good time to make the same change in the <code>Makefile</code> in the root of your site, since you'll be needing it there too (note that the double quotes around the value are left out):</p>
<div class="highlight"><pre><span></span><span class="nv">PELICANOPTS</span><span class="o">=</span>--theme-path ~/github/billagee/pelican-octopress-theme
</pre></div>
<p>Restart your dev server after making those changes, and reload the site in your browser - you should then see the Octopress theme in place:</p>
<div class="highlight"><pre><span></span>make stopserver <span class="o">&&</span> make devserver
</pre></div>
</li>
<li>
<h4>Making a test post</h4>
<p>At this point I added a new step to the original plan: Learning how to create a post.</p>
<p>To do that using Markdown (adapted <a target="_blank" href="http://docs.getpelican.com/en/3.6.3/content.html">from the content docs</a>), you can cat out a post file to disk like so, if you're in your site's root dir:</p>
<div class="highlight"><pre><span></span><span class="c1"># Again, this assumes your current working dir is your blog's root dir:</span>
cat > content/2016-08-20-my-test-post.md <span class="s"><< EOL</span>
<span class="s">Title: My title</span>
<span class="s">Date: 2016-08-20 12:30</span>
<span class="s">Category: Python</span>
<span class="s">Tags: pelican, publishing</span>
<span class="s">Slug: my-test-post</span>
<span class="s">Authors: Your Name Here</span>
<span class="s">Summary: Short version for index and feeds</span>
<span class="s">This is the content of my test post.</span>
<span class="s">EOL</span>
</pre></div>
</li>
<li>
<h4>Bring my old blog's color and sidebar customizations into pelican-octopress-theme</h4>
<p>This is, of course, an optional step, but I wanted to change the header background color of the Octopress theme, and rearrange the sidebar.</p>
<p>This required a bit of tinkering, but it wasn't too bad.</p>
<p>To change the header background color in pelican-octopress-theme, I first made <a target="_blank" href="https://github.com/billagee/pelican-octopress-theme">my own fork</a> of <a target="_blank" href="https://github.com/duilio/pelican-octopress-theme">the original repo</a>.</p>
<p>Then, I cloned my fork, and changed the <code>$header-bg</code> value in the SASS file where it lives: <a target="_blank" href="https://github.com/billagee/pelican-octopress-theme/blob/master/sass/base/_theme.scss">sass/base/_theme.scss</a></p>
<div class="highlight"><pre><span></span><span class="c1">//$header-bg: #333 !default; // default octopress gray</span>
<span class="na">$header-bg</span><span class="o">:</span> <span class="mh">#35206f</span> <span class="nv">!default</span><span class="err">;</span> <span class="c1">// likewise purple</span>
</pre></div>
<p>There's one more step required - recompiling the theme's SASS files using the <code>compass</code> util. To install sass and compass and update the theme with your changes, <code>cd</code> to the root of your pelican-octopress-theme fork, and:</p>
<div class="highlight"><pre><span></span><span class="c1"># Run this in your pelican-octopress-theme root dir</span>
gem install sass compass
compass compile
</pre></div>
</li>
<li>
<h4>Create a <a target="_blank" href="https://github.com/billagee/blog.likewise.org">public github repo</a> containing the work so far</h4>
<p>Not much to say here, except that you might be interested in the <a target="_blank" href="https://github.com/billagee/blog.likewise.org/blob/master/.gitignore">.gitignore</a> file I created before pushing my repo:</p>
<div class="highlight"><pre><span></span>*~
._*
*.lock
*.DS_Store
*.swp
*.out
*.py<span class="o">[</span>cod<span class="o">]</span>
output
env
srv.pid
pelican.pid
</pre></div>
<p>Omit the <code>output</code> dir from your .gitignore if you want to keep your generated site files under version control.</p>
</li>
<li>
<h4>Copy the old Markdown post files from my Octopress dir to my pelican repo</h4>
<p>This is where things began to get real. I started by coping the post files from my <code>sources/_posts</code> Octopress dir to <code>content/</code>. I then committed everything as-is before making edits.</p>
<p>The first step was to convert the old Octopress post metadata, for example:</p>
<div class="highlight"><pre><span></span>---
layout: post
title: "Dockerized Ghostdriver Selenium Tests"
date: 2016-02-14 00:23
comments: true
categories: Docker Linux Selenium WebDriver Automation PhantomJS Python Testing
---
</pre></div>
<p>...to the corresponding Pelican Markdown:</p>
<div class="highlight"><pre><span></span><span class="n">Title</span><span class="o">:</span> <span class="n">Dockerized</span> <span class="n">Ghostdriver</span> <span class="n">Selenium</span> <span class="n">Tests</span>
<span class="n">Date</span><span class="o">:</span> <span class="mi">2016</span><span class="o">-</span><span class="mi">02</span><span class="o">-</span><span class="mi">14</span> <span class="mi">00</span><span class="o">:</span><span class="mi">23</span>
<span class="n">Tags</span><span class="o">:</span> <span class="n">Docker</span><span class="o">,</span> <span class="n">Linux</span><span class="o">,</span> <span class="n">Selenium</span><span class="o">,</span> <span class="n">WebDriver</span><span class="o">,</span> <span class="n">Automation</span><span class="o">,</span> <span class="n">PhantomJS</span><span class="o">,</span> <span class="n">Python</span>
<span class="n">Slug</span><span class="o">:</span> <span class="n">dockerized</span><span class="o">-</span><span class="n">ghostdriver</span><span class="o">-</span><span class="n">selenium</span><span class="o">-</span><span class="n">tests</span>
</pre></div>
<p>Note the addition of the <code>Slug</code> field, and replacing <code>categories:</code> with <code>Tags</code>, since Pelican offers both categories AND tags. And Pelican tags seem akin to Octopress categories.</p>
</li>
<li>
<h4>Edit posts as needed to get them to display properly</h4>
<p>I noticed a few indentation changes to existing code blocks in my posts were required to get them to display properly.</p>
<p>In the end, I switched over to using code blocks prefixed with <code>:::</code>, for syntax highlighting with <a target="_blank" href="http://docs.getpelican.com/en/3.6.3/content.html#syntax-highlighting">CodeHilite</a>. Here's an example:</p>
<p><script src="https://gist.github.com/billagee/b9bf022f1ffbc1c1d3e04e0c49c18ac5.js"></script></p>
</li>
<li>
<h4>Converting Octopress image tags</h4>
<p>There was another extra step required to get images in old posts to display - moving Octopress Markdown posts to Pelican means embedded images that use the Octopress <code>{% img %}</code> tag won't carry over directly without edits (or the <a target="_blank" href="https://github.com/jakevdp/pelican-plugins/tree/liquid_tags/liquid_tags">liquid_tags Pelican plugin</a>).</p>
<p>I didn't have too many images in my old posts, so I just replaced my old Octopress image tags, such as:</p>
<div class="highlight"><pre><span></span>{% img /images/fancybox.png 'fancybox screenshot' %}
</pre></div>
<p>...with the Pelican attach syntax:</p>
<div class="highlight"><pre><span></span><span class="o">!</span><span class="p">[</span><span class="n">Fancybox</span> <span class="n">Screenshot</span><span class="p">]({</span><span class="n">attach</span><span class="p">}</span><span class="n">images</span><span class="o">/</span><span class="n">fancybox</span><span class="p">.</span><span class="n">png</span><span class="p">)</span>
</pre></div>
<p>This was also the point when I finished copying all the image files from my old Octopress dir (<code>source/images/</code>) into my Pelican blog's <code>content/images/</code> dir.</p>
</li>
<li>
<h4>Back up the contents of my existing s3 bucket before copying any new files to the bucket</h4>
<p>Not much to say here. <code>s3cmd sync</code> was my means of doing this.</p>
</li>
<li>
<h4>Publish the new files to s3!</h4>
<p>With your AWS credentials in <code>~/.s3cfg</code>, just set your s3 bucket's name in your Pelican dir's <code>Makefile</code>, in the <code>S3_BUCKET</code> variable.</p>
<p>Then, to publish your changes to your s3 bucket:</p>
<div class="highlight"><pre><span></span>make clean
make s3_upload
</pre></div>
</li>
<li>
<h4>Manually fixing the <code>Content-Type</code> value for main.css</h4>
<p>I ran into a hiccup here - after publishing my files to s3, the MIME type guessing done during the upload apparently didn't work, so my main.css file was served by s3 as text/plain.</p>
<p>This resulted in the published copy of the blog in s3 displaying without CSS being applied; so, the appearance of the blog was totally broken in all browsers I tried. Oops.</p>
<p>Presumably this will only affect you if you're publishing to s3 like me; I didn't try deploying with Dropbox or the other options.</p>
<p>I worked around the problem by using the AWS console to manually change the Content-Type header value S3 serves up for my <code>main.css</code> file. (I still need to find a real fix for this, or at least a way of doing it with <code>s3cmd</code>.)</p>
<p>Steps for the fix were:</p>
<ul>
<li>
<p>After <code>make s3_upload</code> completes, log in to the AWS s3 console.</p>
</li>
<li>
<p>In the UI, navigate to: <em>All Buckets > /YOUR_BUCKET/theme/css</em></p>
</li>
<li>
<p>Right-click <code>main.css</code>, and view its properties</p>
</li>
<li>
<p>Under "Metadata", set the Content-Type key to <code>text/css</code> and save.</p>
</li>
</ul>
</li>
<li>
<h4>Using the pelican-alias plugin to create .html aliases to posts</h4>
<p>I'd forgotten that my very oldest Octopress posts were actually raw HTML migrated from Blogger years ago - long ago I jammed that content into Markdown files, and each Markdown file contained one massive, single-line <code><div></code> exported from Blogger.</p>
<p>Ugh! I couldn't let that stand, so I took the time to convert those old HTML posts to proper Markdown.</p>
<p>I then realized a few of those old Blogger-era HTML posts were still showing up in Google results, with their original .html extensions (which Pelican and Octopress don't create).</p>
<p>In the Octopress blog I'd resorted to creating s3 aliases for those old posts at publish time, in the bash script I used to upload everything with s3cmd.</p>
<p>That resulted in posts whose URLs ended in <code>2012-08-foo/</code> getting matching <code>2012-08-foo.html</code> siblings, in case someone visited the old Blogger URL.</p>
<p>In Pelican, you can use the <a target="_blank" href="https://pypi.python.org/pypi/pelican-alias">pelican-alias plugin</a> to create .html aliases for posts, which worked fine for me. See that plugin's docs for more info.</p>
<p>For more it boiled down to installing and activating the plugin, then adding an extra bit of metadata to the ancient posts that needed an .html alias:</p>
<div class="highlight"><pre><span></span>Alias: /2011/07/selenium-2-net-test-drive.html
</pre></div>
</li>
<li>
<h4>Google Analytics</h4>
<p>To set up Google Analytics, I put my GA ID in the <code>pelicanconf.py</code> file in the site's root dir, in the <code>GOOGLE_ANALYTICS</code> var.</p>
</li>
<li>
<h4>Modifying the order of items in the Octopress theme sidebar</h4>
<p>My old blog had some custom tweaks to the sidebar; preserving those (and adding some new ones) was mainly a matter of editing this file in my pelican-octopress-theme fork:</p>
<p><code>pelican-octopress-theme/templates/_includes/sidebar.html</code></p>
<p>The <a target="_blank" href="https://github.com/billagee/pelican-octopress-theme/commits/master">commit history for my pelican-octopress-theme</a> fork tells the full story of the sidebar.</p>
</li>
<li>
<h4>Github sidebar fix</h4>
<p>Either due to a bug in the theme, or as a result of my sidebar tweaks, the Github chunk of the sidebar was refusing to fetch the list of my github repos.</p>
<p>To get that working, I made this JS tweak in my fork's copy of <code>pelican-octopress-theme/templates/_includes/github.html</code> - namely pulling in jQuery.</p>
<p>This seems rather...not great, but it works for now:</p>
<div class="highlight"><pre><span></span> {% if GITHUB_SHOW_USER_LINK is defined %}
<a href="https://github.com/{{ GITHUB_USER }}">@{{ GITHUB_USER }}</a> on GitHub
{% endif %}
<span class="gi">+ <script src="https://code.jquery.com/jquery-3.1.0.min.js" integrity="sha256-cCueBR6CsyA4/9szpPfrX3s49M9vUU5BgtiJj06wt/s=" crossorigin="anonymous"></script></span>
<script type="text/javascript">
<span class="gd">- $.domReady(function(){</span>
<span class="gi">+ $(document).ready(function(){</span>
if (!window.jXHR){
var jxhr = document.createElement('script');
jxhr.type = 'text/javascript';
</pre></div>
</li>
</ol>
<h2>Whew</h2>
<p>That's all for now, I suppose! This concludes my first Pelican post.</p>Dockerized Ghostdriver Selenium Tests2016-02-14T00:23:00-08:002016-02-14T00:23:00-08:00Bill Ageetag:blog.likewise.org,2016-02-14:2016/02/dockerized-ghostdriver-selenium-tests/<p>(tl;dr: This post describes how to build a Docker image for running Python GhostDriver/PhantomJS tests in a container.)</p>
<h3>Background</h3>
<p>In <a target="_blank" href="/2013/04/webdriver-testing-with-python-and-ghostdriver/">a previous post</a> I described how to set up
an environment to run automated Selenium WebDriver tests using the Ghostdriver/PhantomJS/Python web testing stack.</p>
<p>But these days, that's the sort of setup chore which you might consider Dockerizing, to avoid needless manual repetition of your setup recipe. (Because if your testing project gets any sort of traction at all, you'll inevitably need to replicate your environment on multiple machines.)</p>
<h3>A side note about reusing images</h3>
<p>In this post we'll be building (from scratch) a Docker image capable of running GhostDriver tests, but in other situations, consider searching for an existing Docker image that does what you want - you might find one and be that much closer to your goal, whatever it may be!</p>
<p>To get started searching for images, see <a target="_blank" href="https://hub.docker.com/">https://hub.docker.com/</a></p>
<h2>Outline</h2>
<p>Down to business - let's build a Docker image.</p>
<p>This is an outline of the steps we'll be performing:</p>
<ol>
<li>
<p>We'll create a Docker image which contains:</p>
<ul>
<li>Ubuntu 14.04 (as the underlying base image)</li>
<li>PhantomJS</li>
<li>Python 2.7 and pip</li>
<li>The Selenium WebDriver Python bindings</li>
<li>A Python script that uses Ghostdriver and PhantomJS to perform a Google search test</li>
</ul>
</li>
<li>
<p>We'll then use that image to run a container that:</p>
<ul>
<li>Executes the Python Google search test script</li>
<li>Exits with an error if the test fails</li>
<li>Automatically removes the container when the test is complete</li>
</ul>
</li>
<li>
<p>Next, we'll do some interactive work in a container, by:</p>
<ul>
<li>Launching bash in a container (instead of the search test script)</li>
<li>In the containerized bash shell, we'll edit and manually run the modified test script</li>
</ul>
</li>
<li>
<p>We'll create a Makefile to repeat the tasks above with fewer keystrokes.</p>
</li>
<li>
<p>Finally, we'll push the image to a public repo on Docker Hub.</p>
</li>
</ol>
<p>Whew! Let's begin.</p>
<h2>1. Creating your Docker image</h2>
<ul>
<li>First, if you don't have Docker installed, follow the Docker Engine install guide for your OS, at:</li>
</ul>
<p><a target="_blank" href="https://docs.docker.com/engine/installation/">https://docs.docker.com/engine/installation/</a></p>
<p>I'm using a Mac to write this guide - specifically, I have Docker Toolbox 1.8.2a installed on OS X 10.11.</p>
<ul>
<li>
<p>For the first step, create a new dir and cd into it:</p>
<div class="highlight"><pre><span></span>mkdir myimage <span class="o">&&</span> <span class="nb">cd</span> myimage
</pre></div>
</li>
<li>
<p>Now create a file named <code>Dockerfile</code> in the <code>myimage</code> dir.</p>
</li>
</ul>
<p>Inside the empty Dockerfile, paste these lines:</p>
<div class="highlight"><pre><span></span>FROM ubuntu:14.04
<span class="c1"># Install the phantomjs browser, Python, and the Python Selenium bindings</span>
RUN apt-get update <span class="o">&&</span> apt-get install -y <span class="se">\</span>
phantomjs <span class="se">\</span>
python2.7 <span class="se">\</span>
python-pip <span class="se">\</span>
<span class="o">&&</span> pip install selenium
<span class="c1"># Run a Ghostdriver demo script</span>
ENV <span class="nv">my_test_script</span><span class="o">=</span>google-search-test.py
COPY <span class="si">${</span><span class="nv">my_test_script</span><span class="si">}</span> /
CMD <span class="s2">"/</span><span class="si">${</span><span class="nv">my_test_script</span><span class="si">}</span><span class="s2">"</span>
</pre></div>
<p>Notice the line <code>ENV my_test_script=google-search-test.py</code></p>
<p>That sets the <code>my_test_script</code> environment variable to the name of an executable script, which subsequently gets copied to <code>/</code> in your image (via the <code>COPY</code> instruction on the next line).</p>
<p>And eventually when we reach the point of launching a container, that Python script will be executed by way of the <code>CMD</code> instruction you see at the last line of the Dockerfile.</p>
<ul>
<li>Now, create the file <code>google-search-test.py</code> in the same dir as your Dockerfile, so that the <code>COPY</code> command has something to act on.</li>
</ul>
<p>For that script's content, you can start with a simple hello world example:</p>
<div class="highlight"><pre><span></span><span class="ch">#!/usr/bin/env python</span>
<span class="k">print</span> <span class="s2">"Hello world!"</span>
</pre></div>
<p>Or, you could go for the gusto and use a more complete GhostDriver script, such as the one from the Github repo related to this post:</p>
<p><a target="_blank" href="https://github.com/billagee/ghostdriver-py27/blob/master/google-search-test.py">https://github.com/billagee/ghostdriver-py27/blob/master/google-search-test.py</a></p>
<ul>
<li>
<p>Once the <code>google-search-test.py</code> file has been created, you need to make it executable so that it will also be executable in the container:</p>
<div class="highlight"><pre><span></span>chmod <span class="m">755</span> google-search-test.py
</pre></div>
</li>
<li>
<p>Now, try building your image:</p>
<div class="highlight"><pre><span></span>docker build --rm --force-rm -t myrepo/ghostdriver-py27 .
</pre></div>
</li>
</ul>
<p>Make sure not to omit the build command's trailing <code>.</code></p>
<p>The build command output should show the <code>apt-get update</code> and <code>apt-get install</code> output, and eventually show your Python script being copied into the image.</p>
<h2>2. Running a Docker container</h2>
<p>Now try executing your Python script in a container, with <code>docker run</code>.</p>
<div class="highlight"><pre><span></span>docker run -it --rm myrepo/ghostdriver-py27
</pre></div>
<p>When the container exits, you should see the output of your Python script.</p>
<p>e.g., if your Python script is the hi world example, the run output should resemble:</p>
<div class="highlight"><pre><span></span>$ docker run -it --rm myrepo/ghostdriver-py27
Hello world!
</pre></div>
<p>And here is the output when running the GhostDriver example script from <a target="_blank" href="https://github.com/billagee/ghostdriver-py27/blob/master/google-search-test.py">https://github.com/billagee/ghostdriver-py27/blob/master/google-search-test.py</a>:</p>
<div class="highlight"><pre><span></span>$ docker run -it --rm myrepo/ghostdriver-py27
Navigating to <span class="s1">'http://www.google.com'</span>...
Checking search box presence...
Performing search request...
current_url is now <span class="s1">'http://www.google.com/search?hl=en&source=hp&biw=&bih=&q=selenium&gbv=2&oq=selenium'</span>
.
----------------------------------------------------------------------
Ran <span class="m">1</span> <span class="nb">test</span> in <span class="m">2</span>.530s
OK
</pre></div>
<p>Note that if you make changes to the Python script, re-running the <code>docker build</code> command will add your new changes to the image.</p>
<p>Also take note of the <code>--rm</code> option, which causes <code>docker run</code> to destroy the container on exit. This is nice when rapidly making changes and re-running containers - when working in that fashion, it's better to have less container cruft to clean up later.</p>
<h2>3. Getting an interactive shell in a Docker container</h2>
<p>If you're new to Docker, you might be wondering how to launch a shell in a container and use it interactively.</p>
<p>Here's one way to do it - you can pass the name of an executable to run to <code>docker run</code>, which will override the Python script payload specified in the Dockerfile's <code>CMD</code> line.</p>
<p>Passing <code>bash</code> as the executable and using the <code>-it</code> options to <code>docker run</code> will give you a bash shell with which you can do anything you like - for example, installing more packages, modifying and re-running your test script, or experimenting with other changes you're considering adding to your Dockerfile.</p>
<p>The full command to get an interactive shell in a container looks like:</p>
<div class="highlight"><pre><span></span>docker run -it myrepo/ghostdriver-py27 bash
</pre></div>
<p>You should then see a shell prompt, which you can use to run arbitrary commands (as root) in your Ubuntu container.</p>
<p>For example, you might check the container's phantomjs version, or check the kernel and OS versions:</p>
<p>(Note I'm testing on a Mac running Docker Toolbox, so the <code>uname</code> output may differ from yours.)</p>
<div class="highlight"><pre><span></span>root@9ed850542508:/# phantomjs --version
<span class="m">1</span>.9.0
root@9ed850542508:/# python --version
Python <span class="m">2</span>.7.6
root@9ed850542508:/# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu <span class="m">14</span>.04.3 LTS
Release: <span class="m">14</span>.04
Codename: trusty
root@9ed850542508:/# uname -a
Linux 9ed850542508 <span class="m">4</span>.0.9-boot2docker <span class="c1">#1 SMP Thu Sep 10 20:39:20 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux</span>
</pre></div>
<p>In the container, the Python script that you placed in the image with <code>COPY ${my_test_script} /</code> can be found at <code>/google-search-test.py</code>:</p>
<div class="highlight"><pre><span></span>root@05f05f9d7537:/# ls -la /google-search-test.py
-rwxr-xr-x <span class="m">1</span> root root <span class="m">1065</span> Feb <span class="m">14</span> <span class="m">04</span>:08 /google-search-test.py
</pre></div>
<p>Another useful thing to do is to install your favorite editor, edit the container's Python script, then run the modified script manually:</p>
<div class="highlight"><pre><span></span>root@05f05f9d7537:/# apt-get install vim -y
<span class="c1"># ...snip package installation output...</span>
root@05f05f9d7537:/# vim /google-search-test.py
<span class="c1"># Make some edits, then launch your modified script:</span>
root@05f05f9d7537:/# /google-search-test.py
</pre></div>
<p>NOTE: When you exit the containerized shell, if the container was launched with <code>docker run --rm</code>,
the container will be destroyed, along with any changes to files you made while interactively working within it.</p>
<p>But if you don't use <code>docker run --rm</code>, once you exit the container shell, you'll see the container in the output of <code>docker ps -a</code>:</p>
<div class="highlight"><pre><span></span>$ docker run -it myrepo/ghostdriver-py27 bash
root@8a563421bdb3:/# <span class="nb">exit</span>
<span class="nb">exit</span>
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8a563421bdb3 myrepo/ghostdriver-py27 <span class="s2">"bash"</span> <span class="m">7</span> seconds ago Exited <span class="o">(</span><span class="m">0</span><span class="o">)</span> <span class="m">2</span> seconds ago mad_curie
</pre></div>
<p>To remove the container manually you can pass its CONTAINER ID or NAME to <code>docker rm</code>:</p>
<div class="highlight"><pre><span></span>$ docker rm 8a563421bdb3
8a563421bdb3
</pre></div>
<h2>4. Creating a Makefile</h2>
<p>This step is completely optional, but you may find it convenient.</p>
<p>If you're going to be frequently building your image and running containers on the command line, a Makefile can provide convenient shorthand commands to accomplish those tasks.</p>
<p>For example, building your image could look like:</p>
<div class="highlight"><pre><span></span>$ make build
</pre></div>
<p>And to run a container:</p>
<div class="highlight"><pre><span></span>$ make
<span class="c1"># ...or `make run` if you want to be explicit</span>
</pre></div>
<p>Launching a containerized shell could look like:</p>
<div class="highlight"><pre><span></span>$ make shell
</pre></div>
<p>If you're not familiar with Makefiles, setting one up simply involves creating a file named <code>Makefile</code> (in this case, you should put it in the same dir with your <code>Dockerfile</code>).</p>
<p>The example Makefile in the gist below provides <code>build</code>, <code>run</code>, <code>shell</code>, and <code>clean</code> targets - the latter deletes your local image using <code>docker rmi</code>.</p>
<p>If you don't want your Makefile to use the example image name (<code>myrepo/ghostdriver-py27</code>) used in this post, just change the value of the <code>repo_name</code> variable in the Makefile.</p>
<ul>
<li>NOTE! If running make gives you a separator error like:<div class="highlight"><pre><span></span>make
Makefile:11: *** missing separator. Stop.
</pre></div>
</li>
</ul>
<p>...then check to make sure all indentation in your Makefile is done with tab characters. If all else fails, use wget or curl to download the Makefile gist shown below. For example:</p>
<div class="highlight"><pre><span></span>wget https://gist.githubusercontent.com/billagee/a11874bb83d54ffcfaf8/raw/f4cd1e0bd88d56959286774adba77a81e7d2f20d/Makefile
</pre></div>
<script src="https://gist.github.com/billagee/a11874bb83d54ffcfaf8.js"></script>
<h2>5. Pushing your image to Docker Hub</h2>
<p>If you want to take the next step toward sharing your image with other users via Docker Hub, here's how to do that.</p>
<ul>
<li>
<p>First, create a Docker Hub account at <a target="_blank" href="https://hub.docker.com/">hub.docker.com</a>.</p>
</li>
<li>
<p>With that done, you can log in to Docker Hub's web UI and use the <code>Create Repository</code> button to make a new repo.</p>
</li>
</ul>
<p>Set the repository name to whatever you like (e.g., <code>experiment</code>), and choose whether to make the repo visibility public or private. Clicking the Create button wraps things up.</p>
<ul>
<li>
<p>To push your existing local image (<code>myrepo/ghostdriver-py27</code>) to Docker Hub without rebuilding it under a new name, you can perform these steps on the command line:</p>
<ul>
<li><code>docker login</code></li>
<li>Tag your existing image with the new repo's name: <code>docker tag myrepo/ghostdriver-py27 YOUR_DOCKER_USERNAME/YOUR_REPO_NAME</code></li>
<li>Push the image to its new repo in Docker Hub: <code>docker push YOUR_DOCKER_USERNAME/YOUR_REPO_NAME</code></li>
</ul>
</li>
</ul>
<p>Note you'll need to replace <code>YOUR_DOCKER_USERNAME/YOUR_REPO_NAME</code> with your Docker username and the Docker Hub repo name you chose - e.g., I used <code>billagee/experiment</code>, which looks like this on the CLI:</p>
<div class="highlight"><pre><span></span><span class="n">docker</span> <span class="n">tag</span> <span class="n">myrepo</span><span class="o">/</span><span class="n">ghostdriver</span><span class="o">-</span><span class="n">py27</span> <span class="n">billagee</span><span class="o">/</span><span class="n">experiment</span>
<span class="n">docker</span> <span class="n">push</span> <span class="n">billagee</span><span class="o">/</span><span class="n">experiment</span>
</pre></div>
<p>Once that step is completed, others will be able to <code>docker pull</code> your image.</p>
<p>Also on the topic of sharing images, the Github repo linked here shows an example of the finished Dockerfile, Makefile, and GhostDriver script produced by completing the steps in this post:</p>
<ul>
<li><a target="_blank" href="https://github.com/billagee/ghostdriver-py27">https://github.com/billagee/ghostdriver-py27</a></li>
</ul>
<p>And here's a Docker Hub repo linked to that Github repo - you can retrieve the latest image from this repo with <code>docker pull billagee/ghostdriver-py27</code></p>
<ul>
<li><a target="_blank" href="https://hub.docker.com/r/billagee/ghostdriver-py27/">https://hub.docker.com/r/billagee/ghostdriver-py27/</a></li>
</ul>
<p>An interesting feature to point out: A Docker Hub repo (like the one above) linked to a Github repo can be set up to build and push your image automatically when changes are made to the Github repo. You can also manually trigger builds in the Docker Hub web UI, or with an API call.</p>
<p>As an example, here are the results of a manually-triggered build of my image:</p>
<p><a target="_blank" href="https://hub.docker.com/r/billagee/ghostdriver-py27/builds/bycpbhriwttas2uuxkmbcu4/">https://hub.docker.com/r/billagee/ghostdriver-py27/builds/bycpbhriwttas2uuxkmbcu4/</a></p>
<p>For more info on the topic of automated builds, see <a target="_blank" href="https://docs.docker.com/docker-hub/builds/">https://docs.docker.com/docker-hub/builds/</a></p>
<p>Signing off until next time - and viva la containerism!</p>
<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">Word of the day: containerism<br><br>A social and economic order and ideology encouraging the deployment of containers in ever-increasing amounts.</p>— Dave Cheney (@davecheney) <a target="_blank" href="https://twitter.com/davecheney/status/692111975162236932">January 26, 2016</a></blockquote>
<script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script>Scrolling to an Element with the Python Bindings for Selenium WebDriver2015-04-26T23:43:00-07:002015-04-26T23:43:00-07:00Bill Ageetag:blog.likewise.org,2015-04-26:2015/04/scrolling-to-an-element-with-the-python-bindings-for-selenium-webdriver/<p>When using <a href="http://docs.seleniumhq.org/docs/03_webdriver.jsp">Selenium WebDriver</a>, you might encounter a situation where you need to scroll an element into view.</p>
<p>By running the commands in the following steps, you can interactively try out a solution using Google Chrome.</p>
<h3>1. Set up a scratch environment and install the selenium package</h3>
<p>My usual habit when starting a scratch project like this one is to set up a clean Python environment with virtualenv:</p>
<div class="highlight"><pre><span></span><span class="c1"># In your shell:</span>
mkdir scrolling
<span class="nb">cd</span> scrolling/
virtualenv env
. env/bin/activate
pip install selenium
</pre></div>
<h3>2. Launch the Python interpreter, open a Chrome session with the selenium package, and navigate to Google News</h3>
<div class="highlight"><pre><span></span><span class="err">$</span> <span class="n">python</span>
<span class="o">...</span>
<span class="o">>>></span> <span class="kn">from</span> <span class="nn">selenium</span> <span class="kn">import</span> <span class="n">webdriver</span>
<span class="o">>>></span> <span class="n">d</span> <span class="o">=</span> <span class="n">webdriver</span><span class="o">.</span><span class="n">Chrome</span><span class="p">()</span>
<span class="o">>>></span> <span class="n">d</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"http://news.google.com/"</span><span class="p">)</span>
</pre></div>
<h3>3. Identify the element that serves as the heading for the "Most popular" section of the page, then scroll to it by executing the JavaScript scrollIntoView() function</h3>
<div class="highlight"><pre><span></span><span class="o">>>></span> <span class="n">element</span> <span class="o">=</span> <span class="n">d</span><span class="o">.</span><span class="n">find_element_by_xpath</span><span class="p">(</span><span class="s2">"//span[.='Most popular']"</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">element</span><span class="o">.</span><span class="n">text</span>
<span class="sa">u</span><span class="s1">'Most popular'</span>
<span class="o">>>></span> <span class="n">d</span><span class="o">.</span><span class="n">execute_script</span><span class="p">(</span><span class="s2">"return arguments[0].scrollIntoView();"</span><span class="p">,</span> <span class="n">element</span><span class="p">)</span>
</pre></div>
<p>Note that the Python WebDriver bindings also offer the <a href="https://selenium-python.readthedocs.org/api.html#selenium.webdriver.remote.webelement.WebElement.location_once_scrolled_into_view">location_once_scrolled_into_view</a> property, which currently scrolls the element into view when retrieved.</p>
<p>However, that property is noted in the selenium module docs as subject to change without warning - and it also places the element at the bottom of the viewport (rather than the top), so I prefer using scrollIntoView().</p>
<h3>4. Scroll the element a few px down toward the center of the viewport, if necessary</h3>
<p>After the code above scrolls the element to the top of the window, you may find you need to scroll the document backwards to scoot the element slightly towards the center of the window - this can be necessary if the element is hidden under another element (for example, a toolbar that blocks clicks to the element you're interested in).</p>
<p>Such scrolling is easy to do - this JS scrolls the document backwards by 150px, placing your element closer to the center of the viewport:</p>
<div class="highlight"><pre><span></span><span class="o">>>></span> <span class="n">d</span><span class="o">.</span><span class="n">execute_script</span><span class="p">(</span><span class="s2">"window.scrollBy(0, -150);"</span><span class="p">)</span>
</pre></div>
<p>That's all for now; I suspect I'll continue to run into other types of element scrolling issues when using WebDriver, so this post may become the first in a series!</p>Setting Up ChromeDriver and the Selenium-WebDriver Python bindings on Ubuntu 14.042015-01-25T01:55:00-08:002015-01-25T01:55:00-08:00Bill Ageetag:blog.likewise.org,2015-01-25:2015/01/setting-up-chromedriver-and-the-selenium-webdriver-python-bindings-on-ubuntu-14-dot-04/<p>This post documents how to set up an Ubuntu 14.04 64-bit machine with everything you need to develop automated tests with <a href="http://docs.seleniumhq.org/docs/03_webdriver.jsp">Selenium-WebDriver</a>, Google Chrome, and <a href="https://sites.google.com/a/chromium.org/chromedriver/">ChromeDriver</a>, using the Python 2.7 release that ships with Ubuntu.</p>
<p>These steps might be useful to someone in the near term, and perhaps in the future this post could make for an interesting time capsule - remembering the WebDriver that was!</p>
<p>All steps assume you've just booted a fresh Ubuntu 14.04 64-bit machine and are at the command prompt:</p>
<h3>1. Download and install the latest Google Chrome release</h3>
<div class="highlight"><pre><span></span>wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
sudo dpkg -i --force-depends google-chrome-stable_current_amd64.deb
</pre></div>
<h3>2. Download and install the latest amd64 chromedriver release</h3>
<p>Here we use wget to fetch the version number of the latest release, then plug the version into another wget invocation in order to fetch the chromedriver build itself:</p>
<div class="highlight"><pre><span></span><span class="nv">LATEST</span><span class="o">=</span><span class="k">$(</span>wget -q -O - http://chromedriver.storage.googleapis.com/LATEST_RELEASE<span class="k">)</span>
wget http://chromedriver.storage.googleapis.com/<span class="nv">$LATEST</span>/chromedriver_linux64.zip
</pre></div>
<p>Symlink chromedriver into /usr/local/bin/ so it's in your PATH and available system-wide:</p>
<div class="highlight"><pre><span></span>unzip chromedriver_linux64.zip <span class="o">&&</span> sudo ln -s <span class="nv">$PWD</span>/chromedriver /usr/local/bin/chromedriver
</pre></div>
<h3>3. Install pip and virtualenv</h3>
<p>Using virtualenv allows you to install the Selenium Python bindings (and any other Python modules you might want) into an isolated environment, rather than the global packages dir, which (among other benefits) can help make your test environment easily reproducible on other machines:</p>
<div class="highlight"><pre><span></span>$ python -V
Python <span class="m">2</span>.7.6
$ sudo apt-get install python-pip
$ sudo pip install virtualenv
</pre></div>
<h3>4. Create a dir in which to install your virtualenv environment, and install and activate a new env</h3>
<p>More documentation on what's being done here is available in the <a href="https://virtualenv.pypa.io/en/latest/">virtualenv docs</a>.</p>
<div class="highlight"><pre><span></span>mkdir mytests <span class="o">&&</span> <span class="nb">cd</span> <span class="nv">$_</span>
virtualenv env
<span class="nb">source</span> env/bin/activate
</pre></div>
<h3>5. Install the Selenium bindings for Python in your virtualenv</h3>
<div class="highlight"><pre><span></span>$ pip install selenium
Collecting selenium
Downloading selenium-2.44.0.tar.gz <span class="o">(</span><span class="m">2</span>.6MB<span class="o">)</span>
<span class="m">100</span>% <span class="p">|</span><span class="c1">################################| 2.6MB 1.8MB/s </span>
Installing collected packages: selenium
Running setup.py install <span class="k">for</span> selenium
Successfully installed selenium-2.44.0
</pre></div>
<h3>6. Launch Python in interactive mode, and briefly ensure you can launch a browser with ChromeDriver</h3>
<p>Once the browser is open, navigate to www.google.com and print the document title:</p>
<div class="highlight"><pre><span></span><span class="err">$</span> <span class="n">python</span>
<span class="n">Python</span> <span class="mf">2.7</span><span class="o">.</span><span class="mi">6</span> <span class="p">(</span><span class="n">default</span><span class="p">,</span> <span class="n">Mar</span> <span class="mi">22</span> <span class="mi">2014</span><span class="p">,</span> <span class="mi">22</span><span class="p">:</span><span class="mi">59</span><span class="p">:</span><span class="mi">56</span><span class="p">)</span>
<span class="p">[</span><span class="n">GCC</span> <span class="mf">4.8</span><span class="o">.</span><span class="mi">2</span><span class="p">]</span> <span class="n">on</span> <span class="n">linux2</span>
<span class="n">Type</span> <span class="s2">"help"</span><span class="p">,</span> <span class="s2">"copyright"</span><span class="p">,</span> <span class="s2">"credits"</span> <span class="ow">or</span> <span class="s2">"license"</span> <span class="k">for</span> <span class="n">more</span> <span class="n">information</span><span class="o">.</span>
<span class="o">>>></span> <span class="kn">from</span> <span class="nn">selenium</span> <span class="kn">import</span> <span class="n">webdriver</span>
<span class="o">>>></span> <span class="n">d</span> <span class="o">=</span> <span class="n">webdriver</span><span class="o">.</span><span class="n">Chrome</span><span class="p">()</span>
<span class="o">>>></span> <span class="n">d</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"http://www.google.com/"</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">d</span><span class="o">.</span><span class="n">title</span>
<span class="sa">u</span><span class="s1">'Google'</span>
</pre></div>
<p>That's all you need to get started - the next step I would suggest is to explore how to run
Selenium scripts using <a href="http://pytest.org/latest/">pytest</a> or <a href="https://docs.python.org/2/library/unittest.html">unittest</a>. That sounds like good territory to cover in a subsequent post, so perhaps I'll revisit it!</p>Automatically capture browser screenshots after failed Python GhostDriver tests2015-01-22T01:37:00-08:002015-01-22T01:37:00-08:00Bill Ageetag:blog.likewise.org,2015-01-22:2015/01/automatically-capture-browser-screenshots-after-failed-python-ghostdriver-tests/<p><a href="https://github.com/detro/ghostdriver/">GhostDriver</a> is a fantastic tool, one which I've been happily using for a while now (and have <a href="http://blog.likewise.org/2013/04/webdriver-testing-with-python-and-ghostdriver/">briefly written about</a> before).</p>
<p>I feel it's worth mentioning that troubleshooting GhostDriver tests can seem like a challenge in and of itself if you're used to having a browser GUI to help you visually pinpoint problems in your tests.</p>
<p>This post describes a technique intended to make GhostDriver troubleshooting easier: How to capture a screenshot automatically if your test raises an exception.</p>
<p>Just as in <a href="http://darrellgrainger.blogspot.ca/2011/02/generating-screen-capture-on-exception.html">this blog post by Darrell Grainger</a>, we'll be using the <a href="http://selenium.googlecode.com/svn/branches/safari/docs/api/java/org/openqa/selenium/support/events/EventFiringWebDriver.html">EventFiringWebDriver</a> wrapper to take screenshots after test failures; but here we'll be using the Python WebDriver bindings rather than Java.</p>
<p>On that note, it's worth linking to the <a href="https://code.google.com/p/selenium/source/browse/py/test/selenium/webdriver/support/event_firing_webdriver_tests.py">unit test script for EventFiringWebDriver</a> found in the WebDriver Python bindings repo.</p>
<p>Here's the GhostDriver screenshot demo code - after running it, you should have a screenshot of the google.com homepage left behind in <tt>exception.png</tt>:</p>
<div class="highlight"><pre><span></span><span class="ch">#!/usr/bin/env python</span>
<span class="c1"># * Note: phantomjs must be in your PATH</span>
<span class="c1">#</span>
<span class="c1"># This script:</span>
<span class="c1"># - Navigates to www.google.com</span>
<span class="c1"># - Intentionally raises an exception by searching for a nonexistent element</span>
<span class="c1"># - Leaves behind a screenshot in exception.png</span>
<span class="kn">import</span> <span class="nn">unittest</span>
<span class="kn">from</span> <span class="nn">selenium</span> <span class="kn">import</span> <span class="n">webdriver</span>
<span class="kn">from</span> <span class="nn">selenium.webdriver.support.events</span> <span class="kn">import</span> <span class="n">EventFiringWebDriver</span>
<span class="kn">from</span> <span class="nn">selenium.webdriver.support.events</span> <span class="kn">import</span> <span class="n">AbstractEventListener</span>
<span class="k">class</span> <span class="nc">ScreenshotListener</span><span class="p">(</span><span class="n">AbstractEventListener</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">on_exception</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">exception</span><span class="p">,</span> <span class="n">driver</span><span class="p">):</span>
<span class="n">screenshot_name</span> <span class="o">=</span> <span class="s2">"exception.png"</span>
<span class="n">driver</span><span class="o">.</span><span class="n">get_screenshot_as_file</span><span class="p">(</span><span class="n">screenshot_name</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="s2">"Screenshot saved as '</span><span class="si">%s</span><span class="s2">'"</span> <span class="o">%</span> <span class="n">screenshot_name</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">TestDemo</span><span class="p">(</span><span class="n">unittest</span><span class="o">.</span><span class="n">TestCase</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">test_demo</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">pjsdriver</span> <span class="o">=</span> <span class="n">webdriver</span><span class="o">.</span><span class="n">PhantomJS</span><span class="p">(</span><span class="s2">"phantomjs"</span><span class="p">)</span>
<span class="n">d</span> <span class="o">=</span> <span class="n">EventFiringWebDriver</span><span class="p">(</span><span class="n">pjsdriver</span><span class="p">,</span> <span class="n">ScreenshotListener</span><span class="p">())</span>
<span class="n">d</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"http://www.google.com"</span><span class="p">)</span>
<span class="n">d</span><span class="o">.</span><span class="n">find_element_by_css_selector</span><span class="p">(</span><span class="s2">"div.that-does-not-exist"</span><span class="p">)</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="n">unittest</span><span class="o">.</span><span class="n">main</span><span class="p">()</span>
</pre></div>Installing Sikuli 1.0.1 on Ubuntu 12.042014-03-26T13:53:00-07:002014-03-26T13:53:00-07:00Bill Ageetag:blog.likewise.org,2014-03-26:2014/03/installing-sikuli-1-dot-0-1-on-ubuntu-12-dot-04/<p>While working on a <a href="http://stackoverflow.com/questions/22651721/sikuli-automation-in-ubuntu/22672339#22672339">stackoverflow answer</a> about Sikuli today, I noted that installing Sikuli on Ubuntu 12.04 isn't a one-step process - there are a few dependencies that need manual intervention before you even install it.</p>
<p>Here's the rundown of the steps that worked for me to get a simple Sikuli script working:</p>
<h3>1. Install the Oracle JRE</h3>
<p>I used version 1.7.0_51:</p>
<div class="highlight"><pre><span></span>$ java -version
java version <span class="s2">"1.7.0_51"</span>
Java<span class="o">(</span>TM<span class="o">)</span> SE Runtime Environment <span class="o">(</span>build <span class="m">1</span>.7.0_51-b13<span class="o">)</span>
Java HotSpot<span class="o">(</span>TM<span class="o">)</span> <span class="m">64</span>-Bit Server VM <span class="o">(</span>build <span class="m">24</span>.51-b03, mixed mode<span class="o">)</span>
</pre></div>
<p>Make sure java is in your PATH, or else the Sikuli IDE will have issues.</p>
<h3>2. Install OpenCV 2.4.0</h3>
<div class="highlight"><pre><span></span>sudo add-apt-repository ppa:gijzelaar/opencv2.4
sudo apt-get update
sudo apt-get libcv-dev
</pre></div>
<p>Alternatively, you can probably achieve the same by building/installing OpenCV 2.4.0 from source. I went the package route, though.</p>
<h3>3. Install Tesseract 3</h3>
<div class="highlight"><pre><span></span>sudo apt-get install libtesseract3
</pre></div>
<h3>4. Download and launch sikuli-setup.jar</h3>
<p>As recommended in the Sikuli install guide, I saved the installer to ~/SikuliX and ran it there as well.</p>
<div class="highlight"><pre><span></span>mkdir ~/SikuliX
<span class="nb">cd</span> ~/SikuliX <span class="o">&&</span> java -jar sikuli-setup.jar
</pre></div>
<p>From there, I selected the "Pack 1" option in the GUI and let setup proceed normally.</p>
<h3>5. Launch the Sikuli IDE, create a Sikuli script, and run it.</h3>
<p>To launch the IDE, I'm using the command:</p>
<div class="highlight"><pre><span></span>~/SikuliX/runIDE
</pre></div>
<p>If the IDE dies without an error after you try running your script with the <strong>Run</strong> button in the GUI, running your .sikuli project on the command line may help uncover what's going wrong.</p>
<p>To do so, you can use the "runIDE -r" option; you'll hopefully get much more info about the error.</p>
<p>For example, running the project "foo.sikuli" on the command line is as simple as:</p>
<div class="highlight"><pre><span></span>~/SikuliX/runIDE -r foo.sikuli
</pre></div>Using 7zip in lieu of GNU tar on the command line2014-03-25T21:55:00-07:002014-03-25T21:55:00-07:00Bill Ageetag:blog.likewise.org,2014-03-25:2014/03/using-7zip-in-lieu-of-gnu-tar-on-the-command-line/<p>These days I'm accustomed to having the <strong>7z</strong> command available on Unix-like systems (thanks to the <a target="_blank" href="http://p7zip.sourceforge.net/">p7zip</a> project).</p>
<p>On top of that, 7zip is always one of the first utils I install on any Windows machine I work with.</p>
<p>So as an exercise in cross-platform style (or just for the heck of it), I sometimes use 7z instead of tar when working with archive files.</p>
<p>Here's a list of basic file archiving tasks, with a comparison of how each is tackled with GNU tar versus 7zip:</p>
<h2>Compress and archive a directory, preserving paths</h2>
<p>Imagine you want to compress and archive the directory "foo/" and its contents:</p>
<div class="highlight"><pre><span></span>foo/
foo/level1/
foo/level1/level2/
foo/level1/level2/hi.txt
</pre></div>
<h3>tar</h3>
<p>With GNU tar you can create such an archive with:</p>
<div class="highlight"><pre><span></span>tar czf foo.tar.gz foo
</pre></div>
<h3>7zip</h3>
<p>To create a similar archive with 7zip (specifically, the <strong>7z</strong>, <strong>7z.exe</strong>, or <strong>7za.exe</strong> binaries), use the <strong>7z a</strong> command:</p>
<div class="highlight"><pre><span></span>7z a foo.7z foo
</pre></div>
<p>Interestingly, with 7zip you can also omit the name of the archive file to create; this results in an archive file with a .7z extension, otherwise named after the archived dir:</p>
<div class="highlight"><pre><span></span>7z a foo
</pre></div>
<p>Also note that the 7z format is the default archive type created, unless you specify an alternative type with the <strong>-t</strong> option.</p>
<h2>Extract an archive, recreating paths</h2>
<p>This is simple enough, and quite similar between the two tools:</p>
<h3>tar</h3>
<div class="highlight"><pre><span></span>tar xf foo.tar.gz
</pre></div>
<h3>7zip</h3>
<div class="highlight"><pre><span></span>7z x foo.7z
</pre></div>
<p>Note that the <strong>7z e</strong> command (which you may discover before <strong>7z x</strong>) will ignore the directory structure inside the archive, and extract every file and dir into your current dir. That behavior will come in handy for a later task.</p>
<h2>Determine whether a given file is in the archive</h2>
<h3>7zip</h3>
<p>With 7z, this is pretty straightforward when using the <strong>7z l</strong> (list) command combined with the <strong>-r</strong> (recurse) option:</p>
<div class="highlight"><pre><span></span>7z l -r foo.7z hi.txt
</pre></div>
<h3>tar</h3>
<p>With GNU tar, there are several ways to approach this task.</p>
<p>You can pass the full path to the file to <strong>tar tf</strong>, along with the archive file name, and tar will error out if there's no match inside the archive:</p>
<div class="highlight"><pre><span></span>tar tf foo.tar.gz foo/level1/level2/hi.txt
</pre></div>
<p>Or, if the original, unarchived dir structure is still present on disk, you can pass it to <strong>tar d</strong> (--diff), and tar will compare the archive with the unarchived dir:</p>
<div class="highlight"><pre><span></span>tar df foo.tar.gz foo/level1/level2/hi.txt
</pre></div>
<p>Note that BSD tar does not appear to have anything like the d/--diff option.</p>
<p>After all is said and done, piping <strong>tar t</strong> output to grep may be the most suitable option here:</p>
<div class="highlight"><pre><span></span>tar tf foo.tar.gz | grep hi.txt
</pre></div>
<h2>Extract a single file from an archive into the current dir</h2>
<p>This scenario is interesting, in that the task is noticeably simpler when using 7zip.</p>
<p>Let's say you want to extract <strong>hi.txt</strong> from the archive, placing the file in your current dir.</p>
<h3>7zip</h3>
<p>With 7z, you can use <strong>7z e -r</strong> to retrieve the file (in this case <strong>hi.txt</strong>), even if it's several levels down in the archive:</p>
<div class="highlight"><pre><span></span>7z e -r foo.7z hi.txt
</pre></div>
<h3>tar</h3>
<p>With GNU or BSD tar you'll need to count how many levels deep in the archive's dir hierarchy your file lives, and pass that number of leading dirs to remove from the output, using <strong>--strip-components</strong>:</p>
<div class="highlight"><pre><span></span>tar --strip-components=3 -xf foo.tar.gz foo/level1/level2/hi.txt
</pre></div>Using cURL to Access Bugzilla's XML-RPC API2013-09-17T14:43:00-07:002013-09-17T14:43:00-07:00Bill Ageetag:blog.likewise.org,2013-09-17:2013/09/using-curl-to-access-bugzillas-xml-rpc-api/<p>Today I had the chance to briefly explore <a target="_blank" href="https://bugzilla.readthedocs.io/en/latest/api/index.html">Bugzilla's API</a>.</p>
<p>I used curl to experiment with the XML-RPC API a bit - in the end I just scratched the surface of what's possible, but it was interesting nonetheless.</p>
<p>Here are a few examples of things you can do:</p>
<h2>Hello World</h2>
<p>A nice hello world example for the Bugzilla API is to query your Bugzilla server for its version, as <a target="_blank" href="http://pivotallabs.com/setting-up-and-troubleshooting-the-bugzilla-integration-in-tracker/">documented a while back</a> on the Pivotal Labs blog.</p>
<p>This example uses Mozilla's public server at https://bugzilla.mozilla.org.</p>
<p>Note the use of the <tt>Bugzilla.version</tt> methodName, and the way the output is piped to tidy for indentation and pretty-printing:</p>
<div class="highlight"><pre><span></span>curl --silent --insecure \
https://bugzilla.mozilla.org/xmlrpc.cgi \
-H "Content-Type: text/xml" \
-d "<span class="cp"><?xml version='1.0' encoding='UTF-8'?></span><span class="nt"><methodCall><methodName></span>Bugzilla.version<span class="nt"></methodName></span> <span class="nt"><params></span> <span class="nt"></params></span> <span class="nt"></methodCall></span>" \
| tidy -xml -indent -quiet
</pre></div>
<p>That command should output:</p>
<div class="highlight"><pre><span></span><span class="cp"><?xml version="1.0" encoding="utf-8"?></span>
<span class="nt"><methodResponse></span>
<span class="nt"><params></span>
<span class="nt"><param></span>
<span class="nt"><value></span>
<span class="nt"><struct></span>
<span class="nt"><member></span>
<span class="nt"><name></span>version<span class="nt"></name></span>
<span class="nt"><value></span>
<span class="nt"><string></span>4.2.6+<span class="nt"></string></span>
<span class="nt"></value></span>
<span class="nt"></member></span>
<span class="nt"></struct></span>
<span class="nt"></value></span>
<span class="nt"></param></span>
<span class="nt"></params></span>
<span class="nt"></methodResponse></span>
</pre></div>
<h2>XPath expressions</h2>
<p>To reduce visual clutter, and select specific elements, it's handy to use an XPath expression to extract values you're interested in.</p>
<p>For example, to select the version value from the above query, you can pipe curl's output to the xpath command-line program (which appears to ship with OS X):</p>
<div class="highlight"><pre><span></span>curl --silent --insecure \
https://bugzilla.mozilla.org/xmlrpc.cgi \
-H "Content-Type: text/xml" \
-d "<span class="cp"><?xml version='1.0' encoding='UTF-8'?></span><span class="nt"><methodCall><methodName></span>Bugzilla.version<span class="nt"></methodName></span> <span class="nt"><params></span> <span class="nt"></params></span> <span class="nt"></methodCall></span>" \
| xpath '//name[contains(text(), "version")]/../value/string/text()'
</pre></div>
<p>That command should print:</p>
<div class="highlight"><pre><span></span>Found 1 nodes:
-- NODE --
4.2.6+
</pre></div>
<h2>Getting bug data</h2>
<p>To take things a step further, here's another example - looking up the summary and creation_time values of a given bug ID.</p>
<p>The Bug.get method makes this possible, and an XPath expression that prints the text of the bug summary and creation_time values slims down the blob of XML returned by the API call.</p>
<p>This example will return information on bug 9940. Note how the bug ID is passed in the params list:</p>
<div class="highlight"><pre><span></span>curl --silent --insecure \
https://bugzilla.mozilla.org/xmlrpc.cgi \
-H "Content-Type: text/xml" \
-d "<span class="cp"><?xml version='1.0' encoding='UTF-8'?></span><span class="nt"><methodCall><methodName></span>Bug.get<span class="nt"></methodName></span> <span class="nt"><params><param><value><struct><member><name></span>ids<span class="nt"></name><value></span>9940<span class="nt"></value></member></struct></value></param></span> <span class="nt"></params></span> <span class="nt"></methodCall></span>" \
| xpath '//name[contains(text(), "summary")]/../value/string/text() | //name[contains(text(), "creation_time")]/../value/dateTime.iso8601/text()'
</pre></div>
<p>The result should show you bug 9940's creation date and awesome summary:</p>
<div class="highlight"><pre><span></span>Found 2 nodes:
-- NODE --
19990715T20:08:00-- NODE --
Bugzilla should have a party when 1,000,000 bugs get entered
</pre></div>
<p>Party like it's 1999!</p>
<p>Note that if your bugzilla server has authentication enabled, logging in via the API is also possible. A cookie can be obtained and used in subsequent requests.</p>OpenSSL oneliner to print a remote server's cert validity dates2013-07-31T18:27:00-07:002013-07-31T18:27:00-07:00Bill Ageetag:blog.likewise.org,2013-07-31:2013/07/openssl-oneliner-to-print-a-remote-servers-cert-validity-dates/<p>Today I wanted to check the notBefore and notAfter validity dates of an SSL cert installed on a remote server.</p>
<p>I immediately wondered if there was an easy way to use the <a target="_blank" href="http://www.openssl.org/docs/apps/openssl.html">OpenSSL command line tool</a> to accomplish this.</p>
<p>And there is - you just have to pass the output of <tt>openssl s_client</tt> to <tt>openssl x509</tt>, and away you go:</p>
<div class="highlight"><pre><span></span>echo |\
openssl s_client -connect www.google.com:443 2>/dev/null |\
sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' |\
openssl x509 -noout -subject -dates
</pre></div>
<p>That command should print the subject, notBefore, and notAfter dates of the certificate used by www.google.com:</p>
<div class="highlight"><pre><span></span>subject= /C=US/ST=California/L=Mountain View/O=Google Inc/CN=www.google.com
notBefore=Jul 12 08:56:36 2013 GMT
notAfter=Oct 31 23:59:59 2013 GMT
</pre></div>
<p>I picked up the specifics of how to do this over at the very useful <a target="_blank" href="http://www.madboa.com/geek/openssl/">OpenSSL Command-Line HOWTO</a> site. It's worth reading in depth.</p>Adding a fancybox gallery to a Rails 3.2 app in 5 steps2013-07-30T22:51:00-07:002013-07-30T22:51:00-07:00Bill Ageetag:blog.likewise.org,2013-07-30:2013/07/adding-a-fancybox-gallery-to-a-rails-3-dot-2-app-in-5-steps/<p>I was interested in seeing how quickly one can add a lightbox gallery to a Rails app nowadays.</p>
<p>As it happens, there's really not much to it, especially when using the <a target="_blank" href="https://github.com/hecticjeff/fancybox-rails">fancybox-rails gem</a>.</p>
<p>This post describes how to bring up an existing image viewer app (the "gallery-after" app from the github repo for <a target="_blank" href="http://railscasts.com/episodes/381-jquery-file-upload">Railscasts episode # 381</a>), then add fancybox support to it.</p>
<p>Here's what the end result will look like:</p>
<p><img alt="Fancybox Screenshot" src="http://blog.likewise.org/2013/07/adding-a-fancybox-gallery-to-a-rails-3-dot-2-app-in-5-steps/images/fancybox.png" /></p>
<h2>Setting up a Rails app that displays images</h2>
<ul>
<li>First order of business: We need a Rails app that displays images so we can fancybox it up!</li>
</ul>
<p>Rather than create one from scratch, let's grab an existing app.</p>
<p>As mentioned above, one of the apps from Railscasts episode # 381 will do nicely. To get the files from github:</p>
<div class="highlight"><pre><span></span>git clone https://github.com/railscasts/381-jquery-file-upload.git
</pre></div>
<p>When that completes, cd into the "gallery-after" app dir we'll be using:</p>
<div class="highlight"><pre><span></span>cd 381-jquery-file-upload/gallery-after/
</pre></div>
<ul>
<li>Note that the app depends on rmagick, and rmagick depends on ImageMagick.</li>
</ul>
<p>So next, install imagemagick. On OS X, you can use this homebrew command:</p>
<div class="highlight"><pre><span></span>brew install imagemagick
</pre></div>
<p>On Linux distributions, ImageMagick will more than likely be available in your package management system.</p>
<ul>
<li>This is the point where you'd normally do nothing more than type <tt>bundle</tt>, and the app would be usable in short order.</li>
</ul>
<p>But I ran into a snag:</p>
<p>On my system, trying to 'bundle install' failed on the rmagick gem, during extension compilation, with this error:</p>
<div class="highlight"><pre><span></span>"An error occurred while installing rmagick (2.13.1), and Bundler cannot continue."
</pre></div>
<p>The fix:</p>
<p>I modified <tt>gallery-after/Gemfile</tt> to make bundler fetch rmagick 2.13.2 - a version of the gem that resolves the install issue:</p>
<div class="highlight"><pre><span></span># In gallery-after/Gemfile, specify rmagick "2.13.2":
gem 'rmagick', '2.13.2'
</pre></div>
<p>Then:</p>
<div class="highlight"><pre><span></span>bundle
</pre></div>
<p>...and the installation of rmagick should succeed.</p>
<p>Side note: Near as I could tell, the rmagick problem is due to an incompatibility between rmagick 2.13.1 and the latest version of ImageMagick available via homebrew.</p>
<p>And the <tt>gallery-after/Gemfile.lock</tt> comes configured to install rmagick version 2.13.1, leading to the problem.</p>
<ul>
<li>After your 'bundle' command succeeds, configure your sqlite database:</li>
</ul>
<div class="highlight"><pre><span></span>bundle exec rake db:setup
</pre></div>
<ul>
<li>Launch the app:</li>
</ul>
<div class="highlight"><pre><span></span>bundle exec rails s
</pre></div>
<p>Point a browser at localhost:3000 and drag-and-drop some image files into your browser window.</p>
<p>This will insert the images into your DB, which will come in handy later so we have something to view in fancybox.</p>
<h2>Adding fancybox-rails to the app</h2>
<ul>
<li>Stop the running app, and edit your Gemfile. Add the fancybox-rails gem:</li>
</ul>
<div class="highlight"><pre><span></span># In Gemfile
gem 'fancybox-rails'
</pre></div>
<p>Then tell bundler to install it:</p>
<div class="highlight"><pre><span></span>bundle
</pre></div>
<ul>
<li>Next, per the steps on the <a target="_blank" href="http://hecticjeff.net/fancybox-rails/">fancybox-rails README</a>, add fancybox to your app's main JavaScript file:</li>
</ul>
<p>Edit <tt>app/assets/javascripts/application.js</tt> and add the fancybox line just under the jquery require statement that will already be in the file:</p>
<div class="highlight"><pre><span></span>//= require jquery
//= require fancybox
</pre></div>
<ul>
<li>Next, take care of the fancybox CSS file.</li>
</ul>
<p>Edit <tt>app/assets/stylesheets/application.css</tt> and add the fancybox line above the require_tree line:</p>
<div class="highlight"><pre><span></span><span class="cm">/*</span>
<span class="cm"> *= require_self</span>
<span class="cm"> *= require fancybox</span>
<span class="cm"> *= require_tree .</span>
<span class="cm"> */</span><span class="w"></span>
</pre></div>
<ul>
<li>Now, edit <tt>app/assets/javascripts/paintings.js.coffee</tt>, and at the end of the file, add the code to initialize fancybox for links that have the class value <tt>grouped_elements</tt>:</li>
</ul>
<div class="highlight"><pre><span></span>jQuery ->
$("a.grouped_elements").fancybox({
'transitionIn' : 'elastic',
'transitionOut' : 'elastic',
'speedIn' : 600,
'speedOut' : 200,
'overlayShow' : false
});
</pre></div>
<ul>
<li>Almost done!</li>
</ul>
<p>The last step is to add a gallery link to the paintings partial, where the link's <tt>class</tt> attribute value is set to the "grouped_elements" identifier we added to <tt>paintings.js.coffee</tt>.</p>
<p>Also, the gallery link's <tt>rel</tt> attribute value needs to be defined; in fancybox <a target="_blank" href="http://fancybox.net/howto">elements with the same rel value are considered part of the same gallery</a>, which enables flipping between the images without having to close the fancybox viewer.</p>
<p>To take care of those steps, edit <tt>app/views/paintings/_painting.html.erb</tt> and insert the "view in gallery" link shown below, above the existing edit/remove links:</p>
<div class="highlight"><pre><span></span><span class="x"> <div class="actions"></span>
<span class="cp"><%#</span><span class="c"> This is the line to add: </span><span class="cp">-%></span><span class="x"></span>
<span class="x"> </span><span class="cp"><%=</span> <span class="n">link_to</span> <span class="s2">"view in gallery"</span><span class="p">,</span> <span class="n">painting</span><span class="o">.</span><span class="n">image_url</span><span class="p">,</span> <span class="p">{</span> <span class="ss">:class</span> <span class="o">=></span> <span class="s2">"grouped_elements"</span><span class="p">,</span> <span class="ss">:rel</span> <span class="o">=></span> <span class="s2">"zomg_awesome_images"</span> <span class="p">}</span> <span class="cp">%></span><span class="x"> |</span>
<span class="x"> </span><span class="cp"><%=</span> <span class="n">link_to</span> <span class="s2">"edit"</span><span class="p">,</span> <span class="n">edit_painting_path</span><span class="p">(</span><span class="n">painting</span><span class="p">)</span> <span class="cp">%></span><span class="x"> |</span>
<span class="x"> </span><span class="cp"><%=</span> <span class="n">link_to</span> <span class="s2">"remove"</span><span class="p">,</span> <span class="n">painting</span><span class="p">,</span> <span class="ss">:confirm</span> <span class="o">=></span> <span class="s1">'Are you sure?'</span><span class="p">,</span> <span class="ss">:method</span> <span class="o">=></span> <span class="ss">:delete</span> <span class="cp">%></span><span class="x"></span>
</pre></div>
<p>That's all there is to it.</p>
<p>When you restart your Rails app, each image the app displays should now have a "view in gallery" link below it that launches fancybox, with navigation controls to skip from image to image!</p>
<p>Not too shabby for just a handful of extra lines of code.</p>Using Heroku Postgres as a Free Cloud Database Sandbox2013-06-16T13:51:00-07:002013-06-16T13:51:00-07:00Bill Ageetag:blog.likewise.org,2013-06-16:2013/06/using-heroku-postgres-as-a-free-cloud-database-sandbox/<p>Need a place to experiment with PostgreSQL, but not in the mood to set up the server locally?</p>
<p>Then try out the free dev plan on Heroko Postgres. No configuration or credit card required.</p>
<p>Creating a DB and manipulating it with the psql CLI can be done in just a few steps:</p>
<ul>
<li>If you don't already have a favorite postgres client, get the psql command-line program.</li>
</ul>
<p>If you're using OS X Lion or later, you already have psql; for older OS X installs (or if you want the server binaries too) you can install Postgres via Homebrew with:</p>
<div class="highlight"><pre><span></span>brew install postgresql
</pre></div>
<p>For other platforms, <a href="http://www.postgresql.org/download/">download and install the PostgreSQL binaries</a> for your machine.</p>
<ul>
<li>
<p>Next you'll need a Heroku account - so if you don't already have one, <a href="https://id.heroku.com/signup">sign up</a>.</p>
</li>
<li>
<p>Then hit <a href="https://postgres.heroku.com/databases">https://postgres.heroku.com/databases</a>.</p>
</li>
<li>
<p>Next, click the "Create Database" button.</p>
</li>
</ul>
<p>If you're shown a pricing page with plans to choose from, first click "Dev Plan (free)", then click "Add Database".</p>
<ul>
<li>To get a convenient command you can copy and paste to launch the Postgres CLI on your local machine, click the name of your database, then click the connection settings button.</li>
</ul>
<p>You should see something along the lines of:</p>
<p><img alt="heroku postgres connection settings" src="http://blog.likewise.org/2013/06/using-heroku-postgres-as-a-free-cloud-database-sandbox/images/heroku/conn_settings.png" /></p>
<ul>
<li>
<p>In the menu, click PSQL, and a command will appear (already selected!) that you can copy and paste into your terminal to connect the psql command-line program to your database.</p>
</li>
<li>
<p>That's it! Assuming psql is in your path, pasting the psql command will put you at an interactive prompt, and you'll be ready to create tables and experiment as you like.</p>
</li>
</ul>
<p>Here's an example session, in which a crude music database is created and queried:</p>
<div class="highlight"><pre><span></span>$ psql <span class="s2">"dbname=YOUR_DB_NAME host=YOUR_EC2_HOST user=YOUR_USER password=YOUR_PASS port=5432 sslmode=require"</span>
<span class="o">=</span>> <span class="se">\d</span>
No relations found.
<span class="o">=</span>>
CREATE TABLE artists <span class="o">(</span>id int, name varchar<span class="o">(</span><span class="m">80</span><span class="o">))</span><span class="p">;</span>
CREATE TABLE releases <span class="o">(</span>id int, name varchar<span class="o">(</span><span class="m">80</span><span class="o">))</span><span class="p">;</span>
CREATE TABLE recordings <span class="o">(</span>id int, artist_id int, release_id int, name varchar<span class="o">(</span><span class="m">80</span><span class="o">))</span><span class="p">;</span>
INSERT INTO artists <span class="o">(</span>id, name<span class="o">)</span> VALUES <span class="o">(</span><span class="m">1</span>, <span class="s1">'Underworld'</span><span class="o">)</span><span class="p">;</span>
INSERT INTO releases <span class="o">(</span>id, name<span class="o">)</span> VALUES <span class="o">(</span><span class="m">1</span>, <span class="s1">'Oblivion With Bells'</span><span class="o">)</span><span class="p">;</span>
INSERT INTO recordings <span class="o">(</span>id, artist_id, release_id, name<span class="o">)</span> VALUES <span class="o">(</span><span class="m">1</span>, <span class="m">1</span>, <span class="m">1</span>, <span class="s1">'To Heal'</span><span class="o">)</span><span class="p">;</span>
INSERT INTO artists <span class="o">(</span>id, name<span class="o">)</span> VALUES <span class="o">(</span><span class="m">2</span>, <span class="s1">'Stars'</span><span class="o">)</span><span class="p">;</span>
INSERT INTO releases <span class="o">(</span>id, name<span class="o">)</span> VALUES <span class="o">(</span><span class="m">2</span>, <span class="s1">'In Our Bedroom After the War'</span><span class="o">)</span><span class="p">;</span>
INSERT INTO recordings <span class="o">(</span>id, artist_id, release_id, name<span class="o">)</span> VALUES <span class="o">(</span><span class="m">2</span>, <span class="m">2</span>, <span class="m">2</span>, <span class="s1">'The Night Starts Here'</span><span class="o">)</span><span class="p">;</span>
/* Get all recordings of each artist, and show the release */
SELECT rec.name AS recording, a.name AS artist, rel.name AS release
FROM recordings AS rec
INNER JOIN artists AS a
ON rec.artist_id <span class="o">=</span> a.id
INNER JOIN releases AS rel
ON rec.release_id <span class="o">=</span> rel.id<span class="p">;</span>
recording <span class="p">|</span> artist <span class="p">|</span> release
-----------------------+------------+------------------------------
To Heal <span class="p">|</span> Underworld <span class="p">|</span> Oblivion With Bells
The Night Starts Here <span class="p">|</span> Stars <span class="p">|</span> In Our Bedroom After the War
<span class="o">(</span><span class="m">2</span> rows<span class="o">)</span>
</pre></div>Headless Selenium WebDriver Testing With Python and Ghost Driver2013-04-16T23:34:00-07:002013-04-16T23:34:00-07:00Bill Ageetag:blog.likewise.org,2013-04-16:2013/04/webdriver-testing-with-python-and-ghostdriver/<p><a target="_blank" href="https://github.com/detro/ghostdriver/">GhostDriver</a> is a project that lets you write Selenium WebDriver automation tests that run using the <a target="_blank" href="http://phantomjs.org/">PhantomJS</a> headless WebKit, instead of a traditional web browser.</p>
<p>Put another way, PhantomJS can replace Firefox and friends in your WebDriver scripts - and it doesn't require a display, so testing complex web apps from the command line is just about as easy as using a GUI browser. Very cool!</p>
<p>Getting your system ready to run Python scripts that use GhostDriver can be done in a few brief steps, if you have homebrew on OS X.</p>
<p>First you'll need the Selenium python package:</p>
<div class="highlight"><pre><span></span>sudo pip install selenium
</pre></div>
<p>Then, use homebrew to install PhantomJS:</p>
<div class="highlight"><pre><span></span>brew install phantomjs
</pre></div>
<p>If you don't want to use homebrew (or you're not on a Mac) you can simply <a target="_blank" href="http://phantomjs.org/download.html">download the latest PhantomJS build manually</a> and install it.</p>
<p>Believe it or not, that is all. GhostDriver is integrated into PhantomJS, so you should now be set up to take a test drive.</p>
<p>A typical Hello World program in the web automation world is one that performs a Google search. So, here's what that looks like in Python and GhostDriver, using Python in interactive mode:</p>
<div class="highlight"><pre><span></span><span class="err">$</span> <span class="n">python</span>
<span class="o">>>></span> <span class="kn">from</span> <span class="nn">selenium</span> <span class="kn">import</span> <span class="n">webdriver</span>
<span class="o">>>></span> <span class="n">driver</span> <span class="o">=</span> <span class="n">webdriver</span><span class="o">.</span><span class="n">PhantomJS</span><span class="p">(</span><span class="s1">'phantomjs'</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">driver</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"http://www.google.com"</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">driver</span><span class="o">.</span><span class="n">title</span>
<span class="sa">u</span><span class="s1">'Google'</span>
<span class="o">>>></span> <span class="n">driver</span><span class="o">.</span><span class="n">current_url</span>
<span class="sa">u</span><span class="s1">'http://www.google.com/'</span>
<span class="o">>>></span> <span class="n">driver</span><span class="o">.</span><span class="n">find_element_by_name</span><span class="p">(</span><span class="s2">"q"</span><span class="p">)</span><span class="o">.</span><span class="n">is_displayed</span><span class="p">()</span>
<span class="bp">True</span>
<span class="o">>>></span> <span class="n">driver</span><span class="o">.</span><span class="n">find_element_by_name</span><span class="p">(</span><span class="s2">"q"</span><span class="p">)</span><span class="o">.</span><span class="n">send_keys</span><span class="p">(</span><span class="s2">"selenium"</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">driver</span><span class="o">.</span><span class="n">find_element_by_name</span><span class="p">(</span><span class="s2">"btnG"</span><span class="p">)</span><span class="o">.</span><span class="n">click</span><span class="p">()</span>
<span class="o">>>></span> <span class="n">driver</span><span class="o">.</span><span class="n">current_url</span>
<span class="sa">u</span><span class="s1">'http://www.google.com/search?hl=en&source=hp&q=selenium&gbv=2&oq=selenium'</span>
</pre></div>
<p>A natural next step, when developing automated test cases based on experiments like the one above, is to start storing your code into a Python unittest script.</p>
<p>Here's an example of how one might start organizing the code above:</p>
<p>{% gist 5402386 test_google_ghost_driver.py %}</p>
<p>If all is right with the world, running the above script will print output along the lines of the text below.</p>
<div class="highlight"><pre><span></span>$ ./test_google_ghost_driver.py
current_url is now <span class="s1">'http://www.google.com/search?hl=en&source=hp&q=selenium&gbv=2&oq=selenium'</span>
.
----------------------------------------------------------------------
Ran <span class="m">1</span> <span class="nb">test</span> in <span class="m">2</span>.770s
OK
</pre></div>
<p>That's all for the moment. Now, go forth and Ghost Drive!</p>Using TextMate to rapidly test Java code examples/SSCCEs2013-04-14T11:26:00-07:002013-04-14T11:26:00-07:00Bill Ageetag:blog.likewise.org,2013-04-14:2013/04/using-textmate-to-rapidly-test-java-code-examples-slash-sscces/<p>My last post was about executing small Java programs from within vim, without leaving the editor to manually open a shell.</p>
<p>The goal was to rapidly execute example code for your own edification, or when showing code to other people - basically any time you need a <a target="_blank" href="http://sscce.org/">Short, Self Contained, Correct (Compilable), Example</a> (also known as an SSCCE).</p>
<p>While vim does indeed work for that purpose, I feel <a target="_blank" href="http://macromates.com/">TextMate</a> has the edge when it comes to running Java examples.</p>
<p>Here's what setting up and using TextMate to run Java programs on a Mac looks like:</p>
<ul>
<li>After you install TextMate, open the Bundle settings (in 'TextMate > Preferences > Bundles') and make sure the 'Java' checkbox is set:</li>
</ul>
<p><img alt="TextMate bundle dialog" src="http://blog.likewise.org/2013/04/using-textmate-to-rapidly-test-java-code-examples-slash-sscces/images/textmate/java-bundle.png" /></p>
<ul>
<li>
<p>Open or compose your demo program, and make sure you've saved the file to disk.</p>
</li>
<li>
<p>In the bottom status bar, Make sure the Java bundle is selected:</p>
</li>
</ul>
<p><img alt="TextMate status bar with Java" src="http://blog.likewise.org/2013/04/using-textmate-to-rapidly-test-java-code-examples-slash-sscces/images/textmate/java-status-bar.png" /></p>
<ul>
<li>Now simply use Command-R to run your code! A new window should open to display the output.</li>
</ul>
<p><img alt="TextMate Java output" src="http://blog.likewise.org/2013/04/using-textmate-to-rapidly-test-java-code-examples-slash-sscces/images/textmate/java-output.png" /></p>Testing Java snippets with Vim and GroovyConsole2013-03-28T07:54:00-07:002013-03-28T07:54:00-07:00Bill Ageetag:blog.likewise.org,2013-03-28:2013/03/testing-java-snippets-with-vim-and-groovyconsole/<p>For instructional purposes (either when experimenting on your own, or when demonstrating code to others) it's always useful to be able to run snippets of code in a <a target="_blank" href="http://en.wikipedia.org/wiki/REPL">REPL</a>, or a similar environment allowing fast turnaround in the edit/compile/run cycle.</p>
<p>When using Java, other IDEs offer ways to get REPL-like behavior, but what if you don't want to use a traditional Java IDE?</p>
<p>Perhaps you just want to demonstrate a trivial bit of code without much overhead.</p>
<p>In that situation, a couple of nice options for Java are:</p>
<ul>
<li>Use GroovyConsole as a Java REPL</li>
<li>Edit your code in Vim, and compile and run it without leaving the editor</li>
</ul>
<h3>1. Using GroovyConsole</h3>
<p>If you're on a Mac, GroovyConsole can be installed via the homebrew groovy formula (or, just get it from <a target="_blank" href="http://groovy.codehaus.org">http://groovy.codehaus.org</a>):</p>
<pre>brew install groovy</pre>
<p>Launch GroovyConsole with this command:</p>
<pre>groovyConsole</pre>
<p>Then simply type in a code snippet and run it with <Command-R> (or on Windows, <CTRL-R>):</p>
<p><img alt="groovyConsole screenshot" src="http://blog.likewise.org/2013/03/testing-java-snippets-with-vim-and-groovyconsole/images/groovyConsole.png" /></p>
<h3>2. Using Vim as an improvised Java IDE</h3>
<p><img alt="foo dot java" src="http://blog.likewise.org/2013/03/testing-java-snippets-with-vim-and-groovyconsole/images/vim/foo-dot-java.png" /></p>
<p>First, launch vim and write a small program - for example, Foo.java:</p>
<p><img alt="running javac from vim" src="http://blog.likewise.org/2013/03/testing-java-snippets-with-vim-and-groovyconsole/images/vim/foo-javac.png" /></p>
<p>Then, compile your program without leaving vim by passing the file open in your vim buffer to javac, using the <tt>:!</tt> command sequence and <tt>%</tt></p>
<p><img alt="ENTER prompt from vim" src="http://blog.likewise.org/2013/03/testing-java-snippets-with-vim-and-groovyconsole/images/vim/foo-enter-to-continue.png" /></p>
<p>If all goes well, you'll temporarily be dropped to the shell, with no visible errors, and get prompted to press ENTER to continue back to vim:</p>
<p><img alt="running Foo.class in java" src="http://blog.likewise.org/2013/03/testing-java-snippets-with-vim-and-groovyconsole/images/vim/java-foo.png" /></p>
<p>Back in the editor, use <tt>:!java Foo</tt> to invoke the Java class file you just created with javac:</p>
<p><img alt="Foo.class output" src="http://blog.likewise.org/2013/03/testing-java-snippets-with-vim-and-groovyconsole/images/vim/output.png" /></p>
<p>Finally, you'll see your program's output in the console.</p>
<p>For further fun, try compiling with <tt>javac -g %</tt>, then launch your class file with <tt>jdb Foo</tt> to debug your program from within vim.</p>
<p>Also, you might consider taking the <a target="_blank" href="http://www.vim.org/scripts/script.php?script_id=1785">javacomplete omni-completion plugin</a> for a spin.</p>Raspberry Pi terminal screenshots with fbgrab2013-01-13T11:26:00-08:002013-01-13T11:26:00-08:00Bill Ageetag:blog.likewise.org,2013-01-13:2013/01/raspberry-pi-terminal-screenshots-with-fbgrab/<p>Say you're on the console on your Raspberry Pi, and you want to take a screenshot. But without X running, what does one do?</p>
<p>Simple: Use fbgrab. To install it, just:</p>
<div class="highlight"><pre><span></span>sudo apt-get install fbgrab
</pre></div>
<p>Pass fbgrab the name of the virtual terminal/tty you want to snapshot, and it'll spit out a PNG file.</p>
<p>For example, say you have an awesome console program running on /dev/tty1, and want to screenshot it - just run:</p>
<div class="highlight"><pre><span></span>sudo fbgrab -c 1 screenshot.png
</pre></div>
<p>And that's it! Here's an example of the output:</p>
<p><img alt="cmus screenshot" src="http://blog.likewise.org/2013/01/raspberry-pi-terminal-screenshots-with-fbgrab/images/screenshot.png" /></p>Overview of configuring a static website in Amazon S32013-01-12T01:58:00-08:002013-01-12T01:58:00-08:00Bill Ageetag:blog.likewise.org,2013-01-12:2013/01/overview-of-configuring-a-static-website-in-amazon-s3/<p>Lately I've been looking for reasons to use <a href="http://www.omnigroup.com/products/omnigraffle/">OmniGraffle</a> for diagramming.</p>
<p>And with the steps to set up static web hosting in S3 (as mentioned in my last post) still fresh in mind, it seems like a good time to document what I did. And the steps are simple enough that a diagram might just be enough to cover the important bits.</p>
<p>So here it is - someday it'll be interesting to look back on this and see what's easier/more streamlined in this process, and what remains the same:</p>
<p><img alt="S3 hosting howto" src="http://blog.likewise.org/2013/01/overview-of-configuring-a-static-website-in-amazon-s3/images/s3hosting-howto.png" /></p>Hello Octopress!2013-01-11T00:08:00-08:002013-01-11T00:08:00-08:00Bill Ageetag:blog.likewise.org,2013-01-11:2013/01/hello-octopress/<p>New year, new hosting stack! This blog is now brought to you by <a target="_blank" href="http://octopress.org/">Octopress</a>, hosted on Amazon S3, instead of Blogger.</p>
<p>After spending some quality time with the tools I used in the move from Blogger (Octopress, S3, Route 53, git), I feel like the proverbial new pair of glasses has descended and shown me a new world.</p>
<p><a target="_blank" href="http://en.wikipedia.org/wiki/They_Live"><img alt="they life gif" src="http://blog.likewise.org/2013/01/hello-octopress/images/they_live_deal_w_it.gif" /></a></p>
<p>So what are the benefits of Octopress? To list just a few:</p>
<ul>
<li><a target="_blank" href="http://octopress.org/docs/blogging/code/">Solarized syntax highlighting built in</a>. This was all I had to hear to get really interested.</li>
<li>Easy editing of posts in the editor of your choice, using markdown, instead of wrestling the blogger tools.</li>
<li>Cheap! For my minimal custom domain hosting needs, S3/Route 53 costs should be super low.</li>
<li>Applying third-party themes is super easy, should you get the urge to <a target="_blank" href="https://github.com/imathis/octopress/wiki/3rd-Party-Octopress-Themes">check out alternatives</a>.</li>
<li>Storing everything in a git repo is painless, so no worries when playing around with new themes, or making other significant edits. In blogger-land this didn't feel anywhere near as easy.</li>
</ul>
<p>To sum up, maintaining this blog just became a lot more enjoyable.</p>
<p>Some helpful Octopress references were:</p>
<ul>
<li><a href="http://approache.com/blog/migrating-from-blogger-to-octopress/">http://approache.com/blog/migrating-from-blogger-to-octopress/</a></li>
<li><a href="http://mattgemmell.com/2011/09/12/blogging-with-octopress/">http://mattgemmell.com/2011/09/12/blogging-with-octopress/</a></li>
<li><a href="http://www.julianbonilla.com/blog/2012/11/05/octopress-on-amazon-s3/">http://www.julianbonilla.com/blog/2012/11/05/octopress-on-amazon-s3/</a></li>
</ul>Pure Ruby example of calling functions from the MS IUIAutomation COM interface2012-04-14T04:35:00-07:002012-04-14T04:35:00-07:00Bill Ageetag:blog.likewise.org,2012-04-14:2012/04/pure-ruby-example-of-calling-functions/<p><img alt="Placeholder screenshot" src="http://blog.likewise.org/2012/04/pure-ruby-example-of-calling-functions/images/print_root_element_name_cmd.png" /></p>
<p>A while back I was interested in writing some Ruby code that used functions from the Windows <a target="_blank" href="http://msdn.microsoft.com/en-us/library/windows/desktop/ee671406(v=vs.85).aspx">IUIAutomation COM interface</a>.</p>
<p>But since IUIAutomation is a custom COM interface that doesn't implement <a target="_blank" href="http://en.wikipedia.org/wiki/IDispatch">IDispatch</a>, Ruby's Win32OLE module won't work with it.</p>
<p>And I wanted to avoid using a C extension that wraps UI Automation - see <a target="_blank" href="https://github.com/jarmo/RAutomation">RAutomation</a> for an example of a nice library that takes the C extension approach.</p>
<p>Without using a C extension, the only way I've found to use IUIAutomation in Ruby is to use Windows::COM - which is included in the awesome <a target="_blank" href="https://github.com/djberg96/windows-pr">windows-pr</a> project.</p>
<p>Then, one must pore over the C header file containing the function prototypes you want to use (UIAutomationClient.h in this case) and port each prototype to Ruby one at a time (!).</p>
<p>Note you have to install the <a target="_blank" href="http://www.microsoft.com/download/en/details.aspx?id=3138">the Windows 7 SDK</a> to get a copy of that header file, but you don't need the SDK or .h to simply run the Ruby script shown below.</p>
<p>(For what it's worth, my copy of the header file is at \Program Files\Microsoft SDKs\Windows\v7.1\Include\UIAutomationClient.h)</p>
<p>This is all quite a far cry from how easy the same task is in the CPython world thanks to <a target="_blank" href="http://starship.python.net/crew/theller/comtypes/">comtypes</a>. But I digress - doing the same thing in Python deserves its own post.</p>
<p>Below is a Ruby script that obtains and prints the name of the root (Desktop) UI automation element on a Windows box - the string should always be "Desktop".</p>
<p>Doing so shows how to use IUIAutomation::GetRootElement() and IUIAutomationElement::get_CurrentName().</p>
<p>Apologies for the convoluted hacks on display - stuff like having to hardcode the number of each function in the IUIAutomationVtbl struct will hopefully have a better solution someday!</p>
<p>NOTE: Before running this you must:</p>
<ul>
<li><code>gem install windows-pr</code></li>
<li>If you're using XP or Vista, install the <a target="_blank" href="http://support.microsoft.com/kb/971513">Windows Automation API</a> update from MSFT</li>
</ul>
<script src="https://gist.github.com/billagee/2383726.js"></script>How to set up the Solarized color scheme for vim and iTerm22012-04-09T22:18:00-07:002012-04-09T22:18:00-07:00Bill Ageetag:blog.likewise.org,2012-04-09:2012/04/how-to-set-up-solarized-color-scheme/<p><img alt="Solarized Vim Screenshot" src="http://blog.likewise.org/2012/04/how-to-set-up-solarized-color-scheme/images/solarized.png" /></p>
<p>When you stare at a display full of text for hours at a time, a nice looking color scheme is worth the time it takes to set up.</p>
<p>Enter <a target="_blank" href="http://ethanschoonover.com/solarized">Solarized</a>, a great option for improving your overall text-editing life.</p>
<p>My officemate <a target="_blank" href="http://www.kevinbeddingfield.com/">Kevin</a> has been evangelizing Solarized for a while, so today I took the plunge and set it up, and man do I wish I had done this a while ago. Much of the content described below is straight from his setup - I definitely owe him.</p>
<p>The set of software I'm currently using with Solarized is:</p>
<ul>
<li><a target="_blank" href="https://iterm2.com/">iTerm2</a> (since in my experience it handles Solarized better than Terminal.app)</li>
<li>The command-line vim that ships with OSX</li>
<li><a target="_blank" href="https://github.com/tpope/vim-pathogen">pathogen.vim</a> (for easy installation of vim plugins)</li>
<li>The Solarized config files for iTerm2 and vim</li>
<li><a target="_blank" href="https://github.com/scrooloose/nerdtree">NERDTree</a> (a tree explorer for vim)</li>
</ul>
<p>For future reference, here's how I set everything up:</p>
<ol>
<li>
<p>Download the stable version of iTerm2 from <a target="_blank" href="https://iterm2.com/">iterm2.com</a></p>
</li>
<li>
<p>Download and unzip <a target="_blank" href="http://ethanschoonover.com/solarized/files/solarized.zip">the latest version of the Solarized .zip file</a> (it contains the iTerm2 preset files you'll need)</p>
</li>
<li>
<p>In iTerm2, open <em>iTerm2 > Preferences > Profiles > Colors</em>, and click <em>Load Presets...</em> to load the Solarized color schemes (light and dark) that are found in the .zip in <code>solarized/iterm2-colors-solarized/</code></p>
<p>For more info see <a target="_blank" href="https://github.com/altercation/solarized/blob/master/iterm2-colors-solarized/README.md">the iterm2-colors-solarized README</a>.</p>
</li>
<li>
<p>Follow <a target="_blank" href="https://github.com/tpope/vim-pathogen#readme">these instructions from the pathogen github README</a> to install pathogen. In the next step, we'll be using pathogen to install more bits.</p>
</li>
<li>
<p>Install solarized.vim using pathogen:</p>
<p>(For more info see <a target="_blank" href="http://ethanschoonover.com/solarized/vim-colors-solarized">http://ethanschoonover.com/solarized/vim-colors-solarized</a>)</p>
<div class="highlight"><pre><span></span><span class="nb">cd</span> ~/.vim/bundle
git clone git://github.com/altercation/vim-colors-solarized.git
</pre></div>
<p>In the parent directory of vim-colors-solarized:</p>
<div class="highlight"><pre><span></span>mv vim-colors-solarized ~/.vim/bundle/
</pre></div>
</li>
<li>
<p>Install NERDTree:</p>
<p>(See also <a target="_blank" href="http://programming34m0.blogspot.com/2011/04/nerd-tree-file-explorer-with-mac-vim.html">http://programming34m0.blogspot.com/2011/04/nerd-tree-file-explorer-with-mac-vim.html</a>)</p>
<div class="highlight"><pre><span></span><span class="nb">cd</span> ~/.vim/bundle
git clone git://github.com/scrooloose/nerdtree.git
</pre></div>
</li>
<li>
<p>Set up your .vimrc appropriately - here's mine:</p>
<div class="highlight"><pre><span></span><span class="k">set</span> <span class="nb">ruler</span>
<span class="k">set</span> <span class="nb">cursorline</span>
<span class="k">call</span> pathogen#infect<span class="p">()</span>
<span class="nb">syntax</span> <span class="k">on</span>
<span class="k">filetype</span> plugin indent <span class="k">on</span>
<span class="nb">syntax</span> enable
<span class="c">" Solarized stuff</span>
<span class="k">let</span> <span class="k">g</span>:solarized_termtrans <span class="p">=</span> <span class="m">1</span>
<span class="k">set</span> <span class="nb">background</span><span class="p">=</span><span class="nb">dark</span>
<span class="k">colorscheme</span> solarized
</pre></div>
</li>
<li>
<p>OPTIONAL: A nice choice for a terminal font is Inconsolata-dz - you can download it here, and configure iTerm2 to use it:</p>
<p><a target="_blank" href="http://nodnod.net/2009/feb/12/adding-straight-single-and-double-quotes-inconsola/">http://nodnod.net/2009/feb/12/adding-straight-single-and-double-quotes-inconsola/</a></p>
</li>
</ol>SHDocVw.ShellWindows and IWebBrowser2 in C#, the easy way2012-03-31T10:54:00-07:002012-03-31T10:54:00-07:00Bill Ageetag:blog.likewise.org,2012-03-31:2012/03/shdocvwshellwindows-and-iwebbrowser2-in/<p>Not too long ago I needed to write some test setup code in C# to check for open IE windows.</p>
<p>The idea was to kill any running IE instances before entering the main portion of the test case - and just for reference, I wanted to record the URL that each closed IE window had been viewing.</p>
<p>My first thought was to use this common technique:</p>
<ul>
<li>Enumerate the SHDocVw.ShellWindows collection, looking for IE processes</li>
<li>Use IE's IWebBrowser2 COM interface to interact with any IE instances found</li>
</ul>
<p>Turns out this is a common question for C# projects - and there seem to be some overly convoluted solutions floating around.</p>
<p>However, one solution is quite easy - the key point is that the friendly name of the SHDocVw type library is "Microsoft Internet Controls".</p>
<p>So, if you add a reference to the "Microsoft Internet Controls" COM component in your C# project, you'll be able to use SHDocVw.ShellWindows and SHDocVw.IWebBrowser2.</p>
<p><img alt="Visual Studio screenshot" src="http://blog.likewise.org/2012/03/shdocvwshellwindows-and-iwebbrowser2-in/images/ms_int_controls-big.png" /></p>
<p>This example code (for VS 2012) shows how to access the IWebBrowser2 interface of each running IE:</p>
<script src="https://gist.github.com/billagee/46a1ea83b59f13146567cb779dd003b0.js"></script>
<p>Back in the VS 2008 days, different code was used to accomplish the same thing:</p>
<script src="https://gist.github.com/2267093.js?file=gistfile1.cs"></script>
<p>Full disclosure - I also wrote a <a target="_blank" href="http://stackoverflow.com/questions/6530083/cannot-add-c-windows-system32-shdocvw-dll-to-my-project/8541532#8541532">stackoverflow answer about this topic</a>.</p>Quick and dirty port scan with netcat2012-03-28T22:01:00-07:002012-03-28T22:01:00-07:00Bill Ageetag:blog.likewise.org,2012-03-28:2012/03/quick-and-dirty-port-scan-with-netcat/<p><a target="_blank" href="http://nc110.sourceforge.net/">netcat</a> (or "nc" on the command line) is a useful tool for many reasons.</p>
<p>One common test automation task I've found it handy for is polling a port, waiting for a service to start responding.</p>
<p>While there are many ways to do that, using nc is a handy way to get it done.</p>
<p>And it's easy to script - here's a short Perl snippet showing how to wrap nc to poll a port on a host:</p>
<script src="https://gist.github.com/2233382.js"> </script>Brushing up on cURL2011-08-06T09:51:00-07:002011-08-06T09:51:00-07:00Bill Ageetag:blog.likewise.org,2011-08-06:2011/08/brushing-up-on-curl/<p>The <a target="_blank" href="http://curl.haxx.se/">curl command line tool</a> is a great addition to your toolbox when working on web-related testing projects.</p>
<p>It's difficult to add up all the ways curl is useful for testing tasks. Here's an attempt at documenting a few simple scenarios.</p>
<h3>Installing the curl command line tool</h3>
<p>OSX ships with a copy of curl pre-installed, so Mac users have nothing to do here.</p>
<p>Linux distributions typically ship with curl or make it very easy to install it with a package management system.</p>
<p>On Windows, you can download a binary release of curl to get up and running quickly - I recommend using cygwin, or I've also had good luck with the curl 7.21.7 build found in the "Win32 Generic" section (provided by Günter Knauf) at the bottom of this page:</p>
<p><a target="_blank" href="http://curl.haxx.se/download.html">http://curl.haxx.se/download.html</a></p>
<h3>To make a GET request for a file, and save a local copy:</h3>
<p>Sometimes it's useful to fetch a file from a server and compare it (or its md5 hash) to a known file, to make sure you received the right content - so curl to the rescue:</p>
<div class="highlight"><pre><span></span>curl --remote-name http://www.w3schools.com/images/pulpit.jpg
</pre></div>
<p>The <code>--remote-name</code> option tells curl to save a local copy of the file using the same name found on the remote server.</p>
<p>After running the command above, a copy of pulpit.jpg should appear in your current local directory.</p>
<h3>Using the --silent option for minimal output</h3>
<p>You most likely noticed the progress output that curl prints when the command above is run.</p>
<p>The handy --silent option allows one to suppress that information:</p>
<div class="highlight"><pre><span></span>curl --silent --remote-name http://www.w3schools.com/images/pulpit.jpg
</pre></div>
<h3>Using curl's --write-out option to print extra information about a request</h3>
<p>curl's <code>--write-out</code> option is extremely useful. It allows you to specify extra information to print, using special variables.</p>
<p>For example, to print the HTTP response code of your request, you use the <code>http_code</code> variable.</p>
<p>You must enclose the variables in <code>%{}</code>. For example:</p>
<div class="highlight"><pre><span></span>--write-out %<span class="o">{</span>http_code<span class="o">}</span>
</pre></div>
<p>Here's a real example of using the option. Note the <code>200</code> response code that curl prints:</p>
<div class="highlight"><pre><span></span>curl --write-out %<span class="o">{</span>http_code<span class="o">}</span> <span class="se">\</span>
--silent --remote-name http://www.w3schools.com/images/pulpit.jpg
<span class="m">200</span>
</pre></div>
<p>You can also pass multiple variables to <code>--write-out</code>, and create your own format string, so that it's easy to tell what each value represents.</p>
<p>You can even use newlines in your format string, using <code>\n</code> as shown here:</p>
<div class="highlight"><pre><span></span>--write-out <span class="s2">"Response code: %{http_code}\nTotal time: %{time_total}"</span>
</pre></div>
<p>Example output from the above command:</p>
<div class="highlight"><pre><span></span>curl --write-out <span class="s2">"Response code: %{http_code}\nTotal time: %{time_total}"</span> <span class="se">\</span>
--silent --remote-name http://www.w3schools.com/images/pulpit.jpg
Response code: <span class="m">200</span>
Total time: <span class="m">0</span>.797
</pre></div>
<h3>Using curl to test redirects</h3>
<p><code>--write-out</code> can also be used to make sure that a URL that should redirect the client is successfully doing so.</p>
<p>Even better, you can see what URL the server is telling you to redirect to.</p>
<p>As an example, the URL <em>http://www.gmail.com/</em> currently redirects to <em>http://mail.google.com/</em>.</p>
<p>To check that on the command line, you can use the <code>redirect_url</code> variable with <code>--write-out</code>:</p>
<div class="highlight"><pre><span></span>curl --write-out <span class="se">\</span>
<span class="s2">"\nResponse code: %{http_code}\nRedirect URL: %{redirect_url}\n"</span> <span class="se">\</span>
http://www.gmail.com/
</pre></div>
<p>Running the above command should result in this output:</p>
<div class="highlight"><pre><span></span><HTML><HEAD><meta http-equiv<span class="o">=</span><span class="s2">"content-type"</span> <span class="nv">content</span><span class="o">=</span><span class="s2">"text/html;charset=utf-8"</span>>
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
>The document has moved
<A <span class="nv">HREF</span><span class="o">=</span><span class="s2">"https://mail.google.com/mail/"</span>>here</A>.
</BODY></HTML>
Response code: <span class="m">301</span>
Redirect URL: https://mail.google.com/mail/
</pre></div>
<h3>Using --include and --verbose</h3>
<p>More verbose ways of scrutinizing the server's response are available, via the <code>--include</code> and <code>--verbose</code> options.</p>
<p><code>--include</code> prints all the headers sent by the server in response to the request. Very cool!</p>
<div class="highlight"><pre><span></span>curl --include http://www.gmail.com/
HTTP/1.1 <span class="m">301</span> Moved Permanently
Location: https://mail.google.com/mail/
Content-Type: text/html<span class="p">;</span> <span class="nv">charset</span><span class="o">=</span>UTF-8
X-Content-Type-Options: nosniff
Date: Tue, <span class="m">16</span> Aug <span class="m">2016</span> <span class="m">13</span>:05:21 GMT
Expires: Thu, <span class="m">15</span> Sep <span class="m">2016</span> <span class="m">13</span>:05:21 GMT
Server: sffe
Content-Length: <span class="m">226</span>
X-XSS-Protection: <span class="m">1</span><span class="p">;</span> <span class="nv">mode</span><span class="o">=</span>block
Cache-Control: public, max-age<span class="o">=</span><span class="m">2592000</span>
Age: <span class="m">364604</span>
<HTML><HEAD><meta http-equiv<span class="o">=</span><span class="s2">"content-type"</span> <span class="nv">content</span><span class="o">=</span><span class="s2">"text/html;charset=utf-8"</span>>
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A <span class="nv">HREF</span><span class="o">=</span><span class="s2">"https://mail.google.com/mail/"</span>>here</A>.
</BODY></HTML>
</pre></div>
<p><code>--verbose</code> expands on that by also printing the headers sent by you (the client) and some other diagnostic info:</p>
<div class="highlight"><pre><span></span>curl --verbose http://www.gmail.com/
* Trying <span class="m">216</span>.58.195.229...
* Connected to www.gmail.com <span class="o">(</span><span class="m">216</span>.58.195.229<span class="o">)</span> port <span class="m">80</span> <span class="o">(</span><span class="c1">#0)</span>
> GET / HTTP/1.1
> Host: www.gmail.com
> User-Agent: curl/7.48.0
> Accept: */*
>
< HTTP/1.1 <span class="m">301</span> Moved Permanently
< Location: https://mail.google.com/mail/
< Content-Type: text/html<span class="p">;</span> <span class="nv">charset</span><span class="o">=</span>UTF-8
< X-Content-Type-Options: nosniff
< Date: Mon, <span class="m">15</span> Aug <span class="m">2016</span> <span class="m">01</span>:39:32 GMT
< Expires: Wed, <span class="m">14</span> Sep <span class="m">2016</span> <span class="m">01</span>:39:32 GMT
< Server: sffe
< Content-Length: <span class="m">226</span>
< X-XSS-Protection: <span class="m">1</span><span class="p">;</span> <span class="nv">mode</span><span class="o">=</span>block
< Cache-Control: public, max-age<span class="o">=</span><span class="m">2592000</span>
< Age: <span class="m">493768</span>
<
<HTML><HEAD><meta http-equiv<span class="o">=</span><span class="s2">"content-type"</span> <span class="nv">content</span><span class="o">=</span><span class="s2">"text/html;charset=utf-8"</span>>
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A <span class="nv">HREF</span><span class="o">=</span><span class="s2">"https://mail.google.com/mail/"</span>>here</A>.
</BODY></HTML>
* Connection <span class="c1">#0 to host www.gmail.com left intact</span>
</pre></div>
<h3>To sum up:</h3>
<p>This only scratches the surface of what is possible with curl. For more information, see:</p>
<p><a href="http://curl.haxx.se/docs/manpage.html" target="_blank">http://curl.haxx.se/docs/manpage.html</a></p>
<p><a href="http://curl.haxx.se/docs/httpscripting.html" target="_blank">http://curl.haxx.se/docs/httpscripting.html</a></p>Selenium 2 .NET test drive2011-07-10T22:29:00-07:002011-07-10T22:29:00-07:00Bill Ageetag:blog.likewise.org,2011-07-10:2011/07/selenium-2-net-test-drive/<p>Inaugural post!</p>
<p>Big news - Selenium 2 was released a few days ago:</p>
<p><a target="_blank" href="http://seleniumhq.wordpress.com/2011/07/08/selenium-2-0/">http://seleniumhq.wordpress.com/2011/07/08/selenium-2-0/</a></p>
<p>This is a big deal, considering the impressive results Selenium 2 can yield, not to mention the time and effort that went into the project.</p>
<p>So, what better way to launch a blog about testing than with an Se2 trip report?</p>
<p>To take Selenium 2 for a spin, I settled on these requirements for using it with my web app under test:</p>
<ul>
<li>Use the C# client (since I already had an existing NUnit project that would be a good place to store Selenium test code)</li>
<li>Use IE8 on Windows XP (I know, I know...but why not start with a challenge?)</li>
</ul>
<p>Here are my notes on a few interesting portions of the test drive:</p>
<h3>Installation and first steps</h3>
<p>Selenium 2 really shines in this phase. Setup is painless - I just had to:</p>
<ul>
<li>Visit <a target="_blank" href="http://seleniumhq.org/download/">http://seleniumhq.org/download/</a></li>
<li>Find the C# (Selenium WebDriver) 2.0.0 download link</li>
<li>Download and unzip selenium-dotnet-2.0.0.zip</li>
<li>Make a new C# console project (I used Visual Studio 2008)</li>
<li>In the project, add references to all the .dlls from the zipfile</li>
</ul>
<p>Here's a simple C# program that launches IE and requests www.google.com:</p>
<div class="highlight"><pre><span></span><span class="n">using</span> <span class="n">OpenQA</span><span class="p">.</span><span class="n">Selenium</span><span class="p">;</span>
<span class="n">using</span> <span class="n">OpenQA</span><span class="p">.</span><span class="n">Selenium</span><span class="p">.</span><span class="n">IE</span><span class="p">;</span>
<span class="n">using</span> <span class="n">System</span><span class="p">;</span>
<span class="n">namespace</span> <span class="n">SeFoo</span>
<span class="p">{</span>
<span class="n">class</span> <span class="n">Program</span>
<span class="p">{</span>
<span class="k">static</span> <span class="kt">void</span> <span class="n">Main</span><span class="p">(</span><span class="n">string</span><span class="p">[]</span> <span class="n">args</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">IWebDriver</span> <span class="n">driver</span> <span class="o">=</span> <span class="n">new</span> <span class="n">InternetExplorerDriver</span><span class="p">();</span>
<span class="n">driver</span><span class="p">.</span><span class="n">Navigate</span><span class="p">().</span><span class="n">GoToUrl</span><span class="p">(</span><span class="s">"http://www.google.com"</span><span class="p">);</span>
<span class="n">driver</span><span class="p">.</span><span class="n">Quit</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
<p>More info on getting started can be found in the Selenium docs:</p>
<p><a target="_blank" href="http://seleniumhq.org/docs/03_webdriver.html">http://seleniumhq.org/docs/03_webdriver.html</a></p>
<h3>Cleanup</h3>
<p>Note that when using the .NET bindings, it's important to use:</p>
<div class="highlight"><pre><span></span><span class="n">driver</span><span class="p">.</span><span class="n">Quit</span><span class="p">();</span>
</pre></div>
<p>...when wrapping up your test, because otherwise you can leave a large temp file behind on disk.</p>
<p>For more on that, see this bug report:</p>
<p><a target="_blank" href="http://code.google.com/p/selenium/issues/detail?id=1140">http://code.google.com/p/selenium/issues/detail?id=1140</a></p>
<h3>Switching between frames</h3>
<p>Right off the bat, I needed to access an element in an iframe, and discovered that frames require special treatment. Searches for elements in iframes fail unless you explicitly tell Selenium that you need to work within a frame.</p>
<p>I recall frames were always a special case in Watir as well.</p>
<p>This isn't really an issue, though - just use SwitchTo(). This code tells Selenium to target the first iframe in the document, so elements in the iframe will become accessible:</p>
<div class="highlight"><pre><span></span><span class="n">IWebElement</span> <span class="n">iframe</span> <span class="o">=</span> <span class="n">driver</span><span class="p">.</span><span class="n">FindElement</span><span class="p">(</span><span class="n">By</span><span class="p">.</span><span class="n">TagName</span><span class="p">(</span><span class="s">"iframe"</span><span class="p">));</span>
<span class="n">driver</span><span class="p">.</span><span class="n">SwitchTo</span><span class="p">().</span><span class="n">Frame</span><span class="p">(</span><span class="n">iframe</span><span class="p">);</span>
</pre></div>
<h3>IE8 Click() issue:</h3>
<p>Next I hit a roadblock after discovering IE8 appears to ignore Click() calls on form submit buttons, at least on my WinXP machine.</p>
<p>For more info, see this bug report:</p>
<p><a target="_blank" href="http://code.google.com/p/selenium/issues/detail?id=1415">http://code.google.com/p/selenium/issues/detail?id=1415</a></p>
<p>A very brief search turned up a few other Selenium bug reports that may or may not be duplicates of that one, too.</p>
<p>Issue 1415 is marked WONTFIX at the moment. Regardless, this simple workaround worked for me:</p>
<div class="highlight"><pre><span></span><span class="n">submitbutton</span><span class="p">.</span><span class="n">SendKeys</span><span class="p">(</span><span class="n">Keys</span><span class="p">.</span><span class="n">Enter</span><span class="p">);</span>
</pre></div>
<h3>Waiting with WebDriver</h3>
<p>After using the <code>SendKeys()</code> workaround, my program promptly broke down on the next statement, due to searching for an element that hadn't rendered in the browser yet.</p>
<p>Since <code>SendKeys()</code> doesn't block, you must handle polling for the next element on your own.</p>
<p>Turns out this is pretty easy to do, using the <code>WebDriverWait</code> support class.</p>
<p>Just add this <code>using</code> statement:</p>
<div class="highlight"><pre><span></span><span class="n">using</span> <span class="n">OpenQA</span><span class="p">.</span><span class="n">Selenium</span><span class="p">.</span><span class="n">Support</span><span class="p">.</span><span class="n">UI</span><span class="p">;</span>
</pre></div>
<p>Then add your wait code:</p>
<div class="highlight"><pre><span></span><span class="c1">// Wait 10s for the element to appear</span>
<span class="n">WebDriverWait</span> <span class="n">tenSecWait</span> <span class="o">=</span> <span class="n">new</span> <span class="n">WebDriverWait</span><span class="p">(</span>
<span class="n">driver</span><span class="p">,</span> <span class="n">new</span> <span class="n">TimeSpan</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">10</span><span class="p">));</span>
<span class="c1">// Find element with ID "foo"</span>
<span class="n">IWebElement</span> <span class="n">theElement</span> <span class="o">=</span> <span class="n">tenSecWait</span><span class="p">.</span><span class="n">Until</span><span class="p">(</span>
<span class="n">x</span> <span class="o">=></span> <span class="n">x</span><span class="p">.</span><span class="n">FindElement</span><span class="p">(</span><span class="n">By</span><span class="p">.</span><span class="n">Id</span><span class="p">(</span><span class="s">"foo"</span><span class="p">)));</span>
</pre></div>
<h3>Locating elements by text with XPath versus CSS selectors</h3>
<p>I needed to locate an h3 element with no attributes at all - just unique inner text. No name or ID, etc.</p>
<p>Unless I missed something, there doesn't seem to be a built-in WebDriver method for this scenario.</p>
<p>So, I used an XPath query:</p>
<div class="highlight"><pre><span></span><span class="n">IWebElement</span> <span class="n">myElement</span> <span class="o">=</span> <span class="n">driver</span><span class="p">.</span><span class="n">FindElement</span><span class="p">(</span>
<span class="n">By</span><span class="p">.</span><span class="n">XPath</span><span class="p">(</span><span class="s">"//h3[text() = 'someUniqueText']"</span><span class="p">));</span>
</pre></div>
<p>A little digging revealed that in Selenium 1, an alternate way to do this was apparently this equivalent CSS selector:</p>
<div class="highlight"><pre><span></span>h1:contains<span class="o">(</span><span class="s2">"someUniqueText"</span><span class="o">)</span>
</pre></div>
<p>But the latter doesn't appear to work in Selenium 2 since <code>contains()</code> is not actually part of CSS3; it's just sugar available in Selenium 1 (and jQuery), from what I can tell.</p>
<p>More info on that can be found at:</p>
<p><a target="_blank" href="http://groups.google.com/group/webdriver/browse_thread/thread/a8d4541bdb3ed439">http://groups.google.com/group/webdriver/browse_thread/thread/a8d4541bdb3ed439</a></p>
<p><a target="_blank" href="http://stackoverflow.com/questions/4781141/why-h3nth-child1containsa-selector-doesnt-work">http://stackoverflow.com/questions/4781141/why-h3nth-child1containsa-selector-doesnt-work</a></p>
<p>So, in the end I kept using XPath in this situation.</p>
<p>NOTE: Another useful fix in this situation is to modify the app under test, if possible, so that the element to locate has a unique attribute. Then there's no need to use XPath or CSS selectors at all. :)</p>
<h3>Scrolling elements into view</h3>
<p>A hiccup arose when I tried to <code>Click()</code> an element that was scrolled slightly out of view inside a scrolling div.</p>
<p>For some reason the click didn't bring the element into focus; instead the click went to an element that was immediately "on top", so to speak, of the element I wanted.</p>
<p>I assume this might be because the browser (or WebDriver) thinks the element is displayed even though moving a scrollbar a bit is necessary to see it.</p>
<p>This JS workaround solved the problem (where <code>theLink</code> was the IWebElement object I wanted to click):</p>
<div class="highlight"><pre><span></span><span class="n">IJavaScriptExecutor</span> <span class="n">js</span> <span class="o">=</span> <span class="n">driver</span> <span class="n">as</span> <span class="n">IJavaScriptExecutor</span><span class="p">;</span>
<span class="n">js</span><span class="p">.</span><span class="n">ExecuteScript</span><span class="p">(</span><span class="s">"arguments[0].scrollIntoView(true);"</span><span class="p">,</span> <span class="n">theLink</span><span class="p">);</span>
</pre></div>
<h3>In Summary</h3>
<p>With all the issues above dealt with, I ended up with a fairly robust and useful test program, in a surprisingly short time. And with more polish things will just get better.</p>
<p>I have to say I'm really impressed with the Selenium 2 .NET bindings. Very very cool stuff!</p>