Bill Agee's blog

Technology musings with a twist of user empathy.

Headless Selenium WebDriver Testing With Python and Ghost Driver

GhostDriver is a project that lets you write Selenium WebDriver automation tests that run using the PhantomJS headless WebKit, instead of a traditional web browser.

Put another way, PhantomJS can replace Firefox and friends in your WebDriver scripts - and it doesn’t require a display, so testing complex web apps from the command line is just about as easy as using a GUI browser. Very cool!

Getting your system ready to run Python scripts that use GhostDriver can be done in a few brief steps, if you have homebrew on OS X.

First you’ll need the Selenium python package:

1
sudo pip install selenium

Then, use homebrew to install PhantomJS:

1
brew install phantomjs

If you don’t want to use homebrew (or you’re not on a Mac) you can simply download the latest PhantomJS build manually and install it.

Believe it or not, that is all. GhostDriver is integrated into PhantomJS, so you should now be set up to take them for a test drive.

A typical Hello World program in the web automation world is one that performs a Google search. So, here’s what that looks like in Python and GhostDriver, using Python in interactive mode:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$ python

>>> from selenium import webdriver
>>> driver = webdriver.PhantomJS('phantomjs')
>>> driver.get("http://www.google.com")
>>> driver.title
u'Google'
>>> driver.current_url
u'http://www.google.com/'
>>> driver.find_element_by_name("q").is_displayed()
True
>>> driver.find_element_by_name("q").send_keys("selenium")
>>> driver.find_element_by_name("btnG").click()
>>> driver.current_url
u'http://www.google.com/search?hl=en&source=hp&q=selenium&gbv=2&oq=selenium'

A natural next step, when developing automated test cases based on experiments like the one above, is to start storing your code into a Python unittest script.

Here’s an example of how one might start organizing the code above:

If all is right with the world, running the above script will print output along the lines of the text below.

1
2
3
4
5
6
7
$ ./test_google_ghost_driver.py
current_url is now 'http://www.google.com/search?hl=en&source=hp&q=selenium&gbv=2&oq=selenium'
.
----------------------------------------------------------------------
Ran 1 test in 2.770s

OK

That’s all for the moment. Now, go forth and Ghost Drive!

Using TextMate to Rapidly Test Java Code examples/SSCCEs

My last post was about executing small Java programs from within vim, without leaving the editor to manually open a shell.

The goal was to rapidly execute example code for your own edification, or when showing code to other people - basically any time you need a Short, Self Contained, Correct (Compilable), Example (also known as an SSCCE).

While vim does indeed work for that purpose, I feel TextMate has the edge when it comes to running Java examples.

Here’s what setting up and using TextMate to run Java programs on Mac looks like:

  • After you install TextMate, open the Bundle settings (in ‘TextMate > Preferences > Bundles’) and make sure the ‘Java’ checkbox is set:

  • Open or compose your demo program, and make sure you’ve saved the file to disk.

  • In the bottom status bar, Make sure the Java bundle is selected:

  • Now simply use Command-R to run your code! A new window should open to display the output.

Testing Java Snippets With Vim and GroovyConsole

For instructional purposes (either when experimenting on your own, or when demonstrating code to others) it’s always useful to be able to run snippets of code in a REPL, or a similar environment allowing fast turnaround in the edit/compile/run cycle.

When using Java, the customary IDEs offer ways to get REPL-like behavior, but what if you don’t want to use a traditional Java IDE?

Perhaps you just want to demonstrate a trivial bit of code without much overhead.

In that situation, a couple of nice options for Java are:

  • Use GroovyConsole as a Java REPL
  • Edit your code in Vim, and compile and run it without leaving the editor

1. Using GroovyConsole

If you’re on a Mac, GroovyConsole can be installed via the homebrew groovy formula (or, just get it from http://groovy.codehaus.org):

brew install groovy

Launch GroovyConsole with this command:

groovyConsole

Then simply type in a code snippet and run it with <Command-R> (or on Windows, <CTRL-R>):

2. Using Vim as an improvised Java IDE

First, launch vim and write a small program - for example, Foo.java:
Then, compile your program without leaving vim by passing the file open in your vim buffer to javac, using the :! command sequence and %
If all goes well, you’ll temporarily be dropped to the shell, with no visible errors, and get prompted to press ENTER to continue back to vim:
Back in the editor, use :!java Foo to invoke the Java class file you just created with javac:
Finally, you’ll see your program’s output in the console.

For further fun, try compiling with javac -g %, then launch your class file with jdb Foo to debug your program from within vim.

Also, you might consider taking the javacomplete omni-completion plugin for a spin.

Raspberry Pi Terminal Screenshots With Fbgrab

Say you’re on the console on your Raspberry Pi, and you want to take a screenshot. But without X running, what does one do?

Simple: Use fbgrab. To install it, just:

sudo apt-get install fbgrab

Pass fbgrab the name of the virtual terminal/tty you want to snapshot, and it’ll spit out a PNG file.

For example, say you have an awesome console program running on /dev/tty1, and want to screenshot it - just run:

sudo fbgrab -c 1 screenshot.png

And that’s it! Here’s an example of the output:

Overview of Configuring a Static Website in Amazon S3

Lately I’ve been looking for reasons to use OmniGraffle for diagramming.

And with the steps to set up static web hosting in S3 (as mentioned in my last post) still fresh in mind, it seems like a good time to document what I did. And the steps are simple enough that a diagram might just be enough to cover the important bits.

So here it is - someday it’ll be interesting to look back on this and see what’s easier/more streamlined in this process, and what remains the same:

Hello Octopress!

New year, new hosting stack! This blog is now brought to you by Octopress, hosted on Amazon S3, instead of Blogger.

After spending some quality time with the tools I used in the move from Blogger (Octopress, S3, Route 53, git), I feel like the proverbial new pair of glasses has descended and shown me a new world.

So what are the benefits of Octopress? To list just a few:

  • Solarized syntax highlighting built in. This was all I had to hear to get really interested.
  • Easy editing of posts in the editor of your choice, using markdown, instead of wrestling the blogger tools.
  • Cheap! For my minimal custom domain hosting needs, S3/Route 53 costs should be super low.
  • Applying third-party themes is super easy, should you get the urge to check out alternatives.
  • Storing everything in a git repo is painless, so no worries when playing around with new themes, or making other significant edits. In blogger-land this didn’t feel anywhere near as easy.

To sum up, maintaining this blog just became a lot more enjoyable.

Some helpful Octopress references were:

Pure Ruby Example of Calling Functions From the MS IUIAutomation COM Interface


A while back I was interested in writing some Ruby code that used functions from the Windows IUIAutomation COM interface.

But since IUIAutomation is a custom COM interface that doesn’t implement IDispatch, Ruby’s Win32OLE module won’t work with it.

And I wanted to avoid using a C extension that wraps UI Automation - see RAutomation for an example of a nice library that takes the C extension approach.

Without using a C extension, the only way I’ve found to use IUIAutomation in Ruby is to use Windows::COM - which is included in the awesome windows-pr project.

Then, one must pore over the C header file containing the function prototypes you want to use (UIAutomationClient.h in this case) and port each prototype to Ruby one at a time (!).

Note you have to install the the Windows 7 SDK to get a copy of that header file, but you don’t need the SDK or .h to simply run the Ruby script shown below.

(For what it’s worth, my copy of the header file is at \Program Files\Microsoft SDKs\Windows\v7.1\Include\UIAutomationClient.h)

This is all quite a far cry from how easy the same task is in the CPython world thanks to comtypes. But I digress - doing the same thing in Python deserves its own post.

Below is a Ruby script that obtains and prints the name of the root (Desktop) UI automation element on a Windows box - the string should always be “Desktop”.

Doing so shows how to use IUIAutomation::GetRootElement() and IUIAutomationElement::get_CurrentName().

Apologies for the convoluted hacks on display - stuff like having to hardcode the number of each function in the IUIAutomationVtbl struct will hopefully have a better solution someday!

NOTE: Before running this you must:

- Do a “gem install windows-pr”
- If you’re using XP or Vista, install the Windows Automation API update from MS

How to Set Up the Solarized Color Scheme for Vim and iTerm2


When you stare at a display full of code for hours at a time, a nice looking color scheme is worth the time it takes to set up.

Enter Solarized, a great option for improving your overall text-editing life.

My officemate Kevin has been evangelizing Solarized for a while, so today I took the plunge and set it up, and man do I wish I had done this a while ago. Much of the content described below is straight from his setup - I definitely owe him.

The set of software I’m currently using with Solarized is:

  • iTerm2 (since in my experience it handles Solarized better than Terminal.app)
  • The command-line vim that ships with OSX (since MacVIM appears to have weird problems with :shell)
  • pathogen.vim (for easy installation of vim plugins)
  • The Solarized config files for iTerm2 and vim
  • NERDTree (a tree explorer for vim)

The results look fairly nice, IMO.

For future reference, here’s how I set everything up:

  1. Download the stable version of iTerm2:

    http://iterm2.googlecode.com

  2. Download and unzip the latest version of the Solarized .zip file (it contains the iTerm2 preset files you’ll need):

    http://ethanschoonover.com/solarized/files/solarized.zip

  3. In iTerm2, open “iTerm2 > Preferences > Profiles > Colors”, and click “Load Presets…” to load the Solarized color schemes (light and dark) that are found in the .zip in “solarized/iterm2-colors-solarized/”

    For more info see the iTerm2 README from solarized.zip - the path to it in the zip file is:

    solarized/iterm2-colors-solarized/README.md

  4. Use these instructions from the pathogen github README to install it - pathogen will be used further down to install solarized.vim and NERDTree:

    https://github.com/tpope/vim-pathogen#readme

  5. Install solarized.vim using pathogen:

    (For more info see http://ethanschoonover.com/solarized/vim-colors-solarized)

    $ cd ~/.vim/bundle
    $ git clone git://github.com/altercation/vim-colors-solarized.git
    In the parent directory of vim-colors-solarized:
    $ mv vim-colors-solarized ~/.vim/bundle/

  6. Install NERDTree:

    (See also http://programming34m0.blogspot.com/2011/04/nerd-tree-file-explorer-with-mac-vim.html)

    cd ~/.vim/bundle
    git clone git://github.com/scrooloose/nerdtree.git

  7. Set up your .vimrc appropriately - here’s mine:

    set ruler
    set cursorline
    "set number
    call pathogen#infect()
    syntax on

    filetype plugin indent on

    syntax enable

    " Solarized stuff
    let g:solarized_termtrans = 1
    set background=dark
    colorscheme solarized

  8. OPTIONAL: A nice choice for a terminal font is Inconsolata-dz - you can download it here, and configure iTerm2 to use it:

    http://nodnod.net/2009/feb/12/adding-straight-single-and-double-quotes-inconsola/

SHDocVw.ShellWindows and IWebBrowser2 in C#, the Easy Way

Not too long ago I needed to write some test setup code in C# to check for open IE windows.

The idea was to kill any running IE instances before entering the main portion of the test case - and just for reference, I wanted to record the URL that each closed IE window had been viewing.

My first thought was to use this common technique:

  • Enumerate the SHDocVw.ShellWindows collection, looking for IE processes
  • Use IE’s IWebBrowser2 COM interface to interact with any IE instances found

Turns out this is a common question for C# projects - and there seem to be some overly convoluted solutions floating around.

However, one solution is quite easy - the key point is that the friendly name of the SHDocVw type library is “Microsoft Internet Controls”.

So, if you add a reference to the “Microsoft Internet Controls” COM component in your C# project, you’ll be able to use SHDocVw.ShellWindows and SHDocVw.IWebBrowser2.


This example code shows how to access the IWebBrowser2 interface of each running IE:



Full disclosure - I also wrote a stackoverflow answer about this topic.

Quick and Dirty Port Scan With Netcat

netcat (or “nc” on the command line) is a useful tool for many reasons.

One common test automation task I’ve found it handy for is polling a port, waiting for a service to start responding.

While there are many ways to do that, using nc could hardly be simpler.

And it’s easy to script - here’s a short Perl snippet showing how to wrap nc to poll a port on a host: