Testing with Selenium

Anyone who has worked on Javascript eccentric web applications knows how much of a hassle it can be. Either you're stuck manually testing endless possible combinations of actions, or you're writing them for your Selenium plugin. Things get even worse when your client wants to support browsers like IE6. RSpec has encapsulated the behavior driven development of models, controllers and views, and user stories have integrated testing between the layers, but unfortunately neither has done much in the development of interactive web applications.

Working on Javascript applications after learning RSpec can be a painful experience. Your fingers want to write simple tests, whereas Javascript wants to you painfully point and click until everything works together. For years I have been nagging myself to find a headless browser that could be integrated into my development environment, but I simply never had the time, energy or justification to take action. Tired of pointing and clicking repeatedly after every change, I recently became filled with angst and decided it was time to do something about it.

I was first exposed to Selenium a few semesters back, and while I was unimpressed with the end product, it was the first and only project of its kind I was aware of. My first impression of Selenium was based on personal preferences and not scientific merits, so I thought a second impression was due. To my dismay, Selenium still has an interface that appears to have been designed only for Windows, and still exists as a glorified macro editor. While Selenium continues to be a bust, I did come across SeleniumRC which is built to test multiple browsers on multiple platforms.

A Better Selenium?

SeleniumRC exists as a server that acts as a proxy between a HTTP client and a browser. Clients send commands to the SeleniumRC server, which passes those actions on to the SeleniumCore inside of the browser window with the matching session id. SeleniumRC returns each request with the result of the command, allowing the client to control and test pages just as a user would. Therefore SeleniumRC can be scripted using any language that can send an HTTP request, like Ruby, JRuby, or Intel Assembly.

Immediately seeing the possibilities, I set out to plug the SeleniumRC Ruby client into RSpec, which would allow me to write user interaction specs in Ruby. While plugging the SeleniumRC client into RSpec proved to be almost as easy as drag and drop, the honeymoon quickly faded. Getting the tests to run and pass turned into a seemingly endless adventure, where sometimes XPaths wouldn't work, sometimes browsers wouldn't work... a big mess. A good portion of my time was spent testing if my behavior tests would work rather then writing the tests and functional code.

I tried two different approaches in overcoming the issues with XPath that I experienced. The first involved passing Prototype strings to be evaluated to avoid SeleniumCore all together. Unfortunately, for some reason I was unable to discover, none of the Prototype strings were returning values. The second approach used Hpricot to assert the presence of elements or values, as well as generating the XPaths for those elements, which could then be passed to SeleniumCore. Alas, XPath selectors were still not working when they were generated by Hpricot.

In addition to having difficulties in getting the tests to work, the syntax provided by the Ruby client is not very pleasing to the Rubyist's eye. I never expect a 'bonus' piece of code, packaged with a free product still in beta, to be the cat's meow. Still, it is always nice when the syntax is clean and all of the pieces work correctly. Does Ruby written to test Javascript have to look like Javascript?

Another issue I've found with SeleniumRC is that it is as slow as my grandma driving a Cadillac down the expressway on Sunday. Even when running the server and client from your local machine, the tests take an extraordinary amount of time to run. I believe this is due to the manner in which SeleniumRC makes it all possible, and while it may be tolerable when doing full scale testing, it simply is not when practicing BDD.

SeleniumRC is still a beta product, listed as version 1.0 beta. While I agree with the term beta, I feel that in today's day and age software should be roughly usable at version 0.1. Typically I would have no problem working around these issues for something I really want; however active development turns into more of a requirement then a wish. There have only been two releases of SeleniumRC since 2006, with the most work being done in the first half of 2006.

A New Hope

Although SeleniumRC hacked my enthusiasm into pieces, it did manage to further motivate my quest for a headless browser. Taking the ideas I got from my time spent with SeleniumRC and RSpec, I set out to create a class that would allow me to control a virtual browser instance from a Ruby object. I decided to look back at my ObjectiveC/Cocoa experience and poke at the WebKit API for a bit and see what sort of trouble I could get myself into.

With a little bit of elbow grease and Google, I have been able to get a working instance of a WebKit browser neatly bundled as a Ruby object. Currently Javascript strings can be passed to the browser to be evaluated, with their string result being returned.

While there are a few technical details to be worked out, with any luck the power of WebKit in Ruby combined with the magic of RSpec should free the masses from the infinite loop of edit-reload-edit. Of course visual aspects will still need manual testing, as well as user interaction on other browsers. With a little finesse, the same tests written to be tested locally with WebKit could also be used to test remote browsers using SeleniumRC.

Next time I hope to have a working demonstration and sample code, in the meantime here is some eye candy you can feast on!