---
title: thoughtbot and the Holy Grail
teaser:
tags: web,testing,javascript
author: Chad Pytel
published_on: 2010-11-23
---

_Update:_ Our quest for full-stack testing continued with writing the
[capybara-webkit driver](https://github.com/thoughtbot/capybara-webkit).  We
also [wrote about capybara-webkit more
recently](https://thoughtbot.com/blog/post/4583605733/capybara-webkit).

> **Stop! Who would cross the Bridge of Death must answer me these questions
> three, 'ere the other side he see.**
>
> Ask me the questions, bridge-keeper. I'm not afraid.
>
> **What is your name?**
>
> My name is cpytel of thoughtbot.
>
> **What is your favorite color?**
>
> Red.
>
> **What is your quest?**
>
> To seek a reliable, robust way of integration testing our entire application
> to prevent against regressions and to build better systems.

With apologies to Monty Pyton, we've been on this quest for several years now.
After many false starts and tribulations I feel we've reached an important
plateau in this quest, and I feel its prudent to take a step back, look at where
we've been, where we are now, and pass on some of this information for the
benefit of all.

### Selenium

Throughout this quest, [Selenium](http://seleniumhq.org/) has been the siren
song that continually calls out to us. Unfortunately, in practice we've been
unable to get Selenium to run reliably for real applications, on both developers
machines and on the continuous integration server.

This failure with Selenium has caused us to search for alternative solutions.

### Holy Grail

The first promising solution was the aptly named, [Holy
Grail](http://github.com/mynyml/holygrail). This library uses
[Harmony](http://github.com/mynyml/harmony), which in turn wraps
[Johnson](http://github.com/jbarnette/johnson/),
[env.js](http://github.com/thatcher/env-js) and
[Envjs](http://env-js.appspot.com/) to execute browser-less, console-based,
javascript and DOM code right from within your Rails test suite. Our own Jason
Morrison then wrote
[cucumber-holygrail](http://github.com/jasonm/cucumber-holygrail) to allow
Cucumber to drive the Holy Grail integration tests.

Unfortunately, this solution wasn't really designed to accomplish what we
wanted; Holy Grail lets you run javascript on a single page and routes xhr
requests to a controller in a functional test, and therefore it didn't work like
we expected: we wanted to drive a virtual browser session from action to action
that supported javascript and ui interactions.

### Envjs

From there, we discovered
[capybara-envjs](https://github.com/smparkes/capybara-envjs).
[Capybara](http://github.com/smparkes/capybara-envjs) is a replacement for
Webrat (the driver underneath Cucumber) which has a more flexible driver
subsystem. capybara env-js provides more of what we needed, it had the goal of
being a virtual browser, like we wanted. In order to make it work like we
wanted, we needed to provide a number of fixes which we encapsulated into the
[capybara-envjs-fixes](http://github.com/thoughtbot/capybara-envjs-fixes) gem
(as a holding place while we waited for pull requests to be merged in).

Capybara and capybara-envjs, and capybara-envjs-fixes were (and mostly still
are) a deadly combination. We're actively using it on projects still. However,
the more we used it, the more we started to discover cracks in the foundation.

* It doesn't work with all javascript (for example, we had issues on sites that
  still use Prototype)
* It doesn't support jQuery live
* It is sometimes very inconsistent with how real-world browsers behave, or how
  things should properly work, forcing us to write our code like we wouldn't
  normally write it

The three above points, particularly the lack of support for jQuery live (which
is used by more and more of the internal of jQuery, jQuery mobile, and even the
new Rails unobtrusive javascript) caused us to continue to search for a better
solution.

### Akephalos

Thankfully, we then found
[Akephalos](http://github.com/bernerdschaefer/akephalos). Akephalos provides a
Capybara driver that allows you to run your cucumber integration tests in the
headless browser [HtmlUnit](http://htmlunit.sourceforge.net/). HtmlUnit is a
"GUI-Less browser for Java programs". It models <abbr title="HyperText Markup
Language">HTML</abbr> documents and provides an <abbr title="Application
Programming Interface">API</abbr> that allows you to invoke pages, fill out
forms, click links, etc... just like you do in your "normal" browser. With our
fork of Akephalos to resolve a couple of issues that we ran into along the way,
we were up and running with very reliable, headless browser tests.

HtmlUnit is written in Java, and Akephalos uses jruby-jars to start up and
interact with the HtmlUnit browser. It has fairly good JavaScript support (it
was able to deal with everything we were able to throw at it, including jQuery
1.4.2 and 1.4.3, jQuery Mobile, and jQuery live).

_Note: We also tried [culerity](https://github.com/langalex/culerity) which is a
cucumber driver that works with Celerity, which also is underpinned by HtmlUnit,
but we couldn't get it to really work well._

One of the great things about HtmlUnit and Akephalos is that it uses technology
which has been existence in a while, is actively developed, and well documented.
We were able to look at the code and understand what was going on and
contributate changes if necessary much more easily than we were able to do with
any of the previous attempts.

If you want to get started with Akephalos, I recommend that you use [our fork on
github](http://github.com/thoughtbot/akephalos) until the changes are pulled in.
We've used Akephalos on both Rails 2 and 3.

The relevant part of our Gemfile looks like this:

    group :test, :cucumber do
      gem 'akephalos', :git => 'git://github.com/thoughtbot/akephalos.git'
      gem "cucumber-rails"
      gem "capybara"
      gem "database_cleaner"
      gem "treetop"
      gem "launchy"
    end

Then, to a features/support/akephalos.rb add:

    require 'akephalos'

You can then tag any scenarios you want to run with Akephalos with the
@akephalos tag. If you want to be able to tag your scenarios with @javascript
and have it execute in Akephalos (the default javascript driver in Capybara is
Selenium) you'll want to add the following additional line to your
features/support/akephalos.rb file.

    Capybara.javascript_driver = :Akephalos

There are a couple of important gotchas right now that will effect you,
particularly if you have an existing application.

There is a known bug in HtmlUnit where any jQuery live events bound to the click
event stop any jQuery live events bound to submit defined after them. In
particular, this means that if you're using Rails 3 unobtrusive jQuery form
helpers, you'll want to edit your `rails.js` file and move the definition of the
follow event:

    $('form[data-remote]').live('submit')

Up above any jQuery live click bindings.

Additionally, ajax requests execute asynchronously, just like in a real browser.
In order for the tests to not get out of sequence and fail, ajax requests need
to be executed synchronously instead.  You can do this by adding the following
code after your inclusion of jQuery in only your test or cucumber environments:

    $.ajaxSetup({ async: false });

Akephalos doesn't have the ability to set cookies, or access the session. These
are capabilities in other drivers, and so you may find that your cucumber steps
are doing this, and they'll need to be rewritten. For example, the built in
clearance step definitions read the cookies in order to expose a current_user
helper for step definitions to use, and deleted cookies in order to force a sign
out. There is newly committed support for reading cookies in Akephalos, but I
couldn't get it to work, and I figured as a matter of practice it'd be better
not to rely on it anyway.

### Have we reached the end of our quest

We're not the only ones on this quest, I'm sure, and I encourage those of you
who are to take a look at Akephalos and give it a try, I don't think you'll be
disappointed.

Unfortunately, I don't think the quest is over. But Akephalos represents a
significant step forward in a reliable headless browser that can allow us to
fully integration test our applications. There are probably otther options
available to us to try, and I expect there will be more as time goes on. For
example, we've recently started to use
[evergreen](https://github.com/jnicklas/evergreen) to do javascript unit testing
right alongside the normal tests.

We'll continue to report back on what we find and consider the current
state-of-the-art.  Let us know if you've found and love something we've missed
here.