I’ve been a happy user of Capybara-WebKit for many years now, but its dependence on Qt can make it frustrating to install on macOS, particularly following macOS or Xcode updates. One such recent issue lead me to experiment with running my tests in Chrome via ChromeDriver and Selenium.
I found the installation of ChromeDriver and Capybara-Selenium to be significantly faster than the installation of Qt and Capybara-WebKit, and my teammates reported no difficulties with those steps either.
While tests executing with Capybara-WebKit are headless, I initially did not
have this option when configuring ChromeDriver. In this initial configuration, I
was reminded that it is occasionally useful to watch a test execute without
having to resort to
save_and_open_screenshot. I was also pleasantly surprised
to find that ChromeDriver never seems to steal focus from my active Chrome
session or any other applications, which is a refreshing change from my memories
of executing tests in Firefox via Selenium.
Last week saw the stable channel release of Chrome 59, which supports headless operation on macOS. Headless operation on Linux was already possible as of Chrome 57 and will be coming to Windows soon as well. I updated my Capybara configuration to run ChromeDriver with headless support and am now a mostly satisfied ChromeDriver convert.
Executing your feature specs in Chrome requires that you have Chrome and ChromeDriver installed. Web developers and designers are likely to have Chrome installed already, so that leaves us needing to install ChromeDriver.
ChromeDriver is installed via Homebrew with
brew install chromedriver and is
similarly available in your package manager of choice on Linux. If you already
have ChromeDriver installed, be sure to install an up-to-date version for
headless support. I’m currently using ChromeDriver version 2.30.
If you’re not comfortable making this a prerequisite to running your
application’s tests, you can also install ChromeDriver by adding
chromedriver-helper to your
Gemfile. On install this will download a
platform-appropriate binary for ChromeDriver and add it to your gem path. If you
opt for this approach, be sure to read the documentation on updating
ChromeDriver and the open issue on Windows support.
capybara-selenium to the
test group of your
Gemfile and optionally
capybara-webkit while you’re at it. All that’s left for us to do now is
to configure our drivers. In your
rails_helper.rb or some file required by
that file, add the following:
headless_chrome drivers and sets Capybara to use
tests execute while debugging, you can change the driver to
You may notice that the
headless_chrome driver also passes the
option. The documentation for the headless Chrome indicates this
is only temporarily necessary but does not specify why. It’s not clear if this
is necessary now that the feature is stable, but it doesn’t seem to hurt
If you use the
headless_chrome driver on an older version of Chrome or on
Windows, you will get a
NetReadTimeout error. I had hoped ChromeDriver would
ignore the option on older browsers, but this is unfortunately not the case. You
may need to make
variable until such time as Chrome 59 is ubiquitous across your team.
The project I tried this on uses CircleCI which required no changes to its configuration. Chrome and ChromeDriver are already available in that environment.
If your project uses Travis, you will need to enable the Chrome addon. If you’re trying out Heroku CI, you can use the Chrome buildpack.
Beyond Installation, How Does ChromeDriver Stack Up?
Capybara-WebKit runs our tests on a fork of the WebKit browser engine via Qt. This engine is generally “close enough” but is not functionally equivalent to Safari (built on WebKit), Chrome (built on Blink, another fork of WebKit), or any other browser your users are likely to be using. This has occasionally caused issues in tests. By executing our tests directly in Chrome we are testing with the exact browser many of our users will be using. We’re also just steps away from executing those same tests in Firefox or even in Safari via SafariDriver.
I don’t have any hard science to offer on the matter of performance. I can
anecdotally report that Capybara-Webkit seems significantly faster. I used an
found that Capybara-Webkit runs
rspec --tag js in about 16 seconds, while the
same command takes about 22 seconds using ChromeDriver. It’s not clear to me if
the performance difference is dominated by startup cost or if it will scale
linearly with the number of specs.
Capybara-Webkit also offers the
block_unknown_urls configuration setting which
prevents loading potentially slow external assets in your tests, such as
external web fonts or analytics packages. I’ve yet to find a similar
configuration in ChromeDriver. ChromeDriver allows specifying a proxy which
could be used to accomplish this, but it would mean needing to run that proxy
process as well.
Finally, I’ve noticed that
save_and_open_screenshot produces an empty, gray
screenshot when running in headless mode, while it works as expected on
Capybara-Webkit. When the need for visual inspection of a test arises, I switch
chrome Capybara driver, but this may be a significant detriment to your
workflow if you rely on automated screenshots after test failures, for example.
What Does This Mean for Capybara-WebKit?
We’re not sure. We’re still in the experimentation phase of our use of ChromeDriver. At this time, our new projects still default to using Capybara-WebKit though this may change as more projects try out headless ChromeDriver.
If you’re a Capybara-Webkit user and give headless Chrome a try, we’d love to hear your experiences. How did the performance of your test suite differ? Did you find Chrome via Selenium and ChromeDriver to be lacking any features you count in Capybara-WebKit? Have you found a way to take screenshots? Tweet us and let us know!