Are your test suites slow? Do you continually refactor your test suites so can get your TATFT on, but still have slow, slow tests? Do you isolate your tests from the database where possible to reduce disk I/O but find yourself mocked by the long runtime of CPU-bound suites?
Maybe it’s time for you to install… PARALLEL SPECS! Originally
Remember all that excitement from years ago about multi-core processors? You can burn a CD while you check your email! You can email pictures TWICE AS FAST!! Well, hold onto your butts, because - yep, that’s right - you can use them to run your tests faster! See for yourself!
This is what your two cores looks like when they’re BOTH RUNNING TESTS AT THE SAME TIME!
Now how about some numbers?!
30% speedup - REAL SPEEDUP! 30% to 40% was typical for our projects. Think your tests are largely I/O bound due to extreme database interaction and that using two cores won’t speed it up? THINK AGAIN! Our tests were running so fast that the database was like SLOWWWW DOWWWWWN!
Installation
Do you use RSpec? Do you use Test::Unit? Then you can get in on the action! FAST TESTING ACTION! Fast tests need a fast installation. Get parallel_specs up and running in no time flat - here’s how I did it for our existing Test::Unit applications:
Baseline benchmarking
Let’s see how slow our tests are when they are run one! at! a! time!
$ time rake test:units test:functionals
Started
.......... (abbreviated... FOR BLOG READING SPEED!!)
415 tests, 898 assertions, 0 failures, 0 errors
29.14s user 3.74s system 87% cpu 37.614 total
Plugging in
Are you ready? Let’s install a plugin! IN OUR APP!
script/plugin install git://github.com/jasonm/parallel_specs.git
More tests need MORE DATABASES
More databases… it’s what concurrent environments crave!
$ rake db:structure:dump
$ echo "create database myapp_test2;" | mysql -uroot
$ mysql -uroot -Dmyapp_test2 < db/development_structure.sql
$ vim config/database.yml # add some ERB to config/database.yml
test:
adapter: mysql
encoding: utf8
database: myapp_test<%= ENV['TEST_ENV_NUMBER'] %>
username: root
Now it’s time for testing
POWER TESTING!!
$ time rake test:parallel
(in /Users/jasonmorrison/dev/myapp/trunk)
2 processes for 49 tests, ~ 24 tests per process
Loaded suite -e
Started
.................................. (abbreviated)
29.32s user 3.31s system 143% cpu 22.669 total
BLAZOW! That’s like 15 more seconds I can spend working out. OR REFACTORING!
USING ALL AVAILABLE HARDWARE, I CAN DELIVER BUSINESS VALUE FASTER!!
USE PARALLEL_SPECS!!
Fine print
parallel_specs does not run cucumber features in parallel. You might look at testjour which is aimed at running cucumber features, although it is intended to run them in a distributed fashion across multiple machines, rather than on one multicore machine.
If you have multiple disks, spreading databases out across them will likely yield even more performance.
The solution isn’t originally mine - the original idea made its way to me from Pivotal Labs via Michael Grosser. Currently, I’ve forked Michael’s code to add Test::Unit support, and plan to send a pull request after I clean up some of the duplication I added.
When bears say deliver business value, they mean eating your whole campsite.
Do not drink parallel_specs.
