Parallel Gem Installing Using Bundler

Do you realize how much time you’ve spent running bundle install?

]

No more sword fighting! Bundler 1.4.0 adds support for parallel installation. You can pass in --jobs SIZE as a parameter to bundle config¹. I recommend setting the size to one less the number of CPU cores on your machine².

But Prem, how much time could I save

Let’s benchmark this! I’m going to run bundle install on a freshly-created Rails app.

Before:

$ rvm gemset use j1 --create
Using ruby-2.0.0-p247 with gemset j1
$ time bundle install
# ... snip ...
bundle install  5.75s user 1.76s system 24% cpu 30.679 total

After:

$ rvm gemset use j8 --create
Using ruby-2.0.0-p247 with gemset j8
$ bundle config --global --jobs 8
$ time bundle install
# ... snip ...
bundle install  7.48s user 2.59s system 86% cpu 11.681 total

That’s about 19 seconds faster (~61.90% improvement) over regular bundle install!!!

That’s awesome! How can I use this now

Simple! Just install a prerelease version of Bundler, and you should be good to go:

gem install bundler --pre

Then, configure Bundler to parallelize on your machine globally:

bundle config --global jobs 7

If you’re not sure how many cores you have, you can find out dynamically. Here’s how we set up that configuration automatically on OS X in our laptop script:

number_of_cores=`sysctl -n hw.ncpu`
bundle config --global jobs `expr $number_of_cores - 1`

If you want to edit or delete the configuration directly, it can be located here:

vim ~/.bundle/config

And now, you can spend less time on sword fighting³, and more time on writing actual code.

Credit

I would like to give my thanks to Kohei Suzuki who submitted this patch to Bundler and Murahashi Kenichi for bringing this into my attention on Appraisal.

1. Sadly, this option will not be a default, since it does not work every time and you might want to fall back to do sequential installation in that case. ↩
2. I originally recommended you to set the size to be equal to the number of your CPU cores. However, it turned out that setting the number to be N-1 will yield a better result.↩
3. Well, that could also be considered as a downside as well. ↩