---
title: 'Recipe: A/B Testing With KISSMetrics And The Split Gem'
teaser: How do you set up A/B testing in a Rails app?
tags: web,redis,ruby
author: Dan Croak
published_on: 2011-08-30
---

A/B testing can turn skeptics into believers. Jamie, a designer at 37signals,
recently [shared a great case
study](http://37signals.com/svn/posts/2991-behind-the-scenes-ab-testing-part-3-final)
of A/B testing Highrise.

So, what are the mechanics of actually setting up an A/B test in a Rails app?

One approach we're trying right now on
[Trajectory](https://www.apptrajectory.com) is using the `split` gem with
KISSMetrics. For the first few months of Trajectory's life, we felt it was
necessary to introduce Trajectory and explain why we made it in the face of
existing tools.

Now, it's time to explain its benefits on its own merits and see how well that
resonates with potential users versus the old copy. This recipe will show that
example. We'll also be testing more dramatic layouts, which this combination
can also handle.

* [KISSMetrics](http://kissmetrics.com)
* [split gem](https://github.com/andrew/split)
* [Redis](http://redis.io)

## Setup

`Gemfile`:

```ruby
gem "split"
```

`config/initializers/split.rb`:

```ruby
Split.redis = ENV['REDISTOGO_URL'] || 'redis://localhost:6379'
Split.redis.namespace = "split:trajectory"
```

## Red

Let's use the [Cucumber directory
convention](https://thoughtbot.com/blog/post/164479801/cucumber-directory-convention).

`features/visitor/views_homepage.feature`:

    Scenario: Original landing page
      When I go to the home page with the "original" alternative for the "landing_page" experiment
      Then I should see "Over the past 8 years, we've used many tools"
      And KISSmetrics receives the following properties:
        | property     | value    |
        | landing_page | original |

    Scenario: New copy on landing page
      When I go to the home page with the "new_copy" alternative for the "landing_page" experiment
      Then I should see "One gorgeous tool that everyone actually LIKES to use"
      And KISSmetrics receives the following properties:
        | property     | value    |
        | landing_page | new_copy |

The [split gem's documentation](https://github.com/andrew/split) provides a way
to override the alternatives.

`features/support/paths.rb`:

    when /^the home page with the "([^"]+)" alternative for the "([^"]+)" experiment/
      "/?#{$2}=#{$1}"

We can figure out the expected Javascript from the [KISSMetrics API documentation](http://support.kissmetrics.com/a-b-testing/).

`features/step_definitions/kissmetrics_steps.rb`:

    Then /^KISSmetrics receives the following properties:$/ do |table|
      table.hashes.each do |hash|
        property = hash['property']
        value = hash['value']
        expected_javascript = %Q{_kmq.push(["set","#{property}","#{value}"]);}

        page.should have_content(expected_javascript)
      end
    end

## Green

In this example, we set up two partials that simply contain a translation.

The translation uses the Rails i18n <abbr title="Application Programming
Interface">API</abbr> and is backed by [Copycopter](https://copycopter.com) in
order to make changes to it without re-deploying.

`app/views/homes/_new_copy.html.erb`:

```html
<%= t(".letter-new-copy", :default => %{
  <p><strong>Less frustration, more joy: there's a better way to build software.</strong></p>
  <p>You know that your software planning tools aren't perfect.  It's not clear what to do next.</p>
  <p>We had the same problems.  We’re thoughtbot, a web design and development agency, and that's why we made Trajectory.</p>
  <p>Imagine if you could build better software, faster.  Imagine your teammates not waiting on each other, having a clear sense of what to do next.</p>
  <p>It's super easy to try out for free.  Hit the ground running by importing your Pivotal Tracker project or invite your current team members.</p>
}) %>
```

`app/views/homes/_original.html.erb`:

```html
<p><strong>Hi, we&rsquo;re thoughtbot, a web design and development agency.</strong>
<%= t(".letter", default: %{
  <p>Over the past 8 years, we've used many tools for project communication and planning.</p>
  <p>Basecamp was great for discussion and communication. Pivotal Tracker was great for user stories and emergent planning.</p>
  <p>We've grown tired of having one tool that designers love, one tool that developers love, and no tool that clients love.</p>
  <p>We created Trajectory to solve our own problems. We now use it on all of our projects. Maybe it can solve your problems too.</p>
}) %>
```

Using `split` is pretty simple.

`app/views/homes/show.html.erb`:

    <%= render partial: ab_test("landing_page", "original", "new_copy") %>

`split` provides a web interface but we have all our funnel metrics in
KISSMetrics so we want to send the data there.

`app/views/shared/_javascript.html.erb`:

```html
<script type="text/javascript">
  <% Split::Experiment.all.each do |experiment| %>
    _kmq.push(['set', { '<%= experiment.name %>': '<%= ab_test(experiment.name, *experiment.alternative_names) %>' }]);
  <% end %>
</script>
```

## Putting A/B testing in context

This is a recipe for "how" to A/B test but if you're interested in "when" to
test, see our [A/B
testing](http://playbook.thoughtbot.com/validating-customers/a-b-testing/) page
in our playbook.