Running Acceptance with Heroku Review Apps

Last week, Heroku announced that Review Apps were generally available. Review apps are instant, disposable Heroku app environments that spin up automatically with each GitHub pull request. They allow teams to automatically build and test any pull request, updated at every push, at a temporary, shareable URL. GitHub users are notified of all this, right in the pull request web interface.

Using Heroku Pipelines and Review Apps improved our code review and acceptance test workflows.

Before Review Apps

When we wanted to share work in progress with the team (developers, designers, product owners, etc.) we had to:

  • ask them to come over and take a look at our development environment, or
  • take over the staging environment with our work-in-progress branch, or
  • set up a development Heroku app where we can share early work (effectively, a second staging environment), or
  • send a pull request, apply code review feedback, merge into master and publish in staging. If undiscovered bugs arise during acceptance, we had to start over the development, code review, and acceptance cycle, with a master that’s not deployable anymore, violating a rule of GitHub flow.

After Review Apps

With Heroku Review Apps, each time we create a pull request, a review app will be automatically created in Heroku, which runs the code in a disposable app environment. Each review app has a temporary URL that we can access from the GitHub pull request or the Heroku Dashboard. Collaborators can follow the link to inspect and test the generated app.

Whenever the branch is updated, Heroku updates the related review app. When the pull request is merged or closed, Heroku destroys the review app.

When we want to share work in progress with the team, we push our branch and open a pull request, as early as needed. An application is generated for everyone to see and to provide relevant feedback: how it feels, how it looks, how it behaves, everything while code review might also be happening. If during acceptance people discover bugs, good! We fix them right in the feature branch.

We configure Circle CI to auto-deploy to staging when new commits are merged into master and the test suite passes. That way, when the pull request is merged (and assuming a green CI run) we keep staging up to date, for final testing before promoting to production. Because staging always has the stable master branch, the product owners can reliably use it to showcase upcoming features to their stakeholders.

When ready, any person with access to the Heroku app can promote staging to production, so that the new features are made available to end users.

Enabling Heroku pipelines

Create a pipeline

We first create the pipeline and set the first applications in it (in this case, the staging app).

$ heroku pipelines:create upcase -a upcase-staging --stage staging
Creating upcase pipeline... done
Adding upcase-staging to upcase pipeline as staging... done

We then add the production app to the pipeline, as the production stage:

$ heroku pipelines:add upcase -a upcase-production --stage production
Adding upcase-production to upcase pipeline as production... done

We can also do this through Heroku’s web interface (see video).

Enable Review Apps

In the pipeline view for our application, we see a “Connect to GitHub” button. After doing so, we see a button “Enable Review Apps”:

Review apps enabled

The button opens a pop up with the title “Before you enable review apps, you need to create an app.json file”:

Pick staging as parent app

Create an app.json file

An app.json file is required for review apps to work in the root of the GitHub repository. This file is used to configure the new apps that are created when we open a new pull request.

Review apps can inherit config vars from their parent. We choose staging as the parent app. After clicking the “Create an app.json File” button, we see a form that allows us to configure the environment variables and scripts the app needs before creating the file.

We can define variables as required, “secret”, or not needed. For example, we want RACK_ENV to be required and copied over from the staging environment, but we want to generate a new secret for the SECRET_KEY_BASE variable, used for the secret_access_key Rails configuration option. Variables like DATABASE_URL should be generated, and not copied over.

In this pop-up, check the “Create new review apps for new pull requests automatically” option. Submit, and done!

It might cost money

We received an email from Heroku that says:

As you test using review apps, note that nominal Heroku charges may be incurred for usage and any paid add-ons that may be required. Review apps will be automatically deleted when the associated pull request is closed.

Heroku charges for dynos and add-ons used by review apps in the same way as for regular apps. Costs are pro-rated, and charges apply only for the time that the review app exists.

Whenever possible we use free plans for our staging and review apps’ add-ons, so we get the benefits of Heroku Pipelines at no cost.

It saves time

This feature is in suspenders, and it has made our code review and acceptance more productive. If you deploy to Heroku, you should definitively check it out.