I hope we can agree that Using a CI system for automating your tests is important. Unfortunately, continuous integration for iOS apps hasn’t always been a great experience. Here at thoughtbot, we’ve run the gamut of services as I’m sure most have. Jenkins, Travis, Xcode bots, Circle. We’ve tried them all. We ended up choosing Circle for a number of reasons. A big reason was Circle’s ability to cache directories, which improves build times immensely.
By default, Circle already supports caching builds with CocoaPods. This is fantastic, and for a lot of users will be more than enough. However, we’ve been using Carthage more and more internally, due to the lightweight nature of the tooling, and the ability to use binary frameworks.
Unfortunately, trying to use Carthage caused problems with CI. Of note, the need to code-sign frameworks as they were built meant that we’d need to share signing certificates with our CI service. Additionally, Carthage is itself written in Swift. That means that it needs to be built with a specific version of Xcode, which complicates the build process using Homebrew.
Thankfully, these issues have been fixed! Carthage has been updated to Swift 2.0, which means that Homebrew can build with the latest stable version of Xcode. In addition, since Carthage version 0.11, it no longer requires code-signing when building frameworks, so there’s no need to pass around signing certificates. Even better, Circle recently started pre-installing Carthage on their Mac build systems so we no longer have to worry about manually installing it.
So where does all of this leave us? We can now build iOS projects on Circle, and we can use Carthage as our dependency management tool of choice. But there’s a problem: Circle doesn’t know how to cache Carthage dependencies by default, and as a result, our builds are really really slow. Since Carthage has to re-build all of our dependencies every time, it could easily take over 10 minutes to build a project and all of its dependencies.
Luckily for us, Circle’s build process is easily customized in way that will let us work around this issue.
Note that we’re assuming that you have added your Carthage/ directory to
your .gitignore for the purposes of this post. If you are checking in
Carthage/Checkouts or Carthage/Build, this post might not be super
useful for you.
Improving Carthage
To begin, we need to look at adding some smarts around Carthage. By design,
Carthage is a simple tool, but that same simplicity will let us write a thin
wrapper to add custom behavior. In this case, what we’d like to do is to only
run carthage bootstrap if the dependencies have changed. Carthage doesn’t do
this by default, but it’s fairly trivial to handle ourselves.
Carthage generates a file named Cartfile.resolved that declares the exact
dependencies it expects to be installed, similar to Podfile.lock or
Gemfile.lock. We can use that to determine which dependencies we have
locally and which ones we expect based on the current state of the repo.
To do this, we will first wrap the carthage bootstrap command to perform an
additional action. We’ll save this as bin/bootstrap:
#!/bin/sh
carthage bootstrap
cp Cartfile.resolved Carthage
Don’t forget to chmod +x bin/bootstrap so that this becomes executable.
This small script will run carthage bootstrap, and then copy the
Cartfile.resolved into our (gitignored) Carthage/ directory. This means
that Carthage/Cartfile.resolved will always reflect the currently
downloaded/built dependencies, while ./Cartfile.resolved will reflect the
dependencies that the project expects.
Now we can write another small script that will use this new
Cartfile.resolved file to determine if it needs to update the dependencies.
We’ll save this as bin/bootstrap-if-needed:
#!/bin/sh
if ! cmp -s Cartfile.resolved Carthage/Cartfile.resolved; then
bin/bootstrap
fi
Don’t forget to chmod +x bin/bootstrap-if-needed so that this becomes
executable.
This script will compare ./Cartfile.resolved and
Carthage/Cartfile.resolved. If they are different (or if
Carthage/Cartfile.resolved doesn’t exist), we will run our bin/bootstrap
script, which will in turn update the dependencies and move the new
Cartfile.resolved into place.
You can now test this. Running bin/bootstrap-if-needed should update your
dependencies the first time, but running it a second time should become a
no-op. Updating the dependencies (or deleting Carthage/Cartfile.resolved)
should also result in the script re-installing the dependencies.
Caching with Circle
So now comes the fun part. Since we’re now able to determine if we need to update our dependencies, we can leverage Circle’s built in (and fantastic) support for caching to speed up our builds.
To accomplish this, we’re going to add some config to our circle.yml
file. Specifically, we’re going to override the dependencies
key:
dependencies:
override:
- bin/bootstrap-if-needed
cache_directories:
- "Carthage"
Seriously, that’s it. The override key tells Circle what command to run when
installing dependencies. We tell it to use our smarter version of carthage
bootsrap. Then, we tell it to cache the entire Carthage directory. Circle
automatically moves the cache into place before the dependency step, and saves
it at the end. By the time we run bin/bootstrap-if-needed, everything should
be in place.
Since we’re caching the entire directory, that means we’re also caching the
Carthage/Cartfile.resolved file we create with bin/bootstrap. So when
Circle runs bin/bootstrap-if-needed, it will only build our dependencies if
the cached dependencies are out of date.
And there you have it. Once you push this up and Circle starts to use the new config, you should see a dramatic decrease in build times. We saw our times drop from 14 minutes to under 2 minutes.