---
title: Technical Considerations to Help Scale Your Product
teaser: What should you be thinking about when scaling your product?
tags: web,product
author: Joël Quenneville
published_on: 2019-10-29
---

You've had a successful product launch and now you're seeing explosive growth.
All that growth is putting a strain on your technology.  Recently it's getting
harder to add more users to the application. There have been a few small
outages. Your tech team is throwing around the word **"scaling"**. What does
that actually mean, and how can you make informed decisions around scaling a
product?

![](https://images.thoughtbot.com/master/0feM2B4bSEODHrBlAXbk_exponential-growth.png)

## What is scaling

> **Ask “Will this scale?” no matter what it is**
>
> It’s important to find out if things will scale no matter what it is you’re
> discussing. No one even really knows what that means, but it’s a good
> catch-all question that generally applies and drives engineers nuts.
>
> -- Sarah Cooper in [10 tricks to appear smart during meetings]

[10 tricks to appear smart during meetings]: https://medium.com/conquering-corporate-america/10-tricks-to-appear-smart-during-meetings-27b489a39d1a

"Scaling" is somewhat of a weasel word. It means everything and nothing at all.
What are your trying to scale? The number of users the app can support? Data
throughput? The hardware used by the application?

It's also important to note the difference between **performance** and
**scaling**. Performance is the amount of work you can do with the _existing_
resources. Scaling itself has multiple meanings, sometimes referring to
increasing the amount of work the application can do or increasing the amount of
resources available to the app. Improving an application's performance is one of
several tools at your disposal to help solve scaling problems.

## How to scale

There are many different levers you can pull on to help meet your scaling goals,
some of which are vastly more **expensive** than others.

Below are a list of actions you can take, ordered from cheapest to most
expensive and I recommend following them in order. If you accomplish your goals
before getting to the end then congratulations! You've just saved yourself work
that's expensive both in money and in terms of increased code complexity.

1. Define Goals ($)
2. Profile and identify bottlenecks ($)
3. Hardware ($$)
4. Performance optimizations ($$)
5. Architecture changes ($$$$)

This list reflects the approach we take when working with [our clients] to solve
scaling problems.

[our clients]: https://thoughtbot.com/work

## Define Goals ($)

Before you do any work it's important to **know what you are working towards**.
Any talk of "scaling" an app is meaningless without any numbers attached.
Contrast the following:

> We need to scale this app

vs

> What changes do we need to make to be able to scale up to 100K daily users?
>
> We're onboarding a new supplier. We'll need to be able to handle importing
> their catalogue of 10K parts into our system.
>
> Tickets for a hotly anticipated event are going on sale next week. Our
> ticketing system needs to be able to support an expected 1200 transactions in
> the first five minutes.

Better performance and more resources _might_ help you reach that goal. However,
they might just add extra cost and complexity if they don't address the
application's **bottlenecks**.

## Identify bottlenecks ($)

The worst thing you can do is blindly make changes hoping for the best. It's
important to figure out _what_ is causing the issues you've been seeing before
making any changes.

Is your database overwhelmed? Is your web server running out of memory?  Are you
receiving too many simultaneous web requests? Are you getting rate-limited by a
third-party service?

For the vast majority of web-based applications, the first bottleneck you see
will be the **database**.

Once you know _what_ part of your system is slow, then you need to understand
_why_ and _how slow_. And not just in a qualitative "this is slow" sense. You
want **actual measurements** that you can use to compare before and after you
make changes.

Let's say you've identified the database is your bottleneck. Then you should
start asking  yourself questions like: Are you dealing with too many queries?
How many? Are your queries taking a long time? How long?

## Hardware ($$)

![Slider to add more dynos on Heroku](https://images.thoughtbot.com/master/v3gn4FVxT5OglqeQE1wX_heroku-dynos.png)

If you keep adding users, there will come a time when you outgrow your hardware.
Hardware is cheap, and cloud providers make it trivial to add more. Some places
allow you to scale up hardware simply by dragging a slider.

So go ahead and upgrade to that database plan with a higher row limit. Increase
your max RAM. Get a bigger connection pool.

Just make sure you **identify your bottlenecks** first. Putting a faster web
framework in front of a slow database may not have any measurable speed or
throughput increases. Some bottlenecks can't be easily be solved by throwing
hardware at them. For these, you may want to look at doing some performance
optimizations.

## Performance tuning ($$)

Once you understand the problem, it's time to make some changes! The cheapest
solution is often to do some performance tuning. These are usually small code
changes that allow you to get more work done with your existing architecture and
hardware.

For example, adding an [**index**] might dramatically improve the speed of that
query.  Eliminating an [**N+1 query loop**] could dramatically reduce the number
of queries sent to the database. A little **caching** can do wonders! All of a
sudden your app can deal with more users while using the same amount of
resources.

Performance tuning can also have some downsides. For example, introducing some
caching can dramatically increase the performance of your application. That's a
win! It also opens you to the possibility of _cache invalidation bugs_, a
notorious class of bugs often described as [one of the hardest problems in
computer science]. The general advice around caches (and performance work in
general) is [don't add one until you need it]. Otherwise you pay their cost
without getting any value in return.

[**index**]: https://thoughtbot.com/blog/a-grand-piano-for-your-violin
[**N+1 query loop**]: https://semaphoreci.com/blog/2017/08/09/faster-rails-eliminating-n-plus-one-queries.html
[one of the hardest problems in computer science]: https://martinfowler.com/bliki/TwoHardThings.html
[don't add one until you need it]: https://thoughtbot.com/blog/ruby-memoization-and-alternatives

## Architecture ($$$$)

Eventually you'll reach the point where your code is as efficient as it can be
and adding yet more hardware is getting prohibitively expensive. Now is the time
to revisit the architecture of your application.

Perhaps you want to start [sharding] your database or add a [read-replica].
It might make sense to move to a [streaming data pipeline].

Re-architecting is expensive so you really don't want to get this one wrong.
**Always profile first** and make sure you don't have problem that can be solved
with some smaller-scale performance tuning.

Adding an index to your database is _far_ cheaper than introducing sharding and
may solve your current issue.

[sharding]: https://medium.com/@jeeyoungk/how-sharding-works-b4dec46b3f6
[read-replica]: https://www.brianstorti.com/replication/
[streaming data pipeline]: https://thoughtbot.com/blog/flatmap-your-job-queue

## Scaling too early

> The real problem is that programmers have spent far too much time worrying
> about efficiency in the wrong places and at the wrong times; premature
> optimization is the root of all evil
>
> -- Donald Knuth in _Computer Programming as an Art_

Scaling is often a set of tradeoffs. In addition to actual dollar costs for
development time and better hardware, performance and scaling changes often
introduce code complexity which impacts future development speeds and the rate
of bugs introduced into the application.

This complexity means your code is now harder to understand for new hires,
harder to extend, and more prone to bugs. Because the  system now has more
moving parts, there are more places where things could go wrong.

Greater capacity comes at a cost of greater program complexity and slower
development speeds. Don't pay these costs if you don't need to.

## Common scaling myths

> Google and Neflix do X. Should we do the same?

No. Google and Netflix operate at such unimaginably huge scales. Their problems
are not your problems.

> But we need to support a billion users on launch!

You almost certainly don't have 15% of the planet's population waiting to sign
up for your app on day one. By targeting capacity for a reasonable amount of
traffic and then scaling from there, you can iterate on your product faster for
a lower cost.

> Should we do a rewrite?

Probably not. Rewrites are expensive and often don't solve your original problem
or just give you a new set of problems. Make sure you understand your
application's bottlenecks and have hard numbers before even looking at a
rewrite.

## Continuing the conversation

Scaling is an incredibly nuanced topic that involves a lot of tradeoffs. There
are countless technical solutions that can help you scale, ranging from
single-line code changes to massive re-architectures of your application. This
article gave a high-level overview of different families of solutions.

If you are facing scaling challenges and have questions about specifics or maybe
aren't even sure where to start, [talk to thoughtbot]!

[talk to thoughtbot]: https://thoughtbot.com/hire-us