System Design Basics: Scaling From One User to a Million

Learning Material · System Design

Stage one: one server, one database

Every system starts here, and for a while, it's genuinely enough — a single server running the app and a single database handling reads and writes. Don't add complexity ahead of an actual, measured bottleneck; premature scaling is its own kind of waste.

Stage two: separate the database

The first real bottleneck is usually the database competing with the app for the same server's resources. Moving the database to its own server is a simple, high-value first step, letting each scale independently of the other.

Stage three: add a load balancer

            ┌── App Server 1
Client ──→ LB ─┼── App Server 2
            └── App Server 3

A load balancer distributes incoming requests across multiple app servers, so one server going down doesn't take the whole app offline, and traffic beyond one server's capacity gets spread out. This is also the point where "sessions stored in server memory" becomes a real bug — a user's second request might land on a different server that's never heard of their session, which is why session data needs to move to a shared store (Redis, the database) at this stage.

Stage four: cache what's expensive to recompute

function getUser(id) {
  const cached = redis.get(`user:${id}`)
  if (cached) return cached

  const user = db.query('SELECT * FROM users WHERE id = ?', [id])
  redis.set(`user:${id}`, user, { ex: 300 })
  return user
}

A cache trades a small amount of staleness for a large amount of speed — this example serves from Redis for 5 minutes before hitting the database again. Caching the results of expensive queries or computations is usually the single highest-leverage performance change available, because it reduces load on your slowest component (the database) directly.

Stage five: scale the database itself

Once one database server can't keep up, two common patterns emerge: read replicas (copies that handle read queries, while the primary handles writes — useful since most apps read far more than they write), and sharding (splitting data across multiple databases by some key, like user ID range — necessary once a single database's data no longer fits comfortably on one machine).

The pattern underneath all of this

Every stage above follows the same idea: identify the actual bottleneck, then add just enough infrastructure to remove it — not infrastructure for problems you don't have yet. Systems that scale well tend to grow this way, one measured bottleneck at a time, rather than being over-engineered for a million users on day one.