Why Your CPU Has Three Levels of Cache

A modern CPU can execute an instruction in under a nanosecond. Fetching data from RAM takes roughly a hundred times longer. Cache exists to close that gap — and the three levels are a tradeoff between speed and size.

L1: smallest and fastest

L1 cache sits directly on each CPU core, typically 32–64KB, and responds in a handful of clock cycles. It's split into an instruction cache and a data cache, so the core can fetch the next instruction and the data it needs at the same time. It's small because faster memory is physically more expensive to build at scale.

L2: the middle ground

L2 is larger — usually 256KB to a few MB — and a bit slower, often shared between a small cluster of cores or dedicated per-core depending on the chip. It catches what L1 missed before the CPU has to go further out.

L3: shared, and the last stop before RAM

L3 is shared across all cores on the chip, ranges from several to tens of megabytes, and is noticeably slower than L1 or L2 — but still far faster than RAM. It's the CPU's last chance to avoid a trip to main memory.

Why this matters for real performance

Code that repeatedly touches the same small chunk of memory — a tight loop over a small array, say — stays in L1 or L2 and runs fast. Code that jumps around a large, scattered dataset causes constant "cache misses," forcing the CPU back to slower memory over and over. This is a big part of why the same algorithm can run several times faster depending purely on how its data is laid out in memory.