Feeling the Speed: Understanding the Cache Hierarchy

While reading about the query processing layer in ClickHouse, I came across this detail:

To keep CPU caches hot, the plan contains hints that the same thread should process consecutive operators in the same lane. This got me thinking: CPU caches are clearly critical for performance—but what exactly makes them so special? Are they really that much faster than RAM to justify this added complexity?

It reminded me of a design review meeting where I confidently suggested computing directly over data on disk because “SSDs are fast enough.” (Yes… shame on me 😵‍💫)

The bigger problem wasn’t just my idea — it was that none of us had a solid intuition about the massive latency differences in the cache hierarchy. That detail from ClickHouse reminded me how important it is to deeply understand caching fundamentals when designing performant systems and led me to an interactive and insightful article by Ben Dicken.

This beautifully written article from PlanetScale takes you back to those fundamentals, explaining everything from L1 CPU cache to CDNs, covering temporal, spatial, and geospatial locality, eviction policies, and why caches are so vital — all with clear explanations and interactive animations.

If you want to build or refresh your intuition around performance and caching, this is a must-read 🔥