What impact does inclusive vs exclusive cache design have on DB buffer cache efficiency?
In the world of high-performance database systems, the architecture of the CPU cache is just as important as the size of the DB buffer cache itself. When choosing hardware for an Oracle, SQL Server, or PostgreSQL workload, you’ll encounter two competing philosophies: Inclusive and Exclusive cache designs.
Each impacts how your database handles its most frequent operation—moving data from the buffer cache in RAM into the CPU for processing.
In an inclusive design, the Lower Level Cache (L3) must contain a copy of everything stored in the Higher Level Caches (L1 and L2). If a line is in L1, it must also be in L3.
The "Snoop" Advantage: For a database engine, the biggest benefit is coherency speed. If Core A needs a data block that might be in Core B’s private L1 cache, the CPU only has to check the L3 "directory." If it’s not in the L3, the hardware knows for a fact it’s not in any L1/L2, saving a costly "snoop" of every other core.
Wasted Capacity: The downside is "redundancy." If you have a 45MB L3 cache, a portion of that is "wasted" holding duplicates of what is already in the L1 and L2. For databases with massive metadata or tiny, hot lookup tables, this reduces the effective amount of data that can sit close to the execution units.
The "Back-Invalidation" Risk: If a line is evicted from the L3 to make room for new data, the hardware must also evict it from L1 and L2. This can cause "silent" performance drops in databases during heavy scans, as the L3 fills up and starts "kicking out" hot pointers from the L1.
In an exclusive design, the L3 cache only holds data that is not in the L1 or L2 caches. It acts as a "victim cache." When a data block is evicted from L2, it gets pushed down into L3.
Maximum Effective Capacity: This is the primary winner for databases. If you have 32MB of L3 and 8MB of L2, you have a total of 40MB of unique data near the core. This allows more of the database’s "hot set"—index branch blocks or frequently accessed latches—to stay in the silicon rather than falling back to RAM.
The "Fill" Latency: When a database thread requests a block that isn't in L1, it checks L3. If found, the data is moved up to L1 and usually removed from L3 (or swapped). this "shuffling" can add a few cycles of latency compared to a simple inclusive read.
Complex Coherency: Because L3 doesn't have everything, the CPU has to work harder to track where a specific database buffer resides across a 64-core or 128-core chip.
| Feature | Inclusive (e.g., Intel) | Exclusive (e.g., AMD) |
| Total Cache Utility | Lower (Duplication) | Higher (Unique data) |
| Snoop Latency | Lower (Faster Coherency) | Higher (Complex tracking) |
| Heavy Random I/O | Better for "Ping-pong" workloads | Better for "Large Hot-set" workloads |
| Performance Jitter | Can suffer from back-invalidations | More consistent under capacity pressure |
Choose Inclusive (Intel) if your database workload is highly contended. If you have many threads fighting over the same internal locks and latches (high buffer busy waits), the faster snoop filtering of an inclusive L3 reduces the "latency tail" of those conflicts.
Choose Exclusive (AMD/ARM) if your database workload is memory-intensive but horizontally scaled. The larger "effective" cache size acts like a bigger "L4 buffer" for your index pages, keeping more of your working set out of the system RAM ($10\text{ns}$ vs. $100\text{ns}$).