How does cache line bouncing impact sequence-heavy workloads?
While we’ve discussed how Hardware Atomic Increments make these sequences fast, there is a hidden physical tax that can still bring a high-core-count server to its knees: Cache Line Bouncing.
At 1M TPS, your sequence isn't just a number; it's a "hot potato" being slapped back and forth between CPU cores at the speed of the interconnect.
CPUs don’t move data in bits or bytes; they move it in 64-byte chunks called Cache Lines.
The Scenario: Your sequence (a 8-byte BIGINT) lives inside one of these 64-byte lines.
The Conflict: To increment the sequence, Core A must "own" that cache line in Exclusive (E) or Modified (M) state (per the MESI protocol).
The Bounce: When Core B (on a different socket) wants to increment the same sequence, it sends a "Snoop" request. Core A must flush the line to L3 or send it directly to Core B.
The Result: The cache line "bounces" from Core A’s L1 cache to Core B’s L1 cache. At 1M TPS, this happens millions of times per second across the UPI or Infinity Fabric.
Every time a cache line bounces between sockets, the CPU pipeline stalls.
Local L1 Hit: ~1ns.
L3 Cache Hit: ~10–20ns.
Cross-Socket Bounce: 100ns – 300ns.
In a sequence-heavy workload, if every transaction requires an ID, and that ID causes a cross-socket bounce, you are adding a fixed 200ns tax to every single transaction. This creates a "Performance Ceiling" where adding more CPU cores actually decreases total throughput because the interconnect becomes saturated with "Ownership" traffic.
Cache line bouncing becomes even more destructive due to False Sharing.
The Problem: Your sequence is 8 bytes. The cache line is 64 bytes. If you store another frequently updated variable (like a "Last Update Timestamp") in the same 64-byte chunk as the sequence, they share the same fate.
The Interaction: Even if Core B only wants to update the Timestamp, it must snatch the entire cache line away from Core A, which was trying to increment the Sequence.
The Result: You get massive cache bouncing even though the two threads aren't even touching the same data.
| Technique | Hardware Interaction | 1M TPS Benefit |
| Sequence CACHE | Reduces the frequency of "Global" updates. | Each core "claims" 1,000 IDs at once, reducing bounces by 1,000x. |
| Padding (Alignment) | Moves other variables to different cache lines. | Eliminates False Sharing. |
| NUMA Pinning | Keeps the sequence-heavy threads on one socket. | Limits bounces to high-speed L3 instead of slow Interconnect. |
| Scalable Sequences | Appends the Instance_ID to the sequence. | Each node/socket has its own "Hot Line," eliminating cross-node bouncing. |
At 1M TPS, the Interconnect Bandwidth is often more precious than CPU Ghz.
Padding is Mandatory: In high-concurrency code, always pad your atomic counters to 64 bytes (or 128 bytes on some ARM chips) to prevent False Sharing.
Batching is King: A sequence with CACHE 20 is a recipe for a 1M TPS bottleneck. Use CACHE 1000 or higher to ensure the "Bounce" only happens once every thousand transactions.
Monitor "Remote Hits": Use hardware counters (like MEM_LOAD_RETIRED.REMOTE_HITM on Intel) to see how often your cores are fighting over the same cache lines.
Cache line bouncing is the "friction" of parallel computing. In a sequence-heavy workload, failing to account for it is like trying to run a marathon in sand—the harder you push (more cores), the more the environment resists you. By using Sequence Batching and Cache Line Padding, you grease the wheels of the silicon, allowing your IDs to flow at the speed of the L1 cache.