How do you model hardware-level contention for 1M TPS workload?
Modeling a system to handle 1 million Transactions Per Second (TPS) is less about software logic and more about the physics of the machine. At this scale, the "invisible" bottlenecks—cache misses, context switching, and bus saturation—become the primary constraints.
Here is how you model hardware-level contention for a high-frequency workload.
Before diving into the silicon, you need to establish the baseline relationship between arrival rate ($\lambda$), latency ($W$), and concurrency ($L$).
If your target is $\lambda = 1,000,000$ TPS and your average processing time is 1ms, you must support 1,000 concurrent requests perfectly in flight. If latency spikes to 10ms due to contention, you suddenly need to manage 10,000 concurrent requests, which often leads to a "death spiral" as resource contention increases latency further.
At 1M TPS, hitting Main Memory (RAM) is often too slow. You are modeling for L1/L2 cache residency.
The Problem: False sharing. This occurs when multiple cores attempt to modify different variables that reside on the same cache line (usually 64 bytes).
The Model: Use the MESI protocol (Modified, Exclusive, Shared, Invalid) state machine to estimate overhead. If two threads frequently "fight" over a cache line, the hardware must broadcast invalidations across the interconnect, killing throughput.
Optimization: Model data structures with cache-line padding to ensure high-velocity counters sit on their own lines.
A "transaction" isn't just one block of code; it's a series of instructions.
The Problem: Deep pipelines hate surprises. If your code has complex if/else logic that the CPU mispredicts, the pipeline flushes, wasting dozens of clock cycles.
The Model: Calculate the Cycles Per Instruction (CPI). At 1M TPS on a 3GHz CPU (3,000,000,000 cycles/sec), you have roughly 3,000 cycles total across all available cores to finish one transaction.
Optimization: Use branchless programming and favor linear data arrays over linked lists to keep the prefetcher happy.
If your model relies on standard OS threads for 1M TPS, you will likely fail due to context switch overhead.
The Problem: Every time the OS swaps a thread, it saves registers and flushes part of the TLB (Translation Lookaside Buffer).
The Model: Estimate a cost of 1–5 microseconds per context switch. Multiply that by 1M transactions, and the OS consumes 100% of your CPU just doing "bookkeeping."
Optimization: Model a User-Space Runtime (like DPDK for networking or LMAX Disruptor for logic) that pins threads to specific cores and uses busy-waiting (polling) instead of interrupts.
To see how much you can actually parallelize, apply Amdahl's Law. It defines the maximum speedup ($S$) based on the serial portion of your code ($s$).
Where $n$ is the number of cores. If even 5% of your transaction logic requires a global lock (serial), you can never achieve infinite scaling, no matter how many cores you add. At 1M TPS, your goal is to get $s$ as close to 0 as possible using lock-free data structures.
| Component | Contention Source | Modeling Metric |
| L3 Cache | Multiple sockets sharing cache | Cache Miss Ratio |
| Memory Bus | Excessive RAM R/W | GB/s vs. Max Bandwidth |
| NIC | Interrupt Storms | Packets Per Second (PPS) |
| PCIe Bus | NVMe/GPU data transfer | Lane Saturation |
To hit 1M TPS, you don't build a "web app"; you build a mechanical symphony. You must model your data flow to stay within the CPU cache, avoid the kernel's overhead, and eliminate global locks that trigger Amdahl's Law.