What is east-west vs north-south traffic modeling at rack scale?
In a 1M TPS (Transactions Per Second) architecture, the network is no longer a "pipe"—it is a high-performance backplane. At this scale, how you model traffic determines whether your system remains snappy or collapses under the weight of its own internal chatter.
We categorize this movement into two orthogonal vectors: North-South and East-West.
North-South traffic represents the data flowing into and out of the data center or rack.
Direction: Client $\leftrightarrow$ Server.
The "North": Users, mobile apps, and external APIs (Ingress).
The "South": The internal application endpoints (Egress).
Characteristics: High security requirements (WAF, TLS termination, DDoS protection) and often higher latency due to the "Internet hop."
1M TPS Impact: This is your front door. If your North-South gateway (Load Balancer) can't handle the 1M TPS handshake overhead, the transactions never even reach your compute layer.
East-West traffic is the communication between servers, microservices, or databases within the same environment.
Direction: Server $\leftrightarrow$ Server (or Pod $\leftrightarrow$ Pod).
The Reality: In modern distributed systems, East-West traffic accounts for 80%+ of all data center traffic. * The Multiplier Effect: A single "North-South" user request (e.g., "Buy Now") can trigger 20–50 "East-West" calls (checking inventory, verifying credit, updating loyalty points, logging analytics).
1M TPS Impact: If you have 1M TPS at the edge, your internal network might be handling 20M to 50M PPS (Packets Per Second).
Traditional data center designs (Three-Tier) were optimized for North-South traffic. However, this created "choke points" for internal chatter.
| Feature | Legacy Three-Tier (North-South focus) | Modern Leaf-Spine (East-West focus) |
| Traffic Path | Data must go "up" to the Core switch to move "sideways." | Any two servers are always a fixed number of hops away. |
| Bottleneck | The "Core" switch becomes a single point of congestion. | Bandwidth is distributed across multiple "Spine" switches. |
| Scalability | Scaling up (Bigger switches). | Scaling out (Adding more Leaf/Spine units). |
| 1M TPS Fitness | Poor: High "Tail Latency" due to hop variance. | Excellent: Deterministic latency and massive internal bandwidth. |
When designing for 1M TPS, you must model for "Lateral Movement" risks and performance:
Micro-segmentation: Since most traffic is internal, you can't just trust the "inside." You need host-level firewalls (like eBPF or DPUs) to validate East-West calls.
The "Hairpin" Problem: Avoid "Hairpinning" (sending internal traffic out to a central firewall and back). At 1M TPS, this adds 500$\mu$s–1ms of unnecessary latency.
Service Mesh Overhead: Tools like Istio provide security for East-West traffic but add CPU overhead. In 1M TPS systems, we often use Sidecar-less meshes or DPU-offloaded mTLS to keep the "East-West" path fast.
| Metric | North-South (Edge) | East-West (Internal) |
| Volume | Lower (The "Trigger") | Extreme (The "Engine") |
| Latency Budget | Tens of Milliseconds | Microseconds |
| Primary Tool | API Gateway / WAF | Service Mesh / Leaf-Spine Fabric |
| Security Goal | Perimeter Defense | Zero Trust / Lateral Isolation |
At 1M TPS, you are building a network for East-West traffic. North-South is simply the trigger that starts the fire. If your rack-scale model doesn't prioritize low-latency, high-bandwidth lateral paths (using Leaf-Spine or RDMA), your internal "chatter" will saturate the network long before your CPUs hit 100%.