In the architecture of a modern server, PCIe (Peripheral Component Interconnect Express) is the high-speed highway that connects the CPU to everything from GPUs and NVMe drives to high-speed network cards.

Understanding how to calculate PCIe bandwidth is critical for ensuring you aren't "choking" your hardware by plugging a 100GbE card into a slot that doesn't have enough throughput to support it.

1. The Components of the Calculation

PCIe bandwidth is determined by three factors:

1. PCIe Generation: The "speed limit" of the version (Gen 3, Gen 4, Gen 5, etc.).

2. Lane Count: The "number of lanes" on the highway (x1,x4,x8,x16).

3. Encoding Overhead: The "tax" paid for data integrity (the bits used for signaling rather than actual data).

2. Bandwidth per Generation

Each new generation of PCIe effectively doubles the bandwidth of the previous one. Here is the throughput per single lane (x1):

3. The Formula

To calculate the total theoretical bandwidth of a specific slot, use this formula:

Example 1: An NVMe Drive (PCIe 4.0 x4)

Most modern NVMe SSDs use 4 lanes.

• $1.97\text{ GB/s}\times 4\text{ lanes}=7.88\text{ GB/s}$

• Verdict: This matches the top-tier speeds of Gen 4 SSDs (~7,500 MB/s).

Example 2: A High-End GPU (PCIe 5.0 x16)

Enterprise GPUs for AI/ML require massive pipes.

• $3.94\text{ GB/s}\times 16\text{ lanes}=63.04\text{ GB/s}$

4. The "Encoding Tax" Explained

You might see different numbers online (like 8Gbps vs 1GB/s). This is because of Encoding.

• Gen 1 and 2 used 8b/10b encoding, meaning 20% of your bandwidth was wasted on overhead.

• Gen 3 and newer use 128b/130b encoding, which is much more efficient, wasting only about 1.5%.

• GT/s vs Gbps: GT/s refers to the total "Giga-Transfers" (including overhead). When you subtract the encoding tax, you get the actual data rate in Gbps or GB/s.

5. Why Does This Matter for Servers?

The biggest mistake in server builds is Lane Starvation. A CPU has a limited number of PCIe lanes (e.g., an Intel Xeon might have 80 lanes). If you plug in:

• 2 GPUs (x16 each = 32 lanes)

• 4 NVMe Drives (x4 each = 16 lanes)

• 2 100GbE NICs (x16 each = 32 lanes)

You have used 80 lanes. If you add one more card, the motherboard may "bifurcate" (split) a slot, forcing an x16 card to run at x8 speeds, effectively cutting its performance in half.

6. Summary: Quick Reference Table (GB/s)

The Bottom Line

When sizing hardware, always match the generation and lane count of the card to the slot. Putting a PCIe 4.0 card in a PCIe 3.0 slot will work, but you will be limited to the slower 3.0 speed limit.