CPU dispatch latency is the time a ready-to-run thread spends waiting in the run queue before it actually gets CPU time. In transaction systems, that waiting time directly shapes how many transactions you can complete per second.

1. Why dispatch latency matters for transactions

A typical OLTP transaction is short and has multiple steps:

• parse → compute → I/O → commit

Between these steps, the thread may:

• block (waiting for I/O or locks)
• become runnable again

Every time it becomes runnable, it must:

wait to be dispatched onto a CPU

If that wait is long → throughput drops.

2. Relationship to throughput (core idea)

You can think of transaction throughput as:

Throughput ≈ 1 / (execution time + wait time)

Dispatch latency is part of that wait time.

So:

• Higher dispatch latency → longer total transaction time
• Longer transaction time → fewer transactions per second

3. What causes high dispatch latency

(A) CPU contention

• Too many runnable threads
• Not enough physical CPU capacity

(B) Over-subscription (virtualization)

• More vCPUs than physical CPUs
• LPARs/VMs competing for CPU

(C) Priority imbalance

• Low-priority workloads waiting behind high-priority ones

(D) SMT contention

• Multiple threads competing on same core
• Reduced effective CPU share per thread

4. Impact on OLTP workloads (very sensitive)

OLTP systems are:

• Latency-sensitive
• High concurrency
• Short transactions

Effect:

• Even small increases in dispatch latency → big drop in throughput
• Queueing delays compound across thousands of transactions

👉 Example:

• 2 ms extra dispatch delay per transaction
• At scale → significant TPS loss

5. Impact on OLAP workloads (less sensitive)

OLAP queries:

• Run for seconds/minutes
• Are CPU-bound

Effect:

• Dispatch latency is relatively small compared to total runtime
• Throughput impact is minimal

6. Queueing effect (critical insight)

When dispatch latency increases:

1. Transactions take longer
2. More transactions accumulate in queue
3. CPU demand increases
4. Dispatch latency increases further

👉 This creates a feedback loop (queue buildup)

Result:

• Throughput collapses under load (non-linear degradation)

7. IBM system perspective

On IBM Power (PowerVM)

• Dispatch latency increases when:
  • Entitlement is too low
  • Too many uncapped LPARs compete

👉 Fix:

• Increase entitlement
• Reduce overcommit

On IBM Z

• Hardware-assisted dispatch (PR/SM)
• Prioritization reduces latency for critical workloads

👉 Result:

• More stable throughput even under heavy load

8. Key metrics to watch

• Run queue length
• CPU utilization
• Dispatch wait time
• CPU steal time (in virtual environments)
• Transaction response time

9. How to reduce dispatch latency

(A) Increase CPU capacity

• Add cores or increase entitlement

(B) Reduce overcommit

• Fewer vCPUs per physical CPU

(C) Tune priorities

• Ensure critical workloads get CPU first

(D) Optimize SMT settings

• Reduce contention for latency-sensitive workloads

(E) Improve workload placement

• Better NUMA and cache affinity

10. Simple analogy

Think of a checkout counter:

• CPU = cashier
• Transactions = customers

Dispatch latency = time waiting in line

If the line grows:

• Each customer takes longer to complete
• Fewer customers are served per hour

Key takeaway

CPU dispatch latency directly reduces transaction throughput by increasing the wait time before execution; in high-concurrency OLTP systems, even small increases in dispatch delay can cause significant throughput degradation due to queueing effects and CPU contention.