I/O queuing delay is the time a request spends waiting in a queue before it is actually serviced by a storage device or I/O subsystem. In high-performance systems, this delay is often the dominant contributor to end-to-end latency, especially under load.

The core idea is:

Even fast disks or SSDs can appear slow if requests spend too long waiting in queues.

1. Where I/O queuing delay fits

End-to-end latency of a transaction typically includes:

Total latency = CPU time + lock wait + I/O queue delay + I/O service time + network

👉 I/O queue delay happens before the actual read/write begins.

2. What causes I/O queuing delay

A. High request arrival rate

• too many I/O requests generated
• queue fills faster than device can process

B. Limited device throughput

• disks/SSDs have finite IOPS and bandwidth
• saturation leads to backlog

C. Deep queue depths

• storage controllers buffer many requests
• increases waiting time

D. Mixed workloads

• batch jobs flood I/O
• OLTP requests get delayed

E. Lock/log contention

• many transactions waiting on commit I/O

3. Direct impact on end-to-end latency

A. Latency amplification

Even small queue delays multiply:

• 1–2 ms service time
• 10–20 ms queue delay

👉 Queue dominates total latency.

B. Tail latency explosion (p99/p999)

• most requests fast
• some wait in long queues

👉 causes unpredictable spikes in latency

C. Throughput degradation

As queues grow:

• response time increases
• system spends more time waiting

👉 effective throughput drops

D. Backpressure effects

I/O delay propagates upward:

• threads wait for I/O
• CPU queues grow
• lock hold times increase

👉 system-wide slowdown

4. Queueing theory intuition

Using basic queueing model:

Utilization (ρ) = arrival rate / service rate

As ρ → 1:

• queue length grows rapidly
• waiting time increases exponentially

👉 small overload → huge latency spike

5. Example scenario

Low load:

• queue depth = 1–2
• latency = 2 ms

High load:

• queue depth = 20
• latency = 20–30 ms

👉 Same hardware, very different performance.

6. Interaction with other subsystems

A. CPU impact

• threads blocked on I/O
• increased dispatch latency

B. Memory impact

• buffer pools miss more
• more I/O requests generated

C. Locking impact

• transactions hold locks longer
• contention increases

7. Special case: database systems

In databases:

• commit latency depends on log write
• if log I/O is queued → transaction delay

👉 critical for financial systems

8. Mixed workload effect

Batch jobs:

• generate large sequential I/O

OLTP:

• needs small, fast I/O

👉 Batch floods queue → OLTP latency spikes

9. IBM system perspective

Enterprise systems (IBM Z / Power):

• use multiple I/O paths (channel subsystem)
• parallelize I/O across devices
• isolate workloads via logical partitioning
• prioritize critical I/O requests

👉 goal: minimize queuing delay for critical workloads

10. How to detect I/O queuing delay

Look for:

• high average vs service time gap
• increasing queue depth
• rising I/O wait time
• high p99 latency

11. Simple mental model

Think of I/O queuing delay as:

A checkout line at a supermarket—no matter how fast the cashier is, if too many people are waiting, your total time is dominated by how long you stand in line.

12. Key takeaway

I/O queuing delay affects end-to-end latency by:

• adding waiting time before actual I/O service
• dominating latency under high load
• causing tail latency spikes
• reducing throughput due to backlog
• creating cascading delays across CPU, memory, and locks

👉 In many systems, queue delay—not disk speed—is the real bottleneck