What is impact of CPU topology awareness in Oracle scheduler?
Modern CPUs are no longer uniform blocks of compute; they are complex hierarchies of cores, clusters, and L3 cache slices. CPU Topology Awareness in the Oracle Scheduler ensures that the database doesn't just treat "CPU 0" and "CPU 127" as identical slots, but understands the physical distance and shared resources between them.
Most modern CPUs (like AMD EPYC or Intel Xeon) group cores into CCX (Core Complexes) or Tiles. Cores within the same complex share a massive L3 cache.
Topology-Agnostic Scheduling: The OS might move an Oracle "Foreground" process (the one handling your SQL) to Core 1, while its "Helper" process (like a Parallel Query slave) stays on Core 64.
The Penalty: If these two need to share data, they must exit the L3 cache and travel across the "Infinity Fabric" or "Mesh," adding 50–100ns of latency per request.
Topology-Aware Scheduling: Oracle's internal scheduler (and the underlying OS sched_domain) attempts to keep related processes in the same Cache Neighborhood. This keeps the "working set" of data in the local L3 cache, where access is near-instant.
At 1M TPS, a process migration is a micro-disaster. When the scheduler moves a process from one core to another:
The L1/L2 Flush: The "warm" data in the old core's L1/L2 cache is lost.
The Pipeline Stall: The CPU must rebuild its branch prediction history.
The Result: A transaction that usually takes $10\mu s$ suddenly takes $100\mu s$.
Oracle’s scheduler uses CPU Affinity hints to "stick" high-frequency processes (like the Log Writer (LGWR) or Database Writer (DBWR)) to specific cores. By being topology-aware, it ensures these critical processes don't compete for the same ALU (Arithmetic Logic Unit) or shared execution resources on SMT (Hyper-threading) cores.
In a 1M TPS workload, Hyper-threading (SMT) can be a trap. SMT allows two threads to share a single physical core.
The Conflict: If the Oracle Scheduler places two CPU-intensive "Parallel Query" slaves on the two "sibling" threads of the same physical core, they will fight for the same floating-point units.
The Optimization: A topology-aware scheduler knows which logical CPUs are siblings. It will "fill up" physical cores first (Core 0, Core 2, Core 4) before using the SMT siblings (Core 1, Core 3, Core 5). This ensures each database process gets the full execution bandwidth of a physical core.
| Feature | Topology-Blind | Topology-Aware |
| P99 Tail Latency | High (caused by cache misses) | Ultra-Low / Consistent |
| Inter-Process Latency | Variable (Interconnect speed) | Fixed (L3 Cache speed) |
| Context Switch Cost | High (Cold cache) | Low (Warm cache) |
| Throughput Scaling | Sub-linear (Saturation) | Linear (to core count) |
To maximize Oracle performance on high-core-count servers:
Use _enable_numa_support: Ensure Oracle understands the memory-to-CPU mapping.
Verify lscpu -e: Manually inspect the core-to-socket-to-node mapping to ensure your OS isn't "hallucinating" the topology.
Differentiate LGWR/LMS: In RAC environments, the LMS (Global Cache) processes are extremely sensitive to topology. Pinning them to cores that have direct, "short-path" access to the RoCE NIC can reduce interconnect latency by $15\%$.
In the world of 1M TPS, Silicon Geography is Destiny. If the Oracle scheduler is blind to the CPU topology, it will inadvertently scatter your data across the chip, forcing your transactions to wait on the "interconnect bus." By ensuring topology awareness, you transform your CPU from a collection of isolated cores into a high-speed, cache-coherent engine.