Cache affinity optimization is a performance technique where the system tries to keep a thread or workload running on the same CPU core (or nearby cores) so it can reuse data already loaded in the CPU cache, instead of fetching it again from slower memory.

In simple terms:

“Run work where its data already is.”

1. Why cache affinity matters

Modern CPUs have multiple cache levels:

• L1 (very fast, very small)
• L2 (fast)
• L3 (shared, larger)
• RAM (much slower)

If a thread moves between cores:

• Its cached data is lost (or not locally accessible)
• Data must be reloaded from L3 or RAM
• This increases latency and reduces performance

2. What cache affinity optimization does

The OS scheduler or hypervisor tries to:

• Keep a thread on the same core (strong affinity)
• Or at least within the same CPU socket / NUMA node (soft affinity)
• Avoid unnecessary migrations between cores

3. Types of cache affinity

(A) Temporal affinity

• A thread runs again on the same core it used before
• Its data is likely still in cache

👉 Improves performance for repeated tasks

(B) Spatial affinity

• Threads using related data are placed on nearby cores
• They can share higher-level cache (like L3)

(C) NUMA affinity (extended concept)

• Keep threads close to their memory node
• Reduces remote memory access

4. How systems implement it

(A) Scheduler decisions

Operating systems:

• Track which core last executed a thread
• Prefer scheduling it there again
• Avoid migrations unless necessary (load balancing)

(B) CPU affinity (manual pinning)

Admins or applications can:

• Bind processes to specific cores
• Prevent scheduler from moving them

(C) Hypervisor awareness (enterprise systems)

In systems like IBM PowerVM:

• Virtual CPUs are mapped to physical cores
• Hypervisor tries to preserve cache locality across scheduling cycles

5. Benefits of cache affinity optimization

(A) Lower latency

• Data already in cache → faster execution

(B) Reduced memory traffic

• Less need to fetch from RAM

(C) Better CPU efficiency

• Fewer cache misses
• Higher instruction throughput

6. Trade-offs and challenges

Cache affinity is not always perfect:

(A) Load balancing vs locality

• Keeping threads fixed improves cache use
• But may lead to uneven CPU utilization

(B) SMT contention

• Multiple threads on same core may compete for cache

(C) Migration cost

• Sometimes moving a thread improves overall system balance

So systems balance:

Cache locality vs fair CPU distribution

7. When it matters most

High impact workloads:

• Databases (OLTP, OLAP)
• In-memory applications
• High-frequency trading systems
• HPC workloads

Lower impact:

• Short-lived or stateless processes
• I/O-bound workloads

8. Relationship with NUMA locality

Cache affinity is a micro-level optimization, while NUMA locality is macro-level:

• Cache affinity → keep data in CPU caches
• NUMA locality → keep data in local memory node

Best performance happens when both align.

9. Example

Without cache affinity:

• Thread runs on Core 1 → loads data into cache
• Next run → scheduled on Core 5
• Cache miss → reload from memory

With cache affinity:

• Thread runs again on Core 1
• Data still in cache → faster execution

10. Simple analogy

Think of a worker and their desk:

• Cache = documents on desk
• RAM = archive room

With cache affinity:

• Worker returns to same desk → documents already there

Without it:

• Worker moves desks → must fetch documents again

Key takeaway

Cache affinity optimization improves performance by keeping workloads on the same CPU cores (or nearby ones), maximizing cache reuse and minimizing costly memory accesses.