Workload placement directly determines how well a system takes advantage of NUMA (Non-Uniform Memory Access) locality—that is, whether a thread runs close to the memory it uses or has to fetch it from a remote node at higher latency.

1. Quick refresher: what NUMA locality means

In a NUMA system:

• Each CPU socket/node has its own local memory
• Accessing:
  • Local memory → fast
  • Remote memory (other node) → slower + higher latency + interconnect traffic

So performance depends on:

Keeping compute (CPU) and data (memory) on the same NUMA node

2. What is workload placement?

Workload placement means:

• Which CPU cores a process/thread runs on
• Which NUMA node its memory is allocated from

This is controlled by:

• OS scheduler
• Hypervisor (e.g., PowerVM, VMware)
• Application affinity settings

3. How placement affects NUMA locality

(A) Good placement (NUMA-aware)

• Threads run on cores within one NUMA node
• Memory is allocated from the same node

👉 Result:

• Low latency
• High cache efficiency
• Minimal interconnect traffic

(B) Poor placement (NUMA-unaware)

• Threads run on one node
• Memory resides on another node

👉 Result:

• Remote memory access
• Higher latency (can be 1.5×–3× slower)
• Increased interconnect congestion

(C) Scattered placement

• Threads spread across nodes
• Memory allocated unevenly

👉 Result:

• Cache fragmentation
• NUMA imbalance
• Unpredictable performance

4. Why placement matters more for some workloads

(A) Memory-intensive workloads (very sensitive)

• Databases (OLTP/OLAP)
• In-memory analytics
• Caching systems

👉 Poor placement = major performance drop

(B) CPU-bound workloads (moderately sensitive)

• Heavy computation
• Less dependent on memory locality

👉 Still impacted by cache and bandwidth contention

(C) Distributed workloads (less sensitive)

• Microservices
• Stateless workloads

👉 NUMA effects are smaller but still present

5. Key mechanisms that influence placement

(A) CPU affinity (pinning)

• Bind threads to specific cores or NUMA nodes
• Prevents scheduler from moving them

👉 Improves locality consistency

(B) Memory allocation policy

• First-touch policy (common in Linux)
  • Memory allocated on node where thread first accesses it

👉 Placement at startup is critical

(C) Hypervisor placement (virtualized systems)

In systems like IBM PowerVM:

• LPARs can span multiple NUMA nodes
• Virtual CPUs may map to different physical sockets

👉 Poor VM placement → cross-node penalties

(D) Automatic NUMA balancing

OS may:

• Migrate memory pages
• Move threads to match memory location

👉 Helps, but adds overhead and is not always optimal

6. Typical placement strategies

(A) Compact placement (best for locality)

• Pack workload into one NUMA node
• Use local memory only

👉 Best for:

• Latency-sensitive apps
• Databases

(B) Spread placement (best for bandwidth)

• Distribute workload across nodes

👉 Best for:

• Large parallel workloads
• High memory bandwidth needs

(C) Hybrid placement

• Core workload stays local
• Less critical threads spread out

7. Example scenario

Bad placement:

• Thread runs on Node 0
• Data is on Node 1

Every memory access:

• Goes through interconnect (e.g., Infinity Fabric / UPI)
• Adds latency + reduces throughput

Good placement:

• Thread and memory both on Node 0
• Fast access
• Better cache reuse
• Lower contention

8. NUMA effects in enterprise systems (IBM context)

In IBM Power systems:

• Large multi-socket NUMA architecture
• High SMT levels amplify contention
• PowerVM placement decisions impact locality significantly

Best practice:

• Align LPAR CPU and memory allocation within same NUMA domain when possible

9. Simple analogy

Think of NUMA like offices in different buildings:

• CPU = worker
• Memory = files

Good placement:

• Worker and files in same building → fast

Bad placement:

• Worker constantly travels to another building → slow and inefficient

10. Key takeaway

Workload placement determines NUMA locality by controlling where threads execute relative to where their memory resides; good placement keeps compute and data on the same NUMA node, while poor placement causes remote memory access, higher latency, and reduced performance.