In IBM Parallel Sysplex, cache coherency across nodes is maintained primarily through the Coupling Facility (CF) cache structures, not by traditional hardware cache-coherence protocols (like MESI across CPUs). Instead, it uses a software-hardware hybrid coherence model designed for shared-disk data sharing.

The goal is:

Ensure all sysplex members see a consistent view of shared data blocks even when multiple systems cache them independently.

1. Core idea: directory-based coherence via Coupling Facility

Sysplex does not directly share CPU caches across nodes.

Instead, it uses a directory-based coherence model:

• CF maintains a directory of cached data blocks
• Each block tracks:
  • Which LPARs have a copy
  • Who owns the “current” version
  • Whether copies are valid or stale

👉 CF acts as the central cache authority

2. Main component: CF Cache Structure

A cache structure in CF contains:

• Data blocks (or pages)
• Directory entries (metadata)
• Ownership and state information

Each cached object has states like:

• Owner (dirty/modified copy holder)
• Shared copies (read-only copies in other LPARs)
• Invalidated copies

3. How coherency works (step-by-step)

Case 1: Read request (cache hit or miss)

Step 1: LPAR requests data block

• Request sent to CF cache structure

Step 2: CF checks directory

• If another LPAR has latest version → CF coordinates sharing
• If not present → fetch from DASD (disk)

Step 3: Data returned

• LPAR gets data block
• CF records that LPAR now holds a cached copy

Case 2: Write request (critical path)

Step 1: LPAR requests exclusive update

• CF receives “modify intent”

Step 2: CF invalidates other copies

• All other LPARs holding that block are notified
• Their cached copies become invalid

Step 3: Exclusive ownership granted

• One LPAR becomes the owner (dirty state)

Step 4: Updates occur locally

• Changes made in local cache

Step 5: Write-back coordination

• Eventually flushed back to DASD or shared state

4. Key mechanism: invalidation-based coherence

Instead of constantly syncing data:

• CF uses invalidate-on-write model
• When one system updates:
  • Others are told “your copy is stale”
  • They must refetch if needed

👉 This avoids constant data broadcasting

5. Ownership model (very important)

Each data block in CF cache has:

A. Owner

• Single LPAR allowed to modify data

B. Sharers

• Multiple LPARs can read cached copy

C. Invalid state

• Copy exists but is no longer valid

CF enforces transitions between these states.

6. Role of directory in CF

The CF maintains a directory entry per cached object, including:

• Block identifier
• List of sharers
• Owner identity
• Dirty/clean status
• Sequence/version metadata

👉 This directory is what makes coherence scalable across systems.

7. Write serialization across sysplex

When multiple systems try to update same data:

• CF arbitrates using:
  • Lock structures (for mutual exclusion)
  • Cache structures (for ownership tracking)

👉 Locks ensure who can attempt, cache structure ensures who owns data

8. Interaction with lock structures

Cache coherency is not standalone:

👉 Together they ensure:

• No conflicting updates
• No stale reads after invalidation
• Deterministic ordering of updates

9. Performance optimization techniques

A. Local caching in LPARs

Each system keeps its own buffer cache for speed.

B. CF reduces cross-system traffic

Only metadata + invalidations are exchanged frequently.

C. Write affinity

Some workloads are biased to reduce ownership transfers.

D. Bulk invalidation efficiency

CF can invalidate multiple sharers in one operation.

10. Why this model scales

Unlike distributed cache coherence (like in SMP systems):

• No direct CPU-to-CPU cache snooping
• No broadcast storms
• No full mesh coherence traffic

Instead:

A centralized directory (CF) orchestrates coherence with minimal messaging overhead.

11. Simple mental model

Think of CF cache coherency as:

A central “truth directory” that tracks who owns or shares each data block, and enforces invalidation when updates happen.

Key takeaway

Cache coherency in Parallel Sysplex works by:

• Using CF cache structures as a global directory-based coherence manager
• Tracking ownership and sharing of data blocks across LPARs
• Using invalidation-based updates instead of continuous synchronization
• Coordinating with lock structures to ensure safe updates