Hardware key management in IBM Z (IBM Z mainframe systems) is built around a strict hardware-rooted trust model, where cryptographic keys are never exposed in plaintext outside secure hardware boundaries. It combines specialized cryptographic hardware, hierarchical key encryption, and system software control.

Here’s how it works end-to-end.

🔐 1. Root of Trust: Master Key in Hardware

At the center of IBM Z key management is the Master Key, stored inside secure cryptographic hardware:

• Stored in:
  • Crypto Express adapters (CEX HSMs)
  • On-chip cryptographic coprocessors (CPACF for some functions)

👉 The Master Key is:

• Never visible in plaintext to OS or users
• Entered only through secure procedures (manual or secure load)
• Protected inside tamper-resistant hardware

This creates a hardware root of trust.

🧩 2. Key Hierarchy (Core Concept)

IBM Z uses a layered key structure:

🔑 Master Key (top level)

• Resides in Crypto Express hardware
• Used only to encrypt other keys

↓

🔐 Key Encryption Keys (KEKs)

• Derived/enciphered under the Master Key
• Used to protect operational keys

↓

🔏 Data Keys (working keys)

• Used for actual encryption of data (DBs, files, TLS, etc.)
• Always stored in encrypted (wrapped) form

👉 Important rule:

Keys are always stored “wrapped” (encrypted), never plain.

🏗️ 3. How Key Storage Works (Step-by-Step)

Step 1: Key generation

A key is generated inside:

• Crypto Express HSM or
• ICSF (Integrated Cryptographic Service Facility) using hardware support

Step 2: Key wrapping

• The key is immediately encrypted using a KEK or Master Key
• Result = secure key token

Step 3: Storage

• Wrapped key is stored in:
  • z/OS datasets
  • Databases
  • Security repositories (RACF profiles)

Step 4: Usage

When needed:

1. System sends encrypted key to HSM
2. HSM decrypts it internally
3. Performs crypto operation inside hardware
4. Plain key never leaves hardware boundary

🧠 4. Role of ICSF (Key Management Controller)

On z/OS ICSF, key management is controlled by:

• Key generation
• Key wrapping/unwrapping
• Key lifecycle (rotation, deletion)
• Secure APIs for applications

ICSF acts as the software interface to Crypto Express hardware.

🛡️ 5. Crypto Express (Hardware Security Boundary)

Crypto Express adapters enforce:

• Secure key storage in HSM memory
• Master Key isolation
• Cryptographic domain separation
• Secure execution of cryptographic instructions

They support modes like:

• CCA (banking / enterprise secure key crypto)
• EP11 (PKCS#11 HSM standard)

🔐 6. Key Protection via RACF Integration

Security policy is enforced by:

z/OS RACF

RACF controls:

• Which user or LPAR can use a key
• Key access permissions
• Cryptographic role-based security

👉 Even if a key exists, it cannot be used without RACF authorization.

🧩 7. Secure Key States in IBM Z

Keys typically exist in 3 states:

⚙️ 8. Multi-tenant isolation (LPAR level)

IBM Z supports multiple workloads using the same hardware via:

• LPARs (Logical Partitions)
• Crypto domains in Crypto Express

Each domain:

• Has its own Master Key
• Cannot access other domains’ keys

🚀 9. Real-world usage flow (TLS example)

When a secure connection happens:

1. TLS handshake starts
2. Private key is stored as a wrapped key
3. HSM unwraps key internally
4. Cryptographic signing happens inside hardware
5. Result returned, key never exposed

🧾 Simple Summary

Hardware key management in IBM Z works like this:

• 🔐 Master Key is stored inside Crypto Express HSM
• 🧩 All keys are encrypted under this Master Key
• ⚙️ ICSF manages lifecycle and APIs
• 🛡️ RACF controls who can use keys
• 🔒 Keys are always wrapped outside hardware
• 🚫 Plain keys never leave secure hardware