How does Exadata ensure consistent read across nodes?
In a distributed database system like Oracle RAC running on Exadata, "Consistent Read" (CR) is the mechanism that ensures every user sees a point-in-time, consistent view of the data, even if other nodes are simultaneously updating it.
On standard hardware, this can lead to a "ping-pong" effect where data blocks bounce between servers, creating massive latency. Exadata solves this using a combination of Global Cache Service, RDMA, and Smart Fusion.
In a single-instance database, if you start a long-running query at 10:00 AM, and someone updates a row at 10:05 AM, the database uses Undo data to "roll back" that block in memory so your query still sees the 10:00 AM version.
In an Exadata cluster, the challenge is multiplied:
Node 1 is running the query.
Node 2 has the most recent version of the block in its RAM.
The Storage Cell has the "Undo" records needed to reconstruct the old version.
Exadata uses the Global Cache Service to keep track of who owns what. Every block has a "Master Node." When Node 1 needs a consistent read version of a block:
It asks the Master Node who currently holds the block.
The Master Node points to Node 2.
Node 2 is then asked to provide a "CR version" of that block.
This is where the Exadata RoCE (RDMA over Converged Ethernet) fabric changes the game.
On traditional systems, Node 2 would have to package the block, send it through the OS kernel, and Node 1 would have to unpackage it. In Exadata, Node 1 uses RDMA to read the block directly from Node 2’s memory.
By bypassing the CPU and Kernel on both sides, the time it takes to "ship" a block for a consistent read drops from milliseconds to under 10 microseconds.
As we touched on earlier, Smart Fusion prevents the "Log Write Stall."
In a standard RAC environment, if Node 2 just updated a block, it cannot send it to Node 1 until the Redo Log write is confirmed by storage. This ensures that if Node 2 crashes, Node 1 isn't holding "dirty" data that isn't protected by a log.
Exadata handles this differently: It allows Node 2 to send the block to Node 1 simultaneously while the log write is happening. The storage cell then sends the "Commit Acknowledgment" to both nodes. This eliminates the wait time, ensuring that "Consistent Reads" aren't delayed by "Log Writes."
One of the most unique Exadata features is that Consistent Read logic is offloaded to storage.
When you run a Smart Scan, the Database Server sends the "Snapshot SCN" (System Change Number) to the storage cells. If the storage cell encounters a block that was changed after your query started, the storage cell:
Finds the Undo records.
Applies the Undo to the block locally on the storage server CPU.
Sends the consistent version of the row back to the database.
This prevents the need to ship blocks back and forth between RAC nodes just to satisfy a long-running report.
| Component | Function |
| Undo Segments | Provides the "old" versions of data. |
| RDMA | Moves blocks between nodes in microseconds, not milliseconds. |
| Smart Fusion | Allows block transfers to happen without waiting for log writes. |
| Storage Offload | Reconstructs old versions of data directly on the storage cell during scans. |
Because Exadata makes the cost of "sharing" data between nodes so incredibly low, you can scale your RAC cluster to 4, 8, or even 16 nodes without the "interconnect tax" that usually bogs down large clusters.