What is RoCE networking in Exadata?
If RDMA is the "super-highway" of Exadata, then RoCE (pronounced "rocky") is the high-performance asphalt it’s built on.
In older generations of Exadata, Oracle used a specialized networking technology called InfiniBand. While fast, it required unique cables and switches that didn't play well with standard data centers. Starting with the Exadata X8M, Oracle switched to RoCE (RDMA over Converged Ethernet).
RoCE brings the insane speed of InfiniBand to the universal language of Ethernet.
RoCE stands for RDMA over Converged Ethernet. It is a network protocol that allows Remote Direct Memory Access (RDMA) to run over standard Ethernet cables and switches.
In a standard network, data has to be broken into packets, wrapped in headers (TCP/IP), processed by the Operating System, and then unwrapped at the other end. RoCE skips all of that. It allows the Database Server to talk directly to the Storage Server's memory using the Ethernet wire as a direct straw.
Modern Exadata racks (like the X9M, X10M, and X11M) use a 100 Gbps RoCE Internal Fabric. To put that in perspective:
Standard Office Network: 1 Gbps
High-End Enterprise Network: 10 Gbps or 25 Gbps
Exadata RoCE: 100 Gbps
This isn't just about moving more data; it's about moving it with ultra-low latency (less than 19 microseconds).
The move to RoCE wasn't just for a speed bump; it was about modernizing the "Engineered System."
Universal Compatibility: RoCE uses standard Ethernet physical layers. This makes it easier to integrate Exadata into existing data centers and management tools.
Extreme Scale: RoCE is designed for the cloud. It allows Oracle to scale Exadata Cloud Service (ExaCS) and Autonomous Database to massive levels that InfiniBand struggled to reach.
Better Prioritization: RoCE uses "Priority Flow Control" (PFC). This ensures that critical database "heartbeat" signals aren't delayed by a large data backup happening at the same time.
RoCE is the secret ingredient behind Exadata RDMA Memory (XRMEM).
Because RoCE is so fast and bypasses the CPU (via RDMA), a database server can read a data block from a storage server's memory faster than a standard server can read a block from its own local SSD. This effectively turns the entire Exadata rack into one giant pool of shared memory.
| Feature | Standard Ethernet (TCP/IP) | Exadata RoCE |
| Speed | 10 - 25 Gbps (typically) | 100 Gbps |
| CPU Usage | High (OS must process packets) | Zero (Hardware-to-Hardware) |
| Latency | ~100+ Microseconds | < 19 Microseconds |
| Reliability | Software-based | Hardware-based (Lossless) |
RoCE is the "Nervous System" of the modern Exadata. It provides the massive bandwidth and microsecond latency required to handle the world's most demanding workloads—from high-frequency trading to massive AI data processing. By moving to an Ethernet-based standard, Oracle ensured that Exadata remains the fastest platform for the Oracle Database, whether on-premises or in the Cloud.