What is RDMA over Converged Ethernet?
In the world of high-performance networking, the biggest speed bump isn't the cable—it’s the Operating System. Normally, when data travels from a storage array to a server, the CPU has to stop what it's doing, package the data, and move it through the "network stack."
RDMA over Converged Ethernet (RoCE) is a game-changing technology that allows one computer to reach into the memory of another computer and grab data directly, completely bypassing the CPU and the Operating System of both machines.
To understand RoCE, you have to look at how "Normal" networking (TCP/IP) works compared to RDMA:
Normal Networking (TCP/IP): Imagine a delivery truck that has to stop at every single post office in every town to have its paperwork checked. The driver (CPU) gets exhausted, and the trip takes forever.
RoCE (RDMA): Imagine a private, high-speed rail line where the train has a key to the warehouse. It pulls up, loads the cargo, and leaves. No paperwork, no stops, and the warehouse manager (CPU) doesn't even have to wake up.
By using Remote Direct Memory Access (RDMA) over standard Ethernet cables, we get "Zero-Copy" networking—data goes from Memory A to Memory B with almost zero latency.
The "CE" in RoCE stands for Converged Ethernet. In the past, RDMA required a very expensive and specialized type of cabling called InfiniBand.
RoCE brings that same "supercomputer" speed to the standard Ethernet switches and cables you already have in your data center. It "converges" your standard traffic and your high-speed database traffic onto the same 100GbE wires.
Oracle is the primary champion of RoCE, particularly in the Exadata platform. Here is why it's a requirement for modern databases:
Microsecond Latency: In an Oracle RAC (Real Application Clusters) environment, nodes need to talk to each other constantly. RoCE drops that communication time from milliseconds to microseconds.
CPU Offloading: Because the network card (NIC) handles the data transfer, the server’s CPU is free to focus 100% on running SQL queries rather than moving network packets.
Exafusion: This is Oracle’s special software that talks directly to the RoCE hardware. It allows the database to "pull" a block of data from a remote node's cache faster than it could read it from its own local disk.
| Feature | Traditional TCP/IP | RoCE (RDMA) |
| CPU Usage | High (Interrupt-driven) | Near Zero (Kernel Bypass) |
| Latency | 50–100 Microseconds | < 5 Microseconds |
| Data Movement | Multiple copies in RAM | Direct (Zero-Copy) |
| Reliability | Software-managed | Hardware-managed (Lossless) |
You can't run RoCE on just any old network card. You need a smartNIC (specifically one that supports RoCE v2). These cards have their own onboard processors that handle the "Flow Control" (Priority Flow Control) required to make sure no packets are dropped—because RDMA assumes a "lossless" network.
RoCE is the technology that finally allowed Ethernet to catch up to the speed of the processor. By removing the "middleman" (the CPU and OS) from network transfers, it enables the lightning-fast cluster communication that powers the world's largest Oracle Databases and AI training clusters.