What is hardware–ASM interaction during rebalance storm?
In 2026, we no longer view this as a software "copy" operation. It is a high-intensity Hardware–ASM Interaction that pits the background rebalance utility against the foreground transaction engine for control of the PCIe bus and the CPU's memory controller.
When a rebalance starts, ASM spawns ARB (ASM Rebalance) processes. In a "Storm," these processes attempt to saturate the available hardware bandwidth to restore data redundancy as quickly as possible.
The Transaction Path: CPU $\rightarrow$ NVMe (Small, random 8KB reads/writes).
The Rebalance Path: NVMe $\rightarrow$ RAM $\rightarrow$ NVMe (Large, sequential 1MB "Allocation Unit" moves).
The Hardware Interaction: If not managed, the large sequential rebalance reads create Head-of-Line Blocking on the PCIe lanes, causing the 8KB transaction writes to spike from $100\mu s$ to $2ms$.
ASM_POWER_LIMITThe ASM_POWER_LIMIT parameter isn't just a "speed" setting; it is a throttle for Hardware Resource Consumption.
Low Power (1–11): ASM uses fewer parallel ARB processes, leaving the majority of CPU Execution Ports and Memory Channels free for 1M TPS transactions.
High Power (100+): ASM attempts to hit "Line Rate" on the storage fabric. On an Exadata or high-end NVMe-oF system, this triggers Hardware Backpressure (as discussed earlier), as the rebalance traffic starts to fill the NIC and Switch buffers.
During a rebalance storm caused by a disk failure, the hardware interaction begins at the VMD (Volume Management Device) or NVMe controller level.
Hardware Isolation: The PCIe controller detects a "Surprise Removal" or "Link Failure."
ASM Acknowledgement: The hardware sends an interrupt to the Kernel, which ASM translates into a "Disk Offline" state.
The Storm Begins: ASM immediately begins reading the "Mirrors" of the lost data.
CPU Cache Impact: Because rebalance involves moving massive amounts of data, it can cause L3 Cache Pollution. The CPU's "Prefetcher" starts loading rebalance blocks into the cache, evicting the "Hot" SQL data needed for the 1M TPS workload.
In modern 1M TPS stacks (like Exadata X11M), the hardware is "ASM-Aware" to prevent the storm from sinking the database.
| Hardware Feature | Interaction During Rebalance | 1M TPS Benefit |
| I/O Prioritization | Hardware tags "Rebalance" I/O as Low Priority. | Transactions stay at sub-millisecond latency. |
| RDMA Rebalance | ARB processes move data via Remote Direct Memory Access. | Bypasses the CPU/Kernel on the destination node. |
| Flash Log Protection | Redo writes are protected from Rebalance interference. | Prevents "Log File Sync" spikes during the storm. |
| Predictive Failure | Hardware warns ASM before the disk dies. | Rebalance starts gracefully before a "Storm" hits. |
To survive an ASM rebalance storm at scale, the interaction must be tuned:
Limit the "Blast Radius": Use Flex Disk Groups to ensure a failure in one area doesn't trigger a global rebalance storm.
Monitor "Cell Serv" Latency: Watch for the cell rbal wait events. If rebalance latency is low but transaction latency is high, your hardware isn't correctly prioritizing the transaction path.
Use 2026 "Smart" Rebalance: Ensure your firmware supports NVMe-oF (NVMe over Fabrics), which allows the rebalance to happen "Peer-to-Peer" between storage nodes, completely bypassing the Database Server's CPU.
A Rebalance Storm is the ultimate "Stress Test" for a 1M TPS architecture. By understanding the Hardware–ASM Interaction, you move from a reactive state (hoping the system doesn't crash) to a proactive one (using silicon-level priorities to ensure that while the "house is being repaired," the "business stays open").