RC RANDOM CHAOS

Topology-Aware Scheduling for NVL72 Cluster Efficiency

· via Hacker News

Original source

Grace Blackwell needs a new scheduler: 27% more usable GPUs from the same racks

Hacker News →

NVL72 systems connect all 72 GPUs in a rack through unified NVLink fabric, enabling high-speed communication between nodes that would otherwise be bottlenecked by slower interconnects. Switching from NVLink to InfiniBand for distributed training causes a 43% throughput loss. The rack’s non-blocking fabric topology—built from 18 NVSwitch chips each handling 72 NVLink lines—delivers 130 TB/s of aggregate bandwidth, making it possible to keep critical communication patterns on the fastest path.

Large-scale AI workloads rely on parallelism strategies that impose communication constraints: tensor parallelism requires all-reduce synchronization after each layer, while expert parallelism needs all-to-all token routing between GPUs holding different experts. These operations sit on the critical execution path and resist computation-communication overlap, making network bandwidth and latency paramount. With 64-way expert parallelism on conventional hardware, unoptimized inter-GPU traffic can consume up to 60% of training time. NVL72’s high-speed fabric effectively eliminates this bottleneck.

The challenge is operational: a full 72-GPU rack fails roughly every 27 days (mean time between interruptions). Allocating entire racks to individual jobs means any single component failure idles the entire system until repair. Platforms like SkyPilot improve effective cluster capacity by reserving spare nodes per rack—trading peak utilization for resilience. Topology-aware scheduling keeps distributed jobs rack-local while enabling fast in-place recovery from faults, ultimately delivering more usable GPUs per unit of infrastructure than full-rack allocation schemes.

Read the full article

Continue reading at Hacker News →

This is an AI-generated summary. Read the original for the full story.