NVL72’s unified NVLink fabric keeps all 72 GPUs in a rack on a single high‑bandwidth network, enabling the scheduler’s 27% efficiency gain.
GRACE BLACKWELL'S NEW SCHEDULER YIELDS 27% MORE GPU USAGE
*Grace Blackwell’s SkyPilot platform forces AI jobs to stay rack‑local, unlocking a 27% boost in usable GPUs. The shift hinges on NVIDIA’s NVL72 fabric, which keeps critical communication on NVLink instead of InfiniBand.*
Grace Blackwell faces a familiar bottleneck: AI clusters waste precious GPU cycles because jobs drift off the high‑speed NVLink fabric onto slower InfiniBand links. The problem worsens as hardware failures and staggered job completions scatter free capacity across racks, breaking the tight communication loops that tensor and expert parallelism depend on. SkyPilot’s new scheduler forces every multi‑node job to stay rack‑local, preserving NVLink’s order‑of‑magnitude bandwidth advantage. The result is a 27% increase in usable GPUs without adding a single H100 to the rack.
The breakthrough rests on NVIDIA’s NVL72 architecture, which unifies all 72 GPUs in a rack under a single NVLink domain. This eliminates the historic limitation that confined NVLink to intra‑node traffic and forced cross‑node communication onto InfiniBand or cloud equivalents. By coupling that hardware capability with intelligent, topology‑aware scheduling, Blackwell’s team turns a theoretical bandwidth boost into a measurable productivity gain.
The rack‑locality challenge
AI workloads span thousands of GPUs. When a node fails or a job finishes early, free capacity scatters across racks, breaking the tight NVLink loops that power tensor and expert parallelism. Multi‑node jobs that drift onto InfiniBand suffer a 43% drop in training throughput, according to Hugging Face measurements. The result is under‑utilized hardware and longer time‑to‑model. Preserving rack‑locality is essential, but hardware churn makes static placement impossible. A dynamic scheduler that can react to failures and re‑pack jobs without moving them off the NVLink fabric is the missing piece.
Can NVLink really beat InfiniBand by 43% in real workloads?
NVIDIA’s NVL72 systems connect all 72 GPUs in a rack with a unified NVLink fabric, delivering an order of magnitude more bandwidth than traditional InfiniBand. This eliminates the cross‑rack bottleneck that forces all‑reduce and all‑to‑all steps onto slower links. In expert parallelism, a 64‑way MoE layer normally pushes traffic through InfiniBand, where unoptimized communication can consume up to 60% of training time. On NVL72, the same 64 GPUs stay inside a single NVLink domain, effectively erasing that bottleneck. The performance gap is measurable: moving a tensor‑parallel job off NVLink onto InfiniBand cuts throughput by 43%.
Why the scheduler matters: 27% more usable GPUs
SkyPilot’s platform adds topology‑aware placement, in‑place recovery from hardware failures, and automatic defragmentation of shared clusters. By keeping jobs rack‑local, it prevents the spillover onto InfiniBand that would otherwise throttle communication. The scheduler continuously reshuffles slices as nodes go down or jobs complete, ensuring that each GPU remains in the high‑bandwidth NVLink loop. The net effect, reported by Grace Blackwell, is a 27% increase in usable GPUs from the same rack footprint. No extra hardware is added; the gain comes purely from smarter orchestration.
What does this mean for AI training and inference?
Training pipelines that rely on tensor parallelism and expert parallelism now run on a fabric where critical all‑reduce and all‑to‑all steps stay on NVLink. The 43% throughput loss observed on InfiniBand disappears, and the 60% time penalty for MoE all‑to‑all traffic is eliminated. Inference benefits too: splitting a model across two GPUs freed KV‑cache memory by 13.9× in a vLLM benchmark, delivering 3.9× more throughput. The combined effect is faster model iteration, higher request rates, and lower cost per training run—all without expanding rack space.
Frequently asked questions
How much faster is training on NVLink compared to InfiniBand?
Hugging Face measured a 43% drop in training throughput when moving a tensor‑parallel job off NVLink onto InfiniBand.
What percentage of training time can all‑to‑all traffic consume on InfiniBand?
Unoptimized all‑to‑all communication on InfiniBand can consume up to 60% of training time for MoE models.
How much more usable GPU capacity does SkyPilot’s scheduler provide?
The scheduler yields a 27% increase in usable GPUs from the same rack configuration.
The data is clear: keeping AI workloads on NVLink, not InfiniBand, is no longer optional—it’s a competitive imperative. SkyPilot’s scheduler proves that software can extract the full value of NVL72’s hardware, delivering a 27% uplift in GPU utilization and erasing the communication penalties that have plagued large‑scale training. As AI models grow, the pressure to squeeze every ounce of performance from existing racks will only intensify, making intelligent scheduling the next front‑line weapon in the race for faster, cheaper AI.
Sources: [Grace Blackwell needs a new scheduler: 27% more usable GPUs from the same racks, https://skypilot.ai/blog/grace-blackwell-needs-a-new-scheduler]