Gilad Shainer, SVP of Networking at NVIDIA, breaks down the complex infrastructure architecture required to build modern AI factories. Unlike traditional data centers, AI workloads demand distributed computing across thousands—even hundreds of thousands—of GPUs working as a single unit. Shainer explains NVIDIA’s multi-layered approach to connecting these massive compute resources, from rack-scale GPU clusters to million-scale AI factories spanning multiple locations.
In this video, Shainer reveals how NVIDIA addresses the unique challenges of AI infrastructure through four distinct networking layers: scale-up, scale-out, scale-across, and purpose-built storage. He discusses the critical role of NVLink in creating virtual rack-scale GPUs, the development of Spectrum-X Ethernet specifically for AI workloads, and why co-packaged optics represents a breakthrough in power efficiency. Shainer also explores emerging technologies including liquid cooling systems, confidential computing security, and NVLink Fusion for custom processors. Learn how NVIDIA’s annual platform roadmap continues to push the boundaries of what’s possible in AI infrastructure design.
📚 CHAPTERS:
0:00:00 – Introduction to AI Factory Infrastructure Design
0:00:50 – Scale-Up Architecture: Building Rack-Scale GPUs with NVLink
0:02:34 – Scale-Out Infrastructure: Connecting AI Factories at Massive Scale
0:04:36 – Co-Packaged Optics: Reducing Power Consumption in Optical Connections
0:07:32 – Scale-Across and Gigascale AI Factories
0:09:02 – Purpose-Built Storage Infrastructure for AI Inferencing
0:10:41 – Future Innovations: Liquid Cooling, Security, and NVLink Fusion
0:13:36 – Annual Platform Roadmap and Future Outlook
Want to be involved our video series? Contact [email protected]
https://ngi.fyi/dcnet26-nvidia-gilad
