San Francisco – April 1, 2025– As AI clusters balloon toward the million-GPU mark, the optical and electrical interconnect industry faces immense pressure to keep up with surging bandwidth, power, and reliability demands. At a standing-room-only OFC 2025 Executive Forum panel titled “AI’s Optical Bottleneck: Scaling Networks for the Next Generation of AI Workloads,” industry leaders from NVIDIA, Credo, Marvell, TE Connectivity, and the Optical Internetworking Forum (OIF) broke down the challenges—and innovations—shaping tomorrow’s AI infrastructure.
Moderated by Alan Weckel of the 650 Group, the conversation centered on network architectures, co-packaged optics (CPO), pluggables, copper’s surprising resurgence, and the need for tighter industry collaboration. While optics continues to be essential for scale, new thermal and spatial dynamics are bringing copper back into play for short-reach links.
🔍 Key Takeaways from the Panel Discussion
🔗 AI Is Reshaping Data Center Network Topology
• AI clusters require four distinct network planes:
• Scale-up (intra-rack) – copper-dense connections within nodes.
• Scale-out (east-west) – high-bandwidth inter-node fabrics.
• Front-end (north-south) – user and storage traffic.
• DCI (inter-site) – multi-building and multi-region links.
⚡ Copper’s Resurgence in Short-Reach Interconnects
• Don Barnetson (Credo): AI density and liquid cooling now enable short copper links even in scale-out fabrics.
• “For the first time, we’re converting optics back into copper,” Barnetson said.
• Credo’s AECs reduce soft link errors and offer “zero-flap” operation, improving reliability.
🧠 Optics Still Critical for Scaling AI
• Craig Thompson (NVIDIA): CPO is key for east-west fabric efficiency, reducing power and increasing cluster uptime.
• “We’re simplifying switch-to-optic interfaces and reducing component count with CPO,” Thompson said.
• Front-pluggable optics still have value and will coexist with CPO.
🌐 Data Center Interconnect (DCI) Undergoes 100X Growth
• Josef Berger (Marvell): DCI bandwidth demand has increased 100-fold due to:
1. Growing number of AI training clusters.
2. Need to monetize via inference workloads.
3. Power delivery constraints requiring multi-site clustering.
• Marvell emphasized the importance of ZR optics and silicon photonics.
🧵 Massive Volumes of Interconnect Cable
• Nathan Tracy (TE/OIF):
• Some AI deployments use 60,000–100,000 km of Twinax per year.
• That’s up to 1.5× around the globe—just in backplane cable.
• “Next-gen is 400G electrical—get ready for a wild ride.”
🛠️ Standards and Collaboration
🧩 OIF’s Role in Powering the Future
• Tracy described OIF as the “sports car” standards body, laser-focused on hyperscale needs.
• Key initiatives include:
• 400G electrical (single-lane 400G, enabling 3.2T–6.4T modules).
• CMIS, the protocol “glue” for all interconnects.
• Advancements in CPO, LPO, and coherent optics.
🤝 Cross-Industry Co-Design Is Essential
• Panelists stressed the importance of early engagement:
• NVIDIA provides reference designs; hyperscalers modify them.
• Credo and Marvell work closely with customers and silicon partners to anticipate needs.
• TE adjusts manufacturing and interconnect tech preemptively to meet new form factors and thermal specs.
📈 Speed of Innovation: Faster than Ever
• Development must target N+2 generations in advance.
• Craig Thompson: “GPU bandwidth doubles every two years, and so must the network.”
• Don Barnetson: Long silicon lead times mean design teams must predict customer needs before they even know them.
• Josef Berger: Innovation pace is so fast that new players may struggle to break in.
🔮 Final Thoughts from the Panelists
• Craig Thompson (NVIDIA):
“The network defines how fast we can build computers. We’re headed for million-GPU clusters.”
• Don Barnetson (Credo):
“Copper’s not dead—it’s thriving again. And diversity in interconnects is here to stay.”
• Nathan Tracy (TE/OIF):
“Interconnect is now a first-class citizen in AI infrastructure. It’s our day.”
• Josef Berger (Marvell):
“We’re hitting <5 pJ/bit in short-reach optics. Innovation isn’t slowing down—it’s accelerating.”
