Richard Ho, Head of Hardware at OpenAI, opened his keynote at the Optica Executive Forum in Los Angeles with a clear message: the future of AI will be determined not only by more powerful chips, but by how well the industry co-designs compute, memory, networking, and system architecture to support rapidly evolving AI workloads.
Speaking to an audience of photonics and networking experts gathered alongside OFC 2026, Ho argued that AI infrastructure is undergoing a fundamental architectural shift. Frontier AI systems are increasingly behaving like massive distributed computers, where the performance limits are no longer defined solely by processors but by the interaction between compute, memory bandwidth, power efficiency, and interconnects. In that environment, optical technologies are moving steadily toward the center of the AI stack.
Ho described the extraordinary pace of AI capability growth since the introduction of ChatGPT just three years ago. OpenAI’s model development has moved rapidly from improved foundational models to systems capable of longer context windows, advanced coding capabilities, reasoning with long chains of thought, and the rise of agent-based systems that interact with external tools. Coding agents such as Codex, he said, are already changing how engineers work, even inside hardware teams.
The scale of demand behind these systems is rising just as quickly. Ho noted that ChatGPT now serves more than 900 million weekly users, while enterprise usage continues to accelerate. Enterprise message traffic has grown eightfold, and reasoning tokens — the tokens used when models perform multi-step reasoning — have increased more than 320 times. These trends are placing enormous pressure on the underlying infrastructure needed to train and run models.
AI Workloads Are Reshaping Infrastructure
Ho explained that the infrastructure challenge is no longer limited to training large models. New capabilities such as longer context, reasoning, tool use, and persistent sessions are increasing the computational workload during inference as well.
Longer context windows — now reaching 128K, 512K, and even one million tokens — dramatically increase the size of the KV cache memory required by models. Reasoning models generate additional inference steps as they explore different solution paths before producing an answer. Agents that call tools and external systems introduce additional network and storage activity. And session-based workflows, particularly coding sessions, keep clusters busy over extended periods.
These trends are forcing AI infrastructure to scale to unprecedented levels. As clusters grow, reliability becomes a significant concern. Even if individual components fail infrequently, Ho noted, the sheer number of components in modern AI clusters makes failures inevitable.
He said four factors now determine how far AI systems can scale: raw compute capability, memory bandwidth, power and cooling limits, and networking efficiency. While chip vendors often highlight peak FLOPS performance, real-world system efficiency depends heavily on memory access and data movement across networks.
Copper, Optics, and the Interconnect Transition
A large portion of Ho’s talk focused on the evolving role of interconnect technologies. Inside the rack, he said, copper connections remain extremely attractive because they are simple, inexpensive, and reliable. The industry is currently moving through the transition to 224-gigabit-per-lane signaling, with 448-gigabit lanes under active development.
But electrical signaling faces increasing limitations in reach and signal integrity as speeds climb. As a result, optics will play an expanding role in AI systems, particularly beyond the rack. Ho outlined a gradual evolution from today’s pluggable optical modules toward near-package optics and eventually co-packaged optics integrated closely with compute silicon.
Each approach has advantages and trade-offs. Pluggables are operationally simple but limited in density. Near-package optics promise improved bandwidth density while maintaining some serviceability. Co-packaged optics could offer the highest density and energy efficiency but introduce significant operational challenges.
Ho emphasized that OpenAI evaluates these technologies not just on theoretical performance but also on practical data center operations. Serviceability, reliability, manufacturability, and the ability to replace failed components quickly are critical factors in real-world deployments.
Energy efficiency is another major constraint. Ho suggested that current fast-and-narrow optical approaches may reach a wall at roughly five picojoules per bit at 400-gigabit lanes. Future AI systems will need significantly lower energy per bit, ideally below two picojoules per bit including I/O, to sustain infrastructure growth.
Bandwidth density is also becoming a critical metric. Ho said OpenAI would like to see a doubling of bandwidth density every two to three years along the “shoreline” of an accelerator package — the edge where compute devices connect to the network — in order to keep pace with the growth of compute and memory performance.
Building an Ecosystem for AI Interconnects
Ho also highlighted the importance of industry collaboration. He referenced a new multi-source agreement announced recently with OpenAI as a founding participant, aimed at defining a standardized optical physical layer for AI infrastructure. The effort is intended to encourage interoperability across vendors and build a broader ecosystem capable of supporting the enormous scale of future AI systems.
He concluded by reiterating that the AI industry is entering a period where interconnect design is just as important as advances in compute. Optical technologies will play a central role in enabling the next generation of AI infrastructure, but their adoption will depend on solving practical challenges around reliability, serviceability, and cost.
Key points from the keynote:
- AI infrastructure is evolving into a massive distributed computer spanning racks, data halls, and campuses.
- Rapid growth in reasoning models, agent workflows, and long-context inference is increasing demand on memory, networking, and orchestration systems.
- Copper interconnects remain valuable for short-reach connections, particularly inside racks.
- Optics is expected to move progressively closer to compute through pluggables, near-package optics, and eventually co-packaged optics.
- Energy efficiency and bandwidth density are emerging as critical design metrics for AI networking.
- Standardization and ecosystem collaboration will be essential to scaling AI infrastructure.
“We’re living in a moment where AI capability is increasing exponentially. But the real story is that AI growth is no longer just about bigger models. It’s about longer context, reasoning, agents, and persistent workflows. All of that puts enormous pressure on infrastructure. Interconnect is now central to AI system design. The real question for this industry isn’t whether optics matters to AI — it absolutely will — the question is where and how fast it moves into the critical path.”
