PIC Summit USA | Sunnyvale, California
AI infrastructure has reached a point where electrical interconnects—not compute—are now the dominant limiter of performance, power efficiency, and ultimately profitability, according to Dave Lazovsky, Founder and CEO of Celestial AI, speaking this week at PIC Summit USA in Sunnyvale.
In a keynote that focused less on product announcements and more on architectural inevitability, Lazovsky argued that the rapid shift toward inference-heavy, reasoning-driven AI workloads is forcing data center operators to abandon incremental electrical scaling in favor of deeply integrated optical scale-up fabrics—inside the package, across the rack, and beyond.
The mismatch, he said, is structural. AI model innovation cycles now operate on quarterly—or faster—timeframes, while infrastructure refresh cycles remain anchored at 12 to 18 months. As inference becomes the primary production workload, this gap exposes hard limits in memory bandwidth, latency, and power consumption that cannot be solved by adding more accelerators or by extending copper-based interconnects.
Lazovsky pointed to the accelerating “memory wall” as the most visible symptom. Even excluding KV caches, model parameter growth is outpacing on-package memory capacity by orders of magnitude. With inference-time reasoning models now mainstream, KV cache traffic has become a first-order design constraint, driving demand for extremely low-latency, high-bandwidth access to distributed memory. In this environment, he emphasized, latency is no longer an abstract performance metric—it translates directly into inference throughput and revenue.
A recurring theme throughout the talk was that traditional scale-out networking assumptions no longer hold. While the industry has successfully doubled bandwidth every two years for scale-out fabrics, Lazovsky argued that scale-up domains operate under fundamentally different constraints. Load-store dominated traffic patterns, fine-grained memory access, and tight coupling between accelerators demand interconnect bandwidth measured in tens to hundreds of terabits per second per package, with end-to-end transaction latencies under 200 nanoseconds.
To address this, Lazovsky described Celestial AI’s approach of embedding photonic fabric chiplets directly alongside XPUs, enabling multi-chiplet die complexes with aggregate bandwidths far beyond what electrical links can sustain within realistic power envelopes. In current deployments, individual optical chiplets deliver on the order of 16 Tb/s of bidirectional bandwidth, scaling to tens of terabits per second per XPU package today and significantly more on near-term roadmaps.
Energy efficiency, however, was as central to the argument as bandwidth. Lazovsky cited industry data showing U.S. data centers consuming roughly 5% of grid power today, with projections exceeding 12% by the end of the decade—growth constrained not by capital availability, but by power delivery. For hyperscalers, he said, the unit of deployment has shifted from dollars to gigawatts. In that context, reducing joules per bit at scale becomes mandatory, not optional.
Optical interconnects, he argued, offer a step-function improvement. By eliminating high-loss electrical channels and pushing optics deeper into the system architecture, Celestial AI claims multi-fold reductions in interconnect power while simultaneously extending reach from inside the rack to tens of meters without sacrificing latency. That reach, Lazovsky noted, also introduces architectural flexibility—allowing scale-up domains to span racks while distributing power density more evenly across the data hall.
Beyond interconnect alone, Lazovsky outlined how photonic fabrics enable a rethinking of the memory hierarchy itself. He described optically interconnected memory appliances delivering tens of terabytes per second of aggregate bandwidth, positioned as a low-latency shared memory tier for inference. Earlier attempts at memory disaggregation using PCIe-based CXL, he suggested, were fundamentally bandwidth-limited relative to modern HBM stacks; photonic fabric removes that mismatch by aligning memory access bandwidth with accelerator demand.
Throughout the keynote, Lazovsky emphasized that these architectural shifts are not speculative. They are being driven by a small number of hyperscale operators deploying inference systems at unprecedented scale, where even marginal gains in latency or power efficiency compound into material economic advantages. Rather than pursuing broad product-market fit, Celestial AI has aligned itself with this reality by working directly with a limited set of anchor customers, co-architecting systems backward from application requirements. While this approach extends development timelines, Lazovsky argued it effectively eliminates market uncertainty.
Analysis
Lazovsky’s remarks come as Celestial AI prepares to close its acquisition by Marvell, a deal announced in late 2025 that underscores how strategically important optical scale-up connectivity has become to mainstream silicon vendors. Marvell has positioned the acquisition as a way to extend its data-center portfolio beyond switching and electrical connectivity into tightly integrated optical fabrics that can serve next-generation AI systems, particularly those built around multi-XPU, scale-up architectures.
The strategic logic is clear. As AI systems evolve toward tightly coupled accelerator complexes with shared memory semantics, the boundary between compute, memory, and network continues to blur. Optical interconnect is no longer a peripheral upgrade for extending reach between racks; it is becoming a core architectural element that determines achievable bandwidth density, latency, and power efficiency inside the package itself. For Marvell, integrating Celestial AI’s photonic fabric technology provides a path to participate directly in that architectural transition.
Equally notable is Lazovsky’s repeated reference—without naming—of a hyperscale customer deploying Celestial AI technology at volume. Industry observers widely interpret this as validation from one of the largest AI infrastructure builders, where production inference workloads, not research prototypes, are dictating design choices. Such a customer sets extreme requirements: hundreds of thousands of XPUs per quarter, multi-terabit per-package bandwidth, and power budgets constrained by data-center-level energy availability rather than component cost.
If that hyperscale deployment proceeds as described, it may represent one of the first large-scale commercial validations of optical scale-up fabrics integrated directly into accelerator packages. For the broader networking and systems community, Lazovsky’s keynote suggests that photonic interconnect is moving rapidly from experimental to structural—reshaping how AI systems are designed, deployed, and economically justified.
🌐 We’re tracking the shift toward optical scale-up, co-packaged optics, and photonic fabrics across AI infrastructure—from XPUs and switches to disaggregated memory and system-level architectures. Follow ongoing coverage at ConvergeDigest.com under AI Infrastructure, Optical, and Data Centers.
