Clint Schow, professor in the Electrical & Computer Engineering (ECE) Department at UCSB, has been working with Facebook — now Meta — for more than five years to increase the energy efficiency of the social media giant’s datacenters. The ambitious project spanned the design and integrated packaging of photonics with electronic circuits designed by ECE professor James Buckwalter’s students, and the development of network architectures leveraging photonic switching led by fellow ECE research professor Adel Saleh.
The final deliverable of the project is a coherent optical link, the development of which has resulted through a collaboration with Intel led by Aaron Maharry, who completed his PhD in Schow’s lab in March. The vision for the link, which has a speed of 200 Gb per second (Gbps) single-wavelength transmission and uses just 1.5 watts of power, was put forth about a dozen years ago by now-emeritus ECE professor Larry Coldren, a world pioneer in photonic integrated circuits (PICs) who thought it could be a good fit for datacenters.
At the time, Coldren recalled with a sly grin during a recent Zoom meeting, “No one believed it could work. They thought we were kind of out to lunch and asked us, ‘Why are you trying to do a coherent link in a datacenter? That’s crazy.’”
The Optical Link Landscape
When an optical link is used in a datacenter, information is transmitted from one networking switch to another on the other side of a building by way of an electronic signal that travels from the first switch to a transceiver module, which converts it to an optical signal. That signal is then sent via fiber to be received at the other end, where it is turned back into an electrical signal. “We’re focusing on the transceiver part of the link, because it converts the signal at both ends,” Maharry says.
Maharry, who joined Schow’s group just as the project was starting, presented the research at the Optical Fiber Communication Conference (OFC), the premier meeting for optical communications and networking professionals, held in San Diego in early March. “This is a major step toward next-generation datacenter networks built on coherent optical links,” Maharry said at the time. “These links will enable datacenters to economically scale network throughput with lower energy consumption, leading to a better, more sustainable, and cheaper internet.”
“The power of Aaron’s paper is integration — an extremely high level of optical integration on the optical device side, and then the electrical integration and designing those two things together,” Schow says. “Getting this great result was an impressive achievement, and I think it’s why OFC appreciated and highlighted our paper.”
Schow uses the analog of a car radio to explain what a coherent link does. “An antenna is broadcasting at some frequency, and the signal goes out in every direction,” he says. “By the time it gets to you, it’s very weak, but you can receive it by having a local oscillator — an electrical sine-wave generator in a car radio — that boosts the signal of the station you select. The same principle is at work in coherent optics. While the local oscillator is optical — a laser — it, too, enables much more sensitive signal detection.”
The optical links used in the datacenters that now form the backbone of the internet often rely on a communication approach known as intensity modulation direct detection (IMDD), in which information is encoded by modulating only the power levels of light. Rising data rates have led to increasing interest in replacing IMDD links with more-scalable coherent links, but their high power consumption and cost have prevented their widespread adoption.
At its most basic, coherent optical transmission is a technique based on modulating both the amplitude and the phase of the light while transmitting on two separate axes of polarization, to enable the transport of considerably more information through a fiber optic cable.
Maharry and several fellow PhD students worked with, and at, Intel Corporation to design an analog-coherent detection link architecture; that is, a link that uses analog optical processing to convert received bits directly to digital electrical signals. That architecture removes the power-hungry digital-signal processing functions that are required for longer-reach conventional coherent links, such as those used in transoceanic fiber-optic cables, but are unnecessary for the shorter distances signals travel within a datacenter.
“There is a broad trend across the optical-communication industry, of moving toward link technology that will enable higher data rates and improve the sensitivity of the receiver,” Maharry explains. “Widely used for undersea cables that can go thousands of kilometers, they have been used for shorter and shorter reaches over the past decade or so. We’ve been applying that architecture to lengths of, at most, only a couple of kilometers for high-volume use inside datacenters.”
It’s a technically challenging task, and one that took all of Maharry’s six years as a doctoral student to complete. “It’s very difficult — but essential — to make that technology both energy- and cost-efficient, because it has to compete with simple, robust direct-detection technologies,” he explains. “So, we’ve had to redesign all the chips and optimize them for power efficiency so that it can compete with these other technologies.
“At the highest level, we’ve optimized some of the functions that have to do with how the signal is received, conditioned, and post-processed to recover the data,” he adds. “For longer reaches, those functions are normally performed in a DSP application-specific integrated circuit (ASIC), which accounts for about half the cost and uses half the power consumed by the whole module. We’ve figured out ways to perform these complicated signal-processing functions in the analog-optical domain, so that we can offload them from the DSP chip and save a lot of power.”
One important emerging technology that low-power coherent links can enable is all-optical switching. Though it is not a new technology, using it in huge datacenter networks was unheard of until last year. “Nobody was really doing it until Google took the covers off their deployment of optical switches in datacenter and machine-learning clusters,” Coldren explains. “I think that most thought it was too early to think about it, and that the current technology would scale for the foreseeable future.”
Schow notes that Google recently shook the whole industry when they announced that they had been deploying optical switching in their datacenters at scale for the past five years. “That was kind of our key application from the beginning,” he says. “With heavy hitters like Google having rolled that out, it validates this story we’ve been writing for a while.”
“Google is deploying these optical switches and getting pretty incredible performance and energy-efficiency improvements from them,” Maharry says, “but because the signal has to be passed through a set of optical switching components, it’s harder to detect, so you have to work harder to pick it up on the other side. Our links are more robust than Google’s and can tolerate greater signal loss; that’s why we’re so keen on this optical-switching application.”
Coldren notes, “In the past two to three years, everybody has started saying that coherent is coming, but that the future probably still holds some space for redesigned DSPs. In this paper, though, there is no DSP. We’re doing analog signal processing.”
A Unique Collaboration
Intel partnered with Schow’s group for part of the project, making it possible for several UCSB PhD students, Maharry among them, to do a year-long internship, “actually working in their product group and designing the chips for this project,” he says. “Their technology is ideally suited to this application, because the key feature for this coherent approach is the local oscillator at the receiver, and one of Intel’s huge achievements in recent years has been to integrate indium phosphide lasers directly onto silicon chips, so that the full receiver PIC can be fabricated as one unit. We were able to work with Intel’s engineers to design and build these coherent transmitter and receiver chips.”
“The model of sending interns was unique,” Schow says. “Intel technology is proprietary, and they don’t yet offer it to the outside world, so in order to collaborate so closely, our students had to become Intel employees. It gave Aaron and others the opportunity to work on and design unique technology, an experience you really can’t get any other way.”
“It was great and kind of overwhelming,” says Maharry. “We were working on a UCSB project but were able to interact with all the other groups Intel has within their Silicon Photonics Product Division. It was pretty incredible to be able to draw on their expertise.”
In the end, Maharry says, “We took the same fundamental building blocks that are on the PICs [photonic integrated circuits] and EICs [electronic integrated circuits] — essentially, the transmitter and the receiver — and designed, built, optimized, tested, and measured all four of those blocks to make this link.” (The transmitter has two chips — an electronic driver and a photonic transmitter chip. The receiver also has two chips — a photonic receiver and an electronic receiver chip. Thus, a total of four components comprise the link.)
Another technical aspect involved using a different wavelength band. Until now, coherent links have been designed to use light near the 1550 nm wavelength, which has the lowest loss as the light moves through the optical fiber. “But, in our work, we've built the first coherent links that use 1310 nm light,” Maharry says. “Since we’re redesigning coherent links for short reaches, using 1550 nm light to ensure low fiber loss is no longer critical — and 1310 nm light minimizes distortions from chromatic dispersion.
“We’re moving from direct detection, where only the intensity of the light is modulated, to coherent detection, in which both amplitude and the phase are modulated,” Maharry concludes. “In the first instance, you could encode only a single 0 or 1 transition on the light, but coherent modulation and detection allows you to encode more than just that one bit on that part of the optical signal. For us, the result has been a four-fold increase in the data rate.”