Kenneth FinneganOne of my nice friends at Hurricane Electric gave me a dead 100G-LR4 optic to tear apart for your entertainment, so for the sake of your entertainment, lets dig into it! 🧵
100G-LR4 is a QSFP28 optic that runs over a duplex pair of single mode fiber, so it takes 4 lanes of 25G from the switch, modulates 4 different colors of light, and combines them together over a single strand to the other end up to 10km away.
First comes off the ejection bail. I guess we're committed now since that was rather destructive.
Two tiny screws get us into the case. The grey putty stuff is thermal gap filler, which couples each part that puts off heat to the case for cooling.
Think of it like thermal paste, but it's designed to be thick and span a gap instead of just be a thin film.
Lifting the electrical assembly out of the other half of the metal case, we can start pointing out the major points of interest.
TOSA/ROSA - transmit/ receive optical sub assemblies are where the laser magic happen.
The retimers help 4x25G get to the switch via the QSFP connector
The TOSA and ROSA both need to handle four wavelengths of light moving 25Gbps each, so they're actually connected to the PCB with two layers of ribbon cables. One of the ribbons is just for the 4x25Gbps data channels, and the other is for power, biasing, etc, not high speed stuff to run the OSA parts.
Looking on the other side of the whole assembly, the largest IC in the middle (the silver one) is the microcontroller that talks to the switch over an I2C bus to report telemetry and ID information, and pretty much everything else is voltage regulation for taking the 3.3V from the switch and regulating it down to lower voltages to drive the lasers.
And that microcontroller that runs the whole show is... a Cortex M3! An STM32F103C6 with 32kB of flash, 10kB of SRAM, and can run at up to 72MHz.
Meaning that this optic has more compute power than many early home computers.
You'll note that the ST microcontroller is silver, as are a few other ICs on this optic. This is actually a cost saving measure called "flip chip" packaging, where they don't waste the expense of the black plastic packaging and solder the silicon die straight down onto the board like it was a QFN. Absolutely wild.
While it's fun to think about malware on the ARM core in these, the data path is WAY faster than the Cortex M3 can handle. It's entirely an out-of-band manager of the electronics with no idea of what is going on in the Ethernet link itself.
That being said, some newer optics really DO have interactions in the data path. The most notable is the speed-changing optics which allow modern Ethernet switches (which only support 10G/25G/50G on their front panel ports) to still link up with a 1G Ethernet peer. The optic does the 10:1 speed change and implements clause 37 autoneg entirely inside the optic.
Granted, the firmware on optics actually *IS* field writable. I've coordinated with an optics vendor in the past to help a customer apply a firmware patch to 10,000 optics installed in Arista switches in the field by issuing the right magic sequence of write commands over the I2C bus from EOS.
Remember how I said the rest of the parts were mostly voltage regulation?
Voltage regulators usually need some kind of inductance... so just saying... prime cute coil of wire opportunity.
So this is where things get REAL ugly... we break out the Dremel and cut the lid off the TOSA (Transmit Optical Sub Assembly) to expose the yummy optical bits inside it.
Remember that while there's four different waves for the four 25Gbps streams, 100G-LR4 runs over a single duplex pair of fibers, so all four optical signals need to make it out a single strand of fiber, so... PRISMS.
The colors are all way out in the infrared, so don't put too much stock in the colors I use in the picture, but hopefully you get the point about how the prisms are used as optical combiners.
VE7FIM@kwf Is that grey rectangle on the left a magnet?
@ve7fim It is. You beat me to posting the explanation
@kwf Loving the thread. Thank you.