Kenneth FinneganOne of my nice friends at Hurricane Electric gave me a dead 100G-LR4 optic to tear apart for your entertainment, so for the sake of your entertainment, lets dig into it! 🧵
100G-LR4 is a QSFP28 optic that runs over a duplex pair of single mode fiber, so it takes 4 lanes of 25G from the switch, modulates 4 different colors of light, and combines them together over a single strand to the other end up to 10km away.
First comes off the ejection bail. I guess we're committed now since that was rather destructive.
Two tiny screws get us into the case. The grey putty stuff is thermal gap filler, which couples each part that puts off heat to the case for cooling.
Think of it like thermal paste, but it's designed to be thick and span a gap instead of just be a thin film.
Lifting the electrical assembly out of the other half of the metal case, we can start pointing out the major points of interest.
TOSA/ROSA - transmit/ receive optical sub assemblies are where the laser magic happen.
The retimers help 4x25G get to the switch via the QSFP connector
The TOSA and ROSA both need to handle four wavelengths of light moving 25Gbps each, so they're actually connected to the PCB with two layers of ribbon cables. One of the ribbons is just for the 4x25Gbps data channels, and the other is for power, biasing, etc, not high speed stuff to run the OSA parts.
Looking on the other side of the whole assembly, the largest IC in the middle (the silver one) is the microcontroller that talks to the switch over an I2C bus to report telemetry and ID information, and pretty much everything else is voltage regulation for taking the 3.3V from the switch and regulating it down to lower voltages to drive the lasers.
And that microcontroller that runs the whole show is... a Cortex M3! An STM32F103C6 with 32kB of flash, 10kB of SRAM, and can run at up to 72MHz.
Meaning that this optic has more compute power than many early home computers.
You'll note that the ST microcontroller is silver, as are a few other ICs on this optic. This is actually a cost saving measure called "flip chip" packaging, where they don't waste the expense of the black plastic packaging and solder the silicon die straight down onto the board like it was a QFN. Absolutely wild.
While it's fun to think about malware on the ARM core in these, the data path is WAY faster than the Cortex M3 can handle. It's entirely an out-of-band manager of the electronics with no idea of what is going on in the Ethernet link itself.
That being said, some newer optics really DO have interactions in the data path. The most notable is the speed-changing optics which allow modern Ethernet switches (which only support 10G/25G/50G on their front panel ports) to still link up with a 1G Ethernet peer. The optic does the 10:1 speed change and implements clause 37 autoneg entirely inside the optic.
Granted, the firmware on optics actually *IS* field writable. I've coordinated with an optics vendor in the past to help a customer apply a firmware patch to 10,000 optics installed in Arista switches in the field by issuing the right magic sequence of write commands over the I2C bus from EOS.
Remember how I said the rest of the parts were mostly voltage regulation?
Voltage regulators usually need some kind of inductance... so just saying... prime cute coil of wire opportunity.
So this is where things get REAL ugly... we break out the Dremel and cut the lid off the TOSA (Transmit Optical Sub Assembly) to expose the yummy optical bits inside it.
Remember that while there's four different waves for the four 25Gbps streams, 100G-LR4 runs over a single duplex pair of fibers, so all four optical signals need to make it out a single strand of fiber, so... PRISMS.
The colors are all way out in the infrared, so don't put too much stock in the colors I use in the picture, but hopefully you get the point about how the prisms are used as optical combiners.
The keen eye will also notice that some of the Dremel debris from me cutting open the TOSA is... stuck to this cube and fuzzy... like it's magnetic...
And it it. That is a tiny magnet in the middle of the optical path, but why?
Physics talk. Remember that the transmitters here are lasers, which mean that the four lasers on the right are each their own resonant cavity, which leaks a fraction of the light to be useful. But if there was some kind of partial reflection out in the fiber somewhere, this could cause problems and destabilize these lasers, so we would like some way to let light OUT of the TOSA without letting any reflected light back into the lasers to cause problems, so we make an optical one-way valve...
MORE PHYSICS TALK: Light is a perpendicular E-M field, so if you used a vertical polarizer, and then passed the light through a magnetic field to rotate the polarization 45 degrees to line up with a //// diagonal polarizing filter on the other side of the magnet...
Think about the reverse path. //// polarizing filter, 45 degree rotation, horizontal light hitting a vertical polarizing filtering. BLOCKED.
Putting the TOSA under my microscope, we can start to see some more of the finer details.
This is rotated from the last photos.
I'm not convinced whether the two missing lenses in the middle broke off before I got it, broke off while I was hacking my way into the package, or were just never there. A mystery for the ages.
sen@kwf I think they're optional guides to ensure the lasers hit the prisms correctly as the placement of the optics doesn’t look to be very precise. If you’re cutting costs you could have a human on the assembly line placing them as well as they can and the additional lenses are only added if they don’t pass QA.