Hey @jpm, what kind of fibre and optics do I want? going to do 2-4 ~10m runs.

I am currently getting in way over my head with a 25 gigabit Mikrotik switch but this started as point to point runs.

Think future-proof, not necessarily cheap but of course some of this shit is going to come #StraightOuttaShenzen

Looks like I might have painted myself into a bit of a corner with some "cheap" SFP28 gear, too, but I'm not too attached

@voltagex remind me tomorrow, but in a nutshell go for 10km 1310nm or 1550nm single-mode if it’s not too much more expensive than 850nm multi-mode. You’ll be able to find everything you need on FS.com
@voltagex right, brain engaged, i think, so let's talk high-speed ethernet and the SFP family of optics.
@voltagex (after picking up Little Miss 7 from school early because she fell on her face)
@voltagex Ok, SFPs. Little rectangles, about THAT big, one end goes into network-related thingo, other end goes into cable
@voltagex and that's about it. There's no magic to them, they just translate one signal to another. On the host-side (switch or NIC), it's a high-speed serial connection, and on the cable side it could be anything, limited only by the imagination of a SFP vendor and market demand.
@voltagex In general, you'll only really need to know about SFP, SFP+, and SFP28, and only really need to know about them in the context of Ethernet. QSFP+ and QSFP28 are just 4 SFP+ or SFP28 in a trenchcoat.
@voltagex so, what's up with the cable end? That's where things get deep into "it depends" territory. Luckily, it boils down to 3 different cable families.
@voltagex namely, single-mode fibre, multi-mode fibre, and twin-axial copper (think, coaxial cable, but with 2 cores instead of 1). Even better, the OG 1Gbit/sec Ethernet SFP doesn't even really have the twin-ax option, it only really started appearing from 10GbE.
@voltagex The twin-ax option is the simplest, but the most limited. It literally just electrically connects the high-speed serial links on the host equipment together. Given how fragile these high-speed serial signals are, the cable length is extremely limited to about 10M at most, and anything over about 5M is going to have active signal conditioning and amplification onboard. It works great for anything inside the same rack or the immediately-adjacent rack, but don't go too much further than that.

@voltagex the second option is "multi-mode" fibre optic cables and optics. Traditionally, they were cheaper than "single-mode" but this is no longer true, because cheap VCSEL lasers are now capable of single-mode emission. They operate in the 850nm near-IR light band, and cables are generally constructed from 50um diameter plastic or glass cores. Because of the loose tolerance of the optics and fibre cores, their range is usually limited to 300M, and there's multiple grades of multi-mode fibre available with corresponding limitations on distance vs speed similar to copper ethernet cables. If you're buying new now, just go with OM4. OM1 cables use 62.5um diameter cores, and are only usually found in up to 1Gbit/sec applications. OM2 and later are all 50um diameter cores, but the optical transmission properties vary between generations which will limit transmission length.

You definitely don't need the fancy hybrid single-mode optics that use multi-mode cables unless you're trying to run 10GbE up the side of a building with ancient OM2 multi-mode cable plant.

@voltagex the third option is "single-mode" fibre optic cables and optics. This is where Shit Gets Real, and you do not look at energised optics with remaining good eye. Luckily, the actual cables are already so good and cheap that you just buy whatever from a reputable vendor and it'll work with any new single-mode optical devices you ever want to attach in future.

The optical devices on each end, however, are another matter entirely.

The basic consideration is the range class - typically either 10km, 20km, 40km, 80km, or 120km. Direct correlation to how much power the laser uses, and how expensive the modules is. Faster speeds may not get the 120km range class. Yes, you can have a single piece of glass 120km long to carry your signals.

And then comes the "oh wow holy shit my mind is blown" consideration of WAVELENGTH DIVISION MULTIPLEXING

@voltagex unlike electrical signals where only a single signal can be present on a conductor at the one time, you can send MULTIPLE DIFFERENT SIGNALS, IN DIFFERENT DIRECTIONS, AT THE SAME TIME, ON THE ONE FIBRE CORE
@voltagex you don't NEED to, of course, but you CAN, just by replacing the optical components on both ends.
@voltagex the "basic" single-mode SFPs run at either 1310nm or 1550nm far-IR wavelength, and use a single dedicated fibre core for each transmission direction. Not really any different than a multi-mode fibre connection, except the distance is now at least 10-20km
@voltagex the "bi-directional" single-mode SFPs come in pairs, where one of the pair transmits on 1310nm, the other in the pair transmits on 1550nm, and both transmit on a single fibre core at the same time. The receive path uses a wavelength-selective prism to direct the received wavelength to the receiver inside the SFP module while passing the transmitted wavelength through with little attenuation. This is the basic theory of how GPON networks work over a single fibre core as well.
@jpm @voltagex what the fuck lmao
@arichtman @voltagex they’re just different coloured lights
@jpm @voltagex THATS THE POINT - is this not witchcraft to you? Yes they vibrate at juuuust the right period so they don't interfere.
@arichtman @voltagex not really. It’s the same as 2.4GHz and 5GHz WiFi sharing the same air
16 Channels CWDM Mux or Demux, 1310-1610nm, with Monitor and Expansion Port, 4.5dB Typical IL, LC/UPC, Single Fiber, FMU Plug-in Module - FS.com Australia

16 Channels CWDM Mux or Demux, 1310-1610nm, with Monitor and Expansion Port, 4.5dB Typical IL, LC/UPC, Single Fiber, FMU Plug-in Module, Product Specification:Channel - 16 Channels , Wavelength - 1310-1610nm, Special Port - Monitor, Expansion Port, Technology - TFF (Thin Film Filter), Insertion Loss - ≤5.0dB, 4.5dB Typical, Channel Spacing - 20nm

FS.com
@uep @arichtman @voltagex yep I did touch on those later in the thread. The wildest ones are BiDi CWDM with 8 bi-directional channels on a single fibre core

@jpm @arichtman @voltagex the perils of seeing the thread via a reply and not the main line, and not a tree view.

Also, "dense" rather that "fine", but yes.

The cool thing with these is that because they're a purely optical device, each wl can be a completely different L2. we used to do a bunch of Ethernet and FC over different colours between failover sites.

@jpm @arichtman @voltagex

Oh, and the other discussion about STM across power transmission reminded me of another important use case for CWDM.

If you have multiple colours going down the same fibre, you can immediately tell the difference between a link break caused by a failure in the SFP optics and a fibre cut.

Because the CWDM box is an entirely passive unit, mostly a prism, it's rather unlikely to be the point of failure

Huge benefit that IMHO justifies using the mux for that alone, when one of the ends is hard to get to.