Hey @jpm, what kind of fibre and optics do I want? going to do 2-4 ~10m runs.

I am currently getting in way over my head with a 25 gigabit Mikrotik switch but this started as point to point runs.

Think future-proof, not necessarily cheap but of course some of this shit is going to come #StraightOuttaShenzen

Looks like I might have painted myself into a bit of a corner with some "cheap" SFP28 gear, too, but I'm not too attached

@voltagex remind me tomorrow, but in a nutshell go for 10km 1310nm or 1550nm single-mode if it’s not too much more expensive than 850nm multi-mode. You’ll be able to find everything you need on FS.com
@voltagex right, brain engaged, i think, so let's talk high-speed ethernet and the SFP family of optics.
@voltagex (after picking up Little Miss 7 from school early because she fell on her face)
@voltagex Ok, SFPs. Little rectangles, about THAT big, one end goes into network-related thingo, other end goes into cable
@voltagex and that's about it. There's no magic to them, they just translate one signal to another. On the host-side (switch or NIC), it's a high-speed serial connection, and on the cable side it could be anything, limited only by the imagination of a SFP vendor and market demand.
@voltagex In general, you'll only really need to know about SFP, SFP+, and SFP28, and only really need to know about them in the context of Ethernet. QSFP+ and QSFP28 are just 4 SFP+ or SFP28 in a trenchcoat.
@voltagex so, what's up with the cable end? That's where things get deep into "it depends" territory. Luckily, it boils down to 3 different cable families.
@voltagex namely, single-mode fibre, multi-mode fibre, and twin-axial copper (think, coaxial cable, but with 2 cores instead of 1). Even better, the OG 1Gbit/sec Ethernet SFP doesn't even really have the twin-ax option, it only really started appearing from 10GbE.
@voltagex The twin-ax option is the simplest, but the most limited. It literally just electrically connects the high-speed serial links on the host equipment together. Given how fragile these high-speed serial signals are, the cable length is extremely limited to about 10M at most, and anything over about 5M is going to have active signal conditioning and amplification onboard. It works great for anything inside the same rack or the immediately-adjacent rack, but don't go too much further than that.

@voltagex the second option is "multi-mode" fibre optic cables and optics. Traditionally, they were cheaper than "single-mode" but this is no longer true, because cheap VCSEL lasers are now capable of single-mode emission. They operate in the 850nm near-IR light band, and cables are generally constructed from 50um diameter plastic or glass cores. Because of the loose tolerance of the optics and fibre cores, their range is usually limited to 300M, and there's multiple grades of multi-mode fibre available with corresponding limitations on distance vs speed similar to copper ethernet cables. If you're buying new now, just go with OM4. OM1 cables use 62.5um diameter cores, and are only usually found in up to 1Gbit/sec applications. OM2 and later are all 50um diameter cores, but the optical transmission properties vary between generations which will limit transmission length.

You definitely don't need the fancy hybrid single-mode optics that use multi-mode cables unless you're trying to run 10GbE up the side of a building with ancient OM2 multi-mode cable plant.

@voltagex the third option is "single-mode" fibre optic cables and optics. This is where Shit Gets Real, and you do not look at energised optics with remaining good eye. Luckily, the actual cables are already so good and cheap that you just buy whatever from a reputable vendor and it'll work with any new single-mode optical devices you ever want to attach in future.

The optical devices on each end, however, are another matter entirely.

The basic consideration is the range class - typically either 10km, 20km, 40km, 80km, or 120km. Direct correlation to how much power the laser uses, and how expensive the modules is. Faster speeds may not get the 120km range class. Yes, you can have a single piece of glass 120km long to carry your signals.

And then comes the "oh wow holy shit my mind is blown" consideration of WAVELENGTH DIVISION MULTIPLEXING

@voltagex unlike electrical signals where only a single signal can be present on a conductor at the one time, you can send MULTIPLE DIFFERENT SIGNALS, IN DIFFERENT DIRECTIONS, AT THE SAME TIME, ON THE ONE FIBRE CORE
@voltagex you don't NEED to, of course, but you CAN, just by replacing the optical components on both ends.
@voltagex the "basic" single-mode SFPs run at either 1310nm or 1550nm far-IR wavelength, and use a single dedicated fibre core for each transmission direction. Not really any different than a multi-mode fibre connection, except the distance is now at least 10-20km
@voltagex the "bi-directional" single-mode SFPs come in pairs, where one of the pair transmits on 1310nm, the other in the pair transmits on 1550nm, and both transmit on a single fibre core at the same time. The receive path uses a wavelength-selective prism to direct the received wavelength to the receiver inside the SFP module while passing the transmitted wavelength through with little attenuation. This is the basic theory of how GPON networks work over a single fibre core as well.

@voltagex and finally the "wavelength division multiplexing" single-mode SFPs specify their wavelength, which is then optically combined into a single fibre core and transmitted to the other end, where the multiplexer uses wavelength-selective optics to split the signals back out into separate wavelengths on multiple fibres again.

Oh yeah, and there's 2 kinds of WDM, "coarse" and "fine", but they both work the same way, the only difference is how closely the wavelengths are packed together.

Almost certainly you don't need this, but you can just attach WDM MUXes to the single-mode fibres you already have whenever you want!

@voltagex and for extra lols, you can combine CWDM optics and muxes and BiDi operation, so you end up with 8 x 25Gbit/sec full-duplex links running across a single fibre core.
@voltagex and finally, 10GbaseT SFP+ modules are an abomination and should never be used if you can help it, because they generate so much heat and eat so much power it's not funny.
@voltagex As for where to buy this stuff, just go to https://fs.com/ I run their stuff in production environments, and they're just as good as the vendor-blessed optics at 1/10th the price. They also have a bunch of exotic optics that vendors don't even offer, but work perfectly well because SFPs are just an abstraction layer between layer 2 and layer 1
FS.com Deutschland - Rechenzentrum, Technologie-Lösungen für Unternehmen und Internetdienstanbieter

Bereitstellung skalierbarer Netzwerktechnologie-Lösungen für Rechenzentren, Unternehmen und Internetdienstanbieter. Entdecken Sie Hochgeschwindigkeits-Netzwerklösungen zum Aufbau einer zuverlässigen und skalierbaren Verbindungsinfrastruktur. Weltweite Lieferung, professioneller Support.

FS.com

@jpm @voltagex oh hey Joel want to see my favourite things that render your last sentence hilariously inaccurate when it comes to weird edge cases in broadcast/AV industry?

  • DirectOut USB.MADI operates a high channel count USB sound card converter on FPGA in the SFP
  • Riedel MuoN HDMI/SDI to/from lossless broadcast Video-over-IP converter
  • @s0 @voltagex but that’s the neat* thing about SFPs, they can mangle any data stream to/from a high speed serial data stream
    @jpm @s0 @voltagex or low speed - I’ve recently been testing out little SFP pluggables that are actually single-port terminal servers for OOB management.
    @ThermiteBeGiants @s0 @voltagex SGMII that links up at 1Gbit/sec always runs at just over 600MHz…
    @jpm @s0 @voltagex oh while we’re talking about weird SFPs - there’s also ones that convert SGMII 1Gbe into LCAS’d STM-16 SDH out there, should you wish to run your traffic into an SDH ADM with some TDM traffic smushed into it
    @ThermiteBeGiants @jpm @s0 @voltagex ooh, I'd heard about those for 10G (10GBase-LW etc.), but never for 1G!
    @ThermiteBeGiants @oddline @jpm @s0 @voltagex If you want to play with a MuoN A10 or A1, I'm sure i can find one kicking about for a day or 2
    MiCLK Grandmaster on an SFP

    MiCLK 1588 Grandmaster by RAD offers precise PTP and GPS timing in a compact SFP. Ideal for 5G and LTE-A deployments to support IEEE.

    RAD
    @ThermiteBeGiants @oddline @jpm @voltagex I love to read a Wikipedia article and basically everything in it is completely new to me
    @s0 @oddline @jpm @voltagex if you want to learn about “dead” telecoms technology I’m your guy apparently
    @ThermiteBeGiants @s0 @oddline @voltagex it’s not dead, there’s still customers paying 6 figures annually for a service on that line!
    @jpm @ThermiteBeGiants @s0 @oddline @voltagex fibers installed along the tops of HV transmission lines, originally just to detect breaks if towers fall, later found themselves being used to carry data. Lots of SONET/SDH.
    @uep @jpm @s0 @oddline @voltagex GSM-R train radios, as the GSM heritage in the name suggests, use TDM too - E1 into SDH ADMs and then terminating direct into the BSCs as SDH (apologies for the initialism soup there)
    @jpm @s0 @oddline @voltagex nah, the telcos have gotten rid of almost all of their TDM service now, it’s not even a legacy cash cow for them. You either get a leased MPLS-TP service (which IMHO is a big step down from TDM in many ways) or you roll your own physical network - none of the telcos even lease dark fibre to you unless you’ve got a deeeep chequebook
    @ThermiteBeGiants @s0 @oddline @voltagex eh? Dark fibre is still definitely a thing and isn’t even that expensive, only a couple thousand a month from Vocus or PipeNetworks for CBD building connectivity.
    @jpm @s0 @oddline @voltagex for proper regional backhaul though?
    @ThermiteBeGiants @s0 @oddline @voltagex depends on carrier, Vocus claims to have some* regional** backhaul. Optus flatly refuses to sell dark fibre though
    @jpm @s0 @oddline @voltagex dollars to donuts that the Vocus dark fibre is sparkling MPLS
    @ThermiteBeGiants @s0 @oddline @voltagex their metro dark fibre is a single piece of glass fused together from point A to B
    @jpm @s0 @oddline @voltagex out in the regions tho
    @ThermiteBeGiants @s0 @oddline @voltagex maybe to regional cities where their network reaches, but I’ve never needed that
    @ThermiteBeGiants @jpm @s0 @oddline @voltagex AARNet has dark fibre in some interesting places.
    @ThermiteBeGiants @jpm @s0 @voltagex can confirm, I would pay embarrassing sums of money for TDM service, but they're all getting out of that game as fast as they can
    @oddline @ThermiteBeGiants @jpm @voltagex it’s so irritating how hard it is to get dark fibre hookup
    @oddline @jpm @s0 @voltagex closest you can get is CESoP over an MPLS-TP tunnel but the tunnel bit is the crucial bit there - you need equipment at both ends.
    Also it is very funny to have to have a 2.048Mbps E1 frame carried over 10Mbps+ of MPLS-TP provisioned pseudowire