✧✦Catherine✦✧the network card i'm poking at, which has two different CPU clusters with Linux running on it, apparently has eMMC and NVMe drives onboard
also it has docker^W containerd installed by default
the operating system is booted by grub??
somebody here was making a joke about the network card running kubernetes
i regret to inform you that the network card does in fact run kubernetes (there is kubelet in ps)
okay there's "kubelet" in
ps. it's definitely running it
medium rare bird@whitequark nooooooooo
Graham Sutherland / Polynomial@whitequark "Kubernetes running on a network card" is an entire new level of cursed.
@whitequark Dante better get inventing some new layers of hell
Rairii 
@gsuberland @whitequark i'm sure it's only in the six-seventh layers of hell
Pxl Phile@Rairii @gsuberland @whitequark I mean it's on a network card. You know, like
Ma can we have leaders of hell?
- we have layers of hell at home
Hell at home: 8 OSI layers
🇺🇦 haxadecimal 🚫👑@gsuberland @whitequark
(OK, I included "financial" and "political", which OSI didn't officially include. So sue me.)
(OK, I included "financial" and "political", which OSI didn't officially include. So sue me.)
@brouhaha @gsuberland that's the judicial layer! above the other two
artemist@gsuberland @whitequark despite fun problems, like "entire broadcast studio breaks because someone plugged in a misconfigured audio interface", aes67 was surprisingly tame for dante's inferno.
Palmer Dabbelt@gsuberland @whitequark at my first job I had to get PHP running on a network card
4censord 
@whitequark oh then is it running a version of SONiC?
@4censord what is that
@whitequark https://sonicfoundation.dev/
its a linux based switch or router operating system
its quite good and comfy compared to e.g. cisco ios
its a linux based switch or router operating system
its quite good and comfy compared to e.g. cisco ios
Lona Theartlav@whitequark Hmm. That makes me wonder if it would be possible to get it to run on my Xeon Phi 7120A.
Tobias@whitequark Let me tell you about the *other* Mellanox product that comes with docker enabled by default.
Yes, it's their 100/200G Ethernet and 200/400/800G infiniband switches.
Joe Chilliams@whitequark jokes aside fwiw the way to run P4 on the ASIC is to load the compiled P4 using grpc through a service running on kube https://docs.nvidia.com/doca/sdk/loading-dpl-applications/index.html
@joew whats p4
@whitequark it's a weird language to do network packet manipulation in hardware. NICs like bluefield, AMD pensando and even some switches with the right ASIC (tofino) support loading up custom code this way. So you can do things like firewalls, load balancing and etc in hardware rather than in the OS.
Hugo Slabbert ⚠️@joew @whitequark or you can start caching your kv store in-network on your switches
Chris@whitequark i feel vindicated (and I wasn’t the only one joking about k8s)
Greg Bell@whitequark satire has become impossible
Echo Nar@whitequark at first I was surprised then remembered ISPs/carriers. Due to how modern networking systems work (think 5g), they run a lot of service stuff in containers and sticking it on the NIC is probably giving faster network connectivity by bypassing PCIe to CPU translations.
@lethedata the bf-3 is mostly for AI stuff as far as i know. also like. the CPU on this thing is connected to the NIC (the actual NIC part of NIC) over... maybe PCIe, maybe AMBA? not sure. some sort of bus. but you're definitely going to still have NIC to CPU translations, is my point
@whitequark I'd have to look more into that hardware (I was thinking it was a Broadcom BCM95750X or the like) but my hunch is it's hardware offloading the entire network stack to NIC PCIe cards that can fully act as nodes of the cluster. Pin those network service pods to those PCIe cards and offload the entire network speeding it up.
Raven667@lethedata @whitequark I was going to joke that kube was overkill and overhead for a dedicated packet-pushing device, some modern dev FOMO shit that makes everything more complicated and slower than it needs to be (for ~SCALABILITY~) but that makes sense, the more you can push down into the ASIC on the NIC the less CPU you have to burn, freeing it for workload.
Anna@whitequark I wasn't joking! Bluefield Kubernetes is a real thing and can hurt you
@annathyst can confirm, am hurt by this knowledge
clara "parasocialist" sparks@annathyst @whitequark what in tarnation
@mxsparks @whitequark it is absolutely real and can hurt you
rj@whitequark kuberNICes
FxChiP@whitequark the WHAT is running kubernetes?????????? (/hyperbole for comedic effect)