Progress on my latest endeavor: I’m building a #k8s setup centered around #k3s, the #Cilium CNI, and the #Linkerd service mesh. My goal is to run this atop an immutable #Linux distro with #SELinux enabled. Sadly, the Cilium bits don’t work with @fedora #CoreOS without disabling SELinux even without Cilium’s Envoy daemonset. Barring some breakthrough in figuring out why that is, I’m planning to try @opensuse #MicroOS next. Its use of btrfs is quite appealing.
I’ve got these bits running in a vm now so next is to add #Multus CNI, kube-vip, & something to provide for load balancer services. I’m going to check out the option in Cilium and see if it’s a good alternative to #MetalLB. Once that’s sorted, it’s going to be time for #ArgoCD & the #Nginx ingress (ingress-nginx). After that I have a several pieces of the puzzle to finish researching and assembling as a PoC.
@genebean the lb stuff works decent. Using cilium for LB, cni, ingress and gateway api.
@genebean @fedora @opensuse What were the errors reported by Cilium and SELinux? I wouldn't expect anything fundamental preventing the two working together.

@pchaigno the agent pod never gets to a running state and the operator pod never got to the ready state. If there’s something in particular I can check I’ll be happy to do so.

@fedora

@pchaigno I’m doing the full kube-proxy replacement too, fwiw

@fedora

@genebean @fedora I'd check the agent logs.

@pchaigno Here is what I see on @fedora 40.20240701.3.0 (CoreOS) I also found this in the system logs:

Jul 29 01:31:23 fcos-vm1 audit[4196]: AVC avc: denied { transition } for pid=4196 comm="runc:[2:INIT]" path="/usr/bin/dash" dev="overlay" ino=12624438 scontext=system_u:system_r:unconfined_service_t:s0 tcontext=system_u:system_r:spc_t:s0 tclass=process permissive=0

@genebean @fedora kubectl describe may give you better information on why the agent is crashing. The audit log doesn't seem related to me.
@pchaigno That shows "Created container mount-cgroup" followed by "Back-off restarting failed container mount-cgroup in pod cilium-qh4vw_kube-system"
@genebean Ok. We're probably not going to solve it on Mastodon 😅 I'd open an issue on GitHub with a sysdump. Feel free to cc me (same handle).
Cilium Agent fails on Fedora Core 40 if SELinux is enabled · Issue #34068 · cilium/cilium

Is there an existing issue for this? I have searched the existing issues Version higher than v1.16.0 and lower than v1.17.0 What happened? Unless I disable SELinux, I cannot get the cilium pod to s...

GitHub