Running Podman  in production for years now, and I don't miss the Docker daemon one bit.

I just published a deep dive on managing OCI containers the Unix way: daemonless, rootless, and natively integrated with systemd via Quadlets.

I cover:
- Real secrets management
- Auto-updates via systemd timers
- The Docker compatibility layer

This is the guide I wish I had when making the switch.

Read it here: https://blog.hofstede.it/podman-in-production-quadlets-secrets-auto-updates-and-docker-compatibility/

#Podman #Linux #DevOps #Systemd #Homelab #Sysadmin #Containers

Podman in Production: Quadlets, Secrets, Auto-Updates, and Docker Compatibility

An opinionated production-ops guide to Podman on Linux servers - why I prefer it over Docker, how Quadlets replace Compose files, and practical patterns from real deployments including secrets mana...

Larvitz Blog
@Larvitz I'm in a similar place with podman and I love that guide. You introduce everything I like about podman and quadlets in such a nice way, thank you!
@Larvitz been wondering about switching to either podman or libvirt for the plain LXC things I have on a server, because some other admins are not used to it and want GUI tools, but I suppose that means migrating… ?
@mmu_man For GUIs, there's Podman Desktop (https://podman-desktop.io) and also the web-based Cockpit Client for Podman (https://github.com/cockpit-project/cockpit-podman)
Podman Desktop - Containers and Kubernetes | Podman Desktop

Podman Desktop - An open source graphical tool for developing on containers and Kubernetes

@Larvitz yeah but it won't keep containers as is I guess, so I won't be able to keep using lxc commands directly…
@mmu_man Yeah for sure. LXC and Podman are different technologies.
@[email protected] #proxmox have a gui and can spin off lxc container no ?
@oldsysops not sure, I'll have to check that
@Larvitz this is awesome, thanks for putting it together. I've been using podman for a few years and got started after generating the unit files from running containers. I wish I had a guide like this for getting started.
Quick question if you don't mind; I have a separate container running user and put the unit files in ~/.config/systemd/user/ instead. You suggest ~/.config/containers/systemd/ which seems to make sense as a path but I was hoping to understand the difference better. Could you please point me to a resource?

@shom

~/.config/systemd/user/ is for systmd units (podman generate systemd). That was the old way to do it.

~/.config/containers/systemd/ is for Quadlet files, the modern way to describe containers declaratively:

https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html

Quadlets files are similar to Systemd units and describe a container with all it's attributes.

podman-systemd.unit — Podman documentation

@Larvitz ahhhhh perfect, this made it click finally. I was just generically describing how to run an application (happened to be a container) and Quadlets use the unit file approach but describes the container itself (which I read in the unit file but didn't make the connection). Thanks so much!!

@Larvitz another person of culture I see…/me tips hat

I’ve been operating with a mixture of quadlets and manual podman-compose containers for quite some time. I’ve found compatibility issues with some projects, but I decided those do not justify switching to docker. There’s also an annoying race condition with CNI coming up before networkmanager, but manual fix is easy enough for those times.

Great blog post! Thanks

@andrew That blog article took me the longest of them all. A first draft had been lingering in my blog's git repo since November last year, and I went through numerous rewrites of various parts until I found them good enough. Today, I added the final paragraph about Ansible and decided to publish it before I end up waiting another 6 months 😂
@andrew @Larvitz I had a similar issue with NetworkManager and WLAN on my Laptop.
NetworkManager gets an IP-address after KDE login, but the quadlets starts directly after boot.
The dependency on podman-user-wait-network-online.service didn't helps.
So I added an "ExecStartPre" in the network quadlet, which helped for me.
```
[Service]
ExecStartPre=/bin/sh -c 'until hostname --ip-addresses | grep --quiet \"192.168.\"; do sleep 10; echo \"%n: Wait for IPv4 home network address ...\"; done'
```
@Larvitz Maybe you could add the hint, that automatic starting of rootless quadlets needs an user, where lingering is enabled.
It can be found at the examples.
https://docs.podman.io/en/latest/markdown/podman-system-service.1.html
```
loginctl enable-linger <USER>
```
podman-system-service — Podman documentation

@Larvitz I love podman too, but recently I’ve been wondering about running rootless containers that aren’t tied to a specific host user. I’ve posted this as a discussion topic here - thoughts? https://github.com/containers/podman/discussions/28445
Rootless Podman with Ephemeral User · containers podman · Discussion #28445

I love the idea of rootless podman, running as quadlets, as I want to protect against container escape gaining root. That said, I also want to protect against container escape having access to my u...

GitHub
@Larvitz Thanks for the article.
I never went beyond podman compose because I couldn't really find beginner-friendly examples on how to use Quadlets in production, so this is a great reference on how to get started.