Niclas

After quite some time, I finally have all the pieces in place. Over the last 30 minutes, I’ve set up one of my servers from scratch. Here are some key changes:
- Reverse Proxy: Nginx with Modsecurity (WAF)
- Container Isolation: Every container runs in a seperate linux user
- Podman Quadlet: I rewrote all my compose stacks into quadlet files - now all containers are starting probably after reboot 🥳
- Grafana: Grafana's configuration is no managed by Opentofu which provitions at the moment the datasources (Grafana Loki and Prometheus) as well as the dashboards.
- Server hardening: Improved ssh configuration, firewall, permissions in general on this host
- Ansible: Everything is powered by ansible
- Certbot: Use wildcard certificates for my domains / subdomains for easier renew process
- Backups: All those services have proper backups configured which are timed with systemd timer and are replicated into my local homelab.
- Services that are running at the moment
- Grafana
- Prometheus
- Grafana Loki
- Grafana Alloy
- GitLab Runner
- some other services that I wanna migrate to this server

#homelab #sysadmin #linux #ansible #automation #devsecops #selfhosting #declarative #gitops #monitoring

Apr 12 at 8:39pmWeb
Show threadravenlb Apr 12
@niclas quality! awesome work
Show threadVladyslav KApr 12
@niclas how did you manage to run each container as a separate user? I have a few containers in my stack that have to run as uid 1000 (like homepage for example), what do you do in this situation?
Show threadNiclasApr 12

@hotsause If I take Grafana as an example:
1. Create service account user "grafana"
2. su grafana
3. Copy the podman quadlet file to the folder"~/.config/containers/systemd" so the podman quadlet runs rootless as the user "grafana"
4. systemctl --user daemon-reload
5. systemctl --user start grafana

One disadvantage is that "podman ps" no longer shows the status of all running containers of this host, since the containers are no longer running under the same user.

Edit: Also enable linger for the user, otherwise you get warnings and the user is not able to autostart the containers on boot

Show threadVladyslav KApr 12
@niclas ah, thanks for the explanation! I'm currently running everything in docker but I'll have a closer look at podman now
Show threadLuke EbersoleApr 14

@niclas I had never thought about adding a WAF to nginx in my homelab.

I spent a little bit of time today and got ModSecurity added to it, plus a bit of tuning and defining exclusions for a couple apps.

Thanks for mentioning it! This feels like a good addition to the stack.

Show threadNiclasApr 14
@soleluke I'm glad I was able to give you a little inspiration. But having something like a WAF usually just means more work. I often ended up shooting myself in the foot.