a few people in the replies are asking why someone might want to use nix as an OS / to manage your python packages / to manage your config files.
does anyone want to share their reasons for doing those things? I don't really know.
(please give examples of some specific problems Nix solves for you and what you like about it, don't just say "it's reproducible/declarative” :) )
lots of great answers to this “can you explain why you use nix to manage your config with specific examples" post https://social.jvns.ca/@b0rk/111409710819501159
not going to try to summarize but I loved how folks talked about their feelings & problems they wanted to solve (“it feels less cluttered", "I'm scared of accidentally breaking stuff & nix helps me feel safer”) instead of why nix is Objectively The Best. great responses all around
a few people in the replies are asking why someone might want to use nix as an OS / to manage your python packages / to manage your config files. does anyone want to share their reasons for doing those things? I don't really know. (please give examples of some specific problems Nix solves for you and what you like about it, don't just say "it's reproducible/declarative” :) )
@b0rk Nix has made computing fun again!
I run NixOS on my laptop, desktop, NAS, home server, and multiple VPSs. There are a couple reasons why I switched away from other OSs and tools:
- Everything is configured with code, stored in Git
- This code can be shared easily between machines
- Upgrades to machines are incredibly safe, with the safety net of rollbacks or even booting an older config directly!
- One and done! Now that I have these configs, OS install is a single command.
@b0rk I have however found that there are a number of things that you'll end up implementing yourself that are necessary for managing these more complex setups. There are a handful of tools in the ecosystem to try and make this easier like flake-utils, flake-utils-plus, flake-parts, etc. They all try to solve different problems in different ways, Nix is still finding its footing here. Though I'm happy with Snowfall Lib :)
@b0rk in case anyone is curious what this setup looks like, it's all on GitHub!
This repo has all of my system configs and some packages: https://github.com/jakehamilton/config
I made my own custom neovim that comes with my config and plugins: https://github.com/jakehamilton/neovim
I also did the same for tmux: https://github.com/jakehamilton/tmux
@b0rk I’m somebody who does a lot of exploratory programming and Nix is awful for that because stuff is always kinda broken. That’s why I use `nix develop` for a couple of projects (which can be nice) but for everything global on my system I’m still mostly on brew.
Not an answer to your question but a counterfactual from somebody who looked into this.
@b0rk Two things that stood out for me:
override away. Need to build a master branch of a package? It's one overrideAttributes away (when they work, of course!). Compared to my past experience with apt/pacman/homebrew this has probably changed how I manage software on my machine... (did I mention they sync too?)@b0rk If I’m deploying machines to the cloud with Terraform and Ansible, I need the right versions of those tools for my project. Ansible can be installed via pip in a virtual environment, but Terraform is a binary. I might also need some CLI tools to play with the cloud API (s3_client, openstack_client, …) that might not all be Python tools.
So I’d use Nix to have something similar to a Python virtual environment, but for any tool that I need for a given project.
@b0rk The premise being that different projects might use different versions of the tools.
I learned about Nix from a freelance developer. For him, it’s precious, because every customer needs their versions of the tools.
@b0rk I find it compelling for cases where you are doing any of:
- FFI
- linking to system dependencies
- depending on stuff that are appropriate for your use case, but that have a complex dependency footprint themselves, and you don't really care about it beyond needing to satisfy so you can Do Your Thing
I face these concerns working in Elixir/Rust, but I think those are commonly encountered in Python ecosystem too. Things like CUDA or TensorFlow might be popular examples of the third point.
@b0rk these are sort of pathological examples where a single programming language's dependency handling is not totally sufficient to fight off complexity, bitrot, etc. Going to a project months later may not work, & might be sensitive to other things you've done since then.
Having the flake content in question versioned right alongside your project is a super power, because you can enter your dev env and satisfy prerequisites as the project understood them at any point in the commit history.
I'm also curious about the use case of nix as a python packager.
On the one hand, I like Nix.
On the other, it feels...like adding Yet Another way to manage python dependencies.
I saw myself having to ~~torture~~ teach my poor colleagues about nix and gave up on the idea.
It just feels saner to use pip/poetry; ie: a program specific to this programming language.
@b0rk I use NixOS and home-manager. The number one reason I like this setup is that I have two places (my global nixos config and my home-manager config) that together completely explain why my system is the way it is and how it got this way, and those config files have comments and git history. Why do I have obs installed? Oh, right, it was for when I was messing around with setting up a virtual camera for streaming my desktop. What set of packages and incantations did I end up with for getting power management working well on my laptop? They're not split across several packages' config files, they're all in one place. Config files can also reference packages, so you don't end up with a situation in which you put stuff in your dotfiles that no longer works because you've uninstalled something: for example, if I put something in my vimrc that's going to call LaTeX or pandoc, I can reference those packages directly in vimrc so that they are always there regardless of whether I have them installed "globally" or not.
There's also this magic, which is life-changing: https://discuss.systems/@ricci/110441452541988807
Attached: 1 image Yesterday I discovered comma https://github.com/nix-community/comma and it is going to change my life forever, I think. It allows you to run basically anything on #NixOS without installing it. (This is possible without comma, but comma makes it trivial). This is amazing for those things that you need to use like once a year.
@b0rk I have ADHD and have maintained a weird soup of internet-hosted stuff served from a VPS for decades at this point.
What used to happen before I switched to Nix is that, when I needed to upgrade, I would spin up a new VPS, half-migrate to it, and then never finish, leaving a few things running on the old machine. In the process I'd constantly find I'd forgetten custom configuration tweaks in dusty subdirectories of /etc and end up leaving things half-broken for months before I caught it.
@b0rk Eventually it hit a point where my oldest VPS was running a version of Debian that was so out of date that I _couldn't SSH into it_. I spent two very intense weeks finally migrating everything to NixOS.
NixOS allows me to keep _all_ of my server's configuration in one place. I can easily rebuild my server from scratch. I have a VirtualBox VM that I can test upgrades out on before I push them to the real server, and I can have real confidence that if it works there, it'll work for real.
@b0rk For me it solves these problems:
One way to do things: no longer juggling pyenv, rbenv, homebrew, and remembering how to sync configs for my various apps. All my machines, dev environments, and (mostly) all my apps are configured with nix with everything in a single git repo.
Less conflicts: no longer have issues with one project's setup messing up another. My base machine is pretty bare, with isolated dev environments via nix-shell + direnv (without needing to dockerize everything!).
@b0rk so I've been thinking about this for a little bit, and it's still kinda unformed, but I think the main reason is *emotional* rather than technical.
For my macOS computers, I have 1.6k lines of setup script. It tries to do everything -- configure preferences, set up the dock, use bitwarden's CLI to generate the passwords for GPG keys, which it adds to GitHub and the keychain without interaction, etc. This has not saved me time, I have only ever used it on two computers. But it gives a feeling of safety. For the most part, I could wipe this laptop and the setup wouldn't be terrible.
I've been using NixOS for a month or so for a similar promise. If the server falls over, I don't really have to remember what I did. (Okay well this tool I think needed certain ports open, and then that tool needed to be told about the port... and how did I even install it in the first place?) In that way, it feels less precarious. (In other ways, more, due to the instability of the nix ecosystem.)
I always kind of feel like computers I manage are only fine right now by the grace of god, and any update could break them, or *I* could break them and then forget how to get back to where I was. Changes in NixOS feel both more and less permanent. More in that I've succeeded in pinning down an aspect of the system so that it can't wiggle away, and less in that because everything is version controlled, I can experiment however I want and always be able to go back.
As for specific problems solved: the *arr stack has community written configuration for something called quality profiles. This is synced from the community to the software using a tool called recyclarr. Recyclarr needs to be run regularly, so I made a systemd unit. It also needs a config file, which it expects to be in one of several places. Unfortunately no matter where it is (/etc), it expects a second config file with secrets to be in a ~/.config/ directory. This took me a little while to figure out. Now I can say the problem is solved. Everything is written down exactly as it should be, and I can even keep the secrets in the git repo, which get decrypted by the server's keys. I don't have to remember where I put any of the files, or remember that I need to write the systemd unit if I'm reinstalling or helping someone else get set up. I can change the files by editing my nixos config instead of going "wait I thought the secrets were over here, where are they??" The existing solution for solving this uses containers for all the parts of the system, which I try to avoid when possible.
@b0rk I have switched to #NixOS on my desktop a few months ago, wrote a blog post about it too, but to summarize it here, my major reasons for using NixOS - as it seems to be a common theme in this thread - is organization. I had over two decades of cruft on my workstation, it was impossible to figure out what's where and why, and how, because it was built ad-hoc, and things were all over the place.
I had stuff installed from official Debian repositories. I had packages from upstreams. I have built some of my own. I had some in /usr/local, and some in ~/install, with a few Docker containers, and flatpaks sprinkled on top.
With NixOS, my entire configuration is in one, well managed, over-documented place. This lets me, a rather unorganized person, keep his system sane. So much so, that I could make both / and /home tmpfs, they're rebuilt on every single boot, and whatever I did not explicitly mark as persistent, is lost. This way the amount of cruft that can accumulate is greatly reduced, and I always have that freshly installed system feel.
It also lets me explore freely: want to try something? nix shell nixpkgs#foo, try it. Done with it? Just exit the shell, and it will get grabage collected eventually. No more forgetting to uninstall things! On top of that, with the combination of Nix + dotenv makes it very convenient for me to have low-effort & high-reward development environments. I make a flake with all the dependencies and whatnot (including stuff like starting up, say, a combination of minio + writefreely + caddy for local testing of a project, with one simple command: nix run .#run-local-servers), my editor (#Emacs) picks it up, and I can work as if I was working against production, because the tooling automatically set things up for me.
This lets me make sense of my configuration, too, because my NixOS config inclues the configuration for almost every program I use (the few where it doesn't, will eventually move into the NixOS config too - I just didn't get there yet).
With the combination of Org Mode + Org Roam + NixOS, my configuration is a very verbose knowledge base, organized in a way that makes sense to me, with related things either documented together, or cross-linked aggressively.
I'm not a big fan of Nix, the language (hence my use of Org to organize my config, rather than organizing it via Nix's own stuff), but I am a huge fan of what it allows me to do. Being a declarative configuration is a key part of this, that enables the whole Org setup in my case, where I can just tangle <<foo>> out in one place, and then target that references from a zillion other places, and the ordering doesn't matter.
Phew. This became a huge wall of text. I'll go and shut up now, sorry.
@b0rk my old config's readme (https://github.com/yusefnapora/nixos-system-flake) has some rambling at the end that explains what I get out of it.
Short version is that I start out hacking up some duct-tape script, get it working by making a bunch of ad-hoc changes to my environment, and come back in a year having completely forgotten everything. With nix I can forget all the details and still set it up on a new machine.
- I can actually see everything I've manually installed in one place, which helps me remember CLI tools I added but forgot were lying around (e.g. I forgot I had inferno installed)
- nixpkgs is really big and has CLI tools that aren't easy to get elsewhere, outside of direct binary downloads that are hard to update (haven't had to manually download binaries in ages)
- patching / tweaking packages is really easy, so I can inject extra deps into a Python env or downgrade one thing etc etc
Christian Theune has been working in IT since the late-1990s, and his experience gives him a clear understanding of what it takes to maintain applications and systems over the long term. In this edition of Nix in the Wild, he shares how they use NixOS to support long-term customers.
@b0rk I've always felt funny having multiple sources of packages on my system (apt,cargo,snap,curled things). Nixos makes it more coherent for me.
As for projects managed by nix with flake it's an enablement strategy. New developers don't have to install weird packages on different platforms (Mac, different distributions) just nix develop (automated with direnv) from being able to run whatever ungodly software we have written.
TL;DR I only use it as package manager. I even prefer it over apt to install dev tools. On Linux I also use to install apps like firefox, but not on macOS.
I am not sure this is what you are asking, so ignore me if it is not:
I only use nix as package manager, and I am about to give up on home-manager because it is breaking for some reason.
I avoid building packages myself as much as I can or using flakes.
I also don’t use it on servers because it requires a LOT of memory.
A biggest example I have for me is that I have a complicated email setup and Nix enable me to abstract over it in a simple fashion and have reusable code to generate all kind of relevant files for NeoMutt, mbsync, notification system, in an uniform fashion: https://github.com/RaitoBezarius/nixos-home/blob/master/emails/default.nix
I want a proper configuration language for managing advanced configurations for my own system uniformly.
I have the same for my email servers, etc.
In the end, I can do 10x what I could with other distros.
@flakm If you use Home-Manager, you'll have plenty of knobs to play with depending on what you use for your email setup, e.g. NeoMutt, Mutt, Himalaya, etc.
I'd recommend structuring your configuration to separate
(a) high level information (your email accounts, etc.)
(b) implementation of your email setup (neomutt, mbsync, etc.)
I am not aware of guide including NixOS but I think you can mix'n'match very easily a lot of guides on NeoMutt, mbsync, etc. And feel free to poke me on the matter!
@raito that's the thing I have a pretty nice flaked nix with modular home manager. I develop in a terminal and love it. But for email I use thunderbird and I'd like to move it to a more terminal based flow. I was hoping to hear there is some time worthy blog/tutorial/YouTube channel.
I'll try to set it up, thanks for the recommendations!
@b0rk The amount of work to add a custom package / change version is often trivial, so I've got some custom patches and extra packages included without needing a separate build environment and package repo.
Having the same config installing related software is convenient. For example: The port in caddy and the port in service config come from the same place. The AWS profiles and the aliases using them get generated from the same list.
Generating a container image from a part of the system.
@b0rk I started off making a sever setup that I could replicate easily. With home setups things are cheap/brittle, so being able to rebuild on new hardware was key. I wanted Ansible but baked in to the OS.
I ended up with a configuration that I could use to build a server, very happy about that, but also got the same flake working on Mac with home-manager to configure my user environment.
I got distracted on my pilgrimage, though. Next step is per project envs.
@b0rk I wrote up why I use Nix to manage my project development environments, but with a twist: I use Devbox to make using Nix extremely easy.
https://alan.norbauer.com/articles/devbox-intro.
Your post inspired me to remove several uses of the word "reproducible" from my write-up so thanks! 😅