Jeffrey PerkelMay 1, 2023
Today @Nature, by me: why you might want to do your computing work inside computational environments (e.g., conda, renv). With @ctb @fertiglab @minecr @benmarwick et al. https://www.nature.com/articles/d41586-023-01469-0
The sleight-of-hand trick that can simplify scientific computing

Computational environments and the tools to manage them can help researchers to deliver code that is reproducible, documented and shareable.

Show threadKonrad HinsenMay 1, 2023

@jperkel
"Manage your environments" is definitely good advice. But to share environments with others, or to archive them for future use, the environments themselves must be reproducible. And to ensure that code runs reproducibly inside an environment, the latter must be containerized.

Conda environments are neither reproducible nor containerized. The price to pay for cross-platform and no-root. A compromise. Would have been nice to point that out!

@Nature @ctb @fertiglab @minecr @benmarwick

Show threadKonrad HinsenMay 1, 2023

@jperkel

You can have reproducible and containerized environments, using #Guix. The price to pay: #Linux only, and the manager (Guix) can only be installed by an administrator. It's a different compromise.

@Nature @ctb @fertiglab @minecr @benmarwick

Show threadRaito BezariusMay 1, 2023
@khinsen @jperkel @Nature @ctb @fertiglab @minecr @benmarwick (it does not have to be this way: cross platform is in reach either for Guix or Nix, same for no-root in certain cases which could be most cases.)
Show threadKonrad HinsenMay 2, 2023

@raito

No-root is in principle possible for #Nix and #Guix under #Linux, but not available today.

Cross-platform reproducibility is not possible, period. You can run #Nix under #macOS as well as under #Linux, but you get reproducibility only within each platform.

Moreover, reproducibility with #Nix under #macOS is limited because #Nix depends on code it cannot control (managed by Apple).

@jperkel @Nature @ctb @fertiglab @minecr @benmarwick

Show threadAlper Çuğun-GscheidelMay 5, 2023

@khinsen @raito @jperkel @Nature @ctb @fertiglab @minecr @benmarwick It's quite something. None of the Nix+Docker tutorials work on a Mac.

So what's the point of this thing?

Show threadRaito Bezarius
@alper @khinsen @jperkel @Nature @ctb @fertiglab @minecr @benmarwick While I imagine you had indeed bad experience, I know that a lot of people and companies are running Nix+Docker, so please don't hesitate to report the bugs or your situation so we can look into it. It's the only real way to make this thing have a point for you too.
May 5, 2023 at 12:04pmWeb
Show threadAlper Çuğun-GscheidelMay 5, 2023

@raito @khinsen @jperkel @Nature @ctb @fertiglab @minecr @benmarwick Asking on the Discourse about cross-compilation I'm concluding that it's mostly theoretical…

And on my Mac I'll be stuck with two cross-compiles that both need to work:
- Compile from macOS to Linux container running on my M1
- Compile from macOS to Linux container running on target architecture (x86 or something)

Show threadRaito BezariusMay 5, 2023
@alper @khinsen @jperkel @Nature @ctb @fertiglab @minecr @benmarwick I don't really use Docker but can you even run a cross compiled Docker container on a different arch? What's your usecase? Building golden containers for many arches?
Show threadAlper Çuğun-GscheidelMay 5, 2023

@raito @khinsen @jperkel @Nature @ctb @fertiglab @minecr @benmarwick At this point I'd kill for a fat binary.

The job to be done is: I want to run a command in the Rust project directory on my Mac and then have something that I can deploy somewhere.