Building a #Linuxulator userland from source on #FreeBSD -- I'm not giving up yet. But, starting over 😞

This time, I try building some "complete" #GNU cross-toolchain targeting #Linux first in some dedicated prefix. Maybe this will finally work out. 😜

I really don't get it. I *had* a working port building "some" (with lots of missing features, e.g. threads) cross-#gcc (build: #FreeBSD, host: #Linux). Now I try to build this same port in some separate PREFIX and, it fails 😞

Progress again. 

GCC builds fine when I previously install binutils in the exactly *same* prefix!

Ok, this means I'll need an extra "bootstrapping" port for binutils as well...

Time for bed!

Been a week now I'm working on this idea ...

So far, whenever something looked like good progress, I ran into some dead end one or two ports later.

Should probably decide for some #timebox I will set myself for this. If I can't at least come up with a *working* cross (#FreeBSD -> #Linux) #GNU toolchain within that timebox, the idea should be abandoned as unfeasible 😞

@zirias Just a stupid question here. πŸ˜‰ What exactly is the purpose of the project? Because if it is just cross-compiler from #FreeBSD to #Linux, you can use:

1. Linuxator and GCC there. That could be literally native compilation.

2. Get the proper libraries and use clang cross-compilation abilities. That solution is more native for FreeBSD and require less disk space.

@thindil Background is that #FreeBSD's #Linuxulator userland is in desperate need of an upgrade. The "linux-c7" userland is both too antiquated and incomplete for a lot of #Linux software you might want to run on FreeBSD.

Purpose of my project is to evaluate an approach of building a new userland from source instead of repackaging another Linux #distribution. The rough plan is to create a feature-complete native #GNU toolchain for Linux and then use that to build the Linuxulator userland packages.

The benefit I would *hope* for is that this could allow to relatively quickly create new packages for the Linux userland from existing FreeBSD ports (maybe using "slave ports"), but this has to be proven in practice.

@zirias I agree with upgrade. CentOS is going down in the next year and all that infrastructure will go down with it too.
The idea is interesting, just I'm not sure if not too complicated. πŸ˜‰ Each Linux distro is almost that same different each other, like BSD systems. I would prefer to stick with one distro and just use its internal packaging manager to keep everything up to date. I have bad memories with creating Linuxator packages. 😁 Eventually, you could look at Linux From Scratch.

@thindil I'm actually using #LFS as a reference to get around the weirdest issues "bootstrapping" some #GNU toolchain at all. But of course it can't be used as is, LFS expects to be built on some Linux system and even uses #chroot to build some packages, definitely impossible using #FreeBSD #ports.

I certainly don't want two package managers on my system. Plus, a #Linuxulator userland should only install what's *different* from the normal FreeBSD userland, because /compat/linux works as an "overlay".

Trying to create new ports/packages for the Linuxulator userland is cumbersome, first you have to rely on what the repackaged distro offers, second you have to understand the special repackaging magic used.... that's exactly what I *hope* to improve by just building from source. No, I'm not convinced myself yet this is a good idea, I have to try!

@zirias True, the only way to know do it's a good idea is to test it. πŸ˜‰ And before I forgot again, there is also Chimera Linux: https://chimera-linux.org/ Maybe it will be helpful.

Linuxator isn't exactly an "overlay". #FreeBSD has implemented two APIs in the kernel and can detect, on a binary execution, via magic bits, which API to use. Thus you can't mix, for example FreeBSD library and Linux binary. We need to deliver the whole dependencies chain for that binary.

Chimera Linux

Chimera Linux

Chimera Linux
@thindil I said /compat/linux is an overlay, not #Linuxulator itself. Linuxulator basically consists of the kernel's "Linux personality" (Linux syscalls exposed when the running #ELF binary is a Linux binary) *and* this overlay for the userland. When running a Linux binary, any attempt to access some file is first tried with /compat/linux prepended to the path, only when nothing is found there, it's tried in /.
@zirias Ah, my bad then. 😊 Thank you for the correction. But still, we need to build the whole chain, unless there's a way to mix the different ABIs.

@thindil I can certainly use #FreeBSD "tools" (gmake, bison, gettext, whatever) for building this cross-toolchain. But indeed, for libraries, they need to be built targeting #Linux. And because #GCC with the full feature set needs e.g.#glibc when targeting Linux, but then you need GCC to *build* glibc, I need at least some "bootstrapping" ports. It's really a mess.

Once I have a full-featured cross GCC targeting Linux ready, I'll stop for a while to party πŸ˜‚

@zirias A very long party I suggest. 🀣 The problem is that there's no binary compatibility between distributions, even between older versions of the same distro. That's why even Microsoft supports only selected distros, and not Linux at all. That's why I prefer to use existing distro in Linuxator. After all, you are going to recreate the whole Ubuntu. That's a huge task. πŸ˜€

@thindil Oh there *is* binary compatibility for sure. The #Linux kernel typically doesn't break its userspace-facing #ABI. #Glibc and #GCC's libstdc++ use symbol versioning to provide backwards compatibility.

The issue starts with all the other libs, there's no standard for some "base" GNU/Linux system. That's where all these (IMHO damn broken) ideas like #AppImage, #Flatpak etc come from. Of course, you could just link statically instead, seems people don't get that any more πŸ™ˆ

Anyways, quite some binary #Linux software will work "anywhere" as long as the required libs are not too old (looking e.g. at browsers...). And having a #Linuxulator userland built from source *should* enable you to just add ports for missing libraries. Well, in theory πŸ™ˆ

@zirias I wish that was true. 🀣One of things which I learned as a someone who was creating software for Linux is that binary compiled on Fedora will usually not work on Ubuntu.
1. Contrary to popular believes, Linux pretty often breaks compatibility, mostly by putting some API calls hidden by switches.
2. There's no binary compatibility between GNU lib C releases. Theoretically you have right, but practically it not always works.
3. There's no more standard for Linux base system. LSB is dead.
@thindil
1. The "#Linux" ABI is the syscalls (and a few other things) offered by the kernel ... yes, this thing actually *called* "Linux". It doesn't break, as far as I know.
2. There is. But only in one direction of course. Something built against some newer #glibc won't work on a system with an older version.
3. Yes, definitely yes πŸ™„

@zirias 1. From my experience, breaks. Reason: distros use different build flags. And whole piles of patches. 😁
2. Unless there's some security problems, then they break backward compatibility too. 🀣

Btw, I'm sure that I forgot to mention that there's no standard C library in Linux world? IIRC, there are 3 or 4 incompatible ones. The two major are GNU lib C and musl. 🀯 Ubuntu ships them both. πŸ˜‚

@thindil
1. This doesn't affect the public #ABI of the kernel. Do you have any example of breakage here? πŸ€” Don't get me wrong, I'm really not a fan of the chaos GNU/Linux actually is, but this is still hard to believe.
2. Possible. I don't know any example though ....

Regarding the "standard" #libc, if we're talking about "GNU/Linux", this will always be #glibc. AFAIK, any #Linuxulator userland ever shipped in #FreeBSD ports used it. I plan to do the same trying to build this mess from source πŸ˜‚

BTW, my "lxcross-glibc" package seems to work, and it seems I'm able to build some *almost* complete cross #GCC using it right now ... still need the --disable-libsanitizer configure flag πŸ€”

@zirias 1. Mostly from work. An old binary was depending on some old, insecure syscalls which were disabled by default in later versions of the kernel. Sure, rebuild kernel was fixed the problem. But it's hard to call that: "we don't break API". πŸ˜‰ Sometimes I have feeling, that FreeBSD has better compatibility with Linux than all these distros.

And nice result, congratulations. πŸ₯³ Let's hope the best with the rest. πŸ™‚