Mastodawn

Björkus "No time_t to Die" Dorkus Apr 6

A r e y o u r e a d y t o h a v e s o m e f u n ?

:3

Björkus "No time_t to Die" Dorkus Apr 6

"This is going to destroy my build system"

Nope! It's able to determine everything that will be processed by the Phase 7 compile-time-computed strings by Phase 4, and presents all of that information through already-available means, meaning CMake/build2/meson/make/ninja/etc. can all understand the dependency chain here natively!

Björkus "No time_t to Die" Dorkus Apr 6

Ultimately, this means we can process files -- recursively -- at compile-time, meaning that rather than embedded shaders with #includes that can't be touched, we can process those includes and make true single blobs without extra build steps.

compile-time python with imports is VERY possible.

Martin Uecker Apr 6

@thephd Cool, but why?

Björkus "No time_t to Die" Dorkus Apr 6

@uecker C++ has capabilities that can make great use of this. A perfect example is using (Generative) reflection to generate, at compile-time, the perfect FFI that maps to Python, or Lua, or JavaScript, with all of the utility that comes from having it mapped perfectly to C(++) interfaces and fully type-checked while always being generated directly from said Lua or Python or JavaScript source code.

This also applies to things like e.g. Rust and C++ interop, which has also been the topic of discussion and monetary investment. (Not that they're paying me; I wish they would, I could do a lot more for them than just std::embed.)

C doesn't have the systems in place to do things like this, so in most cases they'd just have to rely on the usual techniques used today: code generators, hand-written parsers, fresh data files and description files used to drive code generation (like e.g. SWIG). Certainly not bad, but not nearly as "automated luxury FFI" as C++ can make it.

Martin Uecker Apr 6

@thephd What I do not understand is why the extra build step is a problem.

@uecker @thephd https://mastodon.online/@manx/116261044523322289

@uecker @thephd This exact extra backwards cyclic dependency is probably the single most significant reason why building C and C++ code is hard.

Martin Uecker Apr 6

@manx @thephd Can you explain what you mean? I do not generally find building C code hard - not even when it runs code (which it rarely does).

Martin Uecker Apr 6

@manx @thephd You could turn this around "Do not feel proud about your clever compile-time meta programming. You should feel ashamed that you have been tricked into using your compiler as a poor interpreter."

@uecker @thephd Well, I am happy to disagree here.

No cyclic dependency is IMHO always better than any cyclic dependency.

This whole problem tends to always bite especially hard when you need to port from one build system to another (because the existing build system does not support your platform or toolchain), like trying to use an Autoconf project with MSVC.

Martin Uecker Apr 6

@manx @thephd I am not entirely sure I understand what is "cyclic" in this context. To me one reason people try to make languages into complete frameworks is that it seems an easier approach to standardize things at the language level than try to harmonize the diverse environments. And this is a valid argument, but not really a technical one.

@uecker @thephd

The build system generates input for the compiler. The compiler generates input for the build system (include dependency graph). This is cyclic, and by itself causes all sorts of complications (which are all solved in current build systems, but which IMHO makes all of them more complex than they should need to be).

This is similar to what Autoconf does (which also induces a cyclic dependency).

It is also remotely similar to what C++ Modules require (but there it is way worse).

Martin Uecker Apr 6

@manx @thephd Thanks! I guess in a reasonable approach you would only have two stages. You build a tool that creates the code you then run to create he code you ultimately compile. This is not fundamentally different to what happens with constexpr. Just there is all part of an in transparent and slow monolithic C++ compiler. The complexity is not gone, but just hidden. But this is not better IMHO.

@uecker @thephd The reasonable approach is only have two stages, I totally agree. But this is not how any build system I know models it. They all support the complexity of arbitrary graphs, which offers enormous flexibility at the cost of unbounded complexity. Unless this gets simplified in one way or another, any build system standardization is IMHO very unlikely to happen.
...

@uecker @thephd ...
I think moving complexity into the language proper is the right approach. It frees developers from learning multiple "languages", and it simplifies porting to other platforms that require different build systems.

Martin Uecker Apr 6

@manx @thephd Maybe you are right, but my perspective after observing the industry for a while is this: Complexity always grows until things suck too much and then people build alternatives and the cycle starts again. At the same time, if some part does not suck yet because it is relatively small and simple, people think that complexity needs to be moved there from the other parts nobody wants to touch. Couldn't this be part of what is happening here?

@uecker @thephd Maybe partly.
I still think that removing cyclic dependencies is something to strive for in general. I see clear practical advantages for use cases that I encounter.
However, I also see problems arising from implementing too much in constexpr, but for me, the advantages from simplifying the build outweigh them clearly here. Other people might weight things differently.
...

@uecker @thephd ...
I also still think that the C/C++ build system ecosystem only has the chance of getting addressed IFF we take away responsibility from them and simplify them first. Standardizing the complexity that currently exist across all of them would just result in an even more complex generalized "solution" that really does not help at all.

@uecker @thephd
make
ninja
cmake
msbuild
premake5
autotools
vcpkg
conan
apt
yum
rpm
...

Having a reverse dependency on the build system multiplies this nonsense, especially when cross-compiling and interacting between different build systems.

Compare this to any (really any) other language.

This is really not fixable if every build system invents its own mechanism for generating code at build time, thereby locking any project into build system vendor specific mechanisms.

Martin Uecker Apr 6

@manx I agree that the lack of standardization of build systems is a problem.

Martin Uecker Apr 6

@manx But I think it partially also reflects the size of the ecosystem. I really do not like many other languages that provide a framework that locks you into a specific way of doing things.

@uecker @thephd If you are cross-compiling, you are running the interpreter and code generator on the build host instead of on the build target, which means, if it requires any platform-specific knowledge (like the size and alignment of fundamental types), you have to manually duplicate this knowledge into your custom interpreter.

Inside the target language itself, this information is naturally trivially available.

Martin Uecker Apr 6

@manx @thephd Cross-compilation is a good argument, but it is still not clear to me that this would not be better solved at a different level.

@uecker @thephd I am also not 100% sure that doing this in constexpr context is necessarily the best approach (mainly because debugging constexpr code is a completely unsolved problem), but it is frankly the best approach I have seen offered so far, and IMHO far better than involving the build system in any way.

Martin Uecker Apr 6

@manx @thephd IMHO if people are unhappy about build systems (which seems to be the case and there are certainly good reasons), one should think hard why this is the case and fix it at that level. This is hard, because it is likely not just a technical issue, but lack of standardization and the need to support annoying environments that intentionally wanted to be different.

@uecker @thephd I agree, and I think reducing the scope and responsibility of what build systems need to do, is a good and necessary step into that direction.