Mastodawn

Björkus "No time_t to Die" Dorkus Apr 6

A r e y o u r e a d y t o h a v e s o m e f u n ?

:3

Björkus "No time_t to Die" Dorkus Apr 6

"This is going to destroy my build system"

Nope! It's able to determine everything that will be processed by the Phase 7 compile-time-computed strings by Phase 4, and presents all of that information through already-available means, meaning CMake/build2/meson/make/ninja/etc. can all understand the dependency chain here natively!

Björkus "No time_t to Die" Dorkus Apr 6

Ultimately, this means we can process files -- recursively -- at compile-time, meaning that rather than embedded shaders with #includes that can't be touched, we can process those includes and make true single blobs without extra build steps.

compile-time python with imports is VERY possible.

Martin Uecker Apr 6

@thephd Cool, but why?

Björkus "No time_t to Die" Dorkus Apr 6

@uecker C++ has capabilities that can make great use of this. A perfect example is using (Generative) reflection to generate, at compile-time, the perfect FFI that maps to Python, or Lua, or JavaScript, with all of the utility that comes from having it mapped perfectly to C(++) interfaces and fully type-checked while always being generated directly from said Lua or Python or JavaScript source code.

This also applies to things like e.g. Rust and C++ interop, which has also been the topic of discussion and monetary investment. (Not that they're paying me; I wish they would, I could do a lot more for them than just std::embed.)

C doesn't have the systems in place to do things like this, so in most cases they'd just have to rely on the usual techniques used today: code generators, hand-written parsers, fresh data files and description files used to drive code generation (like e.g. SWIG). Certainly not bad, but not nearly as "automated luxury FFI" as C++ can make it.

Martin Uecker Apr 6

@thephd What I do not understand is why the extra build step is a problem.

Björkus "No time_t to Die" Dorkus Apr 6

@uecker I don't think the extra build step is the problem. I think the ability to e.g. parse C++ or C code and generate the proper FFI to connect to other languages, or vice versa, is a tooling investment that isn't really a fully solved problem.

Martin Uecker Apr 6

@thephd Ok, but why does it need to be solved by compile-time interpretation and not simply be a tool one runs during built? To me, this seems to solve the problem at the wrong place and using poorly suited tools (a compiler is not a good interpreter). And my only explanation so far is that people are nerd-sniped into doing it.

Björkus "No time_t to Die" Dorkus Apr 6

@uecker If that's all you took from this, okay!

mia (developer mode)Apr 6

@thephd @uecker take a look at fftw3’s build system and i think several use cases for this kind of compile-time functionality will become apparrent. for example, it currently uses some arcane OCaml code generators to turn customizable rules into SIMD code. it’s so unwieldy that the project’s README recommends users only ever build from tarballs with the sources already generated.

(after exchanging some emails i ported that whole mess to meson to make it usable but then that didn’t get reviewed at all and i stopped caring)

mia (developer mode)

@thephd @uecker my personal opinion is that build systems should not generate code, and code should not depend on build system output. the responsibility of a build system is the build/test/deploy process alone.

Martin Uecker Apr 6

@mia @thephd Then let's agree to disagree, because my opinion is that compilers should not be misused as interpreters.

Martin Uecker Apr 6

@mia @thephd (my personal pain with build systems is typically with CMake where it throws some error and I have no idea where in the thousands of lines of CMake I am supposed to change something. Especially if it first compiles for a long time and processing constexpr as part of it just makes it worse. )

mia (developer mode)Apr 6

@uecker @thephd that’s just cmake being utter garbage (without hyperbole, it has more in common with microsoft batch than it does with an actual build system) that for no apparent reason half the software industry agreed to force on each other, the other half being google’s flavor of the week.

qt needs more than 90k lines of code—again, no hyperbole because i’ve ported some of that recently—consisting mostly of cmake workarounds to achieve what meson can express entirely declaratively in just over a hundred. and none of it would have to exist if qt didn’t require 5 or 6 different generators.

so i don’t think “this makes bad build systems even more of a pain to use” is a strong argument

Martin Uecker Apr 6

@mia @thephd I certainly agree that CMake is just bad independently of constexpr. I haven't looked at meson much, but if I needed to build and run a C program to create specialized code it would be a single line in my Makefile. Cross-compilation would be more complicated though.

mia (developer mode)Apr 6

@uecker @thephd meson can also handle that case (plus cross compilation and figuring out how to run it if it needs to compiled for a different system/architecture) in a single line as well.

but you’ve just said it: cross-compilation breaks the assumptions of many build systems. there aren’t many that properly support the canadian cross to begin with, and there’s a lot of platform-specific quirks that build systems need to figure out to make it all work as well.

that’s an incredible amount of jank and moving parts for something that could just be done by the compiler that you already use to compile the generator code.

Martin Uecker Apr 6

@mia @thephd I think "just" having an additional interpreter in the compiler is not generally a good idea. I want things to be more modular and less monolithic. Compilers are already too complex for my taste. If people use modularity to create a mess, then this is IMHO not an argument against modularity but for cleaning up the mess.

mia (developer mode)Apr 6

@uecker @thephd well i think the past 30 years of software engineering have abundantly demonstrated that people are generally bad at cleaning up messes. just look at libc.

i don’t like complexity either, but what we’re talking about here is a problem that is, much like wanting to embed binary files, nearly universal.

and good tools do not foist such things off on the user in the name of simplicity, especially when doing so results in a million ugly hacks downstream.

Martin Uecker Apr 6

@mia @thephd I do not agree that the solution needs to be ugly hacks. I simply do not see why putting the complexity into the language and the compiler is a step to overall improvement. I think it does the opposite.

mia (developer mode)Apr 6

@uecker @thephd i’m arguing in favor of putting that complexity as close to the source of the problem as possible so that it only needs to be dealt with in the one place that is in a very good position to do so, rather than an unbounded number of places that then have to wonder why it is still so hard and still requires all these crutches in the current year. like, so far your argument boils down to aesthetic preferences more than anything else and it makes me wonder who you think the languange and its tools are for

Martin Uecker Apr 6

@mia @thephd You argument is "build systems are a mess, so solve everything in the compiler close to the source of the problem". Somehow this argument is also seems weak. Similar generic arguments always are used to justify centralization of complexity. But I do think code generation is fundamentally different from code translation and should not be done by the same tool and I do not think this is purely aesthetic argument. But let's agree to disagree.

mia (developer mode)Apr 6

@uecker @thephd it isn’t “build systems are a mess”, it’s “this isn’t meeting real-world needs, people have to get overly creative to fill that gap, and that in turn leads to more problems, worse interoperability, more complex dependencies and an overall unpleasant experience”

mia (developer mode)Apr 6

@thephd @uecker i’m NOT saying everything should be centralized and build systems left with no responsibilities because that’s even worse and in practice has resulted in languages that are married to a specific version of LLVM

Martin Uecker Apr 6

@mia The unpleasant experiences I had are not with projects doing run-time code generation by running a program during the build, but were all related to other random build complexities.