John Regehrsoftware is amazing... my otherwise-idle 128 GB machine just started swapping and then invoked the OOM killer to nuke a Debug + code coverage LLVM build
✧✦Catherine✦✧@regehr how much parallelism?
Josh Simmons@whitequark @regehr build system which gracefully handles memory pressure any day now!
Ted Mielczarek@whitequark @dotstdy @regehr ninja has a concept of "pools" for exactly this sort of thing: limiting the number of concurrent linker invocations: https://ninja-build.org/manual.html#ref_pool
@tedmielczarek @whitequark @regehr yeah but that's very much the "problem is users own" approach, since the value for every concurrency limit needs to change depending on the available hardware resources.
@dotstdy @tedmielczarek @regehr but we don't have a way to predict how much memory a linker invocation will consume, do we? in which case that's sort of what we're stuck with, since any solution must necessarily be reactive
@whitequark @dotstdy @regehr now I'm wondering if there are heuristics that would be workable… Number of input objects? Total size (in bytes) of all input objects? Things like LTO complicate matters, but maybe they could be accounted for?
@tedmielczarek @dotstdy @regehr I think it's probably fine to give up on thick LTO and only consider thin LTO, which scales more or less linearly in the size of input code; without specifically commenting on whether I think heuristics will work (as I think only testing will show)