witch_t *naviJul 24, 2025
https://stackoverflow.com/questions/33051108/how-to-get-around-the-linux-too-many-arguments-limit/33278482

> I have to pass 256Kb of text as an argument to the "aws sqs"

what, uhhh, what

> MAX_ARG_STRLEN is defined as 32 times the page size in linux/include/uapi/linux/binfmts.h:
> The default page size is 4 KB so you cannot pass arguments longer than 128 KB.
> I modified linux/include/uapi/linux/binfmts.h to #define MAX_ARG_STRLEN (PAGE_SIZE * 64), recompiled my kernel and now your code produces

casually patching the kernel to send a quarter megabyte as a *single* argument oh my god i'm laughing hard
Show threadkloenkJul 24, 2025
@navi well in the early rust for Linux days we hit this limit with the passing kconfig options to rustc. Fun times
Show threadLaurent BercotJul 24, 2025

@kloenk @navi Back when 128 kB was the limit for argv+envp, Google was hitting it too because they passed all the configuration for their whole software stack on the command line as --long-option=value switches.

Their solution? Compress the command line. So every binary started by ungzipping argv[1] and parsing it to get the configuration.

The person explaining this to me saw my horrified face, and said with the perfect Hide The Pain Harold smile: "a series of individually completely rational and reasonable decisions led to this." and I have been thinking a lot about it since.

Show threadkittenJul 24, 2025
@ska @kloenk @navi nah what the fuck 😭😭😭
Show threadkloenkJul 24, 2025
@kitten @ska @navi if google does it we all are fine :p
Show threadLaurent BercotJul 24, 2025
@kloenk @kitten @navi tbh that was 11 years ago and I have no idea if they're still doing it. I suspect some Googlers were behind the push for Linux to drop the limit, and the whole tech staff breathed a collective sigh of relief when it happened.
Show threadGabeJul 25, 2025

@ska @kloenk @kitten @navi specifically, the arguments were compressed with gzip and then base64 encoded so that they could be passed with reasonable levels of escaping through ssh. This started about 20 years ago, and the kernels at the time were already modified to allow longer command lines.

Frankly, it's just another example of scaling: there's always a bottleneck, sometimes in surprising places.

The line about individual choices is perfect, too: there are always trade-offs to be made.
./.

Show threadLaurent Bercot

@gabe @kloenk @kitten @navi Precisely, I disagree that it's just another example of scaling. Usually, when hitting a limit, Google *reconsiders its approach* and develops a new solution that scales better. Here, the opposite happened: a hack was added to the original solution in order to accommodate the limit. At some point the limit was going to be hit again, even with compressed arguments.

Maybe it was just a way to buy time while waiting for the real solution, i.e. Linux dropping the limit. But it's definitely not the example I think about when mentioning "Google scale" 😅

Jul 26, 2025 at 12:51amWeb
Show threadGabeJul 26, 2025
@ska @kloenk @kitten @navi Google has some odd traditions it still can't quite let go, "put it all on the command line!" Is one of them. It does invent new stuff to fix issues until the next step function of scale, yes, but (even at Google) there's often just a bigger box.