https://stackoverflow.com/questions/33051108/how-to-get-around-the-linux-too-many-arguments-limit/33278482

> I have to pass 256Kb of text as an argument to the "aws sqs"

what, uhhh, what

> MAX_ARG_STRLEN is defined as 32 times the page size in linux/include/uapi/linux/binfmts.h:
> The default page size is 4 KB so you cannot pass arguments longer than 128 KB.
> I modified linux/include/uapi/linux/binfmts.h to #define MAX_ARG_STRLEN (PAGE_SIZE * 64), recompiled my kernel and now your code produces

casually patching the kernel to send a quarter megabyte as a *single* argument oh my god i'm laughing hard
@navi well in the early rust for Linux days we hit this limit with the passing kconfig options to rustc. Fun times

@kloenk @navi Back when 128 kB was the limit for argv+envp, Google was hitting it too because they passed all the configuration for their whole software stack on the command line as --long-option=value switches.

Their solution? Compress the command line. So every binary started by ungzipping argv[1] and parsing it to get the configuration.

The person explaining this to me saw my horrified face, and said with the perfect Hide The Pain Harold smile: "a series of individually completely rational and reasonable decisions led to this." and I have been thinking a lot about it since.

@ska @navi And I guess null bytes in gzipped form must have been funny to handle

@lanodan @navi I don't think that's necessarily a problem. argv[1] doesn't have to be a string, it's a character array. Null is used as a separator when the kernel puts the whole argv on the stack, yes, but argv[1] is still just a pointer and if you know you're expecting a blob and have a way to know where the blob ends, it should work, I think.

Or they could have been base64-encoding the gzip for all I know, it's probably still smaller than the uncompressed argv.

(Edit: typo)

@ska @lanodan @navi Nope, it's a string. execve will stop processing it at the first null byte.

execve syscall essentially acts as a scatter-gather (in this case gather) operation running over the argv and environ pointer arrays in user memory and performing a string-copy-from-user operation for each one to built the object that will be prepopulated into the new process-image.

@ska @lanodan @navi And the since-abolished 128k limit was a very good thing because it put a bound on the burden the kernel could be asked to do on behalf of userspace in one uninterruptible go, and on spraying attacks you could do to suid binaries.

It was probably removed because someone doing the utterly stupid thing Google did here demanded it.

@dalias @ska @lanodan @navi

One of those someones was Rob Pike in 2004.

https://interviews.slashdot.org/story/04/10/18/1153211/rob-pike-responds

And the underlying kernel change happened in 2007.

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b6a2fea39318e43fee84fa7b0b90d68bed92d2ba

Interestingly, modern #FreeBSD has a sysctl() limit (kern.ps_arg_cache_limit) on how large processes can resize their argument block and environment string block and still have it show up in the ps command.

I just raised mine to 768 bytes, coincidentally. I could have got away with just 512, I think.

#Linux

Rob Pike Responds - Slashdot

He starts by clearing up my error in saying he was a Unix co-creator in the original Call For Questions. From there he goes on to answer your questions both completely and lucidly. A refreshing change from the politicians and executives we've talked to so much recently, no doubt about it....

@JdeBP @dalias @ska @lanodan @navi Pike's diplomatic non-answer of "Comparing patents to nuclear weapons is a bit extreme" gave me a good chuckle, lol