daniel:// stenberg://

On #curl's --max-filesize and --compressed. Should we do something about the "compression bomb" risk?

https://curl.se/mail/archive-2026-03/0000.html

curl: --max-filesize and --compressed

Mar 2 at 8:54amWeb
Show threadYonleMar 2
@bagder i mean, the entire goal of --max-filesize is setting the maximum limit of the file, regardless of how small the compressed bytes are. Just like the name implies
Show threadYonleMar 2
@bagder if --max-filesize is relying on ContentLength, then making separate flag like --max-output-size is better.

if there's --max-filesize + --compressed paired together, warn user to use --max-output-size instead
Show thread"Musty Bits" McGeeMar 2
@bagder let's hope this doesn't turn onto a mail-thread bomb 🤗
Show threadStefan EissingMar 2

@bagder It would not contradict the documentation if it applied after decompression.

The crux is backwards compat, but maybe that would be acceptable breakage?

Show threadJyrki Mar 2

@bagder Qt has a decompression ratio check that kicks in after 10MiB by default, maybe something like that would be good enough?
(though this usually catches valid, highly compressable data; e.g. repetitive log files)

https://doc.qt.io/qt-6/qnetworkrequest.html#setDecompressedSafetyCheckThreshold

QNetworkRequest Class | Qt Network | Qt 6.10.2

The QNetworkRequest class holds a request to be sent with QNetworkAccessManager.

Show threadAndreas FinkMar 2
@bagder applying max fiklesize to the decompressed file makes sense. intuitively. however not all compression formats allow you to determine the decompressed size in advance. so you might have to download and decompress in parallel and stop when the filesize is reached
Show threadsodiboo Mar 2
@bagder intuitively, i would assume --max-filesize applies on the size of the file that curl produces. not just the transfer size.
Show threadSikorski ArkadiuszMar 2
@bagder Mechanisms for "killing" a process that behaves incorrectly should be left to the operating system. Such a tool should, it seems, behave according to established standards. A tool that allows for checking, testing, executing, etc., such a scenario should be available.
Show threadAdnanMar 2

@bagder I do like “the --max-filesize limit be applied on the uncompressed size as
well” but I don't know how you would implement it. How will you know in advance what the uncompressed size is going to be?

Therefore, I believe the current behavior is sufficient and portable. More importantly, it is not a surprise.

Show threadMike Fiedler, Code GardenerMar 2
@bagder we apply a compression ratio check prior to decompression for this case
https://github.com/pypi/warehouse/blob/481300222f98d96ce587311fd5a791daeccff791/warehouse/forklift/legacy.py#L82
https://github.com/pypi/warehouse/blob/481300222f98d96ce587311fd5a791daeccff791/warehouse/forklift/legacy.py#L296
warehouse/warehouse/forklift/legacy.py at 481300222f98d96ce587311fd5a791daeccff791 · pypi/warehouse

The Python Package Index. Contribute to pypi/warehouse development by creating an account on GitHub.

GitHub
Show threadmanxMar 2

@bagder

--max-filesize should probably have been called --max-transfersize, if that is what it does

--max-transfersize for the on-wire bytes
--max-filesize for the locally stored bytes

But that would be a breaking change changing the semantics of --max-filesize.

Maybe deprecate it and add --max-transfer-size and --max-file-size (note the extra "-").

Show threadShutterBuggedMar 3
@bagder I'd probably add --max-decompressed-filesize. I can think of a few scenarios where tying it to --max-filesize would cause well-intentioned/reasonable apps to explode. It does strike me as something curl could make a lot easier to mitigate, not just document, and that makes me thing this is well worth doing.
Show threadShutterBuggedMar 3
@bagder As for apps: I could see someone with a package manager or similar downloading compressed archives over a metered connection. They could use --max-filesize to stay under their bandwidth cap. But, if that limit suddenly applies to the decompressed files, valid download-and-unpack operations are going to start failing.