defnullJun 7
The fastest multipart/form-data parser written in python just got 20-37% faster for typical file uploads. I can parse roughly 9GB/s (with Python 3.12) on my laptop now. All tests pass, 100% coverage. Let's see if I can get a release ready this week. Benchmarks will follow. That was some very productive weekend :)
Show threadSchenkl | 🏳️‍🌈🦄Jun 7

@defnull maybe try it with pypy?

Managed get a massive speed boost for fail2ban once, faster and consuming less resources!

Show threaddefnull

@schenklklopfer Huh, interesting.

cPython 3.12.3:
9475.86 MB/s

PyPy (nightly, default settings):
4378.93 MB/s

PyPy (nightly, --jit off):
3676.98 MB/s

My guess: Most of the time is spent in bytes.find(), which is a highly optimized C function in #cPython already. The actual python overhead is so small that my parser even beats #rust (emmett-core) in this benchmark. #PyPy does not have much room for its #JIT to do anything, and seems to have a less optimized stdlib?

Jun 8 at 7:09amWeb
Show threaddefnullJun 8

@schenklklopfer The interesting part for me: The recent optimizations also show up in the PyPy test run, which is good. I had a feeling that I am relying too much on cPython internals.

For example, I assumed that buffer+=new_data (both are byte strings) is WAY cheaper if the buffer is empty. The recent optimization spends extra work to keep this buffer empty more often. And that assumption seems to be correct, for both cPython and PyPy. Yay :)

Show threadSchenkl | 🏳️‍🌈🦄Jun 8
@defnull cPython is Part of Common Python? Or is it Like pypy, an alternative Interpreter?
Show threaddefnullJun 8
@schenklklopfer The default python interpreter, the 'vanilla' python we all know and love, is actually called cPython because it is implemented in C.
Show threadSchenkl | 🏳️‍🌈🦄Jun 8
@defnull aaah, thanks!