I/O Multiplexing (select vs. poll vs. epoll/kqueue)

https://nima101.github.io/io_multiplexing

#HackerNews #IOMultiplexing #select #poll #epoll #kqueue

Please help me spread the link to #swad 😎

https://github.com/Zirias/swad

I really need some users by now, for those two reasons:

* I'm at a point where I fully covered my own needs (the reasons I started coding this), and getting some users is the only way to learn about what other people might need
* The complexity "exploded" after supporting so many OS-specific APIs (like #kqueue, #epoll, #eventfd, #signalfd, #timerfd, #eventports) and several #lockfree implementations based on #atomics while still providing fallbacks for everything that *should* work on any #POSIX systems ... I'm definitely unable at this point to think of every possible edge case and test it. If there are #bugs left (which is somewhat likely), I really need people reporting these to me

Thanks! 🙃

Today, it's exactly one month since I released #swad 0.11. And I'm slowly closing in on releasing 0.12.

The change to a "multi #reactor" design was massive. It pays off though. On the hardware that could reach a throughput of roughly 1000 requests per second, I can now support over 3000 r/s, and when disabling #TLS, 10 times as much. Most of the time, I spent with "detective work" to find the causes for a variety of crashes, and now I'm quite confident I found them all, at least on #FreeBSD with default options. As 0.11 still has a bug affecting for example the #epoll backend on #Linux, expect to see swad 0.12 released very very soon.

I'm still not perfectly happy with RAM consumption (although that could also be improved by explicitly NOT releasing some objects and reusing them instead), and there are other things that could be improved in the future, e.g. experiment with how to distribute incoming connections to the worker threads, so there's not one "loser" that always gets slowed down massively by all the others. Or design and implement alternative #JWT #signature algorithms besides #HS256 which could enable horizontal scaling via load balancing. Etc. But I think the improvements for now are enough for a release. 😉

Getting somewhat closer to releasing a new version of #swad. I now improved the functionality to execute something on a different worker thread: Use an in-memory queue, providing a #lockfree version. This gives me a consistent reliable throughput of 3000 requests/s (with outliers up to 4500 r/s) at an average response time of 350 - 400 ms (with TLS enabled). For waking up worker threads, I implemented different backends as well: kqueue, eventfd and event-ports, the fallback is still a self-pipe.

So, #portability here really means implement lots of different flavors of the same thing.

Looking at these startup logs, you can see that #kqueue (#FreeBSD and other BSDs) is really a "jack of all trades", being used for "everything" if available (and that's pretty awesome, it means one single #syscall per event loop iteration in the generic case). #illumos' (#Solaris) #eventports come somewhat close (but need a lot more syscalls as there's no "batch registering" and certain event types need to be re-registered every time they fired), they just can't do signals, but illumos offers Linux-compatible signalfd. Looking at #Linux, there's a "special case fd" for everything. 🙈 Plus #epoll also needs one syscall for each event to be registered. The "generic #POSIX" case without any of these interfaces is just added for completeness 😆

Now that #swad 0.7 is released, it's time to prepare a new release of #poser, my own lib supporting #services on #POSIX systems, following a #reactor with #threadpool design.

During development of swad, I moved poser from using strictly only POSIX APIs (with the scalability limits of e.g. #select) to auto-detected support for #kqueue, #epoll, #eventports, #signalfd and #timerfd (so now it could, in theory(!), "compete" with e.g. libevent). I also fixed quite some hidden bugs, and added more base functionality, like a #dictionary using nested hashtables internally, or #async tasks mimicking the async/await pattern known from e.g, #csharp. I also deprecated two features, the periodic and global "service tick" (superseded by individual timers) and the "resolve hosts" property of a "connection" (superseded by a separate resolve class).

I'll have to decide on a few things, e.g. whether I'll remove the deprecated stuff immediately and bump the major version of the "posercore" lib. I guess I'll do just that. I'd also like to add all the web-specific stuff (http 1.0/1.1 server) that's currently part of the swad code as a "poserweb" lib. This would get a major version of 0, indicating a generally unstable API/ABI as of now....

And then, I'd have to decide where certain utility classes belong to. The rate limiter is probably useful for things other than web, so it should probably go to core. What about url encoding/decoding, for example? 🤔

Stay tuned, something will come here, maybe helping you to write a nice service in plain #C 😎:

https://github.com/Zirias/poser