a very strange trackball1d ago
Ungh. Gotta do something about iocaine's firewalling performance.
Show threada very strange trackball

Had to restart it on Eru, because 31GiB resident memory was making the server crawl.

Firewalling temporarily disabled, because it's currently causing more trouble than what it is worth. sigh

Mar 27 at 8:01pmWeb
Show threada very strange trackball1d ago
Looks like single-threading it was a bad idea. I either need batching, or a small pool, or both.
Show threada very strange trackball1d ago

A single thread in a release build can do ~4-5 blocks / sec on my development box. That's... piss poor.

On top of that, it slows down over time.

Show threada very strange trackball1d ago
Lets try batching first, because thread pools are complicated.
Show threada very strange trackball1d ago

Hrm. The problem with batching is that I'd need to build separate commands for each set. So what if I made this simpler, and I'd only have two sets: one for IPv4 and one for IPv6 addresses.

No per-ruleset counters that way, but much simpler implementation.

Show threada very strange trackball1d ago
Ow. Yes, batching helps. Debug build happily inserts 1k IPs/sec.
Show threada very strange trackball1d ago
Lets see a release build!
Show threada very strange trackball1d ago

Inserted 120k entries in about a minute, so about 2000 / second, with a batch size of 1000. Not bad. Not perfect, but not bad.

Now to figure out how to make it timeout, so I don't have IPs stuck in the queue if I don't reach the batch limit.

Then clean up the code, and all that.

Show threada very strange trackball1d ago
I think I'll do that tomorrow, though. Feels like this is something that could do with some thinking first.
Show threada very strange trackball1d ago

So... what if I had one thread to send tasks to, which collects them in a pair of HashSets. I can make these tokio tasks, so that I can have an interval ticking too.

When the hashsets reach their quota, or interval expires, this thread will construct an nft command, and send it over to another thread that talks with nftables.

The advantage of this architecture is that I can later turn the nftables thread into a threadpool.

Show threada very strange trackball1d ago

Current downside: no per-ruleset chains or sets. Not sure how I will ferry the table name over, either.

This will break the firewall API on the Roto side (unless I keep it the same, but ignore some params).

Show threada very strange trackball1d ago

For #iocaine 3.4, I think I'm just gonna ignore the table & set name, and use hard-coded ones. Not very nice, but that's a simple to implement and API-compatible way.

Will fix it in 4.0. That'll restore configurable table name, but the per-ruleset sets and chains will be gone (from the API too).

Show threada very strange trackball1d ago
Mmmmm. Yeah. I think this'll work fine. Will hardcode the batch size and the timeout for iocaine 3.4, will figure out how to make them configurable in 4.0.
Show threadVysogota1d ago
@algernon errr... so... an ffi call to add entry to nft ipset can be executed at a rate of max 5 per second... wow...
Show threada very strange trackball1d ago
@petko It's a bit more complicated than that. The library I'm using does quite a bit of work to add an entry. If I batch it, and insert entries by batches of 1000, that's a lot faster. Like, about 1000 times faster. :D
Show threadVysogota1d ago
@algernon maybe some tree balancing or summarisation takes place??
Show threada very strange trackball1d ago

@petko It's mostly parsing. I don't directly talk to netlink, I talk to libnftables, which turns a string into... something, and then sends that to netlink.

There's quite a bit of work involved in that process that has nothing to do with nftables.

I could make it a lot faster if I talked directly to netlink, but the Rust support for that is limited. So: compromises had to be made.

The fault lies not in nft or libnftables, but in the way I use them.