Mastodawn

hi there rust peeps

does anyone have any clue why a tokio runtime would have a hard cap on the number of tasks at 32,768?

hitting this limit silently fails to spawn new tasks and existing tasks hit "device or resource busy" errors when they need to use IO

no ulimits have been hit as far as i can tell (83 open files, well under limit of 16,384, and all other limits are either unlimited or unrelated to this issue)

given the failure mode is completely silent i have no clue where to start debugging this

Show thread

Luck Goddess 🌒🌕🌘 (They/She)Mar 23

@niko I looked into this and it seems the reason is that if all runtime threads are already busy, tokio will push tasks onto a queue, and apparently the size of that queue is bounded now.

not exactly a full solution, but I found an unstable tokio method of spawning a task that returns an error instead of panicking: https://docs.rs/tokio/latest/tokio/task/struct.Builder.html#method.spawn

(side note: goddamn, tokio is such a rust project. this API has been in there for years and there’s no sign of stabilisation.)

Builder in tokio::task - Rust

Factory which is used to configure the properties of a new task.

Show thread

Niko Mar 23

@follpvosten just checked runtime metrics, the global queue depth never goes above 0 according to https://docs.rs/tokio/1.49.0/tokio/runtime/struct.RuntimeMetrics.html#method.global_queue_depth

RuntimeMetrics in tokio::runtime - Rust

Handle to the runtime’s metrics.

Show thread

Luck Goddess 🌒🌕🌘 (They/She)Mar 23

@niko oh, you’re spawning tasks from outside the runtime?

Show thread

Luck Goddess 🌒🌕🌘 (They/She)

@niko I‘m getting more and more intrigued to learn about the X problem here 😅