Evan PhoenixNov 18, 2022

To all mastodon admins:

Hi! I’m Evan Phoenix, the primary author of puma, the ruby webserver that powers mastodon!

Please reply or DM me if you need tuning help! I’ve got no officiation with the mastodon, just want to see you succeed!

(Quick Tip: set WEB_CONCURRENCY to core count * 1.5 and then tune MAX_THREADS. High thread values will see diminishing returns!)

Show threadMike Perham 
@evanphx Careful with that 1.5 value. Sidekiq needs to run too.
Nov 18, 2022 at 5:21pmWeb
Show threadDJ SundogNov 18, 2022
@getajobmike @evanphx this is the most delightful resource contention I've seen all week. if we can get core contributors from postgres and nginx into the thread we'd be full stack! ​
Show threadEvan PhoenixNov 18, 2022
@getajobmike For sure! Setting WEB_CONCURRENCY to cores * 1.5 can result in puma soaking up 100% of the cpu. So if you're running sidekiq on the same cores, use 1 * cores or even 0.8 * cores is prudent.
Show threadMike P.Nov 18, 2022
@evanphx @getajobmike In a fresh install of Mastodon following the guide, Puma doesn't read .env.production on boot, so if you want to change the number of workers you either need to edit config/puma.rb or the mastodon-web.service to set e.g. Environment="WEB_CONCURRENCY=3".
Show threadMike P.Nov 18, 2022

@evanphx @getajobmike Additionally, since y'all are giving formulas with floating point math, make sure to round up or down to an Integer value to keep Puma happy.

If you're running a small VPS with e.g. one core for a small number of users, consider setting WEB_CONCURRENCY=0 to take Puma out of clustered mode.

Show threadEvan PhoenixNov 18, 2022
@mwp @getajobmike Yup! Not being in cluster mode will save a small amount of memory as well.
Show threadRalf StockmannNov 18, 2022
@evanphx @getajobmike see - this is where it gets tricky. We need some sort of „calculator“ that takes cores, RAM and dedicated servers into account and gives advise _where_ to set the values, how to restart etc. This would be super-helpful for our bunch of not-so-expert admins. And we need it fast, for numbers are rising fast. The official documentation is way to fuzzy regarding tuning. 🙏
Show threadEvan PhoenixNov 18, 2022
@rstockm @getajobmike Yeah, totally agree. I'm going to setup a google form to start gathering data now, that we could use for a calculator.
Show threadRalf StockmannNov 18, 2022
@evanphx @getajobmike brilliant, and just for the record: it‘s beyond awesome that you get in touch with us. And the actual documentation was never meant to handle a situation like this: the basic setup was fine for several years. But now we are not facing a wave, but a brick wall of new users - and need to adapt. Thanks again for your effort! ❤️
Show threadRalf StockmannNov 20, 2022

@evanphx @getajobmike maybe worth a read and how you do fit in:

https://nora.codes/post/scaling-mastodon-in-the-face-of-an-exodus/

Scaling Mastodon in the Face of an Exodus | Nora Codes

Show threadSantiago Nov 19, 2022
@evanphx @getajobmike where is this setting? i can't remember
Show threadJanNov 18, 2022

@getajobmike @evanphx

Hey Evan, Hey Mike nice to meet you. Puma and Sidekiq works like a charm 👍

Show threadEvan PhoenixNov 18, 2022
@jan @getajobmike 364M happy customers served.
Show threadMike Perham Nov 18, 2022
@jan @evanphx Not bad. I have a customer doing 22 billion per day, 250k per sec.