École des Bro-ArtsNov 18, 2022
Oh yeah, queues are lagging, like, 18 minutes behind right now. I'm gonna bump us up a tier on hosting, and approve new members once queues calm down. #admin
Show threadÉcole des Bro-Arts

oooookaaay, I'm bumping us up *two* tiers.

Also my god, we've chewed through 40 GB of media storage in the last two weeks. I'm gonna have to set an expiration policy on the remote media cache. #admin

Nov 18, 2022 at 3:26pmWeb
Show threadÉcole des Bro-ArtsNov 18, 2022
Right--had to bump us from 4 sidekiq workers to 25, and queues have recovered. Presently 160 pending users; going to approve them in small batches so we don't knock things over again. #admin
Show threadDarius KazemiNov 18, 2022
@aphyr yeah I had to double DB_POOL and concurrency in my sidekiq push service from 25 to 50 but now we are happily chugging along
Show threadJohnny GearNov 18, 2022
@aphyr Hah, I actually spun down my sidekiq workers from 5 to 3 yesterday. Server kept falling over trying to encode 5 videos at a time... It's been keeping up since sooo 🤷
Show threadB. P. RuggerNov 18, 2022
@aphyr Be sure to let us know if you're going to need contributions. I don't have a lot of cash to spare, but I'd happily chip in $10 a year.
Show threadDaddyBooFFerNov 18, 2022
@DammitMooMoon @aphyr same - with how quickly things are growing, if you need support, let a dog know.
Show threadGoose 🐶Nov 18, 2022
@aphyr you're doing good work 🫡
Show threadmspinNov 18, 2022
@aphyr are you able to see how much of that storage is the cache? I'm likely going to be setting up a kinky instance soon, and want to get an idea what I'm looking at
Show threadpeter hessler @openbsdNov 18, 2022

@Mspin @aphyr not for wolf.group, but for the instance I'm admin for, we only keep remote media for 7 days, but combined local and remote media storage is 167G.

3 weeks ago (so, before the current influx) it was around 90G. Ouch.

Show threadmspinNov 18, 2022
@phessler Good data point. thanks!
Show threadBuckFosterNov 18, 2022
@aphyr Thank you for everything you're doing. Can I help with a donation?
Show threadDeeply AsleepNov 18, 2022
@aphyr very curious what size/system reqs are necessary from what you’re seeing… and is Mastodon able to run load-balanced across multiple servers?
Show threadÉcole des Bro-ArtsNov 18, 2022
@deeplyasleep It's sort of abstract; masto.host handles it for us. And yes, mastodon fans out across lots of webheads/background workers fairly well.
Show threadAndyNov 18, 2022
@aphyr Curious if you have donation pot or something, to help with hosting costs? Appreciate that usage has massively ramped up