Mastodawn

Jerry 🦙💝🦙

Almost to 10000 accounts on Infosec.exchange. Y’all have chewed through 120GB of disk in 2 weeks.

"Offensive" Security Engineer

@jerry wow, scaling sure is a beast!

Graham Watt Nov 6, 2022

@jerry yikes. Maybe ease off on the images folks. Or host them elsewhere and just link?
Reminds me of usenet feeds that had to delete after 3 days because of space issues.

@guamwatt all good. I have plenty of space. And I’m the top offender

Andrew Certain

@guamwatt @jerry back in the mid nineties, the company I worked for subscribed to a Usenet archive service. Every month, they sent you all the posts on all news groups on CD-ROM. A very visceral way to visualize exponential growth! They didn't survive! 😂

Dana Epp

@tacertain @guamwatt @jerry LOL. I remember a similar service for Fidonet. Those were the same days when I used to get binders of CDs from Microsoft for MSDN. Gawd how far we have come.

Dana Epp

@tacertain @guamwatt @jerry Now that I think about it, I don't even have a friggen CD or DVD player anymore. Maybe they are in the closet with the 8tracks and zipdisks. #we'reold

Graham Watt Nov 6, 2022

@danaepp obtained a usb 3.5 floppy drive. Muh ha ha ha.

However, I need an ibm sys 34 mainframe and 8in floppy drive to recover my COBOL code. 😉

Jernej Simončič �Nov 6, 2022

@guamwatt @danaepp I've got an 8″ floppy on my wall, but I only saw an 8″ drive in a museum (the first computer we had at home did have a 5.25″ drive). No idea where that floppy is from, it used to have a sticker, but it fell off years ago.

https://eternallybored.org/imgs/compstuff/floppy.jpg

Dana Epp

@jernej__s @guamwatt Well, at least it isn’t the equal amount of punchcards… (would be interesting wallpaper though)

Tevruden Dawnspear Nov 6, 2022

@tacertain @guamwatt @jerry I wonder if those CD are still floating around anywhere.

Tevruden Dawnspear Nov 6, 2022

@guamwatt @jerry This gets a bit complicated because mastodon will ALSO pull down images when it gets posts, I've made 1 image post in the past day and my instance has pulled down 78,734 from the fediverse.

@tevruden @guamwatt yep. I prune remote images after 7 days. It’s almost mandatory to do that.

Tevruden Dawnspear Nov 6, 2022

@jerry @guamwatt Yeah i've got a ridiculously large nas where i host my mastodon instance and thats the ONLY reason I don't bother

I'm glad to be less reliant on the bird now, and there's so much more content/less ads, and is making my browsing that much more difficult because I'm reading most of the posts now.

Sent something to help with costs. Appreciate you doing this again.

Jef Kazimer😶‍🌫️Nov 6, 2022

@jerry Is there a site for an all up view of the servers across the mastodon federated network? This place has growth, and you clearly have the skills to help it grow. I imagine some others may be struggling with the influx though

CyberGingey 🤖Nov 6, 2022

@jerry nom nom nom. If you are ever looking for some help, happy to try help where I can

Aaron Galbraith Nov 6, 2022

@jerry Thank you for the new home.

hopefully you have the donation you need to handle the maintenance costs that will surely creep up.

#10000NewFriends

Elan 🦃Nov 6, 2022

@jerry what's your architecture like? I'm thinking of running an instance.

Pet VMs feel kinda icky to me :/

A couple of links I have bookmarked 🙂
https://infosec.exchange/@jerry/109296646225734012
https://infosec.exchange/@jerry/109295166927417993

Elan 🦃Nov 6, 2022

@wally3k @jerry cool. VMs I guess and not k8s

@elanhasson @wally3k bare metal. VMs would not have the IO I need.

Elan 🦃Nov 6, 2022

@jerry @wally3k what's the IO like currently?

Andrew Certain

@jerry Thanks so much for providing this service.

Jay Nov 6, 2022

@jerry I will start dropping the resolution on my cat pics.

@jws I was marveling at the number - we have plenty of space and can get more

sysengineer

@jerry awesome! Congrats on hitting 10k!

let me know if there’s anything I can do to help out.

Tyler

@jerry woot! Congratulations on the milestone 🎉

Michael Potts (HMHackMaster)Nov 6, 2022

@jerry I am curious if you have the ratio of attachments (pictures/video) to other user data (and the to logs and whatnot if that's included in that count)?

I wonder at what point does it make sense so shard the attachments onto a different (high storage/high bandwidth/lower compute) server...

This is something I did for static images on a message board I run. Not having the main web server hit a whole bunch just for images and graphics was a pretty noticeable CPU improvement.

Ryan Jamieson Nov 6, 2022

Jesse Brown Nov 6, 2022

@jerry Delicious

@jerry And we're just strarting! I imagine you're going to need a bigger boat.

@ryanjamieson our boat is being upgraded to a warship at the moment

Kalani Helekunihi Nov 6, 2022

@jerry that’s really not bad… under 1MB / user / day of data creation. Especially where users are allowed to upload images! I would have expected more.

Thanks for sharing your experiences running this server. It’s fascinating to see the sorts of problems one runs into while scaling up - especially when it’s one guy dealing with it! Even when I’ve overseen growth at work, it’s always been stuff spread around a team so it never felt “real” in the way that your posts have.

@jerry are you doing anything like a patreon/whatever to help fund the server(s)? would be happy to chip in. I could have sworn I saw something about it a week ago but can't find anything on the 'about' page at the moment.