mookulatorAug 15, 2023

Where can I get the data about Lemmy subscription patterns?

https://mander.xyz/post/2268203

Where can I get the data about Lemmy subscription patterns? - Mander

I have a fair amount of experience with data visualization, analysis etc and bought it would be a fun project to try to visualize the Lemmy network, specifically which instances have strong links to one another via subscriptions from users in one to communities in the other. How/where can I get that data?

Show threadDanteriousAug 19, 2023
Personally I would love to see this kind of project done. But for it to work most people on the servers data that you are working with would have to be informed and given time to answer. I hope you reach out to the administrators of each instance and ask them if they would be ok with this and give them time to ask their users. Knowledge is power and if the visualization was public I think it could be helpful.
Show threadmookulator

Thanks for the support! I thought it would be a useful tool too. People could use it to find instances they didn’t know about, but which are popular in their network, or to find under appreciated instances to build bridges.

Anyway, sounds like database access is a must, and I’m not trying to take on a massive data wrangling exercise. I had it in my mind that the data were just sitting their for download, which was wishful thinking!

Aug 19, 2023 at 4:39amWeb
Show threadDanteriousAug 19, 2023
If you are interested there is a professor named Damon Centola that might be interested in dedicating time to gathering that data and might help with the visualization as well. If you are in the sociologist space or just interested you could reach out to him.
Show threadmookulatorAug 19, 2023
Thanks!
Show threadDanteriousAug 21, 2023
No problem just notify me if you ever decide to follow through.
Show threadDanteriousAug 23, 2023

Sorry for commenting again but I think there is a way for you to do this in a completely open, easy, and privacy-preserving way. You don’t need to access their database.

  • Get a list of instances that you want to look at the subscription patterns for. (All the instances here + Lemmy.world)

  • Go to that instance’s website, click the “Communities” tab at the top, and then click “All” It shows how many users from that instance are subscribed to that community (both communities from that instance and outside of that instance)

    • If you find a way to automatically (or manually) scrape this data from all of those websites you can create the visualization that you were talking about.

    So you were right, the data is open source it is just specific to each website.

    Lemmy Explorer

    Instance and Community Explorer for Lemmy