jwzNov 26, 2022
Mastodon stampede. "Federation" now apparently means "DDoS yourself." Every time I do a new blog post, within a second I have over a thousand simultaneous hits of that URL on my web server from unique IPs. Load goes over 100, and mariadb stops...
https://jwz.org/b/yj6w
Mastodon stampede

"Federation" now apparently means "DDoS yourself." Every time I do a new blog post, within a second I have over a thousand simultaneous hits of that URL on my web server from unique IPs. Load goes over 100, and mariadb stops responding. The server is basically unusable for 30 to 60 seconds until the stampede of Mastodons slows down. Presumably each of those IPs is an instance, none of which ...

Show threadKazinator
@jwz Interesting. Do you have an estimate of how many of those might be "organic" hits: actual people following the link in the toot?
Nov 26, 2022 at 8:12pmWeb
Show threadNemo_bis 🌈Nov 26, 2022
@Kazinator Zero. He's talking about the Sidekiq job in Mastodon which fetches the URL to (potentially) create a preview card.
Show threadKazinatorNov 26, 2022
@nemobis I see. Does that usefully respect robots.txt? If not, it could be blocked at the request level. Easily so if it announces itself as a particuliar agent type.
Show threadRogers CadenheadNov 26, 2022
@Kazinator @nemobis Blocking the traffic would make the post broken in all those Mastodon instances. The post preview would be missing.
Show threadKazinatorNov 27, 2022

@rcade @nemobis

That hardly qualifies as "broken". The link would be there right?

Most hyperlinks on the internet do not come with previews. Is the exception rather than the rule.

Show threadRogers CadenheadNov 27, 2022
@Kazinator @nemobis Preview cards with images make links look better and get far more clicks.
Show threadKazinatorNov 27, 2022

@rcade @nemobis

Even if that is true (where are the numbers to prove it), it seems to me that since toots can include images, you can have that cake and eat it too: you can attach your own screenshot of the page to accompany the link, and block the preview requests.

The image travels through the fediverse without generating redundant hits on your server.

Proof of concept:

Here is hackernews: https://news.ycombinator.com

Hacker News

Show threadRogers CadenheadNov 27, 2022
@Kazinator @nemobis I don't know anything about Mastodon's design yet. Couldn't the Twitter card data be pulled in to the post when it is published to avoid the need to manually add an image?
Show threadUbergeekNov 28, 2022

@rcade @Kazinator @nemobis I don't think "Click through rate" is something we should honestly be caring about.

This isn't an advertising network.

Show threadRogers CadenheadNov 28, 2022
@ubergeek @Kazinator @nemobis You're arguing that Mastodon should not support link previews. It does, so the debate is whether it is supporting them sustainably.
Show threadUbergeekNov 28, 2022

@rcade @Kazinator @nemobis Hardly.

What I said is that "Click through rate" isn't even a metric we should consider.

Again, this isn't an advertising network, and we don't care about CTR or total impressions. We care about socializing with humans.

My suggestion elsewhere in this thread is that link previews should be an optional toggle in the admin UI, to disable it instance-wide, disabled by default. And/or, users should also have that toggle, for their experiences, defaulted to off.

Reason being? Several. One is a the huge traffic surge we're causing to other network citizens. The other is... frankly, goatse, or other gore/shock porn. Shit, even making it so people who may have content sensitivities wouldn't get blasted by things they didn't ask to see.

Bottom line: You metric you suggested above is a bad one that should have no place in discussing features present in social media software.

Show threadRogers CadenheadNov 28, 2022

@ubergeek @Kazinator @nemobis It isn't contrary to the goals of Mastodon to want people to click the links in your posts.

That's one of the only two reasons everybody undertook the hassle of implementing Twitter-style preview cards on their websites.

The other is to give users more information before they decide whether to click the link.

Giving instances and users more control on whether to show link previews is a good idea.

Show threadUbergeekNov 28, 2022

@rcade Yes, many sites undertook the effort to do preview cards, because of capitalist drives for economic return.

There's very little social value in preview cards, really. Sure, the argument could be made "Decide on clicking the link", but most of that decision should be made because of who is providing the link to you, not because of what a preview card shows you.

Ie, gargron shares a link about new Masto changes, I don't need a preview card to give me info: I have if from the person sharing the link. Same with the local antifascist groups. They provided the link, I trust them, and I don't need a preview card to tell me more, truth be told.

Random person sharing link? I probably will start to consider the sources, or open in an incognito window. I don't trust the preview card, anyways, and at worst, it's preloading data in my browser from a potentially malicious site looking to track people.

Show threadChristopher SchmidtNov 26, 2022

@Kazinator @jwz In the first 60 seconds, from skimming the logs, it's very obvious that it's almost none of them. For my account (658 followers), it was 41 servers, each making two requests (one for the page, one for the opengraph preview image), triggering 82 requests... and one human.

And of course, this happens if _anyone_ posts one of my links.