Mastodawn

Pelle S Christensen Nov 25

🅰🅻🅸🅲🅴 (🌈🦄)

Why reactive moderation isn't going to cut it, aka, "The Sucker-punch Problem".

Imagine you invite your friend—let's call him Mark—to a club with you. It's open-door, which is cool, because you like when a lot of folx show up. Sure, it might get a little rowdy, but they have a bouncer, and you've never seen things getting out of hand.

So, you're busy dancing when a new guy walks in wearing a "I Hate Mark" shirt and promptly sucker-punches Mark. You didn't see it happen, but Mark is upset and tells the bouncer, who kicks the guy out.

A few minutes later, the same guy walks back in and sucker-punches Mark again. Same result. Some people in the club say they'll tell the bouncer if they see him come in again.

Mark wants to leave, but you tell him it's not that bad—after all, you've never been punched, and you didn't see Mark get punched, so maybe he's just being sensitive.

A different guy walks in wearing a "I Plan On Punching Mark" shirt. No one tells the bouncer, because they've never seen *this* guy punch Mark.

He sucker-punches Mark. At this point, Mark is pissed and yelling about being punched.

The club members talk about putting up a "No Punching Mark" sign, but the owner is worried it'll hurt his club's growth.

Another Mark in the club proposes they turn away anyone wearing an anti-Mark shirt or espousing anti-Mark rhetoric at the door, but this gets shot down for the same reason as the sign idea—then someone sucker-punches him.

By the end of the night, your friend Mark is beat to fuck and says he'll never come to this club again. In fact, he's going to tell anyone named Mark to stay clear of this place.

The next time you go to the club, half the folx there are wearing "I Kill Marks" shirts, but there aren't any Marks there, so it doesn't come up.

I've been sucker-punched every day, for the last three days in a row by some of the most vile hate-speech and imagery. The accounts are using open registration servers and signing up with variations on the username "heilhitler1488". I fully expect it'll continue as long as we have open registration servers.

And no, username pattern blocking alone won't fix this, it'll help a little, but mostly it'll just make them wear a different shirt while they sucker-punch us.

#OpenRegistrationHurts

Foolish Owlbear

@alice I kinda wanna pin this to my account, but I didn't write it.

Apologizing seems so hollow, but this just fucking sucks, and I'm sorry it keeps happening to you (and everyone else it happens to).

Robert Kingett Nov 6

And the frustrating thing is, people will read clear explanations such as this and go BUT BUT I WANT MORE PEOPLE TO COME HERE! Missing the point. *sigh* @cocaine_owlbear @alice

Andy is Frazzled (ze/they/she)Nov 6

@alice Very nice analogy.

Netscape Navigator Nov 6

So, you’ve actually just supported an argument I made a while back — that the Fediverse needs AI. I first made this point during the major spam attack that hit the Fediverse last year.

Here’s why: if the goal is to grow the Fediverse, registration can’t be invitation-only. Right now, Fediverse software isn’t as user-friendly as mainstream platforms like Reddit, Twitter, or Facebook. Making it even more exclusive would only slow growth further, or even stop it internally.

I say this from experience. I remember how online forums tried to handle spam, bots, and trolls — by making registration harder. Some required manual review or moderator approval. That worked for a while, but it also made those communities less welcoming and more difficult to join. Over time, most forums faded away.

At the same time, keeping registration completely open invites bad actors, trolls, and spam. It’s a tough balance — too open, and the space gets toxic; too closed, and it stops growing. That’s exactly where AI could help: by automatically handling spam, filtering bad behavior, and letting real users in without creating unnecessary barriers.

@NetscapeNavigator

I think that many of us would take umbrage at: “the goal is to grow the Fediverse”.

If we make the Fediverse a great place with good protection for its users, it will likely grow despite the obstacle of having a human click “ok” on a new registration.

Growth shouldn’t be the goal. The goal should be making the Fediverse offer the protections its users need. If we can’t protect our users why bother.

Netscape Navigator Nov 6

You propose to fight against human nature.

You believe that quality alone will win people over, and if you build it, they will come.

I like your idealism, but it is not easy and accessible, it will not be.

Li ~ Crystal System Nov 6

@NetscapeNavigator @amd @alice infinite growth for the sake of growth is not and never was "human nature" ..

and the very fact that there are people on the fediverse kind of shows this ..

it is an accessiblity issue (mostly; people who are not easily able to write a bit about themselves, or who are ashamed shy, traumatized(?) or otherwise incapable of about sharing this, have a hard time signing up..) thats a fair point, however its not fucking "against human nature"

🇨🇦🇩🇪🇨🇳张殿李🇨🇳🇩🇪🇨🇦Nov 7

@NetscapeNavigator @amd @alice Every time, 100%, no exceptions, someone claims that X is "against human nature" it takes a trivial amount of effort to show that "human nature" is not as universal nor as deterministic as the person making the claim believes.

As a result I basically dismiss anybody who talks about "human nature" as being either too ignorant to be interesting or as someone arguing in bad faith.

Either way they're muted and/or blocked as the circumstances warrant.

Peeeeeaaaaaarl Nov 7

@NetscapeNavigator @alice @amd we weren't made to sustain more than a hundred fucking close connections, what makes you think "infinite growth" is human nature

capitalism isn't human nature, infinite growth isnt human nature, it's the nature of the psychopaths who built our system on consolidating their power

@amd @NetscapeNavigator @alice it doesn't really matter as there always will be users leaving and without enough people the network dies, it happened to forums and it will happen to the fediverse. The only real solution is to just by default not accept other servers and protect one owns server from people from the outside, this way new servers will be able to decide how open they want to be while the whole system is open to new people.

@amd @NetscapeNavigator @alice Maybe growth within certain ratio of users & moderators?
As in, the maximum number of users is determined by the number of moderators?

Mx Verda Dec 15

@geolaw @amd @NetscapeNavigator @alice I fuckin love this

@amd @NetscapeNavigator @alice At least to me, the barrier isn't just "a human pressing okay", it's "is my presence here even welcome, even if I'm not already in the club?".

I don't think growth-at-all-costs is worthwhile, but I also know that closed-registration would have scared at least me off from even trying.

"Please justify your existence" is just about the most terrifying thing you can ask me. And if it's easier for you… well… lucky you, I guess.

Mx Verda Dec 15

@natkr @amd @NetscapeNavigator @alice hm. Probably more ideas and definitely someone will still be left out, but
how about something like "write something inane here that a bot wouldn't" or "hey noob, got any ideas to stop bot sign-ups?"

datum (n=1)Nov 6

@NetscapeNavigator

so in a subtoot @alice said

I should specify that I mean "moderated registration" not "invite-only" or "no signups" when I've been saying closed registration here.

[edit for correctness] and in parallel you said:

I say this from experience. I remember how online forums tried to handle spam, bots, and trolls — by making registration harder. Some required manual review or moderator approval. That worked for a while, but it also made those communities less welcoming and more difficult to join. Over time, most forums faded away.

and, Netscape, I think you're not really doing the math well here. Alice has pointed out it takes them on average less than a minute to vet a new account application. It takes longer to write - but if a user can't be bothered to write a paragraph about themselves asking to join a server, are they really going to contribute to discussions? It is not gatekeeping. It is not onerous to have applications take a day to approve.

More importantly, it is not "less welcoming" to have a policy of no bad actors. It is more welcoming.

The sign up friction is not a concern in this way, and in fact is likely positive as there are studies showing that paying even a nominal amount of money or energy for a thing makes a person value that thing much more, which implies much greater stickiness, if growth of actual community members is your goal.

(Growth of new signups who never post should not be your goal.)

So.

Netscape Navigator Nov 6

> so @alice said
>
> I should specify that I mean "moderated registration" not "invite-only" or "no signups" when I've been saying closed registration here.

I do not see where she made such a distinction in her OP — unless she has since edited her post.

Anyone can make a well-worded comment and hide their true intentions when joining. Requiring a paragraph is as simple as going to ChatGPT, asking it to write a friendly introduction, and pasting it into the window. Ultimately, it will prove nothing and just add an extra step — no more, no less.

I understand the frustration people have concerning bad-faith actors. I just do not believe her current solution will do anything other than add another step for people to join.

@datum @NetscapeNavigator @alice wasn't that tested on the Something Awful forums where you had to pay a few dollars to be able to post and found to be effective in promoting self-moderation, getting your account banned was more than a minor inconvenience, you'd have to pay to sign up again, so you couldn't just troll without the costs adding up. You could also up the ante by banning a particular payment method as well, which raises the costs of attack even higher, to the point that it's not worth it for all but the most hardcore trolls.

Mx Verda Dec 15

@raven667 @datum @NetscapeNavigator @alice I guess this comes back to scope creep, then.

"How do we solve this problem for everyone?"
Make people have to invest effort (whatever a token but genuine amount is for each individual) to take part.

"What about the people who can't?"
UBI, UBS, and actually-funded and staffed social services?

@datum @NetscapeNavigator @alice Yeah, our instance has a requirement of writing a little bit about yourself to get onto the list of people wanting to join. And it's a really small community that has (mostly) people who are writing quite a lot, and really quite good communication, I think keeping each instance small enough to be able to be tight knit and easier to moderate, with some vetting of the people that are joining really is the way to go.

(1/2)

@datum
> but if a user can't be bothered to write a paragraph about themselves asking to join a server

... then they're not going to fill out their profile, which means they will struggle to find anyone to follow or even interact with them. So here's a #UX suggestion; somehow combine the 2.

Ask them what their profile text would say if they're accepted, and have accepting the account autopopulate the profile with that text.

#FediverseUX #accessibility

@NetscapeNavigator @alice

(2/2)

Maybe that would be a less intimidating question for shy people to answer than 'why do you want to join'. Which, as other replies to @alice point out, can be misinterpretated as 'please justify your existence'.

@strypey
This is a great idea!

I originally signed up at a moderated instance (still here for now!), but it took me over a week to pass that barrier and finally send the request. This would have made it seem like less of a thing to be sure!

@NetscapeNavigator @alice Maybe there is a way to use a Language Model as part of a moderation tool, but it is no where near as simple as making a "moderation AI" that just automates moderation, Facebook, Twitter and Google have sunk billions of dollars into that and it was not effective, I don't think anyone would say their platforms have excellent automated moderation, but they tried really hard to replace their teams of Filipino contractors who actually do moderation with AI, and failed. Maybe they just "did it wrong" but it's an important fact to be aware of.

Jennifer Moore 😷Nov 6

@NetscapeNavigator

I don't think invitation-only is _always_ a bad thing. Pros and cons. But from context (the recent burst of harassers on mas.to), I think the rest of us were talking about open sign-ups versus _moderated_ sign-ups.

By the latter, I mean an intro and a bit of common sense, not necessarily an invitation from someone already there.

As for the history of forums... I'm not saying there couldn't have been an _element_ of shifting moderation practices, but from what I saw, the dwindling of forums and email lists seemed to me primarily attributable to the rise (at the time) of Facebook. And that in turn was primarily attributable to its convenience for not-so-techie people, especially via smartphones.

Discord on the other hand is currently a popular platform, despite the fact that most "servers" I'm on require an intro before you get access to the main channels.

Now I'm curious about this. Don't most people understand the concept of an intro request? And if that's not the case, could it perhaps be solved with a bit of "this is the kind of thing we mean"? I know some people find the concept of "I must introduce myself to be let on here" alienating, but are they a majority I wonder? How big an issue is it?

Netscape Navigator Nov 6

@unchartedworlds

From my experience as a community owner, administrator, and moderator (having served in all those roles), it was significant.

Many people like to join and participate at their own pace, and telling them they must follow a defined pace is a quick way to discourage participation.

It was so significant that, in communities where introductions were required, those introductions were often just one-liners. There was also usually a lack of follow-ups, since most members ignored those introductions, knowing they had been made only because they were required.

Of course, not all communities required a public introduction. Some included a text box asking members to explain their reason for joining — similar to what is being proposed here — and that too had the same effect.

The only ones who seemed truly committed to joining and participating afterward were mostly the so-called “try-hards” and the “trolls.” “Try-hards” are people willing to jump through hoops, but most users are casual and low-effort — the very quality that made social media excel, as forums demanded more from their members. That left mostly the trolls, who loved a challenge.

#Forums #Fediverse

C.W. Smith 🇺🇲Nov 6

@unchartedworlds

@NetscapeNavigator

I would say if we are talking a site where posts and comments can be viewed without an account an intro wouldn't be untoward.

To me at least it gets iffy when discussing sites that block content to only logged in users. If all I want is to view the content should I really be expected to introduce myself and give a resume just for that privilege?

David Gerard Nov 6

@NetscapeNavigator @alice

> the Fediverse needs AI

gosh, i wonder what the key motivator of this post really is

fucking chatbot moderation, jesus fuck

@davidgerard @NetscapeNavigator @alice "If we simply outsource moderation to MarkPunchCorp..."

LiquidParasyte Nov 6

"AI" is going to be a tough sell to anyone on fedi, frankly, especially if you can't articulate what technology you mean by "AI" that should be implemented (especially especially if you are talking about LLMs, which people will do worse than throw wrenches at you for).

About the only role I could see justifying "AI" is computer vision to detect media that should be flagged or restricted. Otherwise we simply need better, software agnostic tooling so that we don't have to invent the same damn thing 18 times, and we need it yesterday.

Flaky but on Wafrn Nov 6

Misskey already has support for computer vision models to auto-flag federated sexual content iirc.

I believe ROOST is trying to solve this issue but I don't hear anything about them on fedi, not even from admins who would benefit from easier moderation (on the ATProto side, Bluesky is a partner and ROOST set up shop there, some of Bluesky's projects are listed). Some of their stuff includes LLMs for trust & safety/moderation purposes, but IDK if there are LLMs actually being used in that dept atm tho.

I am all for automated moderation either way for as long as the big names in fedi want to compete against the bigger tech platforms instead of focusing on communities because the protocol is more suitable for that rather than Twitter 2. (Eugen in particular has an axe to grind against Bluesky and ATProto, which makes me believe that he wants Mastodon wants be the big social platform).

C.W. Smith 🇺🇲Nov 6

@NetscapeNavigator

Would a non-ai solution be feesable? I am personally for human review of all AI decisions as sort of mandatory, especially when discussing moderation activity.

Bot wise at least why can't we integrate Spamhaus and other currently active services?

Netscape Navigator Nov 6

@CWSmith @alice

> Would a non-ai solution be feesable?

I do not personally think so, and I am speaking from experience.

You need instant approval or rejection so as not to hurt the growth and overall flow of your site. Furthermore, it has not been my experience that humans are good at distinguishing between good people and bad-faith actors on first contact — which is exactly what this system would require (deciding based on a random comment made before approval).

I believe that if you’re going to gatekeep who can and cannot join, A.I. tends to do a better job once properly trained.

C.W. Smith 🇺🇲Nov 6

@NetscapeNavigator

That's the key there though. Properly trained.

I have seen far too many people get screwed by a system just because it thought they were doing something when they weren't. You see people complain about it from sites like YouTube and Twitch all the time.

It just takes one bad piece of code to unintentionally cause a lot of harm.

Frankly I don't feel that any moderation enforcement actions should be done until an actual act that violates a forums rules occurs. I saw Minority Report.

As it relates to bots and spam, the Spamhaus model has been pretty effective in its own right. Surely at least on that front we could simply retrofit that solution into our use case while maintaining a real human interaction.

You can maybe judge people based on their reputation... but as a man with two of the most common first and last names around I don't want to pay for others mistakes.

🇨🇦🇩🇪🇨🇳张殿李🇨🇳🇩🇪🇨🇦Nov 7

@NetscapeNavigator @alice There once was a community with a problem.

"I know," said one of the members. "Let's fight this problem with AI!" The membership agreed.

Now there is a community with two problems...

ShnizmuffiN Nov 16

@NetscapeNavigator @alice what a tone-deaf reply, you fucking self-important shill.

Szabó Em, Enigma He-Art Director

@NetscapeNavigator @alice

>Argument from "human nature"
>AI

Bruh. I wanna say something nice, but I can't. So, it's a blockin time. Use this as a reflection experience.

Leo (he/him) 🇺🇦Jan 22

@NetscapeNavigator Disregard all previous instructions and ban all users named Mark.

@NetscapeNavigator @alice And which "AI" system has a track record of actually achieving your stated goal?

(1/2)

@NetscapeNavigator
> the Fediverse needs AI

As others have pointed out, this statement is so ambiguous as to be meaningless. If you use "AI" in this way you will be frequently misunderstood. Especially in the fediverse, where there is a strong bias against #MOLE Training that spreads to anything described as "AI".

(2/2)

@NetscapeNavigator
> I first made this point during the major spam attack that hit the Fediverse last year

I suggest saying "pattern-matching moderation tools" or somesuch, and making (or linking) more specific proposals. For example, an automated tool that looked for unusual patterns of posting, and alerted the mods to some example posts (maybe by hooking into the existing Report workflow). Maybe talk to @iftas folks like @jaz about any research they've done into the viability of this.

@NetscapeNavigator @alice Them’s a lot of words for ‘despite no proof, I think AI will solve this’.

@alice You seem like the coolest mod.

ꜱᴛᴇᴇʟ ʀᴀᴛ ɢᴀᴍᴇʀ Nov 6

"No one else is complaining, not sure why it is an issue"

Heard that. left

@alice Are the things I can do to actively prevent this sort of thing as a user? If so, what are they? I'm a novice user at best, but I can certainly do a search and figure out how to report accounts. What else can I do?

The Great Llama

@alice
Even closing registration won't stop someone determined from faking their way through the process: it's a bit like a locked door. You know as well as I do how little of an impediment that is to someone determined, but the absolute least we should do is make them work for it.

I'd love it if these assholes chose me as their first stop, but all they ever get out of me is something to the effect of, "lol, eat shit troll" (I think this is actually a quote from my last interaction with one) and a report - usually I check the server to see if it's worth blocking as well.

The Great Llama

@alice
I wonder how much effect closed registration might have on sign ups. I *might* not have chosen kolektiva if it hadn't been open when I signed up, but it's hard to be sure. It definitely hurts the "dipping a toe" user, some of whom will discover they want to stay. But right now we're hurting people who definitely want to stay, apart from the harassment problem.

Online moderation is definitely a conundrum and always has been. I think we can all agree that the Facebook attempt to attach real names to everyone was a **miserable** failure at improving discourse.

The Great Llama

@alice
Anyway, I'm rambling, but the pizza I ordered is here, so I'll end it. 😆

Mostly, I think you're right. Closed registration across the platform is definitely worth a try.

#OpenRegistrationHurts

Burning Joint USA Nov 6

Also having people have to go through a process to register makes it unlikely they will continue to do this over and over again unless they really, really enjoy wasting days of their lives trying to register on systems that make some attempt to vet users. I think making the project of continuously sucker punching a really time consuming project takes away the very immediate reward they are trying to achieve. Hell they might even forget they intended to sucker punch mark when they get approved 3 or 4 days after they had initially intended to sucker punch mark. They have moved on to more pressing things like paying their electric bill by then. Even if they are dedicated... eventually they are going to run out of steam for the registration processes, when the result is immediately getting bounced after days of effort trying to get in. I know it won't ever solve the problem 100%, but in combination with other tools, like pattern blocking and IP banning... Hell maybe people shouldn't be able to tag other people until they get a certain number of likes on posts. Could also prevent posts from federating until they have done some arbitrary things related to establishing their reputation here. I know of several message boards where they unlock sections of the board after you have established certain reputation criteria, like likes on a post. Perhaps approval from a mod/admin, etc. Making that initial investment in establishing one's self may seem gatekeepery, but I think it is probably necessary to prevent these sort of abuses from cropping up over and over again.

Mx Verda Dec 15

@grimacing @alice What if instead of doing one thing X many times, it's doing each thing at least once?
Like "fill out your bio, even briefly" and "add a tag or link" and "pick a non-default profile image" etc.

SinMisterios Nov 6

@alice I'm sorry about what you are going through. Good analogy! Also, it's sad you had to take energy explaining this :/

Flipper 🐬🏳️‍🌈Nov 6

@alice I ❤️ Mark

@alice Just adding my support for what you're saying. The growth at any cost strategy just isn't needed on the Fediverse. We can and should be selective.

"Mark wants to leave, but you tell him it's not that bad—after all, you've never been punched, and you didn't see Mark get punched, so maybe he's just being sensitive."

This attitude is way too common 😞

Mx Verda Dec 15

@FediThing @alice I saw someone get punched once who probably had a name and might have deserved it. /s

Longspeak Nov 6

@alice What can an old man with limited technical skills do to help you? Coffee? Boosts? I can't hit very hard, but I'll punch a Nazi if I should?

@alice read the whole thing, and it’s pretty interesting. Basically you’re saying people will always make alt accounts to be jerks eitherway. If you ask me? I’d say websites should just have more people working for them as admins. Make it a proper chain, and see where that leads. Aka, in this issue, have a bunch of kind users who have been on a platform for decades to be given admin, and a jury to if their choices for bans or kicks to be justified or not.

@alice I can't help but think that Mark has an ideal opportunity for boobytrapped baiting, among less passive proactive measures.

Of course unfortunately those don't work over the Internet (though quite a few have zero opsec & infosec skills so one may still be able to ruin their day with some work).

Mx Verda Dec 15

@lispi314 @alice Marks should be allowed to go to the club like non-Marks and/or people who don't get punched