JL Revilla 
Djoerd Hiemstra ๐I love Mastodon and I research search engines, so when we set up a mastodon instance at the University of Twente, we planned to research federated search, who-to-follow recommendations, and trending topics. Students worked on this in one of our courses:
When engaging with the community, I quickly learnt that many tooters actively oppose search, recommendation and most other "algorithmic" tools. As a researcher in Information Retrieval I was kind of shocked: How could someone NOT like search??
Turns out, search can be easily misused to find persons to harass, searching for instance for "trans rights are human rights" and threaten all those persons is way to easy. Who-to-follow recommendations will then give the trolls more people to harass, as do trending topics. The lack of full-text search is a feature, not a bug or an omission.
If you think this is a relatively small problem, then read up on kiwifarms. (and If you think: "Well, but this doesn't affect me", then for goodness sake, please go back to Musk's sorry birdsite!)
So, I stopped researching (federated) search algorithms for Mastodon and social media in general. OpenWebSearch.eu for the win!
Of course they are, or do you think only your cisgender ass has human rights?
Ruth Mottram@djoerd that sounds really interesting, but I'm not quite sure I understand - is the aim to build a new web search engine?
@Ruth_Mottram We research possibilities for organisations (universities and other public research institutes like Radboud U. and CERN) to collaboratively crawl the web. We will provide an open web index that anyone may use to build a web search engine. Let a thousand googles bloom!
@djoerd that sounds really cool!
Ton Zijlstra@djoerd I like search a lot but the location of where that search is possible matters. Full text search in social media disallows pseudo limited public conversations and creates damaging asymmetries. Unlike on a real public square where many conversations take place in a semipermeable setting and crashing one would immediately carry social consequences for the crasher, in social media that friction needs to be created. Search reduces friction in a way that favours the crashers.
@djoerd If a conversation wants to be findable using # is a way. Boosting is a more natural discovery tool mimicking how it would be on a proper public square. That said I do like search on anything I said , which I can.
@ton search on own posts and hash tags is fine.I like your town square analogy's social consequences analogy.
berkes ๐ ๐ ๐ ๐ฑ@djoerd @ton leaves me wondering: if it is an artificial limitation,it would be easy to circumvent. It's not a limitation of the protocol, but of the client.
An instance could add search or a search tool. Then have their users (Ab)use that to bully, or simply inject themselves in conversations.
All that can be done, is defederate such instances when found out. And block users the misuse such tools.
Hence we should not rely on it for "security", which we currently do.
@berkes @djoerd I'm not sure we rely on it for security. There's never anything inherently private in a public space after all. I see it more as about aiming for friction that balances inherent asymmetries where disbalance favours toxicity. It's what we do offline too and what humans in groups are very good at, attaching a social cost to behaviour, in context and in minute increments even. Circumvention is always possible, but increases social cost to the point of defederation.
@ton @djoerd sure. But the fediverse isn't upping the costs of doing surveillance, it's lowering it.
E.g. each profile has a public RSS feed. But far more is public, in HTML, easily scrapable (courtesy of being accessible & microformats). We have relay servers that are essentially tapping systems (of public communication), etc. etc.
@berkes @djoerd I shouldn't have brought up the analogy. I wasn't making statements about surveillance but about friction. Everything in Mastodon, as on the web in general is public. Scraping etc unpreventable. None of that as such creates toxicity. It being easy or harder to get in my face with something is where upping the friction is a useful tactic as it is offline. Not a cure, just a tool.
@djoerd @ton I'm pretty sure we'll see services soon that offer the following:
* See what people say about your brand (keyword)
* See what posts like to URL
* Find profiles with [advanced search conditions including emoji and flags]
* Reply to posts that mention your brand (keyword)
And we can't do much about it, other than try to hide from it, which is technically impossible.
We're relying on everyone being nice. Which is a very good attitude. Untill it isn't anymore.
@berkes @djoerd We generally always rely on people being nice, when crossing the street, when browsing the shelves in the supermarket. It's a pretty good assumption. Until it isn't. Where introducing higher costs for not being nice becomes necessary. Just browsed my earliest tweets, which were also nice and chatty, like here now, before it became a toxic I-need-to-win-this-conversation thing. I do get your point.
@berkes @djoerd Indeed. It's also why I think defederation is an effective tactic. Esp for smaller instances. I don't need to talk to everyone on the public square, just the _option_ of doing so. I do need the ability to refuse conversation with anyone, and esp to prevent them _joining_ my conversation nonetheless. Making it relatively harder to find my conversations outside my primary circle of interaction helps, granular control over blocking and defederation too.
@berkes @djoerd I just checked during a conversation yday: I've blocked 28 instances in the past 4 years (extremists, sh*tposters, bot-farms etc). About 25% of those in just the weeks since #twittermigration. Which supports your worry.
Dr David R Brake@berkes @ton You are right, this will happen and there are no technical solutions. (I am guilty of doing such analysis with tweers, I admit) Some instances thought about this, though, see:
https://mastodon.social/@socrates@scholar.social/109364972724279253
@djoerd @ton interesting discussion. What strikes me, is the same thing that worries me. We have to ask people to be nice. Rely on ethical researches.
That will get us far.
But we really ought to think about better protection against the inevitable actors that don't listen to our requests. That don't have to adhere to ethical principles. Just saying "we asked not to do this" isn't enough on the internet. B/C all it takes is a single bad actor.
@berkes @ton Others may build different fediverse software, and we will block them, sure. But.. This is not about your usual "they got my password" security. It's about requirements engineering and social design: This software provides hash tag and account name search, but not full text search. Not all that can be done, should be done.
Philip KALUฤERฤIฤ@djoerd I get the argument, but it does sound like an โexcuseโ to tackle a difficult problem. The feature could can be abused, but it would also be helpful to find an old post you didnโt save or other people who share common interests.
IMO one should be open and honest about the shortcomings of decentralisation (it might be a net gain, but it has a cost), instead of trying to making a vice into a virtue.
Cleo of Topless Topics@djoerd that's what I'd assume, but it also makes it ridiculously hard to find people who DO talk about stuff you're interested in. If you could look within an instance without having to join it, that would be lovely... Not like requiring someone to sign up to see it alone stops abuse, they could always lurk then use their other account to abuse
sergio_101 has moved!@djoerd i think it would be really cool to have a "personal" algorithm.. that you could run in the cloud.. that would scour your own profile and the rest of the fediverse and give inferences and recommendations..
@djoerd although i get that there is nothing we can do about it the big picture, search and recommendations tend to be troll bait and waste lots of time.
Frostใ
๏ฝ
๏ฝ้ใฎ็ผ๏ฝไบบ้ข็ฃๅฟใ๐บโ๏ธ@djoerd (just so you know that link is a web UI link, so we can't pop it into our client to pull up the post โ could y'maybe put in the "copy link" link somewhere? It's actually better than URLbar-copying on Mastodon, for once. >,,>)
@frostwolf What do you mean with "copy link"? The instance at mastodon.utwente.nl was shut down in July, so the original post is gone now; it's quite amazing that idf.social still has those posts. Apparently, Mastodon servers hold on to everything they've seen for ever(?)
@frostwolf Now I understand, see:
Djoerd Hiemstra (@[email protected])
Attached: 1 image The link above that does not work shows the following:
@djoerd Ah, yes! That's the actual canonical link. Sorry I didn't get back before, it just totally slipped my mind.
๐๐:~/eu/pl/priv$
@djoerd
i thing search by hashtags is enough because poeple have control over what part of post they want to be searchable.
i thing search by hashtags is enough because poeple have control over what part of post they want to be searchable.
@djoerd unfortunately, that link does not appear to work.
โฆmight work for coffeeโฆNice thread from @djoerd about his work regarding fediverse search and why he stopped researching it. Really worth reading. Thanks!