Dylan Freedman

Anyone familiar with the way Mastodon handles verifying URLs? At the Washington Post we're working on adding verified rel="me" and Mastodon links in our author profiles, but we're seeing mixed results on verification actually going through. It seems non-deterministic(?)

@dylan (me): works
@jeremybmerrill: does not work even though he's on the same server as me
@zubakskees : works
(+ more)

The author pages on the Post are identical in terms of linking back to Mastodon. #mastodon

Dec 22, 2022 at 6:23pmWeb
Show threadBrandon HorstDec 22, 2022
@dylan @jeremybmerrill @zubakskees Mastodon doesn't re-verify unless you update the URL in settings. If you added the rel=me after adding the verification link, it'll never go out and check.
Show threadDylan FreedmanDec 22, 2022
@brandonhorst @jeremybmerrill @zubakskees We are re-adding the links and getting mixed results. It worked for me when I removed it, saved, and added it back in; but alas it did not for Jeremy
Show threadOpenDNA⚙️Dec 22, 2022

@brandonhorst @dylan @jeremybmerrill @zubakskees IMO Brandon's got it. Jeremy should remove the link, save, then add it back, and save. (note: URL *must* include https)

Source: lots of testing while writing the "Simple Mastodon Verification" plugin for Wordpress.

Show threadThomas H Jones IIDec 23, 2022

@opendna @brandonhorst @dylan @jeremybmerrill @zubakskees

Bah... Guess I need to letsencrypt my nearly-abandoned, long fallow vanity domain's homepage. :p

Maybe over the holidays.

Show threadOpenDNA⚙️Dec 23, 2022
@ferricoxide Putting it behind CloudFlare is sufficient... 🫢
Show threadThomas H Jones IIDec 23, 2022
@opendna Letsencrypt is completely free. :)
Show threadDamon KiesowDec 22, 2022
@dylan @jeremybmerrill @zubakskees Dumb question maybe, but has everyone deleted and re-added the links in their profiles? That triggers the verification more quickly.
Show threadDylan FreedmanDec 22, 2022
@dkiesow @jeremybmerrill @zubakskees Yep, we all did that. For me, it verified pretty immediately. Others had less luck
Show threadKatmaDec 22, 2022
@jeremybmerrill @zubakskees @dylan
Can anyone advise? #journalists #journalism #verification #WashingtonPost #TwitterMigration #onboarding
Show threadGlenn FleishmanDec 22, 2022
@dylan @jeremybmerrill @zubakskees Silly to ask, possibly, but are you forcing a secure connections as in typing out https://washingtonpost.com/etc.? I saw one account that someone had entered a URL and it didn't verify but explicitly starting it with https:// instead of leaving it implicit caused the verification to work.
Show threadThat Blair GuyDec 22, 2022
@glennf @dylan @jeremybmerrill @zubakskees That's one I fought with for a bit. Definitely include the https:// on the mastodon side of things.
Show threadryanDec 22, 2022
@dylan @jeremybmerrill @zubakskees @jeremybowers I’ve found it to be very inconsistent from instance to instance and I’m not sure why — I got The Marshall Project hooked up a couple weeks ago and it still doesn’t show as validated on its home server (newsie.social), but when I view the profile from other servers it validates correctly
Show threadJack Daniel (often offline)Dec 22, 2022
@dylan @jeremybmerrill @zubakskees I have very limited experience, but I found one of the wiki platforms I use (PBWorks' PBWiki) doesn't support the "rel" tag and removes it. Worth checking to see if your platform(s) are stripping the tag when you add it.
Good luck.
Show threadAnthony NiximaccoDec 22, 2022

@dylan @jeremybmerrill @zubakskees I had this issue with links not verifying for a little while. My problem ended up being that the link verification only seems to work if it's https:// and does not work with an http:// link.

That solved my issue and my links page was verified. Looking at the profiles you linked shows the links pointing to https:// though. So I'm not sure. Maybe on the front end Mastodon sends all links through https:// ?

Show threadDarnell Clayton Dec 22, 2022
@dylan @jeremybmerrill @zubakskees I think it would be better if Washington Post posted (pun not intended) underneath their own domain. It would be a better way to verify that you are reporters from the publication.
Show threadAmy (she/her)Dec 22, 2022

@darnell @dylan @jeremybmerrill @zubakskees while this isn't a bad suggestion, it is a limiting one.

The argument I've seen against it is if the reporters move to another publication, they don't get to keep their social media without having to move instances, which means they leave their old content behind even if they are allowed to initiate a move and take their followers.

There are pros and cons of both, to be fair.

Show threadDarnell Clayton Dec 22, 2022
@DeathKitten @dylan @jeremybmerrill @zubakskees That is a concern. I guess they could remedy this by running an instance underneath their own personal domain.
Show threadAmy (she/her)Dec 22, 2022

@darnell That's a lot of work though. As I said, there's pros and cons to having a newsroom run instance vs anywhere else on the fediverse, so mostly just wanted to squeak up so it's an informed choice.

Personally, I'd like to see the news sites post their articles on a long form activitypub compatible front end, and verify the reporters and other staff on whatever instance they're on.

Then people can follow the journos and the publication directly as they prefer.

Show threadDarnell Clayton Dec 22, 2022
@DeathKitten I believe the Washington Post could do this via a plugin as I think the site is powered by WordPress.
Show threadAmy (she/her)Dec 22, 2022

@darnell That would be awesome. I installed the activitypub plugin on my wp, but need to poke at it more.

I need to figure out how I broke it, as I got it so people can find *this* account searching any handle @ my domain on mastodon, but not finding my blog when searching the handle the plugin says should find it.

Show threadDarnell Clayton Dec 22, 2022
@DeathKitten I wonder if another plugin is causing interference‽
Show threadAmy (she/her)Dec 22, 2022

@darnell I wouldn't be surprised, since 99% of the time when something breaks with WP it's from plugin conflict. I just need to find time and spoons to troubleshoot it.

For now, wp automagically toots to this account when I make a new post*, and I don't post all that often, so it's good enough.

* I wouldn't be surprised if this plugin is where the problem lies, and I'll probably start there when troubleshooting. 🤞

Show threadMarcus "MajorLinux" SummersDec 22, 2022

@dylan @jeremybmerrill @zubakskees I used a GPG key for my verification.

https://www.redhat.com/sysadmin/mastodon-verification-cryptography

How to verify Mastodon social platform users with cryptography

Use open source GPG key pairs and Keyoxide to prove your identity on Mastodon.

Enable Sysadmin
Show threadNick IstreDec 22, 2022
@majorlinux That is pretty cool! Just setup the same on my own account.
Show threadRairiiDec 22, 2022

@dylan I just tried doing a request with curl, and I noticed I had to ensure curl followed redirects and stored cookies somewhere before it gave me a proper response.

I wonder if it's some weird inconsistent "anti-bot" thing, and that's what's causing the issue.

Show threadfuomag9Dec 22, 2022

@dylan @jeremybmerrill @zubakskees

In my case my instance verified @jeremybmerrill correctly

Show threadRussDec 22, 2022

@fuomag9 @dylan @jeremybmerrill @zubakskees

Weird, didn't happen when I looked at his profile on your server:

Show threadfuomag9Dec 22, 2022
@theruss @dylan @jeremybmerrill @zubakskees that’s so strange!
Show thread[shrisec@parrot]─[~]: $Dec 22, 2022

@dylan Also can anyone list the steps to add "rel=me" and do everything to verify the links.

Or maybe point me to somewhere, from where I can read and follow the steps.

Show threadTim NolteDec 22, 2022
@dylan @jeremybmerrill @zubakskees are you removing it from the profile on Mastodon, saving, then adding it back, and saving again? Verification on Mastodon is a one-time check. if the link is already added on the account nothing will attempt to validate it again unless you do those steps.
Show threadTim NolteDec 22, 2022
@dylan @jeremybmerrill @zubakskees another aspect that might be at play is geographic site caching or paywalls. If the Post is blocking access using a paywall after so many reads from the same source that might be blocking verification as well.
Show threadGuinnessDec 22, 2022
@dylan @jeremybmerrill @zubakskees in itself, the protocol used is XFN (https://gmpg.org/xfn/), so it should "just work" but if I recall correctly you have to add rel=me on you webpage before adding the metadata field in your account.
Also, second idea that comes to my mind, as XFN rel=me is supposed to link one website with one identity is that since you succeeded, the website is linked to you and nobody else.
Worth to check what XFN says about that, and if it's not another check done by masotodon?
XFN - XHTML Friends Network

Show threadGuinnessDec 22, 2022
@dylan @jeremybmerrill @zubakskees (edit, as it works for two of you, the last section of my former message is a wrong assumption)
Show threadRussDec 22, 2022

@dylan @jeremybmerrill @zubakskees

There are a variety of hiccups with link verification (see https://github.com/mastodon/mastodon/issues/22418 and linked issues).

@info @e_urq I don't suppose y'all could take a look at the logs for the fetch requests for Jeremy's verified link? I did a spot check on the linked page and it looks valid, if potentially a bit challenging to parse.

Website verification: add a */* fallback to the Accept request header for improved website compatibility · Issue #22418 · mastodon/mastodon

Steps to reproduce the problem Run a PHP site on Apache HTTP Server Add Options MultiViews to your .htaccess file, but don't add MultiviewsMatch Any and instead configure AddType application/x-...

GitHub
Show threadEvan Urquhart 🏳️‍⚧️Dec 22, 2022
@theruss @dylan @jeremybmerrill @zubakskees @info This is more technical than I go, but I've passed it along to team members who could help.
Show threadJeremy B. MerrillDec 22, 2022
@e_urq thanks! I also emailed y'all's group email this morning in hopes that maybe the server admins could check their own logs.
Show threadSimon Fondrie-TeitlerDec 22, 2022
@jeremybmerrill @e_urq have you tried removing and re-adding it? I know it's cached, so it's helpful to prompt it to recheck the site.
Show threadJeremy B. MerrillDec 22, 2022
@simon @e_urq yes, sadly, several times :(
Show threadSimon Fondrie-TeitlerDec 22, 2022
@jeremybmerrill @e_urq what about linking to https://www.washingtonpost.com/people/jeremy-b-merrill/?cachebust ?
Show threadSimon Fondrie-TeitlerDec 22, 2022
@jeremybmerrill @e_urq (with the ?cachebust at the end)
Show threadJeremy B. MerrillDec 22, 2022
@simon @e_urq I'll give it a try, good idea
Show threadSimon Fondrie-TeitlerDec 22, 2022
@jeremybmerrill @e_urq Seems like that still doesn't work. journa.host is on masto.host, which doesn't allow access to the logs enough to figure out of there's anything in there. Only theory I have right now is the post's site seems to delay returning information by 8 seconds or so if the user agent isn't a browser agent, which might be hitting a timeout.
Show threadSimon Fondrie-TeitlerDec 22, 2022
@jeremybmerrill I wonder if link previews work. https://www.washingtonpost.com/people/jeremy-b-merrill/
Show threadSimon Fondrie-TeitlerDec 22, 2022
@jeremybmerrill vs Dylan https://www.washingtonpost.com/people/dylan-freedman/
Dylan Freedman - The Washington Post

Dylan Freedman is a software engineer at The Washington Post with a focus on elections and campaign finance.

Dylan Freedman
Show threadJeremy B. MerrillDec 22, 2022
@e_urq although let me slightly eat my words and note that the email appears to have just sent just now because Outlook is awful :)
Show threadRussDec 22, 2022

@e_urq @dylan @jeremybmerrill @zubakskees @info

Thanks, Evan.

I've been lobbying @Gargron and the other wonderful folks developing Mastodon to surface more information about what's going on with link validation in the UI, because so many people are trying to do it (which is great) and sometimes it mysteriously doesn't with (which isn't).

I'd be happy to pay a moderate bounty to get it done.

Show threadRussDec 30, 2022

@e_urq @dylan @jeremybmerrill @zubakskees @info @Gargron

@robpc to the rescue!

Jeremy, that link isn't being verified because the page it's on is bigger than 1MB. Check this out:

https://mstdn.social/@[email protected]ocial/109604273331298601

https://mastodon-link-debugger.vercel.app/profile/@[email protected]

Rob Cannon (@[email protected])

At work, we've been experimenting with link validation and there were several more hurdles than I expected. I put together this link debugger tool as a way to show the problems we found that can prevent validation, and as an excuse to start to play around with the Mastodon API. If you are interested in giving it a spin, just enter your username or profile link and it will evaluate the the links in your profile. https://mastodon-link-debugger.vercel.app/ #Mastodon #LinkVerification #Vercel #SvelteKit

Indieweb.Social
Show threadChris Zubak-SkeesDec 30, 2022
@theruss @e_urq @dylan @jeremybmerrill @info @Gargron @robpc Thanks, Russ. Rob's the technical architect on our team. We've got a PR in to Mastodon to remove that limit, but we'll probably reduce the page weight before that one lands.
Show threadcybrkydDec 22, 2022

@dylan @jeremybmerrill @zubakskees

No noticeable differences between your pages and rel="me" links. Sometimes, it can take a few hours to be picked up. If possible, see if the following format can be added instead to make the link cleaner (without any css refs:

<a rel="me" href="https://journa.host/@jeremybmerrill"></a>

This format tends to be picked up in minutes I've noticed.

EDIT: Make sure to include full https - my link above was truncated

Show threadRobert McNeesDec 22, 2022
@dylan @jeremybmerrill @zubakskees Another issue is whether the link points to an http or https address. Only the latter get verified.
Show threadSuganthan MohanadasanDec 22, 2022
@dylan @jeremybmerrill @zubakskees If the page is more than 1MB it won’t work. I have the same problem. You need to find a lighter page.
Show threadDylan FreedmanDec 22, 2022
@sugan @jeremybmerrill @zubakskees This is a very promising lead and may well be the problem! Thank you for the suggestion. We'll look into this 🙌
Show threadChris Zubak-SkeesDec 22, 2022
@dylan @sugan @jeremybmerrill We now think this is it.
Show threadSuganthan MohanadasanDec 22, 2022
@zubakskees @dylan @jeremybmerrill You can see the issue being discussed here: https://github.com/mastodon/mastodon/issues/15316
Link verification fails silently when page size exceeds 1MB · Issue #15316 · mastodon/mastodon

Expected behaviour Adding a link in the specified rel="me" format to a webpage, and then adding that webpage to a user profile, causes the link to be identified as verified. Actual behavi...

GitHub
Show threadChris Zubak-SkeesDec 22, 2022
@sugan @dylan @jeremybmerrill Thank you!
Show threadRubiconDec 22, 2022
@dylan @jeremybmerrill Since you know yours works and jeremy does have a heavy page, why not just add jeremy's rel=me to your page and link to it for testing purposes?
Show threadPaul Chambers🚧Mar 14, 2023

@sugan @dylan @jeremybmerrill @zubakskees
Suganthan Mohanadasan, I believe your suggestion made it into an article and a tool check.

🔗 Here’s how The Washington Post verified its journalists on Mastodon https://washpost.engineering/heres-how-the-washington-post-verified-its-journalists-on-mastodon-7b5dbc96985c

🔗 Mastodon Link Debugger https://mastodon-link-debugger.vercel.app/

Here’s how The Washington Post verified its journalists on Mastodon

A small cross-disciplinary team of engineers worked together to add a feature so journalists at The Washington Post could link their Mastodon profiles from The Post’s website and verify themselves on…

Washington Post Engineering
Show threadChris Zubak-SkeesMar 14, 2023
@paul @sugan @dylan @jeremybmerrill Honestly we need to update the tool. We removed that limit.
Show threadTim CarmodyDec 22, 2022
@dylan @jeremybmerrill @zubakskees @jeremybowers the only trick to it that I’ve found is that the verification only seems to work when the URL is first added. So if the person already had the URL in their profile, and the rel=me was added after, you have to delete and then readd the URL in their profile.