I have been using email for 40 years. It used to work.

As an (independent) academic researcher, I need to contact new people, primarily in universities, to ask questions.

I refuse to use Google, Microsoft or the other American IT giants.

But they are increasingly preventing refuseniks from sending email at all.

I know what RFC, DNS, MX, SPF and DMARC mean. My email goes through small British companies with intelligent, friendly and helpful staff.

mxtoolbox.com says that I must have DMARC to send email to M$. So I set it up. I now get a dozen copies of the same report from G or M$ for each email that I send out.

They show that my email gets to G and M$ sites, but then it is marked as spam.

The stupid senior management of numerous universities has surrendered their staff email to M$.

Web searches and AIs preach about spam. I don't send spam - I want to contact my colleagues.

Rumour has it that previously unknown senders are treated with suspicion and their emails are sent to spam. In other words, it is impossible to **initiate** communication with someone.

Let's be blunt about this. They are a mafia that is enforcing an **oligopoly**. It's got nothing to do with reducing spam --- I have no doubt that they let through emails from "trusted partners", ie companies that bribe them enough to send their spam.

The result of this is that it will only be possible to send emails by paying M$ to do it, and then it will only be allowed to express "approved" opinions.

What can we do about this?

At the very least, those of you with senior positions in universities can tell your management to revert to competent standards-based email systems hosted on Linux systems.

@Paul_Taylor At a "University" I refused to use their M$ "e-mail system" as it failed to be a bonā fidē e-mail system. My supervisor agreed with me that it was not an e-mail system as it did not operate using standard protocols. IIUC, M$ had an exclusivity clause that prevented any other e-mail system from operating within their TLD that did not already predate the contract with M$. Thus there was nothing to be done: use it or else!

@dgb37 @Paul_Taylor they didn't even allow #IMAP & #SMTP?

WTF??

@kkarhan

Not sure how similar it is, but I've had issues with $DAYJOB for the last couple years where, while I can use IMAP/SMTP in _theory_, because they use OAUTH2 which seems to send a client-ID in the request, I can't connect in practice because they refuse to approve any client-IDs that aren't Outlook. 🤬

@dgb37 @Paul_Taylor

@gumnos @dgb37 @Paul_Taylor So you gotta have to use commercial addons like #Owl for @thunderbird

  • Shite like this is why I refuse to work for companies that do this to me as #Sysadmin:
    • If they don't trust me keeping #ITsec up and running and knowing my #TechStack then why should I trust them to pay me on time?
Owl for Office365

Owl connects Thunderbird to your Office365 email account

Owl for Office365

@kkarhan

Yeah, in this case I use mutt(1) and there are OAUTH2 modules to get it connected in theory, but without the corporate-approved app-ID, it's a non-starter. 😑

@dgb37 @Paul_Taylor @thunderbird

@gumnos @dgb37 @Paul_Taylor @thunderbird and I guess you can't just fake "Microsoft Outlook" as "App-ID" because they've to manually add any App...

  • Seriously, this kind of necessity to workaround horseshite needs to be outlawed!

@gumnos @kkarhan @dgb37 @Paul_Taylor @thunderbird

I've been having success with Evolution in the exact same circumstances. But I have not found any other solution that let me spoof the client-ID and UA of "Outlook" to an extent that convinces Microslop to accept my client with those restrictions.

Kevin Karhan (@[email protected])

@[email protected] @[email protected] @[email protected] @[email protected] and I guess you can't just *fake* "Microsoft Outlook" as "App-ID" because they've to manually add any App... - Seriously, this kind of necessity to workaround horseshite needs to be outlawed!

jorts.horse
@kkarhan @dgb37 @Paul_Taylor Proton Mail doesn’t support IMAP and SMTP, yet people still use it. (They offer a “bridge” you can run that accesses the mail and then lets you connect to your bridge machine with a standard mail client.)

@mathew @dgb37 @Paul_Taylor Really?

I mean, I knew #ProtonMail paywalled #IMAP + #SMTP in the past, but this makes it absolutely worse than @monocles / #monoclesMail and even cock.li

https://docs.monocles.eu/services/mail.service/

monocles mail - monocles Documentation

@kkarhan Really. I see no difference between MS, Google and Proton ecosystems. They're the same thing with different marketing campaigns.
@kkarhan @dgb37 @Paul_Taylor
Microsoft tried to stop letting 3rd party clients use IMAP & SMTP to connect to Hotmail/Outlook a couple years ago. Like, WHAT!?!?!?

@TheZwick32 @dgb37 @Paul_Taylor granted #Microsoft's #patents in #MAPI are the reason @thunderbird cannot support it!

MAPI - Wikipedia

@dgb37 @Paul_Taylor Solution: Acquire new TLD and support it officially.

Fuck corposcum.

That they did not immediately resort to this is indicative of malice or complicity.
@Paul_Taylor I run a self hosted mail server. It's kinda hard to get right, but the requirements seem absolutely reasonable and you can fulfil them.
I don't really see them abusing their market power there ... just yet. (I think they are just waiting for email to die the natural death)
@helge @Paul_Taylor no, they're actively taking steps to strangle it. we fulfil the requirements but none of our mail gets through because we aren't big enough to register any reputation in their system.
@atax1a @Paul_Taylor That seems weird, I didn't have issues so far. If the DNS is setup right, trust is essentially granted?
@helge @Paul_Taylor the number of times we have to call people on the phone to tell them to fish our message out of their spam box begs to differ
@atax1a @Paul_Taylor I test that extensively and have no issues for far. If DKIM is not setup properly, you run into this.
@helge @atax1a @Paul_Taylor I've got mixed results from both Google and Microsoft even when DKIM, SPF etc. checks out and everything's set up correctly. In my honest opinion the demand for logging into their postmaster tools to get delivery to their walled garden working is unreasonable and a nuissance at best.
@paavi @helge @atax1a @Paul_Taylor I have no issues even without DKIM.

@helge @atax1a @Paul_Taylor not necessarily. where do you host your mailserver? because reputation of your ip _and_ neighbouring ips are taken into account.

i'd agree that the published requirements like dkim, dmarc, etc are good actually, but there is more filtering happening beyond that :-c

@malte @helge @atax1a @Paul_Taylor

I can live in a beautiful IP neighborhood and keep mine immaculate but a) it takes years to scrub the filth and grime left by the previous tenant, and b) I can't control the dirty deeds going on inside my neighbor's IP, yet I am judged by both. 🤬

@juliewebgirl @helge @atax1a @Paul_Taylor 100%

i hate that the big providers can afford to be lazy enough to judge whole ip-ranges without actually looking what a particular ip is doing, and i only asked because maybe helge is lucky for having a good one 🤷‍♀️

@juliewebgirl @helge @atax1a @Paul_Taylor mild tangent: telekom's mailserver-admins are actually quite accommodating regarding allowlisting single ips. you just have to write them an email, and i guess if you sound vaguely human they just do it ¯\_(ツ)_/¯

@malte @helge @atax1a @Paul_Taylor

I've given up giving the benefit of the doubt and believe everything little thing they do is intentional. I could excuse laziness but they know the little guy can't buy whole blocks and it's the easiest way to... Ok, it's lazy-ish lol Let's call it efficient... to eliminate everyone except the other major player.

@atax1a @helge @Paul_Taylor I got a text message from an organisation that runs summer music camps for children.

We are signed up for two courses this summer so getting details of when, where, how etc is rather important.

The text message said that they had sent an email about one of the courses. Then it went on to say that their emails often end up in spam boxes and asked me to check for this email in my spam box.

They are a reputable charity here in Norway.

@atax1a @helge @Paul_Taylor I am beginning to wonder if, what with proposals for client side scanning of chat systems on devices still on the table and all of the issues with the big email actors blocking others without rhyme nor reason and AI agents with root access to devices perhaps the only way to communicate reliably in the future will be old style SMSs on a dumbphone.

Not terribly secure but neither or any of the above options going to be secure in the future.

#chatcontrol

@atax1a @Paul_Taylor This is good, make sure everything is green here: https://internet.nl
Test for modern Internet Standards like IPv6, DNSSEC, HTTPS, DMARC, STARTTLS and DANE.

Test for modern Internet Standards IPv6, DNSSEC, HTTPS, HSTS, DMARC, DKIM, SPF, STARTTLS, DANE, RPKI and security.txt

@helge buddy we've been doing this for over 20 years and am currently employed professionally as a postmaster
@atax1a I don't have that much experience 🙄, but what is the reason your emails end in spam? Because Google proactively blocks you? I cannot confirm that they arbitrarily do that.
@atax1a What's the domain you are having issues with?
@helge i do not want or need your advice or assistance
@atax1a That's fine, but I'd like to know why Google would be blocking you.
@helge so would i??? they do not explain why??? and their support is proactively useless???
@atax1a By now I guessed it was their fault 🙂
@helge seriously, at our day job they blocked the entire company fleet once, and refused to explain why, tried to insist that the problem was on our end, and generally gaslit us about the situation until we had executives reach out across company lines!
@helge covered this in an earlier post, bye now
@helge @atax1a @Paul_Taylor That's nifty. New to me. Thank you.
@helge @atax1a @Paul_Taylor sometimes yes, sometimes no. basically the problem is that in order for your mail to have a high chance of not being flagged, you must have a good reputation. to get a good reputation, you must prove yourself by sending a bunch of not spam, slowly ramping up the rates.

this doesn't work for small, personal servers because you simply just don't have that much mail to send.

if you get lucky, you might be fortunate to have an ip with good reputation and not have to deal with that
@stag @atax1a @Paul_Taylor My feeling is that for getting a bad reputation you really have to be on a very fishy provider.
In the particular case of the original poster it seems to be a clear miss on the DKIM requirement (which is reasonable). Nothing reputation related.
@helge @atax1a @Paul_Taylor generally from my experience big tech's spam filters will always reject any email from residential ips and reused cloud ips (like those you would find from aws or gcp)

i've had better luck with other providers like hetzner, but it really is a hit or miss
@helge @Paul_Taylor
I agree it's still possible to run your own mail server. It's got harder over the 25 years I've been doing it for... I now have SPF, DKIM, DMARC, DNSSEC and full IPv6 support with rDNS, and I'm just an enthusiastic #HomeLab user with a domestic IP address. As far as I'm aware, my emails are getting delivered to people's inboxes.
@dave @helge @Paul_Taylor I still run my email server too for family and some friends but email as a whole is something I try to get rid of in my life as it has become more or less unusable and mostly an annoyance.
@joacim @dave @helge @Paul_Taylor I too feel exactly the same way, and have given up self-hosting email, after making every valiant attempt.
@dave @helge @Paul_Taylor it's non-deterministic in some cases, even when all the right DNS voodoo is performed (SPF, DKIM, DMARC, DNSSEC and full IPv6 support with rDNS). Please don't make out that it's deterministically reliable *for everyone* with their self-hosted domains, and it's just that they haven't done the right DNS rituals.

@helge @Paul_Taylor As long as the ip range from your provider is not on any blacklist on google or ms servers, you are in luck. But that can change anytime. And neither you or your provider can do anything about it.

But hearing, that universities don't use their own servers makes my blood boil.

@helge @Paul_Taylor I am running stalwart (https://github.com/stalwartlabs/stalwart) on my server and so far it works ok and is quite simple to setup
GitHub - stalwartlabs/stalwart: All-in-one Mail & Collaboration server. Secure, scalable and fluent in every protocol (IMAP, JMAP, SMTP, CalDAV, CardDAV, WebDAV).

All-in-one Mail & Collaboration server. Secure, scalable and fluent in every protocol (IMAP, JMAP, SMTP, CalDAV, CardDAV, WebDAV). - stalwartlabs/stalwart

GitHub
@Paul_Taylor would you mind trying to send a mail to my email at cispa.de? Should be very easy to find from my name, just don't want to get ingested by every mastodon account ever.

@Paul_Taylor DMARC reports don’t tell you whether your mail went to spam or not. They tell you when your mail was not authenticated when it was received.

There’s something to investigate there, but it may not be what you’re describing it as.

@lluad @Paul_Taylor my favorite is getting a DMARC report for an sender my SPF record forbids, thus should not have generated a DMARC report for

@drbrain @Paul_Taylor

DMARC reports are to tell you about delivery attempts “From:” your domain that are not authenticated.

If your SPF records “forbid” a sender that’s exactly the situation you have _explicitly_ asked to be notified about.

@lluad @Paul_Taylor ah, then it seems odd that DMARC would be a requirement to deliver mail

@drbrain @Paul_Taylor It’s not a general requirement.

If you’re sending “bulk” mail then some large consumer ISPs (Google, Yahoo, that sort) require you to have DMARC records in place or your mail is likely to be rejected. “Bulk” doesn’t have a hard volume cut off, for good operational reasons, but unless you’re sending several thousand a month that’s not you.

The only real requirement for modern email is DKIM signatures, ideally aligned with the From: header. Some outliers want SPF too.

@Paul_Taylor A big part of the answer is teaching more folks to run their own email systems. There's a shameful tendency for tech people who should know better to warn people off running their own email servers, which makes it worse.

I teach people to run their own email, and people generally love the freedom it gives them.

@mason @Paul_Taylor MS and Google say they filter spam by ”artificial intelligence”, but they don’t. They just block everyone who doesn’t play by their rules.

@gimulnautti @Paul_Taylor I don't entirely agree. They say what their rules are, and then they disregard them. They're bad email neighbors.

Frankly, I've been running email for years longer than Google has, so I feel justified in this opinion.

@gimulnautti @mason @Paul_Taylor It's worse than that; gmail's filtering is so bad, if I send email to myself, it gos in spam; notifications from Adwords go in spam. They can't even get past their own filters.