Mastodawn

I have been using email for 40 years. It used to work.

As an (independent) academic researcher, I need to contact new people, primarily in universities, to ask questions.

I refuse to use Google, Microsoft or the other American IT giants.

But they are increasingly preventing refuseniks from sending email at all.

I know what RFC, DNS, MX, SPF and DMARC mean. My email goes through small British companies with intelligent, friendly and helpful staff.

mxtoolbox.com says that I must have DMARC to send email to M$. So I set it up. I now get a dozen copies of the same report from G or M$ for each email that I send out.

They show that my email gets to G and M$ sites, but then it is marked as spam.

The stupid senior management of numerous universities has surrendered their staff email to M$.

Web searches and AIs preach about spam. I don't send spam - I want to contact my colleagues.

Rumour has it that previously unknown senders are treated with suspicion and their emails are sent to spam. In other words, it is impossible to **initiate** communication with someone.

Let's be blunt about this. They are a mafia that is enforcing an **oligopoly**. It's got nothing to do with reducing spam --- I have no doubt that they let through emails from "trusted partners", ie companies that bribe them enough to send their spam.

The result of this is that it will only be possible to send emails by paying M$ to do it, and then it will only be allowed to express "approved" opinions.

What can we do about this?

At the very least, those of you with senior positions in universities can tell your management to revert to competent standards-based email systems hosted on Linux systems.

David Berry Apr 12

@Paul_Taylor At a "University" I refused to use their M$ "e-mail system" as it failed to be a bonā fidē e-mail system. My supervisor agreed with me that it was not an e-mail system as it did not operate using standard protocols. IIUC, M$ had an exclusivity clause that prevented any other e-mail system from operating within their TLD that did not already predate the contract with M$. Thus there was nothing to be done: use it or else!

@dgb37 @Paul_Taylor Solution: Acquire new TLD and support it officially.

Fuck corposcum.

That they did not immediately resort to this is indicative of malice or complicity.

Helge Heß Apr 12

@Paul_Taylor I run a self hosted mail server. It's kinda hard to get right, but the requirements seem absolutely reasonable and you can fulfil them.
I don't really see them abusing their market power there ... just yet. (I think they are just waiting for email to die the natural death)

mx alex tax1a - 2020 (6)Apr 12

@helge @Paul_Taylor no, they're actively taking steps to strangle it. we fulfil the requirements but none of our mail gets through because we aren't big enough to register any reputation in their system.

Helge Heß Apr 12

@atax1a @Paul_Taylor That seems weird, I didn't have issues so far. If the DNS is setup right, trust is essentially granted?

mx alex tax1a - 2020 (6)Apr 12

@helge @Paul_Taylor the number of times we have to call people on the phone to tell them to fish our message out of their spam box begs to differ

Helge Heß Apr 12

@atax1a @Paul_Taylor I test that extensively and have no issues for far. If DKIM is not setup properly, you run into this.

DJ Paavi Apr 13

@helge @atax1a @Paul_Taylor I've got mixed results from both Google and Microsoft even when DKIM, SPF etc. checks out and everything's set up correctly. In my honest opinion the demand for logging into their postmaster tools to get delivery to their walled garden working is unreasonable and a nuissance at best.

@paavi @helge @atax1a @Paul_Taylor I have no issues even without DKIM.

@helge @atax1a @Paul_Taylor not necessarily. where do you host your mailserver? because reputation of your ip _and_ neighbouring ips are taken into account.

i'd agree that the published requirements like dkim, dmarc, etc are good actually, but there is more filtering happening beyond that :-c

Julie Webgirl Apr 13

@malte @helge @atax1a @Paul_Taylor

I can live in a beautiful IP neighborhood and keep mine immaculate but a) it takes years to scrub the filth and grime left by the previous tenant, and b) I can't control the dirty deeds going on inside my neighbor's IP, yet I am judged by both. 🤬

@juliewebgirl @helge @atax1a @Paul_Taylor 100%

i hate that the big providers can afford to be lazy enough to judge whole ip-ranges without actually looking what a particular ip is doing, and i only asked because maybe helge is lucky for having a good one 🤷‍♀️

@juliewebgirl @helge @atax1a @Paul_Taylor mild tangent: telekom's mailserver-admins are actually quite accommodating regarding allowlisting single ips. you just have to write them an email, and i guess if you sound vaguely human they just do it ¯\_(ツ)_/¯

Julie Webgirl Apr 13

@malte @helge @atax1a @Paul_Taylor

How refreshing.

Helge Heß Apr 18

@malte @juliewebgirl @atax1a @Paul_Taylor I’m really no expert on that at all but that sounds like a conspiracy theory to me. DKIM establishes cryptographic evidence, IPs shouldn’t matter at all unless you are hosting in an extremely creepy neighborhood.
Is there actual evidence that properly configured hosts (non Russia) are denied? Where is sth like that tracked?

mx alex tax1a - 2020 (6)Apr 18

@helge you're right, you're really no expert. maybe take a step back and stop digging.

@helge @juliewebgirl @atax1a @Paul_Taylor you're shifting goal-posts. there is a difference between being blocked, and being filtered into spam-folders.

i am rarely blocked, but often land in spam-folders.

my guess is, that my domains don't have much weight, and are mostly hosted on hetzner. hetzner has a historically bad reputation from the times when outgoing 25 was open by default.

mx alex tax1a - 2020 (6)Apr 19

@malte if youre going to continue the conversation with the splainy dude please leave me out of it

@helge @juliewebgirl @atax1a @Paul_Taylor exact same setup, also on hetzner, with a more established domain, had none of the beforementioned issues.

Julie Webgirl Apr 19

@helge @malte @atax1a @Paul_Taylor

You left out the word "Actually..."

Julie Webgirl Apr 13

@malte @helge @atax1a @Paul_Taylor

I've given up giving the benefit of the doubt and believe everything little thing they do is intentional. I could excuse laziness but they know the little guy can't buy whole blocks and it's the easiest way to... Ok, it's lazy-ish lol Let's call it efficient... to eliminate everyone except the other major player.

Eggs now in different baskets.Apr 13

@atax1a @helge @Paul_Taylor I got a text message from an organisation that runs summer music camps for children.

We are signed up for two courses this summer so getting details of when, where, how etc is rather important.

The text message said that they had sent an email about one of the courses. Then it went on to say that their emails often end up in spam boxes and asked me to check for this email in my spam box.

They are a reputable charity here in Norway.

Eggs now in different baskets.Apr 13

@atax1a @helge @Paul_Taylor I am beginning to wonder if, what with proposals for client side scanning of chat systems on devices still on the table and all of the issues with the big email actors blocking others without rhyme nor reason and AI agents with root access to devices perhaps the only way to communicate reliably in the future will be old style SMSs on a dumbphone.

Not terribly secure but neither or any of the above options going to be secure in the future.

Helge Heß Apr 12

@atax1a @Paul_Taylor This is good, make sure everything is green here: https://internet.nl

Test for modern Internet Standards like IPv6, DNSSEC, HTTPS, DMARC, STARTTLS and DANE.

Test for modern Internet Standards IPv6, DNSSEC, HTTPS, HSTS, DMARC, DKIM, SPF, STARTTLS, DANE, RPKI and security.txt

mx alex tax1a - 2020 (6)Apr 12

@helge buddy we've been doing this for over 20 years and am currently employed professionally as a postmaster

Helge Heß Apr 12

@atax1a I don't have that much experience 🙄, but what is the reason your emails end in spam? Because Google proactively blocks you? I cannot confirm that they arbitrarily do that.

Helge Heß Apr 12

@atax1a What's the domain you are having issues with?

mx alex tax1a - 2020 (6)Apr 12

@helge i do not want or need your advice or assistance

Helge Heß Apr 12

@atax1a That's fine, but I'd like to know why Google would be blocking you.

mx alex tax1a - 2020 (6)Apr 12

@helge so would i??? they do not explain why??? and their support is proactively useless???

Helge Heß Apr 12

@atax1a By now I guessed it was their fault 🙂

mx alex tax1a - 2020 (6)Apr 12

@helge seriously, at our day job they blocked the entire company fleet once, and refused to explain why, tried to insist that the problem was on our end, and generally gaslit us about the situation until we had executives reach out across company lines!

mx alex tax1a - 2020 (6)Apr 12

@helge covered this in an earlier post, bye now

⠀ ⠀ ⠀ ⠀ stg Apr 13

@helge @atax1a @Paul_Taylor sometimes yes, sometimes no. basically the problem is that in order for your mail to have a high chance of not being flagged, you must have a good reputation. to get a good reputation, you must prove yourself by sending a bunch of not spam, slowly ramping up the rates.

this doesn't work for small, personal servers because you simply just don't have that much mail to send.

if you get lucky, you might be fortunate to have an ip with good reputation and not have to deal with that

Helge Heß Apr 14

@stag @atax1a @Paul_Taylor My feeling is that for getting a bad reputation you really have to be on a very fishy provider.
In the particular case of the original poster it seems to be a clear miss on the DKIM requirement (which is reasonable). Nothing reputation related.

⠀ ⠀ ⠀ ⠀ stg Apr 14

@helge @atax1a @Paul_Taylor generally from my experience big tech's spam filters will always reject any email from residential ips and reused cloud ips (like those you would find from aws or gcp)

i've had better luck with other providers like hetzner, but it really is a hit or miss

Dave Robinson Apr 13

@helge @Paul_Taylor
I agree it's still possible to run your own mail server. It's got harder over the 25 years I've been doing it for... I now have SPF, DKIM, DMARC, DNSSEC and full IPv6 support with rDNS, and I'm just an enthusiastic #HomeLab user with a domestic IP address. As far as I'm aware, my emails are getting delivered to people's inboxes.

Joakim Melin Apr 13

@dave @helge @Paul_Taylor I still run my email server too for family and some friends but email as a whole is something I try to get rid of in my life as it has become more or less unusable and mostly an annoyance.

Owl Eyes Apr 13

@joacim @dave @helge @Paul_Taylor I too feel exactly the same way, and have given up self-hosting email, after making every valiant attempt.

Owl Eyes Apr 13

@dave @helge @Paul_Taylor it's non-deterministic in some cases, even when all the right DNS voodoo is performed (SPF, DKIM, DMARC, DNSSEC and full IPv6 support with rDNS). Please don't make out that it's deterministically reliable *for everyone* with their self-hosted domains, and it's just that they haven't done the right DNS rituals.

Dave Robinson Apr 17

@d1 @helge @Paul_Taylor
Well, that's always been the case. The last stage is the client's inbox, and they can write their own rules for that. If they want to route all emails from people named Dave into spam folders, there's nothing that can be done about it.

In other respects, I don't agree with the use of the term non-deterministic though. That suggests (to me) randomness. You're right in that any email server in a chain can impose its own arbitrary policies, but that's been the case for years. It depends on lots of things such as the training of Bayesian filters, which online block lists are consulted, policies about bulk mailers and so on. Over the years the restrictions that servers apply have become more rigorous, but if your server plays by the rules as a good citizen in my experience it has got more reliable rather than less.

@helge @Paul_Taylor I am running stalwart (https://github.com/stalwartlabs/stalwart) on my server and so far it works ok and is quite simple to setup

GitHub - stalwartlabs/stalwart: All-in-one Mail & Collaboration server. Secure, scalable and fluent in every protocol (IMAP, JMAP, SMTP, CalDAV, CardDAV, WebDAV).

All-in-one Mail & Collaboration server. Secure, scalable and fluent in every protocol (IMAP, JMAP, SMTP, CalDAV, CardDAV, WebDAV). - stalwartlabs/stalwart

GitHub

@Paul_Taylor would you mind trying to send a mail to my email at cispa.de? Should be very easy to find from my name, just don't want to get ingested by every mastodon account ever.

Steve Atkins Apr 12

@Paul_Taylor DMARC reports don’t tell you whether your mail went to spam or not. They tell you when your mail was not authenticated when it was received.

There’s something to investigate there, but it may not be what you’re describing it as.

@lluad @Paul_Taylor my favorite is getting a DMARC report for an sender my SPF record forbids, thus should not have generated a DMARC report for

Steve Atkins Apr 13

@drbrain @Paul_Taylor

DMARC reports are to tell you about delivery attempts “From:” your domain that are not authenticated.

If your SPF records “forbid” a sender that’s exactly the situation you have _explicitly_ asked to be notified about.

@lluad @Paul_Taylor ah, then it seems odd that DMARC would be a requirement to deliver mail