Arint - SEO+KI3d ago

RT @glenngabe: Interessant: Laut den von @AIoriginality verfolgten Top-1000-Websites gab es einen Anstieg bei Seiten, die Semrushbot blockieren, genau als die Adobe-Übernahme bekannt gegeben wurde. Wenn das stimmt, hat die Adobe-Übernahme einige Leute verunsichert. :) Siehe den Anstieg unten, etwa am 20.11.2026...

mehr auf Arint.info

#Adobe #DigitalMarketing #Semrushbot #SEO #WebCrawler #Übernahme #arint_info

https://x.com/glenngabe/status/2051624562673005050#m

Arint - SEO+KI (@[email protected])

<p>RT @glenngabe: Interessant: Laut den von @AIoriginality verfolgten Top-1000-Websites gab es einen Anstieg bei Seiten, die Semrushbot blockieren, genau als die Adobe-Übernahme bekannt gegeben wurde. Wenn das stimmt, hat die Adobe-Übernahme einige Leute verunsichert. :) Siehe den Anstieg unten, etwa am 20.11.2026...</p> <p><a href="https://arint.info/@Arint/116522804274979010">mehr</a> auf <a href="https://arint.info/">Arint.info</a></p> <p>#Adobe #DigitalMarketing #Semrushbot #SEO #WebCrawler #Übernahme #arint_info</p> <p><a href="https://x.com/glenngabe/status/2051624562673005050#m">https://x.com/glenngabe/status/2051624562673005050#m</a></p>

Mastodon Glitch Edition
Marcus AdamsApr 14, 2025
So I decided to check on my server to see how it's handling all the bots and we're up to nearly 10,000 banned IP addresses, plus I caught another one being abusive with thousands of requests in the last 24 hours; #SemrushBot. No idea who #Semrush is, some kind of SEO or marketing outfit looks like, but it seems to have started yesterday. The day before it was just a couple hundred, but yesterday it was several thousand so now they get a 403 and a firewall ban.
EmpathicQubitFeb 19, 2025
#semrushbot blew up my server tonight, so I blocked it. #semrush
Jack Yan (甄爵恩)Oct 10, 2024

#Semrush claims that we need to wait ‘up to one hour or 100 requests for #SemrushBot to discover changes made to your robots.txt’.

I blocked the main SemrushBot in late September.

I blocked every listed Semrush bot on October 6.

Today, they’re still coming. Never believe a listed tech firm.

https://jackyan.com/blog/2024/10/blocking-the-semrush-bots/

Blocking the Semrush bots

It’s not enough to block just Semrush’s bot, SemrushBot, in one’s robots.txt, since they have a whole bunch of them. We’ve usually allowed all bots but with “AI” and Semrush’s terrible programming that wasted hours of my time every day for nine months, there are certain parties no longer welcome to crawl. To block Semrush

Jack Yan: the Persuader Blog
Florian LohoffJun 19, 2024

When did webcrawlers get so dumb and annoying?

#Amazon Bot appending random stuff to urls and putting them into any text input box searching on websites for other websites content, #semrushbot and #mjbot just getching stuff at high frequency although it didnt change, #googlebot always at the top of Useragent requests with also tons of stupid follows into the depths of dynamic/cgi generated content

And all of the crawling with frequencies as if Websites where "High Frequency Trading" websites, although content has shows to change weekly at most for 10+ years.

BalDec 17, 2018
Забавно. Забанил #SemrushBot — отзывчивость сервера улучшилась раза в полтора :) Даже странно, по логам не казалось, что он создаёт такую огромную нагрузку. Главное, совершенно бесполезная для открытого сообщества хрень. Все услуги по SEO-анализу у него платные. Так что забанил с чистой совестью.