Mastodawn

May 14, 2025

Neil Craig May 13, 2025

As part of a time-limited trial, yesterday we removed the www.bbc.co.uk & www.bbc.com robots.txt "block" on *one* GenAI crawler.

Here's a graph of total daily requests from that 1 GenAI crawler by content-type. You'll note, they've retrieved ~460k web pages in ~16 hours, so a mean of ~28,750 web pages per hour. They'll likely pull ~690k pages today, ~32GB egress.

Whilst this is a drop in the ocean versus our other traffic, I can easily see how this could sink a smaller website.

@smatei I am on vacation and nowhere near my computer so there is little I can do right now. This is what I could find when having a quick look.

Show thread

Valentin Pletzer

Jan 22, 2024

@glenngabe Afaik Google worked with more than one company for quality raters. Maybe the reason for cutting ties with Appen is much more trivial, like cutting costs?

@cawer "Auch zwei CDU-Mitglieder sind dabei, Mitglieder der Werteunion."

Valentin Pletzer

Jan 4, 2024

@dusoft Thanks for the heads up. Just to remove any confusion: You are talking about this article? https://valentin.app/uule.html

Which one of the two imlementations are you talking about?

Google UULE parameter – a 2020 guide to geolocationr

The goal of this article is to explain how the UULE parameter in Google Search works. A lot of tools, including the Google Adwords Tool, use it to emulate a physical location of a searcher.

Show thread

Valentin Pletzer

Dec 21, 2023

@optimisey I believe that's a very recent addition. You can now follow searches and entities and those produce this. You can go here on mobile and change what you follow and what to notify you about https://www.google.com/preferences/interests/yourinterests

I totally agree: It's really bad user experience.

Valentin Pletzer

Oct 18, 2023

what does this even mean? cc @methode

Valentin Pletzer

Oct 13, 2023

@ashnallawalla Hi Ash. Thanks for the feedback. Are you talking about the website or the Chrome extension? When using the website I'd recommend to use incognito mode. Google tends to set the location within cookies which sometimes interfere with the location set in the url. When using the extension make sure to use google.com other TLDs like co.in are not supported (and not needed for the extension). Also: Sometimes other extension interfere.

Valentin Pletzer

Oct 5, 2023

"Refine this search" #google #mobile #serp

Github	https://github.com/VorticonCmdr
Bluesky	https://bsky.app/profile/valentin.app