I'm moving many of my projects from github to codeberg.

One of them has 20 years of history, 10K users and a git repository approaching 1GB. A lot of this comes from:

* third-party libraries, from the days before composer
* translation files, that could easily live in a separate repository

Should I take this opportunity to purge the git history and reduce storage/network/environmental costs for everyone?

Several of my sites are now being hit by Microsoft's BingBot (valid, I checked the IPs).

It is abusing my "/search?query=..." URL to search for everything from chinese motorcycle parts to train times.

Is Microsoft really running its searches through every search-form it can find on the internet?

/Sigh

Every website I control is being overwhelmed by robots.

It's all the same pattern: one request per IP address and random user-agent strings.

One site had 1.6 million unique IPs in the last week.

What kind of botnet is this?

Current fix:

If request has no cookies and a chrome/firefox/etc. user-agent, then send 4xx response with a cookie, and a body containing meta-refresh tag.

Browsers see the HTML and reload with a cookie.
Robots see the 4xx.

It works - but the robots keep coming...