"We can't do anything about Google's indexing about the Fediverse, so we're giving them a pass."

No, you can do something about Google: build better Fediverse search.

@atomicpoet there's robots.txt for that
@robUx4 @atomicpoet

Also I find robots.txt to be somewhat unreliable. Some scrapers just ignore it.