@noiob It doesn't actually work. Unscrupulous third-parties ignore robots.txt, and it's trivial to pretend to the server that you're actually a regular ol' human-guided browser.
@Aerdan it will still keep big search engines from showing your profile and toots, and the specific assbutts I'm talking about are definitely too lazy to write their own indexing tools or sth