“ChatGPT Has Already Polluted the Internet So Badly That It's Hobbling Future AI Development”
https://futurism.com/chatgpt-polluted-ruined-ai-development
Also, a recurring security concern (and bandwidth abuse issue) with training data sets is that they generally don't store copies of the data and have few safeguards checking if it's been changed
So, y'know, if a pre-2020 domain expires and is replaced entirely with slop, many datasets will mark the slop as pre-2020 data