mesaThe AI feedback loop: Researchers warn of ‘model collapse’ as AI trains on AI-generated content
Wasn’t there a paper not long time ago that it was possible to generate data with AI as a training set for AI? I was surprised (and the math is to much for me to check out my self) but that seems to solve that problem.
Microsoft’s Phi model was largely trained on synthetic data derived from GPT-4.
I’m to lazy to search for the paper, mot sure it was Microsoft, but with my rather basic knowledge of modeling (studied system biology) - it seemed rather crazy and impossible, so I remembered it.