sayzardJan 14

Rohan Paul (@rohanpaul_ai)

한 논문은 LLM이 주로 자기 생성(self-generated) 데이터로 자체 학습(self-training)할 경우 모델의 다양성이 감소하고 진실성에서 벗어나게 된다고 증명한다. LLM은 자체 텍스트만으로 무한히 부트스트랩할 수 없으며 외부의 현실 검증(fresh reality checks) 데이터가 필요하다는 경고를 담고 있다.

https://x.com/rohanpaul_ai/status/2011336480061288573

#llm #selftraining #research #modelrobustness

Rohan Paul (@rohanpaul_ai) on X

This paper proves LLM self training on mostly self generated data makes models lose diversity and drift from truth. LLMs cannot bootstrap forever on their own text, they need fresh reality checks or they collapse. The problem is that many people expect an AI to learn from its

X (formerly Twitter)
Brian Greenberg May 18, 2025

🧠 Neural networks can ace short-horizon predictions — but quietly fail at long-term stability.

A new paper dives deep into the hidden chaos lurking in multi-step forecasts:
⚠️ Tiny weight changes (as small as 0.001) can derail predictions
📉 Near-zero Lyapunov exponents don’t guarantee system stability
🔁 Short-horizon validation may miss critical vulnerabilities
🧪 Tools from chaos theory — like bifurcation diagrams and Lyapunov analysis — offer clearer diagnostics
🛠️ The authors propose a “pinning” technique to constrain output and control instability

Bottom line: local performance is no proxy for global reliability. If you care about long-horizon trust in AI predictions — especially in time-series, control, or scientific models — structural stability matters.

#AI #MachineLearning #NeuralNetworks #ChaosTheory #DeepLearning #ModelRobustness
https://www.sciencedirect.com/science/article/abs/pii/S0893608025004514