fly51fly (@fly51fly)
2026년 UC Berkeley 연구진(I Abdi, A Gupta, M Mok, A Lu 등)의 arXiv 논문은 진화전략(Evolutionary Strategies)을 LLM 훈련에 적용할 경우 치명적 망각(catastrophic forgetting)이 발생함을 보고합니다. 진화적 최적화 기법이 지속학습 및 기억 유지에 미치는 부작용을 실험적으로 분석한 연구입니다.
https://x.com/fly51fly/status/2016988424327385418
#evolutionarystrategies #catastrophicforgetting #llm #research
Google’s new “nested learning” draws on the brain’s fast‑slow circuits to keep LLMs from catastrophic forgetting. By blending neuroscience insights with back‑propagation tricks, the approach promises more stable, open‑source models. Curious how this works and what it means for the future of generative AI? Dive in! #NestedLearning #CatastrophicForgetting #FastSlowCircuits #OpenSourceLLM
🔗 https://aidailypost.com/news/googles-nested-learning-based-brains-fastslow-circuits-curbs-llm
> “You never forget how to ride a bike, – but how is that possible?"
🧠 How can #brains learn new tasks without forgetting old ones? Barry, Gerstner & Bellec propose GateON, a neuro-inspired learning rule: neurons become context-selective and their plasticity can be frozen or unfrozen. Balances #memory and #flexibility, avoiding #CatastrophicForgetting in #AI models.
🌍 https://www.sciencedirect.com/science/article/pii/S0893608025006082
Two things I got out of this article:
1. There already exist drugs that can wipe long-term memory, like in sci-fi movies. Scary.
2. Unlike artificial neural networks, the brain relies on extremely sparse and permanent updates to synaptic strength. This has implications for "catastrophic forgetting", among other things.
Scientists discover "glue" that holds memory together in fascinating neuroscience breakthrough
https://www.psypost.org/scientists-discover-glue-that-holds-memory-together-in-fascinating-neuroscience-breakthrough/
Scientists found that the molecule KIBRA helps stabilize memory by binding to PKMζ, an enzyme that strengthens brain connections, allowing memories to last for years despite the constant turnover of proteins in the brain.
AI has the potential to learn like humans https://innovationtoronto.com/2023/07/ai-has-the-potential-to-learn-like-humans/
#OhioState #continuallearning #catastrophicforgetting #AIsbehavior
...
Finetuned pretrained models soften suffer f. catastrophic forgetting: performance on pretrained tasks deteriorates when finetuned on new tasks.
As weith most fine-tuning approaches, the performance on pretraining tasks deteriorates significantly.
We propose a novel method, Learning-to-Modulate, that avoids the degradation of learned skills by modulating the information flow of the frozen pre-trained model via a learnable modulation pool.
Mart Beller
sayzard
AI Daily Post
Fabrizio Musacchio
Aaron
Victoria Stuart 🇨🇦 🏳️⚧️