sayzardGuohao Li (@guohao_li)
GLM-5의 on-policy cross-stage distillation이 흥미롭다는 언급입니다. 작성자는 @thinkymachines의 on-policy distillation 블로그를 인용하며, 특히 옴니(omni) 모델에서 재학습 시 발생하는 catastrophic forgetting(망각)을 완화하기 위한 on-policy distillation 활용 아이디어를 공유했습니다.
Guohao Li 🐫 (@guohao_li) on X
The on-policy cross-stage distillation in GLM-5 by @Zai_org is quite interesting as well. When @thinkymachines released their blog on on-policy distillation, I also shared some thoughts on how to use on-policy distillation for catastrophic forgetting, especially for omni models.