Nomad_Sim (@sedonaroxx)
과적합이 아닌 과매개변수화 모델에서 파라미터 수가 증가할수록 더 다양한 방식으로 적합할 수 있어, 훈련에서 발견되지 않은 잠재 구조를 학습할 수 있다는 관점을 설명했다. 로짓 모델과 SVM의 커널 고차원 투영을 예로 들어, 더 큰 모델의 일반화 직관을 논의한 내용이다.
https://x.com/sedonaroxx/status/2049440218634494424
#ml #overparameterization #svm #generalization #theory
Nomad_Sim (@sedonaroxx) on X
@_avichawla We observe this in logit and SVMs (where the kernel projects onto high dims); the intuition is higher parameters help to fit the model in more ways and the model learns underlying latent structure not discovered in training. Happens with over parameterized models. Fascinating
'Local Linear Recovery Guarantee of Deep Neural Networks at Overparameterization', by Yaoyu Zhang, Leyang Zhang, Zhongwang Zhang, Zhiwei Bai.
http://jmlr.org/papers/v26/24-0192.html
#overparameterization #overparameterized #deep
Local Linear Recovery Guarantee of Deep Neural Networks at Overparameterization
'Preconditioned Gradient Descent for Overparameterized Nonconvex Burer--Monteiro Factorization with Global Optimality Certification', by Gavin Zhang, Salar Fattahi, Richard Y. Zhang.
http://jmlr.org/papers/v24/22-0882.html
#optimality #minimizer #overparameterization
Preconditioned Gradient Descent for Overparameterized Nonconvex Burer--Monteiro Factorization with Global Optimality Certification
sayzard
JMLR