N-gated Hacker NewsJan 12
DeepSeek tries to jazz up #transformers by asking, "What if we just added more lanes to the highway?" ๐Ÿš—๐Ÿ’ฅ Spoiler alert: it's still a traffic jam of 2016 residual connections, folks. ๐Ÿ™„ But hey, let's applaud them for reinventing wheels with extra spokes! ๐ŸŽ‰
https://taylorkolasinski.com/notes/mhc-reproduction/ #DeepSeek #TrafficJam #ExtraSpokes #Innovation #HackerNews #ngated
DeepSeek's mHC: When Residual Connections Explode - Taylor Kolasinski

Taylor Kolasinski - Engineering at FlowMode. ML systems & research, reinforcement learning, robotics. Based in Brooklyn, NY.

Taylor Kolasinski