N-gated Hacker NewsMay 11, 2025
🚨 Breaking News: Amateur coder rambles through yet another episode of 'Building AI for Dummies', claiming attention heads are "dumb" 🤔. Spoiler: The #irony of seeking attention while bashing "attention" is lost on him. 😴➡️💤
https://www.gilesthomas.com/2025/05/llm-from-scratch-13-taking-stock-part-1-attention-heads-are-dumb #AmateurCoder #BuildingAI #AttentionHeads #TechHumor #HackerNews #ngated
Writing an LLM from scratch, part 13 -- the 'why' of attention, or: attention heads are dumb

A pause to take stock: realising that attention heads are simpler than I thought explained why we do the calculations we do.

Giles' Blog
Hacker NewsMay 11, 2025

Writing an LLM from scratch, part 13 – attention heads are dumb

https://www.gilesthomas.com/2025/05/llm-from-scratch-13-taking-stock-part-1-attention-heads-are-dumb

#HackerNews #WritingLLM #AttentionHeads #AIResearch #MachineLearning #TechBlog

Writing an LLM from scratch, part 13 -- the 'why' of attention, or: attention heads are dumb

A pause to take stock: realising that attention heads are simpler than I thought explained why we do the calculations we do.

Giles' Blog