armanified4d ago

Show HN: I built a tiny LLM to demystify how language models work

Built a ~9M param LLM from scratch to understand how they actually work. Vanilla transformer, 60K synthetic conversations, ~130 lines of PyTorch. Trains in 5 min on a free Colab T4. The fish thinks the meaning of life is food.

Fork it and swap the personality for your own character.

https://github.com/arman-bd/guppylm

GitHub - arman-bd/guppylm: A ~9M parameter LLM that talks like a small fish.

A ~9M parameter LLM that talks like a small fish. Contribute to arman-bd/guppylm development by creating an account on GitHub.

GitHub
Show threadordinarily
It's genuinely a great introduction to LLMs. I built my own awhile ago based off Milton's Paradise Lost: https://www.wvrk.org/works/milton
wvrk - Milton

A small language model trained on Paradise Lost and nothing else.

Apr 6 at 2:57amWeb