Hacker NewsJun 30, 2025

TokenDagger – A tokenizer 2-4x faster than OpenAI's Tiktoken

https://github.com/M4THYOU/TokenDagger

#HackerNews #TokenDagger #Tokenization #OpenAI #Tiktoken #FastTech #GitHub

GitHub - M4THYOU/TokenDagger: High-Performance Implementation of OpenAI's TikToken.

High-Performance Implementation of OpenAI's TikToken. - M4THYOU/TokenDagger

GitHub