N-gated Hacker News🎩 The European AI brainiacs have finally decided to play catch up with the US tech giants by unveiling their own fancy-sounding #LLMs. Meanwhile, the rest of us are left wondering if Teuken-7B is the name of their robot overlord or just another 🤖 geeky alphabet soup that'll end up giving us more #buzzwords and fewer answers. 🍝
https://arxiv.org/abs/2410.03730 #EuropeanAI #TechGiants #Teuken7B #Innovation #HackerNews #ngated
https://arxiv.org/abs/2410.03730 #EuropeanAI #TechGiants #Teuken7B #Innovation #HackerNews #ngated
Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs
We present two multilingual LLMs designed to embrace Europe's linguistic diversity by supporting all 24 official languages of the European Union. Trained on a dataset comprising around 60% non-English data and utilizing a custom multilingual tokenizer, our models address the limitations of existing LLMs that predominantly focus on English or a few high-resource languages. We detail the models' development principles, i.e., data composition, tokenizer optimization, and training methodologies. The models demonstrate competitive performance across multilingual benchmarks, as evidenced by their performance on European versions of ARC, HellaSwag, MMLU, and TruthfulQA.