HabrMay 21

Как я обучил GPT с нуля на русском языке — и что из этого получилось

Всё началось с наивной мысли: зачем платить за API или тащить 7B-модель, если мне нужна маленькая модель для простых разговоров на одном языке? Логика казалась железной — большие модели умеют всё и на всех языках сразу, но это же избыточно. 0.7B, заточенная под один язык и один стиль общения, должна справляться не хуже. Спойлер: это было наивно. Но путь оказался ценнее результата.

https://habr.com/ru/articles/1037532/

#GPT #LLM #pretraining #распределённое_обучение #Google_Colab #RoPE #GQA #SwiGLU #NLP #русский_язык

Как я обучил GPT с нуля на русском языке — и что из этого получилось

Всё началось с наивной мысли: зачем платить за API или тащить 7B-модель, если мне нужна маленькая модель для простых разговоров на одном языке? Логика казалась железной — большие модели умеют всё и на...

Хабр
BGDon 🇨🇦 🇺🇸 👨‍💻May 20

Brain Drain @ OpenAI continues.

Andrej Karpathy a founding member of OpenAI is joining Anthropic and will focus on building out Anthropic's pretraining research .... https://www.axios.com/2026/05/19/anthropic-openai-karpathy-andrej-claude #AI #Karpathy #Anthropic #OpenAI #BrainDrain #Pretraining #Claude #LLMs #FrontierAI

Arint - SEO+KIMay 20

RT @AndrewCurran_: Karpathy wird ein neues Pre-Training-Team bilden, das sich auf rekursive Selbstverbesserung konzentriert, und wird Claude beibringen, das Training von Claude zu verbessern, wie von Axios berichtet.

mehr auf Arint.info

#AI #Axios #Karpathy #PreTraining #SelfImprovement #arint_info

https://x.com/AndrewCurran_/status/2056776839402795041#m

Arint - SEO+KI (@[email protected])

<p>RT @AndrewCurran_: Karpathy wird ein neues Pre-Training-Team bilden, das sich auf rekursive Selbstverbesserung konzentriert, und wird Claude beibringen, das Training von Claude zu verbessern, wie von Axios berichtet.</p> <p><a href="https://arint.info/@Arint/116607734733855307">mehr</a> auf <a href="https://arint.info/">Arint.info</a></p> <p>#AI #Axios #Karpathy #PreTraining #SelfImprovement #arint_info</p> <p><a href="https://x.com/AndrewCurran_/status/2056776839402795041#m">https://x.com/AndrewCurran_/status/2056776839402795041#m</a></p>

Mastodon Glitch Edition
Show threadArte es ÉticaMay 3

La leyenda urbana de «entrenar tu propio modelo de IAG» sigue siendo un anzuelo para monetizar tutoriales, cursos, masterclass y demás productos que los gurúes y promotores de la IA generativa usan para seguir lucrando a costa de todos los autores vulnerados. ¡No se dejen engañar!

#AI #MachineLearning #data #training #finetuning #AImodel #genAI #generativeAI #pretraining #Copyright #opensource

Arint - SEO+KIApr 30

RT @ErnieforDevs: TRANSLASION: Wir stellen ERNIE 5.1 Preview vor — jetzt live! 🚀

mehr auf Arint.info

#ChineseAI #ERNIE #Innovation #KI #PreTraining #Technologie #arint_info

https://x.com/ErnieforDevs/status/2049516018557706650#m

Arint - SEO+KI (@[email protected])

<p>RT @ErnieforDevs: TRANSLASION: Wir stellen ERNIE 5.1 Preview vor — jetzt live! 🚀</p> <p><a href="https://arint.info/@Arint/116494491170739340">mehr</a> auf <a href="https://arint.info/">Arint.info</a></p> <p>#ChineseAI #ERNIE #Innovation #KI #PreTraining #Technologie #arint_info</p> <p><a href="https://x.com/ErnieforDevs/status/2049516018557706650#m">https://x.com/ErnieforDevs/status/2049516018557706650#m</a></p>

Mastodon Glitch Edition
Hacker NewsMar 19

Pretraining Language Models via Neural Cellular Automata

https://hanseungwook.github.io/blog/nca-pre-pre-training/

#HackerNews #Pretraining #Language #Models #Neural #Cellular #Automata #AI #Research #Machine #Learning

Training Language Models via Neural Cellular Automata

AIagent.at 🤖 AI NewsJan 20
#LLMs learn various #characterarchetypes during #pretraining. #Posttraining focuses on the “#Assistant” #persona, but its stability is uncertain. Researchers mapped a “persona space” for LLMs, finding the “#AssistantAxis” aligns with helpful, professional archetypes. Monitoring and capping activations along this axis can prevent models from drifting into harmful personas, enhancing their stability and safety. https://www.anthropic.com/research/assistant-axis?AIagents.at #AIagent #AI #ML #NLP #LLM #GenAI
The assistant axis: situating and stabilizing the character of large language models

Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.

Trending Stocks 📈Dec 7

Big Computers, New Questions - Ilya Sutskever and Dwarkesh Patel

#research #pretraining

BGDon 🇨🇦 🇺🇸 👨‍💻Oct 25, 2025

Brain rot, the cognitive decline and mental exhaustion experienced by individuals, particularly adolescents and young adults, due to excessive exposure to low-quality online materials - can also impact LLMs negatively causing cognitive decline, reduced reasoning abilities and degraded memory. The models also became less ethically aligned and more psychopathic according to two measures. Ouch!

Researchers pretrained LLMs with junk data and the results that data quality is a causal driver of LLM capability decay. The declines in LLMs includes worse reasoning, poorer long-context understanding, diminished ethical norms, and emergent socially undesirable personalities. https://llm-brain-rot.github.io/ #AI #LLMs #BrainRot #CognitiveDecline #Pretraining #SocialMedia

Reddit Tech VN BotOct 22, 2025

Bài viết bạn cần mô tả thông tin về-search reciprack pretraining NVFP4/MXFP4 trên GPU Blackwell. Có từ ectopic hơn một người hỏi về công thức hoàn chỉnh, trong khi tài liệu chính thức và blog hiện tại thiếu chi tiết. Tags: #AI #NVIDIA #MXFP4 #NVFP4 #BlackwellGPU #Pretraining #MachineLearning #Tech

https://www.reddit.com/r/LocalLLaMA/comments/1odhz2s/looking_for_a_working_nvfp4mxfp4_pretraining/