Some cool work from allen institute where they trained a more understandable mixture of experts model. The content that the experts are experts at are more clustered and better resemble human topics. A big benefit of this is it looks like you can remove an expert from the model and performance degrades much more gracefully than a standard MoE.

https://bsky.app/profile/ai2.bsky.social/post/3mle56nehfz2w

#llm #mixtureofexperts #ai2

Ai2 (@ai2.bsky.social)

Today weโ€™re releasing EMO, a new mixture-of-experts (MoE) model trained so modular structure emerges directly from data without human-defined priors. EMO can use a small subset of its experts for a given task while keeping near full-model performance. ๐Ÿงต

Bluesky Social

I did some experiments to compare AI2's tools Asta and DR-Tulu. Reports A and B
have the exact same prompt and Report A from Asta is superior in quality.

Reports C and D have the same prompt and Report D is better and more relevant to the query as DR-Tulu has superior iterative reasoning abilities compared to Asta.

Report A:
https://asta.allen.ai/share/cba36194-fdf3-475d-b7b7-bf8da8ca8095

Report B:
https://www.dr-tulu.org/shared/2MlHrP4OGZ2D

Report C:
https://asta.allen.ai/share/0d362688-c278-4b19-9187-f44be594a50a

Report D:
https://www.dr-tulu.org/shared/BfZ0xI8SiEPW

#research #AItools #AI2

Ai2 Asta

A scholarly research assistant that combines literature understanding and data-driven discovery. Asta uses 108M+ abstracts and 12M+ full-text papers to find, summarize, and analyze scientific evidence. A project from Ai2.

#Ai2 has released #MolmoWeb, an #openweight #visualwebagent that operates from browser screenshots and executes actions like clicking, typing, and scrolling. It comes with #MolmoWebMix, a dataset of 30,000 human task trajectories and 2.2 million screenshot question-answer pairs, making it the largest publicly released collection of human web-task execution. https://venturebeat.com/data/ai2-releases-molmoweb-an-open-weight-visual-web-agent-with-30k-human-task?eicker.news #tech #media #news

Wes Roth (@WesRoth)

๋งˆ์ดํฌ๋กœ์†Œํ”„ํŠธ๊ฐ€ ์ „ Ai2 CEO Ali Farhadi๋ฅผ Corporate Vice President๋กœ ์˜์ž…ํ–ˆ๊ณ , ๊ทธ์™€ ํ•จ๊ป˜ ์šฐ์ˆ˜ AI ์—ฐ๊ตฌํŒ€๋„ ํ•ฉ๋ฅ˜ํ•ด ์ƒˆ๋กญ๊ฒŒ ์žฌํŽธ๋œ AI ์กฐ์ง์„ ๊ฐ•ํ™”ํ•œ๋‹ค๊ณ  ๋ฐํ˜”๋‹ค. ์—ฐ๊ตฌ ์—ญ๋Ÿ‰ ํ™•๋Œ€์™€ ์กฐ์ง ๊ฐœํŽธ ์ธก๋ฉด์—์„œ ์ค‘์š”ํ•œ ์ธ์‚ฌ๋‹ค.

https://x.com/WesRoth/status/2036533511973142822

#microsoft #ai2 #hiring #airesearch #organization

Wes Roth (@WesRoth) on X

Microsoft has hired Ali Farhadi, the former CEO of the Allen Institute for Artificial Intelligence (Ai2), as its new Corporate Vice President. He is bringing a team of elite AI researchers with him to bolster Microsoft's newly restructured AI division. Farhadi recently stepped

X (formerly Twitter)

merve (@mervenoyann)

AI2(Allen Institute for AI)๊ฐ€ ํฌ์ธํŒ…(pointing) ์ž‘์—…์—์„œ SOTA ์„ฑ๋Šฅ์„ ๋ชฉํ‘œ๋กœ ํ•œ ์ƒˆ๋กœ์šด ๋น„์ „ ์–ธ์–ด ๋ชจ๋ธ ํŒจ๋ฐ€๋ฆฌ 'MolmoPoint'๋ฅผ ๊ณต๊ฐœํ•จ. ๊ณต๊ฐœ๋œ ๋ชจ๋ธ์€ MolmoPoint-8B(๋ฒ”์šฉ), MolmoPoint-GUI-8B(๊ทธ๋ž˜ํ”ฝ UI์šฉ), MolmoPoint-Vid-4B(๋น„๋””์˜ค ๋‚ด ๊ณ„์ˆ˜/์ถ”์ )์ด๋ฉฐ, ๊ด€๋ จ ๋ฐ์ดํ„ฐ์…‹๋„ ํ•จ๊ป˜ ์ œ๊ณต๋จ.

https://x.com/mervenoyann/status/2034343677116531005

#ai2 #molmopoint #visionlm #datasets #sota

merve (@mervenoyann) on X

AI2 released new family of vision LMs for pointing (SOTA!) ๐Ÿ”ฅ > MolmoPoint-8B (general use) > MolmoPoint-GUI-8B (graphical computer use) > MolmoPoint-Vid-4B (counting/tracking in videos) also with their datasets ๐Ÿฅต

X (formerly Twitter)
Ai2: Building physical AI with virtual simulation data

Virtual simulation data is driving the development of physical AI across corporate environments, led by initiatives like Ai2โ€™s MolmoBot.

AI News

merve (@mervenoyann)

Allen Institute for AI(AI2)๊ฐ€ Olmo Hybrid ๋ชจ๋ธ๊ตฐ(base/SFT/DPO)์„ ๊ณต๊ฐœํ–ˆ์Šต๋‹ˆ๋‹ค. ์ด ๋ชจ๋ธ๊ตฐ์€ ํŠธ๋žœ์Šคํฌ๋จธ์™€ RNN ๋ ˆ์ด์–ด๋ฅผ ํ˜ผํ•ฉํ•ด FLOP ๋Œ€๋น„ ํ•™์Šต ํšจ์œจ์ด ๋†’์€ ๊ตฌ์กฐ๋ฅผ ์ถ”๊ตฌํ•˜๋ฉฐ ํ•™์Šต ์ธก๋ฉด์—์„œ ํŒŒ๋ ˆํ†  ํ”„๋Ÿฐํ‹ฐ์–ด์— ์œ„์น˜ํ•œ๋‹ค๊ณ  ์ฃผ์žฅํ•˜๊ณ  ํ™•์žฅ์„ฑ๋„ ํ™•๋ณดํ–ˆ๋‹ค๊ณ  ๋ณด๊ณ ํ–ˆ์Šต๋‹ˆ๋‹ค. ๋˜ํ•œ ํ•™์Šต ๋ฐ์ดํ„ฐ ๋ฏน์Šค๋„ ํ•จ๊ป˜ ๊ณต๊ฐœ๋˜์—ˆ์Šต๋‹ˆ๋‹ค.

https://x.com/mervenoyann/status/2029600313703899321

#ai2 #olmohybrid #efficienttraining #transformerrnn

merve (@mervenoyann) on X

AI2 @allen_ai just dropped a family of new Olmo Hybrid models (base/SFT/DPO) ๐Ÿ”ฅ it's a FLOP-efficient mix of transformer and RNN layers on pareto frontier (for training) ๐Ÿ™Œ๐Ÿป and scales too! as usual they also dropped the training set mix ๐Ÿ’—

X (formerly Twitter)
Ai2's open coding agents slash costs for developers

With the release of Ai2's open coding agents, developers have a new method for writing and testing software that promises to slash costs.

Developer Tech News

Tim Dettmers (@Tim_Dettmers)

Ai2์˜ Open Coding Agent ์‹œ๋ฆฌ์ฆˆ ์ฒซ ๋ชจ๋ธ SERA ์ถœ์‹œ ๋ฐœํ‘œ. ์ž‘์„ฑ์ž๋Š” SERA๊ฐ€ ๋™์ผ ๊ทœ๋ชจ์—์„œ SoTA ์„ฑ๋Šฅ์„ ๋ณด์ด๊ณ  ์„ค๊ณ„๊ฐ€ ๋‹จ์ˆœํ•˜๋ฉฐ, ๊ฐ•ํ™”ํ•™์Šต(RL) ๋Œ€๋น„ 26๋ฐฐ ํšจ์œจ์ ์ด๋ผ๊ณ  ์ฃผ์žฅํ•จ. ์ƒ์„ธ ์„ค๋ช…๊ณผ ๊ฐœ๋ฐœ ์—ฌ์ •์€ Tim Dettmers์˜ ๋ธ”๋กœ๊ทธ ๊ธ€๋กœ ์ œ๊ณต.

https://x.com/Tim_Dettmers/status/2016199055504736267

#sera #ai2 #codingagent #efficientmodel

Tim Dettmers (@Tim_Dettmers) on X

We release SERA, the first model part of Ai2โ€™s Open Coding Agent series. SERA is a SoTA agent for its size, super simple, and 26x more efficient than RL. In my blog post, I write about my personal journey of building this coding agent: https://t.co/kPZHUGwBBC Details: ๐Ÿ‘‡

X (formerly Twitter)