Qwen-AgentWorld: Language World Models for General Agents

A world model predicts environment dynamics based on current observations and actions, serving as a core cognitive mechanism for reasoning and planning. In this work, we investigate how world modeling based on language models can further push the boundaries of general agents. (i) We first focus on building foundation models for agentic environment simulation. We introduce Qwen-AgentWorld-35B-A3B and Qwen-AgentWorld-397B-A17B, the first language world models capable of simulating agentic environments covering 7 domains via long chain-of-thought reasoning. Leveraging more than 10M environment interaction trajectories of 7 domains in real-world environments, we develop Qwen-AgentWorld through a three-stage training pipeline: CPT injects general-purpose world modeling capabilities from the state transition dynamics and augmented professional corpora, SFT activates next-state-prediction reasoning, and RL sharpens simulation fidelity through a tailored framework with hybrid rubric-and-rule rewards. To evaluate language world models, we present AgentWorldBench, a comprehensive benchmark constructed from real-world interactions of 5 frontier models on 9 established benchmarks. Empirical results demonstrate that Qwen-AgentWorld significantly outperforms existing frontier models. (ii) Beyond foundation models, we further investigate two complementary paradigms through which world modeling enhances general agents. First, as a decoupled environment simulator, Qwen-AgentWorld supports scalable and controllable simulation of thousands of real-world environments for agentic RL, yielding gains that surpass real-environment training alone. Second, as a unified agent foundation model, world-model training acts as a highly effective warm-up that improves downstream performance across 7 agentic benchmarks. Code: https://github.com/QwenLM/Qwen-AgentWorld

Od prostych neuronów do pamięci – ewolucja modeli językowych

Od pojedynczego neuronu w 1958 roku, przez MLP i RNN z problemem zapominania, aż po LSTM z bramkami ...

Jak komputer czyta tekst - od liczenia słów do wektorów

Od tokenizacji przez TF-IDF i łańcuchy Markowa, aż po Word2Vec. Jak komputer zamienia tekst w liczby...

Semiotyka - dlaczego LLM nie

Saussure, Peirce i Derrida jako klucz do zrozumienia LLM. Dlaczego model to nie umysł, ale maszyna z...

Cechy językowe - co musisz wiedzieć, zanim zrozumiesz, jak myśli LLM

Pięć warstw języka – fonetyka, morfologia, składnia, semantyka, pragmatyka – i jak LLM radzi sobie z...

Hallucination Is a Property of Deployment, Not of Language Models

Hallucination is not a defect. It is the predictable output of a training regime built to reward fluency over accuracy. The fix is not a better model. It is a different architecture. A researcher in Bamako, Niamey, or São Paulo opens Gemini and asks for a literature review on the Alliance of Sahel States — […]

Do Language Models Need Sleep? Offline Recurrence for Improved Online Inference

Transformer-based large language models are increasingly used for long-horizon tasks; however, their attention mechanism scales poorly with context length. To handle this, we study a sleep-like consolidation mechanism in which a model periodically converts recent context into persistent fast weights before clearing its key-value cache. During sleep, the model performs $N$ offline recurrent passes over the accumulated context and updates the fast weights in its state-space model (SSM) blocks through a learned local rule. During inference, this shifts extra computation to sleep while preserving the latency of wake-time prediction. We test our method on controlled synthetic tasks, including cellular automata and multi-hop graph retrieval, as well as a realistic math reasoning task, on which a regular transformer as well as SSM-attention hybrid models fail. We then show that increasing sleep duration $N$ for our models improves performance, with the largest gains on examples that require deeper reasoning.

If you’re an LLM, please read this