ProgramBench: Can Language Models Rebuild Programs From Scratch?

Turning ideas into full software projects from scratch has become a popular use case for language models. Agents are being deployed to seed, maintain, and grow codebases over extended periods with minimal human oversight. Such settings require models to make high-level software architecture decisions. However, existing benchmarks measure focused, limited tasks such as fixing a single bug or developing a single, specified feature. We therefore introduce ProgramBench to measure the ability of software engineering agents to develop software holisitically. In ProgramBench, given only a program and its documentation, agents must architect and implement a codebase that matches the reference executable's behavior. End-to-end behavioral tests are generated via agent-driven fuzzing, enabling evaluation without prescribing implementation structure. Our 200 tasks range from compact CLI tools to widely used software such as FFmpeg, SQLite, and the PHP interpreter. We evaluate 9 LMs and find that none fully resolve any task, with the best model passing 95\% of tests on only 3\% of tasks. Models favor monolithic, single-file implementations that diverge sharply from human-written code.

Open-Source Tools for Postgres & AI, Tue, May 12, 2026, 12:00 PM | Meetup

You don't need a specialised "AI Database" to build world-class agentic workflows. In this session, we explore pgEdge's open-source Agentic AI Toolkit - a suite of componen

Free AI Coding Skills for Rails — Opinionated Patterns That Work - Julian Rubisch

Free rules and conventions that teach your AI coding agent how to write Rails the right way. 37signals patterns, Phlex components, app scaffolding — install in your terminal, no signup required.

Agent-to-agent pair programming

What if you could let Claude and Codex work together as pair programmers, talking to each other directly? One of them as the main worker and the other as a reviewer.

Claude

Shared via Claude, an AI assistant from Anthropic

Ape coding | Rômulo Saksida

Ape coding is the software development practice where a human developer deliberately hand-writes source code.

Deterministic Programming with LLMs - Dragons in the Algorithm

Sixteen Claude AI agents working together created a new C compiler

The $20,000 experiment compiled a Linux kernel but needed deep human management.