Real-World Superintelligence
"[In] the space of an hour [the novelist] sets free within us all the joys and sorrows in the world, a few of which only we should have to spend years of our actual life in getting to know, and the most intense of which would never be revealed to us because the slow course of their development prevents us from perceiving them." - Marcel Proust, Swann's Way
Nothing in the history of software has been adopted as quickly as AI coding tools. In under two years, they broke every prior record for software company growth and reached daily use among a large share of developers. But I think people are misreading the success of those tools in two (related) ways.
First, we might be overestimating how much economic progress was being bottlenecked by the production of code. In most real-world domains, the harder problem is not writing the software but knowing what the software should do: how to allocate capital, run an organization, control a physical system, or treat a disease. Making code dramatically cheaper does not make those underlying problems dramatically easier.1
Second, we are assuming that the techniques that worked in code will work everywhere else. Much of the progress in code generation has come from scaling reinforcement learning with verifiable rewards: models write programs, execute them, and learn from whether they run correctly. Recent progress in mathematics has followed the same pattern. AlphaProof was trained by generating millions of problem variants and testing candidate proofs through interaction with Lean, a formal proof checker.
I define sandbox superintelligence as this kind of intelligence, produced by repeated interaction with closed, verifiable environments: the ability to reason, code, or solve formal problems better than humans. Real-world superintelligence is the ability to act above human capability in complex, partially observed systems: to predict how the world will respond to intervention, choose actions under uncertainty, and learn from delayed and noisy outcomes.
We are far from real-world superintelligence. Frontier models can solve Olympiad-level math problems yet do very poorly at running a vending machine business. Math and code are domains where perfect simulators already exist (code interpreters and proof checkers) which makes every incremental rollout very cheap. That's not the case for most economically important domains and decisions. Drug trials take years, trading strategies are expensive to test, company decisions take months to show results.
Consider the Wright brothers. In 1901, their glider produced only about a third of the lift predicted by calculations based on the accepted aerodynamic data. Testing a major redesign meant building a full-size machine, transporting it to Kitty Hawk, waiting for suitable winds, and trying to fly it. In order to speed up the process, they built a wind tunnel: a box in which small model wings could be exposed to controlled airflow and their lift and drag measured systematically. The tunnel made experimentation dramatically cheaper, allowing them to screen as many as two hundred model wing shapes over that fall, rather than building and flying a full-size glider for every design.
Scaling interaction means building wind tunnels for every domain in which experimentation is slow and costly. Over the past two years, frontier labs and startups have undertaken a massive build-out of RL environments. Most of these are deterministic environments built as software programs, such as simulations of e-commerce sites and airline-booking systems. Building each environment requires dedicated engineering effort. And for many of the systems we care about, we do not know how to construct a sufficiently faithful simulation.
A more scalable path compared to deterministic environments is learned world models trained on large quantities of existing interactions and observational data. Unlike hand-built environments, learned simulators do not require us to program the rules of the world explicitly. Their fidelity can improve with greater compute and data scale, and we have not yet trained simulators on more than a small fraction of the world's observations. Importantly, a useful world model must remain accurate when an agent takes actions that are rare or absent from its training data, or tries to exploit the simulator's loopholes. This is why real-world deployment and a continuous flywheel of policy-environment updates remains necessary.
World models are now best known in robotics, where they simulate how an environment will change in response to a robot's actions, but the same principle of action-conditioned prediction can be applied to other domains. In social science, models built from large populations of simulated individuals estimate how people may respond to different events, such as elections, news, or economic policies. In biology, virtual-cell models predict how gene expression changes as a result of a genetic perturbation. In financial markets, learned simulators model how trades can affect prices and liquidity.
It might seem that we are already far along the AI exponential, but we are still at the beginning of building this new kind of intelligence. Similar to how a novel compresses years of emotional life into an hour, and the wind tunnel compresses months of dangerous experimentation into days, simulation and world models will dramatically compress slow and expensive feedback loops, to enable AI systems to learn how to take the right actions in an unpredictable world.