Real-World Superintelligence
"[In] the space of an hour [the novelist] sets free within us all the joys and sorrows in the world, a few of which only we should have to spend years of our actual life in getting to know, and the most intense of which would never be revealed to us because the slow course of their development prevents us from perceiving them." - Marcel Proust, Swann's Way
Nothing in the history of software has been adopted as quickly as AI coding tools. In under two years, they broke every prior record for software-company growth and reached daily use among a large share of developers. But I think their success is being misread in two ways.
First, we are overestimating how much broad economic progress was bottlenecked by the production of code. In most real-world domains, the harder problem is not writing the software but knowing what the software should do: how to allocate capital, run an organization, control a physical system, or treat a disease. Making code dramatically cheaper does not make those underlying problems dramatically easier.1 Second, we are assuming that the techniques that worked in code will work everywhere else. They may not.
Much of the progress in code generation has come from scaling reinforcement learning with verifiable rewards: models write programs, execute them, and learn from whether they run correctly. Recent progress in mathematics has followed the same pattern. AlphaProof was trained by generating millions of problem variants and testing candidate proofs through interaction with a formal proof system.
I define sandbox superintelligence as the intelligence produced by repeated interaction with closed, verifiable environments such as the above: the ability to reason, code, or solve formal problems better than humans. Real-world superintelligence is the ability to act above human capability in complex, partially observed systems: to predict how the world will respond to intervention, choose actions under uncertainty, and learn from delayed and noisy outcomes.
We are far from real-world superintelligence. Frontier models can solve famous math problems yet flounder at running a vending machine business. The reason is that math and code are domains where perfect simulators already exist (code interpreters and proof checkers) so a million rollouts cost very little. Most economically important domains offer no such luxury. Drug trials take years. Trading strategies are expensive to test. Organizational decisions take months to pan out. A self-driving car cannot learn from a million crashes.
Consider the Wright brothers. In 1901, their glider produced only about a third of the lift predicted by calculations based on the accepted aerodynamic data. Testing a major redesign could mean building a full-size machine, transporting it to Kitty Hawk, waiting for suitable winds, and eventually putting a pilot aboard. So they built a wind tunnel: a box in which small model wings could be exposed to controlled airflow and their lift and drag measured systematically. The tunnel made experimentation dramatically cheaper, allowing them to screen as many as 200 model wing shapes over several weeks, rather than building and flying a full-size glider for every design.
Scaling interaction means building wind tunnels for every domain where we don't have perfect simulators. Over the past two years, frontier labs and startups have undertaken a massive build-out of RL environments. Most of these are deterministic environments built as software programs, such as simulations of e-commerce sites and airline-booking systems. Building each environment requires dedicated engineering effort, and for many of the systems we care about, we do not know how to construct a sufficiently faithful simulation. We do not know how to build general-purpose simulations of financial markets or human biology.
The alternative to deterministic environments is learned world models trained on large quantities of existing interactions and observational data. Unlike hand-built environments, learned simulators do not require us to program the rules of the world explicitly. Their fidelity can improve with greater compute and data scale, and we have not yet trained simulators on more than a small fraction of the world's observations. Importantly, a useful world model must remain accurate when an agent takes actions that are rare or absent from its training data, or tries to exploit the simulator's loopholes. This is why real-world deployment and flywheel remains necessary.
World models are now best known in robotics, where they simulate how an environment will change in response to a robot's actions, but the same principle of action-conditioned prediction can be applied to other domains. In social science, models built from large populations of simulated individuals estimate how people may respond to elections, news, or economic policies. In biology, virtual-cell models predict how gene expression changes as a result of a genetic perturbation. In financial markets, learned simulators model how trades affect prices and liquidity.
It might seem that we are already far along the AI exponential, but we are still near the beginning of building this kind of intelligence. Similar to how a novel compresses years of emotional life into an hour, and the wind tunnel compresses weeks of dangerous experimentation into an afternoon, learned simulators and world models will enable AI models to learn faster how to act in the world. Building real-world superintelligence will require machines that can learn faster than the world unfolds, without losing contact with the world they are meant to understand.