Abstract
Frontier AI systems learn from human-made targets: labels, rewards, benchmarks, and preference signals. That has produced rapid progress in usefulness and performance, but it has also created a bottleneck. Models get better at matching the surfaces we can score, while remaining weak at building representations that stay reliable under intervention, transfer, and contact with the world. In open-ended domains such as science and real-world reasoning, this matters because the target is not just fluent prediction or benchmark success. The target is a model of reality that survives new evidence.
Current research structures reinforce this bottleneck. Text is cheap, labels are legible, and benchmark gains are easy to measure. Multimodal, embodied, and intervention-based training is slower, harder, and more expensive. I argue that the field should shift from proxy-heavy training toward reality-coupled training, where models learn from temporal structure, multimodal grounding, action, consequence, and uncertainty. I propose a pilot experiment that compares proxy-trained models with models trained on richer world feedback, and evaluates which representations transfer more robustly beyond their training setup.
1. The bottleneck
Frontier AI has a training problem. The field rewards models for hitting human-made targets such as labels, preference scores, and benchmark metrics, so systems get better at matching what we can grade than at building representations that stay reliable when the world changes. That trade worked well for chat assistants and narrow games, but it breaks down in science and open-ended reasoning, where the target is not a fixed answer on a static test but a model of reality that holds up under intervention, transfer, and new evidence.
A model trained mostly on archived human discourse can absorb many real regularities, but it also inherits the shortcuts and blind spots of the channel that produced the data. Reinforcement learning helps in cases where the objective is clean, yet open-ended domains rarely offer rewards that capture what researchers actually care about. The result is a proxy trap: the more the field optimizes for human-defined targets, the more progress can drift away from the underlying structure of the world.
This is not an argument against structure. Learning always requires constraints. A system with no bias at all would have no reason to privilege one pattern over another, no basis for deciding what matters, and no stable path toward generalization. The problem is not that frontier AI uses constraints. The problem is that many of its strongest constraints are designed for human convenience rather than for contact with reality.
2. Why it persists
Current research structures keep this bottleneck in place. Text is cheap to collect, labels are legible, reward signals are scalable, and benchmark gains are easy to measure, while multimodal, temporal, and embodied training is slower, costlier, and harder to run. That pushes labs toward systems that improve quickly on visible metrics instead of systems that learn through action, consequence, and repeated contact with the world.
These incentives shape the science as much as the models. If a team can show benchmark gains in weeks from more scraped text, post-training, or better reward tuning, that work will usually outrun a slower program built around world feedback, physical data collection, and intervention-based evaluation. The field then mistakes convenience for progress. It treats what is easy to optimize as if it were the same thing as what matters.
There is also an institutional reason this pattern survives. Human-defined targets produce outputs that are easy to compare across models, papers, and product cycles. Reality-coupled training does not. Its gains may show up in robustness, transfer, causal stability, or resistance to failure in new conditions, all of which are harder to package into a leaderboard. That makes proxy-heavy training attractive not only because it is cheaper, but because it is easier to publish, easier to sell, and easier to scale.
3. Why this matters in AI research
This bottleneck matters because AI research no longer aims only to build systems that autocomplete text. The field now claims to build systems that can reason, discover, assist in science, and operate in the world. Those claims become weaker when models perform well on human-written benchmarks but fail to keep their bearings once they face new conditions, conflicting evidence, or tasks that require causal understanding instead of pattern recall.
The problem is most visible in domains where truth arrives through consequence. In science, forecasting, robotics, and other open-ended settings, a useful model has to survive contact with experiment, time, and intervention rather than just produce plausible outputs. A system that tracks our labels better than it tracks the world may look strong in evaluation and still fail where it matters most.
This matters for AI research in a deeper way as well. If current systems mostly learn from the outputs humans have already produced, then their ceiling may remain close to the edge of what humans already know how to express. That does not mean models cannot exceed us in narrow tasks. AlphaGo Zero showed that freer search can move beyond human demonstrations when the environment offers clear feedback. But in open-ended domains, progress depends on whether models can learn from richer forms of contact with reality than static labels and chat logs can provide.
4. Hypothesis
My hypothesis is simple. In open-ended domains, models trained with richer world feedback, such as multimodal temporal structure, action-consequence loops, and uncertainty tracking, will learn representations that transfer better and stay more stable under intervention than models trained mainly on text prediction, labels, and preference signals.
The advantage will not come from removing constraints. It will come from replacing weak human proxies with constraints that are more answerable to the world. Temporal coherence is a better teacher than isolated snapshots. Multimodal grounding is a better teacher than language alone. Action and consequence are better teachers than approval signals when the goal is to learn how a system or environment actually works. Uncertainty tracking matters because a model that can represent what it does not know has a better chance of correcting itself when new evidence arrives.
This claim is testable. If two systems are trained on the same domain, and one relies mostly on proxy supervision while the other learns from world-coupled signals, the second should generalize more robustly when the environment changes or when it must act on what it knows. That result would support the broader argument of this essay: the next step in AI is not less structure, but better structure.
5. Pilot experiment
I propose a pilot that tests one narrow question: do models trained with richer contact with the world learn more stable and transferable representations than models trained mainly on human-defined targets?
The goal is not to solve truthful AI in one step. It is to compare two training regimes on the same domain and see which one holds up better when the setting changes. The domain should have three features. It should unfold over time, include more than one modality, and allow intervention with measurable consequences. Good candidates include robotics, controlled simulators, lab video, microscopy, or other settings where action changes what happens next.
The experiment would train two systems on the same task environment. The first would use standard proxy-heavy training: text supervision, labeled examples, and reward signals tied to benchmark performance. The second would use reality-coupled training: multimodal temporal prediction, action-consequence feedback, and uncertainty estimates tied to what the system does not yet know.
Both systems would then face tasks that go beyond their training setup. Instead of asking only which model scores higher on familiar benchmarks, the evaluation would test which one transfers better to new conditions, adapts better after intervention, and preserves more stable internal structure when the environment shifts. A useful version of this test would include three forms of evaluation: out-of-distribution transfer, causal intervention performance, and robustness under missing or conflicting evidence.
Success in this pilot would not mean the second system becomes fully truthful. It would mean that it forms representations that break less easily when pushed outside the narrow channel of human labels and static tests. That is the result that matters, because it would show a practical path away from training systems to match our targets and toward training systems that stay answerable to the world.
Closing
Frontier AI has made striking progress by learning from human-made targets at scale. That strategy has carried the field far. It has also shaped the limits we now run into. If we want systems that do more than imitate the surfaces we can label, reward, and benchmark, then the field has to train them under constraints that are closer to the structure of reality itself.
The question is no longer whether AI needs constraints. It does. The question is which constraints will help models learn representations that survive contact with the world. That is where the next serious progress may come from. Not from removing structure, but from replacing proxies that are easy for us to score with feedback that is harder for reality to fake.