Are We Summoning Ghosts or Building Animals?
I recently went down a rabbit hole after watching a fascinating conversation between Dwarkesh Patel and Richard Sutton, one of the "godfathers" of Reinforcement Learning (RL). Listening to Sutton, I felt like I was hearing from a classicist, someone whose ideas are forged in the foundational principles of AI, stretching back to the era of Alan Turing. His core belief is that Large Language Models (LLMs) can never be truly intelligent because they are products of supervised learning. They mimic the vast repository of human knowledge; they don't learn from scratch through experience, the way animals do.
I get his perspective. It’s pure. It’s elegant. But something about it feels disconnected from the practical reality of building intelligence now. I can’t help but think that LLMs have found a clever, if imperfect, shortcut. They’ve artificially skipped the messy, billion-year evolution phase by being trained on carefully structured human data, the distilled output of our entire intellectual and cultural history.
It’s similar to how a newborn horse isn't a blank slate. It can walk within minutes of birth not because it rapidly learns physics from scratch, but because millions of years of evolution have hard-coded that ability into its DNA, muscles, and brain. The learning is done before it’s even born. Pre-training on the internet feels like our attempt to give an AI that evolutionary head start. Sure, a model that learns everything from the ground up via pure RL might be more robust, more truly intelligent. But what does that even look like? We can’t just spin up a simulation and have a model relive the entire history of life on Earth: learning to see, walk, use tools, and develop language all at once. The constraints of time and resources make that an impossible dream, for now.
Or is it? I recently finished a project training an agent to play Flappy Bird using a genetic algorithm. The AI got better with each generation, its fitness function tied to survival time, using principles like elitism and mutation to evolve. It was a microcosm of this learning process. Just as that agent was trained on the single, constrained task of flying through pipes, couldn't we simulate environments for everything humans do? Robotics is already deep into this with agents learning to manipulate objects, and models like DeepSeek are using RL on text to figure out the intricate rules of language and logic.
This brings us to the heart of the current debate, a tension perfectly articulated by Andrej Karpathy in his thoughts on the Sutton podcast. He notes that Sutton’s famous essay, "The Bitter Lesson," has become almost biblical in LLM circles. The essay’s core idea is that general methods that leverage massive computation will always win out over approaches that rely on handcrafted human knowledge. Researchers now constantly ask if an idea is "bitter lesson pilled," meaning, does it scale with more compute?
The irony, as Karpathy points out, is that Sutton himself doesn't believe LLMs pass this test. They are trained on a finite, human-generated dataset. What happens when we run out of text? How do we overcome the inherent human biases baked into the data? Sutton’s ideal is Turing’s "child machine," an AI that learns dynamically by interacting with the world, not by imitating a static dataset. In his view, all learning should be a continuous loop of action and consequence, driven by rewards. "If we understood a squirrel," Sutton says, "we'd be almost done."
But as Karpathy rightly argues, this purist view overlooks the cold-start problem. A baby zebra doesn't randomly spasm its muscles until it figures out how to walk; it would be eaten in minutes. It relies on the powerful initialization encoded in its genes. For AI, pre-training is our "crappy evolution." It’s our practical solution for injecting billions of soft constraints into a neural network to get it to a useful starting point.
This leads to Karpathy's brilliant metaphor: today's frontier AI research is not about building animals; it is about summoning ghosts. These LLMs are "a kind of statistical distillation of humanity's documents," thoroughly engineered and muddled by our own intelligence. They aren't the pure, world-learning agents Sutton envisions, but they are incredibly powerful nonetheless. They are perhaps not platonically "bitter lesson pilled," but practically so.
So, where does that leave us? The future of AI doesn’t seem to be a choice between Sutton’s pure RL "animals" and the pre-trained "ghosts." Instead, the most promising path forward appears to be a hybrid one, a synthesis of both philosophies that is already taking shape in frontier labs.
The strategy is twofold. First, we summon the ghost. We use massive, pre-trained models as a foundation, giving them a rich, nuanced understanding of human language, concepts, and common sense, which is the evolutionary shortcut. Second, we teach the ghost to act. Using reinforcement learning techniques like RLHF (Reinforcement Learning from Human Feedback), we give these models goals and allow them to interact with environments, refining their behavior based on outcomes. We are essentially taking the statistical ghost distilled from the internet and training it to behave more like a goal-directed animal.
This approach acknowledges the wisdom in both perspectives. It leverages the incredible power and practicality of pre-training while embracing the core RL principle that true agency comes from learning through interaction and feedback. It may not be the pure, from-scratch learning that the classicists dreamed of, but it’s a path that is yielding astonishing results.
It's entirely possible that the intelligences we create this way will diverge from the animal kingdom's blueprint. They may end up being permanently different, un-animal-like, but world-altering. As Karpathy muses, the relationship might be that "ghosts are to animals as planes are to birds." They are a different kind of intelligence, born of a different evolutionary path, but no less transformative. My initial thought was on the right track: we have found a way to skip the line, to artificially jumpstart evolution. The great task ahead is figuring out what to do with the powerful, strange, and ghostly minds we've summoned.