ChatGPT the Memorious

In “Funes the Memorious,” Argentine writer Jorge Luis Borges depicts a series of fictional encounters with Ireneo Funes, an Uruguayan teenager. After a horseback riding accident, Funes becomes hopelessly injured and, secluded in his house, develops an extraordinary memory. He perceives everything in full detail and can (exactly) reconstruct the past. The short story then engages with the absurdity of his “total recall.” His prodigious memory handicaps him: he cannot abstract, cannot generalize, and can’t even sleep.

This story came up in a recent conversation about the significant challenges we face in assessing the capabilities of large language models (LLMs). On the one hand, LLMs have stumped the world by writing code, passing professional exams, and responding fluidly in natural language. Their apparent mastery of complex subjects often feels indistinguishable from human expertise. On the other hand, they frequently fail to maintain coherent abstractions across contexts. Recent work, for example, has shown that transformers fail in retrieving coherent world models in fairly simple scenarios.

It’s tempting for skeptics to point to Funes as an analogy for LLMs and call them “stochastic parrots,” mere mimics without understanding. I don’t think this is fair; LLMs have demonstrated generalization beyond imitation. But the Funes metaphor does capture something about their limitations: an overabundance of detail, and a struggle to forget or compress information in useful ways.

So what would it look like to design language models that forget productively? Models that, like humans, can abstract away from the noise and focus on what matters? Some researchers have begun exploring “forgetting” by fine-tuning models with negative data, teaching them what not to recall. But perhaps there’s room to build the structure of forgetting—and abstraction—deeper into our AI systems.

After all, Borges’ parable reminds us: remembering everything is less a blessing than a curse. For humans and machines alike, the path to intelligence may depend as much on what we forget as on what we remember.