Part 5/15:
Karpathy underscores that many early attempts to create agents were prematurely ambitious. Reinforcement learning in gaming and Atari environments, while influential, diverted focus from foundational issues like representation learning and continual adaptation. He emphasizes: we need neural representations first, which are built through large-scale pretraining, before effective agents can emerge.
He notes that training neural networks on specific tasks or trying to build full agents too early leads to exponential resource expenditure with limited progress. Today, with massive pretraining on internet data, models like GPT-4 have acquired vast representational capabilities that serve as a base for future agent development.