LLM capabilities, limits, and future directions
A useful way to understand a modern LLM is not merely as a text generator but as the central processor of a new computing paradigm — what Andrej Karpathy calls the LLM OS. In this framing, the LLM acts like an operating system kernel: it coordinates a context window (RAM), external databases via retrieval-augmented generation (storage), tool-calling for calculators and code interpreters (peripherals), web search (network), and increasingly multimodal inputs and outputs (vision, audio). This reframing helps explain both the power and the characteristic failure modes of current systems.
The context window as working memory
A Transformer can only attend over its context window — the maximum number of tokens it processes in a single forward pass (2,048 tokens in the original GPT-3; far larger in recent models). Everything the model "knows" about the current conversation must fit here. Long-term storage beyond the context window requires external retrieval systems that page relevant content back in — a pattern formalized as Retrieval-augmented generation (RAG), which embeds source documents in a vector database and injects the most relevant chunks into the prompt at inference time. This hard boundary means LLMs can exhibit impressive coherence within a context while having no persistent memory across conversations.
Tool use and multimodality
Because LLMs are poor at precise arithmetic performed "in their heads," they are trained to recognize when to delegate: writing Python code, invoking a calculator, or calling an API, then reading the result back into the context window. This tool-use pattern substantially extends what LLMs can do reliably. On the input side, vision-language models process image patches alongside text tokens (see Transformer architecture for the patch-embedding approach in Vision Transformers), and speech-to-speech systems allow voice interaction in real time.
System 1 vs. System 2 thinking
In Daniel Kahneman's framework, System 1 thinking is fast and automatic; System 2 is slow, deliberate, and resource-intensive. Current LLMs operate purely in System 1 mode: they generate tokens sequentially, spending the same amount of computation on a trivial function word as on a critical mathematical step, with no mechanism to pause, explore a tree of possibilities, or self-correct before committing to an answer. A major research frontier is developing System 2 capabilities — allowing models to allocate more compute to harder problems, plan across multiple reasoning steps, and verify their own outputs before responding. This connects directly to the challenge described in LLM training and pretraining: reinforcement learning for open-ended tasks requires an automated reward signal, which is exactly what is missing when the task is "reason well about anything."
Self-improvement and the reward problem
In narrow domains with clear win conditions — Chess, Go — AI systems can improve dramatically through self-play RL, as AlphaGo demonstrated. For general language tasks, no equivalent automatic oracle exists. There is no program that can reliably judge whether an argument is sound, a summary accurate, or a plan wise, without human evaluation. Until that problem is solved, general LLM self-improvement at the level AlphaGo achieved remains out of reach.