Understanding Large Language Models: How They Actually Work
A plain-language look at how large language models turn text into predictions, one token at a time.
Predicting the next word, over and over
A large language model (LLM) does one thing at its core: given some text, it predicts which word — or more precisely, which token — is most likely to come next. Everything else, from writing essays to answering questions, is built on repeating that single prediction millions of times.
From text to numbers
Models cannot read letters directly. A tokenizer first splits your text into small chunks called tokens, which can be whole words, parts of words, or punctuation. Each token is then converted into a list of numbers called an embedding, a coordinate that captures something about the token's meaning.
"The cat sat" -> ["The", " cat", " sat"] -> [0.12, -0.04, ...], [0.31, 0.02, ...], [0.08, -0.19, ...]
Layers that build understanding
Those embeddings pass through dozens of stacked layers. Each layer looks at the surrounding tokens and adjusts its understanding of what the sentence means so far. By the final layer, the model has built a rich representation of the entire input, which it uses to score every possible next token.
Sampling the next token
The model does not simply pick the single most likely token every time. A temperature setting controls how much randomness is allowed: low temperature produces safe, repetitive text, while higher temperature produces more varied and creative output at the cost of occasional nonsense.
Why this matters for how you use one
Because an LLM is fundamentally a next-token predictor, it has no built-in sense of truth. It generates text that is statistically plausible given its training data, not text that has been checked for accuracy. Keeping that in mind changes how you write prompts and how much you double-check the results.
Key takeaways
Large language models predict one token at a time, using layered attention over embeddings to decide what comes next. Understanding this mechanism explains both their fluency and their tendency to produce confident-sounding mistakes.