How do LLMs work? Large language models, explained simply

You type a question. Three seconds later, a fluent, structured answer streams onto your screen. What actually happened in between? Understanding it, even loosely, makes you noticeably better at using these tools.

One word at a time

A large language model does exactly one thing: given some text, it predicts which word (technically, token, a word fragment) is most likely to come next. Then it appends that word and predicts the next one. Every essay, poem, and program an LLM has ever produced was generated one token at a time, left to right.

This sounds too simple to work. The magic is in what "predicting well" requires. To continue "France's capital is" correctly, the model must store geography. To continue a half-finished proof, it must have absorbed logic. Trained on trillions of words, the model is forced to compress a working sketch of how the world behaves, or at least how text about the world behaves.

Tokens: the unit everything is measured in

The model does not read words. It reads tokens, which are chunks of text somewhere between a character and a word. In English a token averages about three quarters of a word, so a thousand words is roughly thirteen hundred tokens. Common words are usually one token each; unusual ones get split into pieces.

This sounds like an implementation detail and it explains four things people find baffling.

Why AI is billed the way it is. API pricing is per token, in and out. A long document costs more to process not because it is harder but because it is more tokens.

Why it is bad at counting letters. Ask a model how many times a letter appears in a word and it will often get it wrong, which looks absurd for something that can write code. It never saw the letters. It saw a token, the way you see a word rather than a sequence of strokes.

Why some languages cost more. Tokenisers are trained mostly on English text, so English packs efficiently and many other languages fragment into more tokens for the same meaning. The same sentence can cost noticeably more in one language than another.

Why length limits are stated in tokens. Which brings us to the thing that catches everyone out.

The context window, and why the AI forgot

Everything the model can see at once, your whole conversation plus any documents you pasted plus the reply it is currently writing, has to fit in a fixed budget called the context window. Anything outside it does not exist. Not deprioritised, not summarised. Absent.

That is the honest explanation for the most common complaint about these tools: you had a long conversation, and partway through it started contradicting things you agreed at the start. Nothing went wrong. The start fell off the end.

Two things follow, and both are practical.

Front-load what matters. Give the constraints, the reference material and the goal early and restate them if the conversation runs long. In a very long chat, starting a fresh conversation with a summary you write yourself often works better than continuing.

A big context window is not the same as using it well. Models advertise large windows, and filling one is not the same as the model weighting all of it evenly. Attention across a very long input is uneven in practice, and material in the middle of a long document gets less of it than material at either end. Pasting a hundred pages and asking a narrow question about page fifty is the case most likely to disappoint you.

Why they sound so human

After base training, models are refined with human feedback: people rate candidate answers, and the model learns to prefer responses that are helpful, harmless, and clear. This is why chatbots apologise, hedge, structure answers with headers, and ask clarifying questions. The conversational personality is a learned layer on top of the raw text predictor.

Why they make things up

Hallucination falls straight out of the design. The model always produces the most plausible continuation, and when it doesn't know something, the most plausible continuation is often a confident-sounding invention. A fake citation looks statistically identical to a real one.

This is why the golden rule of AI use is: fluency is not accuracy. Verify anything that matters, especially names, numbers, dates, quotes, and citations.

Where the knowledge comes from, and when it stops

Training happens once, on an enormous pile of text, and then stops. The model is a snapshot. That is why it has a knowledge cutoff, and why it will describe the world as of that snapshot in the present tense without flagging that it is doing so.

Three things get bolted on to work around this, and it is worth knowing which one is operating.

Search. The model is given a tool to fetch pages and reads the results as context. Current, and dependent on whether the fetch succeeded, which is less reliable than it sounds because many sites block automated access.

Retrieval from documents you supplied. The material is put into the context window, so the model is reading rather than remembering. This is the most reliable of the three and the one you have most control over.

Memory features. Notes the product keeps about you between conversations and quietly adds to the context. Useful, and worth knowing about, because an answer shaped by something you said last month can be baffling if you have forgotten you said it.

None of them changes the underlying model. They change what is in the context window when it runs, which is a different thing, and understanding that distinction explains most of what these products can and cannot do.

What this means for how you prompt

• Context is everything. The model only sees what's in the conversation. Paste the relevant document instead of describing it.
• The beginning shapes the rest. Because generation is sequential, asking for the structure you want up front ("answer as a table") works better than fixing it afterwards.
• Ask it to reason first. "Think through this step by step before answering" genuinely improves accuracy on hard problems, because the reasoning tokens become context for the final answer. The model is, in a real sense, thinking out loud because it has nowhere else to think.
• Give examples rather than adjectives. "Write it like this" with a sample beats "write it professionally". The model is matching patterns, so hand it the pattern.
• Length limits are a budget you share. A very long instruction leaves less room for the answer and for the document you wanted it to read. When something gets truncated, this is usually why.

What this does not mean

Two overcorrections, both common, both wrong.

"It is just autocomplete, so it does not do anything real." Predicting the next token well across trillions of words turns out to require internal structure that looks a lot like understanding of the material, and the outputs are useful in ways plain pattern-matching does not account for. Dismissing it on the mechanism misjudges what the mechanism produces.

"It thinks like a person, so I can trust it like one." It has no beliefs, no intentions, no sense of when it is out of its depth, and no experience of the world outside text. A person who sounds confident has usually checked. A model that sounds confident has done nothing of the kind, which is the subject of why AI makes things up.

The useful stance sits between those, and it is not a fudge: treat it as a probabilistic engine with an extraordinary amount of compressed human knowledge, brilliant at things where you can check the answer, unreliable at things where you cannot.

The takeaway

An LLM is autocomplete scaled to the point where it became something qualitatively new. It has no beliefs and no intentions, but it has compressed an extraordinary amount of human knowledge into something you can talk to. Treat it as a probabilistic engine rather than an oracle, and it will serve you remarkably well. Start with the plain-English AI guide if you want the bigger picture, or the agents guide for what happens when one of these is wired up to act rather than answer.