An LLM does not know anything the way a person does — it guesses the next word. That one fact explains why it can still be useful, why every token costs money, and why it sometimes sounds confident while being wrong.
This is part 1 of Learning AI Engineering in Public. I am writing for complete beginners — frontend developers who are new to AI, including me.
One-sentence answer: An LLM is a next-token predictor. Think of it as super-smart autocomplete.
A next-token predictor
When your editor suggests the next bit of code, it is guessing from what you already typed. A large language model does the same job with ordinary language. It looks at the tokens so far, picks the next token, then repeats until the reply is finished.
It is not a database. It does not "look up" a page unless some other step fetched that page first. Autocomplete can invent a plausible line of code that never existed. An LLM can invent a plausible sentence the same way.
Tokens, the prompt, and the context window
A token is a chunk of text the model counts. It is often a word, part of a word, or punctuation. Cost and limits are counted per token — not per message, and not per word.
The prompt is everything you send in: instructions plus the question.
The context window is how many tokens the model can see at once. Your prompt and the tokens it has already written both have to fit. A longer prompt costs more and leaves less room for the reply.
Think of tokens like seats on a bus. The prompt takes some seats. The answer takes the rest. When the bus is full, nothing else can board.
Temperature
Temperature changes how the next token is chosen.
At 0, the model stays with the likely choice, so the output is consistent. Higher temperature lets it pick less likely tokens more often, so the wording varies.
Regenerate the same prompt in ChatGPT and the wording changes each time. That is sampling — the app does not expose a temperature slider, but the variation comes from the same kind of randomness temperature controls when you call the API. In Part 2, you set temperature directly in code.
Hallucination
A hallucination is text that "sounds right" even when it is not a fact. The model is still predicting the next token. Fluency is not a source. If nothing retrieved a real page first, the model is free to autocomplete a confident-looking story.
Retrieval before generation
I asked ChatGPT, with web search on, "Who is Yeremia?" It answered from my website and LinkedIn, and it showed sources.
That order is the point. Something retrieved the pages first. Then the model wrote the reply from those pages. Retrieval before generation is the core idea of RAG — retrieval-augmented generation.
Without retrieval, you get autocomplete. With retrieval, you get autocomplete that is steered by documents you actually fetched.
My portfolio chatbot on /lab does the same thing on this site: it retrieves content from the portfolio first, then generates the reply. When you ship something like that, a pre-launch checklist helps — see What I Checked Before Letting Strangers Talk to My AI Chatbot.
Takeaways
- An LLM predicts the next token, not facts from memory — that explains both useful answers and hallucinations.
- Bills and cutoffs are per token; the prompt and context window share one fixed number of seats.
- Same prompt, different wording on each regenerate: sampling in the app; temperature when you call the API (Part 2).
- Fetch documents first, then generate — web search, /lab, or full RAG all follow retrieval before generation.