Generative AI & LLM Engineering · Chapter 1 · ~15 min · No maths needed
What an LLM Actually Is — Inside the Next-Token Machine
ChatGPT, Claude and Gemini feel like magic. Underneath, they do one simple thing again and again: guess the next small piece of text. Play with the machine and you'll understand AI better than most people who use it daily.
- Explain an LLM in one sentence
- See text split into tokens
- Control randomness with temperature
- Estimate cost & context limits
1Super-powered autocomplete
Your phone keyboard suggests the next word. A Large Language Model (LLM) is the same idea, trained on a huge slice of the internet and books so its guesses are amazingly good. It never “looks up” an answer — it writes one, piece by piece.
Loop: read everything so far → guess the next piece → stick it on the end → repeat. Step 1 of 8.
How does an LLM produce a long answer?
2Tokens: the pieces the model reads
Models don't see letters or whole words. They see tokens — common chunks of text. “the” is one token; a rare word like “unbelievably” may be three. Type below and watch it slice.
14 pieces · 52 characters · “·” marks a space that belongs to the token. Common words stay whole; rare long words get chopped.
Roughly how many tokens is a 400-character English paragraph?
3Probabilities & the temperature dial
For every step the model gives each possible token a score, turned into percentages. Then it picks one. Temperature decides how adventurous that pick is. Drag it and roll the dice several times.
The cat sat on the ___
matsofafloorroofmoonData extraction, code, facts. Same input → nearly same output.
Chat and writing help. Natural but sensible.
Brainstorming, poems. Surprising — and more mistakes.
You need the model to extract invoice numbers into JSON reliably. Which temperature?
4Context window & cost
The context window is the model's short-term memory: your prompt + chat history + its answer must fit inside. You also pay per token — both what you send in and what it writes back. Slide to feel the numbers.
Example prices: $0.50 per million input tokens, $1.50 per million output tokens (~500 tokens per page).
Why might a chatbot ‘forget’ the start of a very long conversation?
5What LLMs are great — and bad — at
Great at
- Writing, summarising, rewriting
- Explaining & generating code
- Turning messy text into structured JSON
- Translation & tone changes
Watch out for
- Hallucinations: confident made-up facts
- No knowledge after its training cutoff
- Exact maths & counting letters (tokens hide them!)
- Your private data it has never seen
6Chapter challenge
Next-token machine check
Answer all four correctly to complete the chapter.
1Which sentence best describes an LLM?
2Why do LLMs often miscount the letter ‘r’ in “strawberry”?
3Raising temperature from 0.2 to 1.5 will…
4Your 900-page manual doesn't fit the context window. Best next step?
7Recap
- An LLM is super-powered autocomplete: predict a token, append, repeat.
- Tokens are text chunks — about 4 English characters each.
- Temperature controls randomness: low for facts/code, higher for creativity.
- The context window limits what the model can see; you pay for input and output tokens.
- LLMs can hallucinate — always verify important facts.
Solve the challenge above to complete the chapter, or