- Practice
- About 16 min
- 5 practice questions
Generative AI basics: next-token prediction, training, temperature and the context window
Explain how generative AI makes text, images, audio and code, how a language model predicts the next token, and what pre-training, fine-tuning, GPT, diffusion, temperature and the context window mean in practice.
If something looks wrong, please tell us.
By the end of this lesson you can
- Explain how a large language model builds an answer one token at a time, and why fluent text can still be false.
- Describe pre-training and fine-tuning, expand GPT, and outline how a diffusion model makes an image.
- Predict what changing the temperature does and what happens when a chat outgrows the context window.
Practice questions on generative AI describe a situation - a teacher using a chatbot, a student comparing two answers - and ask what is happening inside the tool. This lesson gives you that working model.
What generative AI makes
Predictive AI labels or scores what already exists, such as a spam filter. Generative AI creates new content - text, images, audio, video or code - resembling what it learned from. The instruction you give it is the prompt.
| Output | What it makes | Well-known examples |
|---|---|---|
| Text | Answers, summaries, letters, translations | Chatbots such as ChatGPT, Gemini and Claude |
| Images | Pictures from a written description | Image generators such as DALL-E, Midjourney and Stable Diffusion |
| Audio | Speech in a chosen voice, music | Text-to-speech and voice tools |
| Code | Programs, spreadsheet formulas, bug fixes | Coding assistants such as GitHub Copilot |
How a large language model writes
A chatbot runs on a large language model (LLM), which works in tokens - a token is a word or a piece of a word. Given the text so far, the model estimates how likely each possible next token is, picks one, adds it and repeats. The whole answer is built this way, one token at a time. It does not look facts up in a checked database; it produces the continuation that best fits learned patterns - fluent, but not necessarily true.
How an LLM builds an answer, one token at a time
- Take the text so far Your prompt plus everything written up to now
- Estimate how likely each next token is A token is a word or a piece of a word
- Pick one token
- Add it to the text Then the loop repeats until the answer is complete
Then back to step 1: Take the text so far
- Pre-training: the model reads a very large collection of text - books, articles, web pages - and learns to predict the next token. This is where its general knowledge and language ability come from.
- Fine-tuning: it is then trained further on a smaller set of chosen examples, such as instructions paired with good answers, so that it follows requests helpfully.
- Human feedback: reviewers compare answers and the model is adjusted towards the more helpful and safer ones (RLHF).
| Letter | Word | Meaning |
|---|---|---|
| G | Generative | It produces new text rather than only classifying text |
| P | Pre-trained | It is first trained on a huge general text collection, then adapted |
| T | Transformer | A neural-network design that uses 'attention' to weigh how the words in a passage relate to one another |
Diffusion image models in one paragraph
Many image generators use diffusion. In training, noise is added to real images step by step until only static remains, and the model learns to reverse each step. To create a picture it starts from random noise and removes it a little at a time, steered by your text prompt, until an image appears. Because it works from learned patterns rather than an understanding of objects, details such as fingers or lettering on a signboard often come out wrong.
Two settings that change behaviour
| Setting | What it controls | In practice |
|---|---|---|
| Temperature | How much randomness is allowed when the next token is chosen | Lower: more focused, more repeatable wording. Higher: more varied, more surprising wording. It does not change what the model knows. |
| Context window | How much text, counted in tokens, the model can take into account at once: instructions, the conversation, pasted documents and its own reply | Text beyond the limit cannot be considered, so in a very long chat early instructions or details may be lost or need repeating. |
Worked example 1: predicting the next token
Prompt: "The capital of India is". Suppose (illustratively) the model rates " New" as very likely, picks it, and then rates " Delhi" as by far the most likely next token. But if a fact appeared rarely or inconsistently in the training text, the likeliest-sounding continuation may be wrong - and it reads just as confidently. A confident tone tells you nothing about accuracy.
Worked example 2: choosing the temperature
A coaching-centre teacher has two jobs. (a) Twenty different slogans for a library-week poster: variety matters, so a higher temperature suits it. (b) Each day's notes as a five-point summary in the same layout: consistency matters, so a lower temperature suits it. Some tools let you set this; many chat apps choose it for you. Trap: lower temperature makes answers more repeatable, not more true - facts are still checked against the source.
Choosing the temperature
-
Lower temperature
- More focused, more repeatable wording
- Suits: each day's notes as a five-point summary in the same layout
- Does not make answers more true
-
Higher temperature
- More varied, more surprising wording
- Suits: twenty different slogans for a poster
- Does not change what the model knows
Worked example 3: a chat that outgrows the window
A student begins a chat with "Always answer in Hindi", pastes three long chapters and asks forty questions. Later replies drift into English: the opening instruction is now far back or outside what the model can consider. Fixes: repeat key instructions, start a fresh chat with a short summary, or work one chapter at a time.
An LLM writes by predicting the next token, again and again. Temperature changes how varied the wording is, not how true it is. The context window limits how much text it can consider at once.
Key takeaways: an LLM is pre-trained on huge text collections, then fine-tuned, and writes one token at a time. GPT means Generative Pre-trained Transformer; diffusion models turn noise into images. Higher temperature gives variety, lower gives consistency - never truth. Restate what matters in long chats.
Check your understanding
-
1. What does GPT stand for?
- General Purpose Technology
- Generative Pre-trained Transformer
- Graphical Processing Tool
- Generated Predictive Text
Show answer
Answer: B. GPT is Generative Pre-trained Transformer: generative because it produces new text, pre-trained on a huge general text collection, and built on the transformer neural-network design.
-
2. A district office wants a chatbot to suggest many different slogans for a cleanliness drive, as varied as possible. Which setting fits this job best?
- A lower temperature, so every answer is the same
- A smaller context window
- Switching off pre-training
- A higher temperature, so the wording varies more
Show answer
Answer: D. Temperature controls randomness in choosing the next token. A higher value gives more varied, surprising wording, which suits brainstorming. A lower value suits jobs that need consistency. Pre-training cannot be switched off by a user, and the context window does not control variety.
-
3. A user sets a chatbot's temperature to its lowest value, hoping this will make the exam dates it gives correct. Which statement is right?
- Lower temperature makes wording more consistent, but does not make facts correct; the dates must still be checked in the official notification
- Lowest temperature guarantees correct facts, because the model then uses a verified database
- Lower temperature makes the model search the internet for the latest dates
- Temperature has no effect on the output at all
Show answer
Answer: A. Temperature only changes how much randomness is used in picking tokens. A wrong date can be repeated consistently at low temperature. The model does not consult a verified database, so official facts are checked in the official notification.
-
4. In a very long chat, a chatbot stops following an instruction the student gave in the first message. What is the most likely reason?
- The chatbot is refusing on purpose
- The temperature was set too low
- The early instruction is far back or outside the context window, so the model no longer takes it into account
- The model has been fine-tuned again during the chat
Show answer
Answer: C. The context window limits how much text the model can consider at once. When a chat grows very long, early instructions can be lost, so key instructions should be repeated or a fresh chat started with a summary.
-
5. Which statement correctly describes fine-tuning?
- Reading a very large general collection of text to learn to predict the next token
- Further training on a smaller set of chosen examples, such as instructions with good answers, so the model follows requests helpfully
- Adjusting the randomness of each reply
- Typing a better prompt into the chat box
Show answer
Answer: B. Fine-tuning comes after pre-training and uses selected examples to shape behaviour. Learning from a huge general collection is pre-training, randomness is the temperature, and a better prompt guides one reply without training the model.
Sign in to keep track of the lessons you finish.