- Practice
- About 14 min
- 5 practice questions
What generative AI is: models that create text, images, audio and code
Explain what generative AI does, how a large language model writes by predicting the next token, what GPT stands for and how diffusion image generators work.
If something looks wrong, please tell us.
Generative AI now appears in general-studies and computer-awareness papers, and many students already use AI chatbots to study. Questions test whether you know what these tools actually do: how they write, where their knowledge comes from and why they sometimes go wrong. Understanding the mechanism also makes you a better, safer user.
Generative AI versus other AI
Most older AI systems are predictive: they sort or score things that already exist. A spam filter decides whether an email is spam, face unlock decides whether a face matches, and a bank system flags a risky transaction. Generative AI instead creates new content - text, images, audio, video or code - that resembles the material it learned from. The instruction you give it is called a prompt.
| Output | What it makes | Well-known examples |
|---|---|---|
| Text | Answers, summaries, letters, translations | Chatbots such as ChatGPT, Gemini and Claude |
| Images | Pictures from a written description | Image generators such as DALL-E, Midjourney and Stable Diffusion |
| Audio | Speech in a chosen voice, music | Text-to-speech and voice tools |
| Code | Programs, spreadsheet formulas, bug fixes | Coding assistants such as GitHub Copilot |
How a large language model writes
A chatbot runs on a large language model (LLM). It works with tokens - a token is a word or a piece of a word. Given the text so far, the model calculates how likely each possible next token is, picks one, adds it to the text and repeats. A whole answer is built one token at a time. The model does not look each fact up in a checked database; it produces the continuation that best fits the patterns it learned. That is why its writing is fluent, and also why fluent text can still be false.
How an LLM builds an answer, one token at a time
- Take the text so far Your prompt plus everything written up to now
- Rate every possible next token How likely each word or piece of a word is to come next
- Pick one token Usually a likely one; temperature adds some randomness
- Add it to the text Then the loop repeats until the answer is complete
Then back to step 1: Take the text so far
The model does not always pick the single most likely token. A setting often called temperature adds controlled randomness, so the same prompt can give differently worded answers on different tries.
- Pre-training: the model reads a very large collection of text - books, articles, websites - and learns to predict the next token.
- Fine-tuning: it is trained further on examples of good questions and answers so that it follows instructions.
- Human feedback: reviewers rate answers, and the model is adjusted towards the more helpful and safer ones (reinforcement learning from human feedback, RLHF).
What GPT stands for
| Letter | Word | Meaning |
|---|---|---|
| G | Generative | It produces new text rather than only classifying text |
| P | Pre-trained | It is first trained on a huge general text collection, then adapted |
| T | Transformer | A neural-network design that uses 'attention' to weigh how the words in a passage relate to one another |
GPT = Generative Pre-trained Transformer. An LLM writes by predicting the next token, again and again; it predicts likely text, it does not look up verified facts.
Image generators: diffusion in one paragraph
Many image generators use diffusion. During training, noise is added to real images step by step until only static remains, and the model learns to reverse each step - to remove a little noise at a time. To create a new picture it starts from pure random noise and removes noise step by step, steered by your text prompt, until an image appears. Because it builds pictures from learned patterns rather than from an understanding of objects, small details such as fingers or lettering in the background often come out wrong.
How a diffusion model makes a picture
- Start from pure random noise
- Remove a little noise Steered by your text prompt
- Repeat step by step Each step reverses one step of added noise learned in training
- An image appears
Worked example 1: predicting the next token
Prompt: "The capital of India is". Suppose (illustratively) the model rates the next token as " New" very likely, " Delhi" less likely and " Mumbai" unlikely. It picks " New", then for the text "The capital of India is New" it rates " Delhi" as by far the most likely next token and adds it. Result: "The capital of India is New Delhi." Now suppose a fact appeared rarely or inconsistently in the training text. The most likely-sounding continuation may then be wrong, yet the sentence will read just as confidently. Lesson: confidence of tone tells you nothing about accuracy.
Worked example 2: generative or predictive?
| System | Creates new content? | Type |
|---|---|---|
| Email app moving a message to spam | No - it labels an existing email | Predictive |
| Chatbot drafting a leave application | Yes - new text | Generative |
| Phone unlocking when it sees your face | No - it matches a face | Predictive |
| App turning 'a poster for a science fair' into a picture | Yes - new image | Generative |
- Context window: how much text the model can take into account at once - your prompt, the conversation so far and its own reply.
- Multimodal: a model that can take in or produce more than one kind of data, such as text and images.
Key takeaways: generative AI creates new content; predictive AI sorts or scores existing content. An LLM writes by repeatedly predicting the next token after pre-training on huge text collections, then fine-tuning and human feedback. GPT means Generative Pre-trained Transformer. Diffusion models make images by removing noise step by step, guided by the prompt.
Check your understanding
-
1. What does GPT stand for?
- General Purpose Technology
- Generative Pre-trained Transformer
- Graphical Processing Tool
- Generated Predictive Text
Show answer
Answer: B. GPT is Generative Pre-trained Transformer: generative because it produces new text, pre-trained on a huge general text collection, and built on the transformer neural-network design.
-
2. A large language model produces its reply mainly by:
- Looking up each answer in a fixed database of verified facts
- Copying the single closest web page word for word
- Repeatedly predicting a suitable next token from the text so far
- Passing the question to a human operator
Show answer
Answer: C. An LLM builds its answer one token at a time, each time predicting a likely next token from the text so far. It does not consult a verified fact database, which is why fluent answers can still be wrong.
-
3. Which of these is an example of generative AI?
- A tool that writes a leave application from a one-line request
- A spam filter marking an email as spam
- Face unlock on a mobile phone
- A bank system flagging a risky transaction
Show answer
Answer: A. Writing a new leave application creates new content, so it is generative. The spam filter, face unlock and fraud flag only label or match things that already exist, which is predictive AI.
-
4. A diffusion-based image generator creates a picture by:
- Stitching together pieces of photographs found on the internet
- Drawing outlines first and then colouring them by fixed rules
- Searching a stock-photo library for the closest match
- Starting from random noise and removing it step by step, guided by the prompt
Show answer
Answer: D. Diffusion models learn to reverse the adding of noise. To generate, they begin with pure noise and remove a little at a time, steered by the text prompt, until an image emerges.
-
5. A student asks a chatbot the same question twice and gets two differently worded answers. The most likely reason is:
- The chatbot is broken and should not be used
- The model picks next tokens with some randomness, so wording can vary between tries
- The question was stored incorrectly the first time
- The internet connection changed some of the words
Show answer
Answer: B. The model does not always choose the single most likely token; a randomness setting (often called temperature) lets wording vary. Different wording is normal and not a fault.
Sign in to keep track of the lessons you finish.