- Foundation
- About 15 min
- 5 practice questions
Strengths and limits of AI: errors, hallucination, human checking and key terms
Judge what AI does well and where it fails, explain errors such as bias and hallucination, say why a human must check AI output, and use the key AI terms correctly.
If something looks wrong, please tell us.
AI can be a powerful helper for a student or an office, but only if you know its limits. A chatbot can explain a topic in seconds and, in the same answer, state a wrong date with full confidence. This lesson shows what AI is good at, where it goes wrong, and why a person must check its output before relying on it.
What AI does well
- Speed and scale - checks millions of transactions or documents far faster than people.
- Pattern-finding - spots patterns in huge data, such as signs of fraud or early signs of crop disease in photos.
- Consistency - applies the same method every time and does not get tired or bored.
- Repetitive work - sorting, tagging, transcribing and translating routine material.
- Always available - answers at any hour, in many languages.
What AI cannot do (or does poorly)
- True understanding and common sense - it works from patterns, not lived experience, so it can miss what any child would notice.
- Values and ethics - it cannot take moral responsibility or be held accountable; people and organisations are.
- New situations - it is unreliable on cases unlike its training data.
- Explaining itself - many learning models are a 'black box': even their makers cannot fully say why a particular answer came out.
- Knowing recent events - a model knows only what was in its data unless it is connected to fresh sources.
How AI goes wrong
| Error | What happens | Example |
|---|---|---|
| Bias | Unfair results because the training data were unbalanced or reflected past unfairness | A face system trained mostly on one group of faces works worse for others |
| False positive | Says 'yes' when the true answer is 'no' | A genuine payment blocked as fraud |
| False negative | Says 'no' when the true answer is 'yes' | A spam email lets a scam through to the inbox |
| Hallucination | A chatbot gives fluent, confident, but false or made-up information | Invents a book title, a quotation, a date or a law section |
Why does hallucination happen? A chatbot built on a language model produces the words that are most likely to follow, based on patterns in its training text. It is designed to sound right, not to look facts up in a checked record. When the pattern is weak, it still produces something plausible - and that something may be wrong. Confidence of tone is therefore no evidence of accuracy.
Why a chatbot hallucinates
- Learns patterns from text A language model is built from patterns in its training text.
- Produces likely words It writes the words most likely to follow.
- Sounds right It is designed to sound right, not to look facts up in a checked record.
- Plausible but may be wrong When the pattern is weak, it still produces something plausible.
Why a human checks the output
A person must stay in the loop for anything that matters: a medical decision, a loan, an exam answer, a news report. The human catches errors, adds context the AI lacks and takes responsibility for the final decision. For facts such as dates, fees, rules or figures, check against a reliable source - a textbook or the official notice - before you use them.
A person in the loop
- AI gives an output
- Person checks it Facts such as dates, fees and figures are checked against a reliable source.
- Person adds context Context the AI lacks.
- Person decides And takes responsibility for the final decision.
Worked example 1: checking a chatbot's answer
- You ask a chatbot who coined the term 'artificial intelligence'.
- It replies 'Alan Turing, in 1950' in a confident tone.
- You check your notes or a standard textbook: the term was coined by John McCarthy for the 1956 Dartmouth workshop; Turing's 1950 paper proposed the imitation game.
- Conclusion: the chatbot mixed up two real facts - a typical hallucination. Write the checked answer, not the chatbot's.
Worked example 2: false positives and negatives
Suppose a fraud filter checks 1,000 payments (illustrative figures). It flags 30. Of those 30, 24 really were fraud and 6 were genuine (6 false positives). It also missed 4 frauds among the payments it passed (4 false negatives). Working: total real frauds = 24 + 4 = 28. Wrongly blocked genuine payments = 6. Missed frauds = 4. A stricter filter would cut missed frauds but block more genuine customers - a human team decides the right balance and reviews flagged cases.
Key terms glossary
| Term | Meaning |
|---|---|
| Algorithm | A step-by-step method for solving a problem or for learning from data |
| Model | The learned patterns produced by training, used to make predictions |
| Dataset | The collection of examples used to train or test a model |
| Training | The stage in which a model learns patterns from data |
| Inference | Using a trained model on new input to get an output |
| Chatbot | A program that holds a conversation with people in text or voice |
| AGI | Artificial General Intelligence: hypothetical AI able to do any intellectual task a human can |
| Hallucination | Confident but false or made-up output from an AI system |
| Bias | Systematic unfairness in AI output, often caused by unbalanced data |
Use AI as an assistant, not an authority. Fluent is not the same as true: check every fact that matters against a reliable source.
Key takeaways: AI is fast, consistent and good at patterns, but it lacks understanding, common sense and accountability. It makes errors - bias, false positives and negatives, and hallucination. A person checks the output and takes the decision. Learn the glossary terms exactly; they appear in exam questions.
Check your understanding
-
1. In AI, 'hallucination' means:
- The AI system switching itself off
- A camera seeing images in the dark
- The AI giving confident but false or made-up information
- A virus attacking the AI
Show answer
Answer: C. Hallucination is when an AI system, especially a chatbot, produces fluent, confident output that is false or invented, such as a fake reference or a wrong date.
-
2. A fraud filter blocks a customer's genuine payment as suspicious. This is a:
- False negative
- Hallucination
- True positive
- False positive
Show answer
Answer: D. The system said 'fraud' (positive) when the truth was 'not fraud', so it is a false positive. A false negative would be a real fraud let through.
-
3. A face-recognition system trained mostly on photos of one group of people works poorly for others. This problem is called:
- Inference
- Bias
- Reinforcement
- Encryption
Show answer
Answer: B. Unfair, uneven results caused by unbalanced training data are bias. The fix starts with more balanced, representative data and human review.
-
4. Why should a human check AI output before using it for an important decision?
- AI can make errors, lacks context and cannot be held accountable
- AI output is always written in a foreign language
- Checking makes the AI run faster
- The law says AI may never be used
Show answer
Answer: A. AI can be wrong, can miss context a person would notice, and cannot take responsibility. A person catches errors and owns the final decision.
-
5. A fraud filter flags 50 payments: 42 are real fraud and 8 are genuine. It also lets 5 real frauds through. How many real frauds were there in total?
- 42
- 50
- 47
- 55
Show answer
Answer: C. Real frauds = frauds caught + frauds missed = 42 + 5 = 47. The 8 genuine payments flagged are false positives and are not counted as fraud.
Sign in to keep track of the lessons you finish.