• Foundation
  • About 15 min
  • 5 practice questions

Strengths and limits of AI: errors, hallucination, human checking and key terms

Judge what AI does well and where it fails, explain errors such as bias and hallucination, say why a human must check AI output, and use the key AI terms correctly.

If something looks wrong, please tell us.

AI can be a powerful helper for a student or an office, but only if you know its limits. A chatbot can explain a topic in seconds and, in the same answer, state a wrong date with full confidence. This lesson shows what AI is good at, where it goes wrong, and why a person must check its output before relying on it.

What AI does well

  • Speed and scale - checks millions of transactions or documents far faster than people.
  • Pattern-finding - spots patterns in huge data, such as signs of fraud or early signs of crop disease in photos.
  • Consistency - applies the same method every time and does not get tired or bored.
  • Repetitive work - sorting, tagging, transcribing and translating routine material.
  • Always available - answers at any hour, in many languages.

What AI cannot do (or does poorly)

  • True understanding and common sense - it works from patterns, not lived experience, so it can miss what any child would notice.
  • Values and ethics - it cannot take moral responsibility or be held accountable; people and organisations are.
  • New situations - it is unreliable on cases unlike its training data.
  • Explaining itself - many learning models are a 'black box': even their makers cannot fully say why a particular answer came out.
  • Knowing recent events - a model knows only what was in its data unless it is connected to fresh sources.

How AI goes wrong

ErrorWhat happensExample
BiasUnfair results because the training data were unbalanced or reflected past unfairnessA face system trained mostly on one group of faces works worse for others
False positiveSays 'yes' when the true answer is 'no'A genuine payment blocked as fraud
False negativeSays 'no' when the true answer is 'yes'A spam email lets a scam through to the inbox
HallucinationA chatbot gives fluent, confident, but false or made-up informationInvents a book title, a quotation, a date or a law section

Why does hallucination happen? A chatbot built on a language model produces the words that are most likely to follow, based on patterns in its training text. It is designed to sound right, not to look facts up in a checked record. When the pattern is weak, it still produces something plausible - and that something may be wrong. Confidence of tone is therefore no evidence of accuracy.

Why a chatbot hallucinates

  1. Learns patterns from text A language model is built from patterns in its training text.
  2. Produces likely words It writes the words most likely to follow.
  3. Sounds right It is designed to sound right, not to look facts up in a checked record.
  4. Plausible but may be wrong When the pattern is weak, it still produces something plausible.

Why a human checks the output

A person must stay in the loop for anything that matters: a medical decision, a loan, an exam answer, a news report. The human catches errors, adds context the AI lacks and takes responsibility for the final decision. For facts such as dates, fees, rules or figures, check against a reliable source - a textbook or the official notice - before you use them.

A person in the loop

  1. AI gives an output
  2. Person checks it Facts such as dates, fees and figures are checked against a reliable source.
  3. Person adds context Context the AI lacks.
  4. Person decides And takes responsibility for the final decision.

Worked example 1: checking a chatbot's answer

  1. You ask a chatbot who coined the term 'artificial intelligence'.
  2. It replies 'Alan Turing, in 1950' in a confident tone.
  3. You check your notes or a standard textbook: the term was coined by John McCarthy for the 1956 Dartmouth workshop; Turing's 1950 paper proposed the imitation game.
  4. Conclusion: the chatbot mixed up two real facts - a typical hallucination. Write the checked answer, not the chatbot's.

Worked example 2: false positives and negatives

Suppose a fraud filter checks 1,000 payments (illustrative figures). It flags 30. Of those 30, 24 really were fraud and 6 were genuine (6 false positives). It also missed 4 frauds among the payments it passed (4 false negatives). Working: total real frauds = 24 + 4 = 28. Wrongly blocked genuine payments = 6. Missed frauds = 4. A stricter filter would cut missed frauds but block more genuine customers - a human team decides the right balance and reviews flagged cases.

Key terms glossary

TermMeaning
AlgorithmA step-by-step method for solving a problem or for learning from data
ModelThe learned patterns produced by training, used to make predictions
DatasetThe collection of examples used to train or test a model
TrainingThe stage in which a model learns patterns from data
InferenceUsing a trained model on new input to get an output
ChatbotA program that holds a conversation with people in text or voice
AGIArtificial General Intelligence: hypothetical AI able to do any intellectual task a human can
HallucinationConfident but false or made-up output from an AI system
BiasSystematic unfairness in AI output, often caused by unbalanced data

Use AI as an assistant, not an authority. Fluent is not the same as true: check every fact that matters against a reliable source.

Key takeaways: AI is fast, consistent and good at patterns, but it lacks understanding, common sense and accountability. It makes errors - bias, false positives and negatives, and hallucination. A person checks the output and takes the decision. Learn the glossary terms exactly; they appear in exam questions.

Check your understanding

  1. 1. In AI, 'hallucination' means:

    1. The AI system switching itself off
    2. A camera seeing images in the dark
    3. The AI giving confident but false or made-up information
    4. A virus attacking the AI
    Show answer

    Answer: C. Hallucination is when an AI system, especially a chatbot, produces fluent, confident output that is false or invented, such as a fake reference or a wrong date.

  2. 2. A fraud filter blocks a customer's genuine payment as suspicious. This is a:

    1. False negative
    2. Hallucination
    3. True positive
    4. False positive
    Show answer

    Answer: D. The system said 'fraud' (positive) when the truth was 'not fraud', so it is a false positive. A false negative would be a real fraud let through.

  3. 3. A face-recognition system trained mostly on photos of one group of people works poorly for others. This problem is called:

    1. Inference
    2. Bias
    3. Reinforcement
    4. Encryption
    Show answer

    Answer: B. Unfair, uneven results caused by unbalanced training data are bias. The fix starts with more balanced, representative data and human review.

  4. 4. Why should a human check AI output before using it for an important decision?

    1. AI can make errors, lacks context and cannot be held accountable
    2. AI output is always written in a foreign language
    3. Checking makes the AI run faster
    4. The law says AI may never be used
    Show answer

    Answer: A. AI can be wrong, can miss context a person would notice, and cannot take responsibility. A person catches errors and owns the final decision.

  5. 5. A fraud filter flags 50 payments: 42 are real fraud and 8 are genuine. It also lets 5 real frauds through. How many real frauds were there in total?

    1. 42
    2. 50
    3. 47
    4. 55
    Show answer

    Answer: C. Real frauds = frauds caught + frauds missed = 42 + 5 = 47. The 8 genuine payments flagged are false positives and are not counted as fraud.

Sign in to keep track of the lessons you finish.

Next lesson → Final self-assessment: AI basics

Privacy · Terms · Refunds · Delivery