• Practice
  • About 15 min
  • 5 practice questions

Limits of AI chatbots: hallucination, training cut-off, bias and keeping data safe

Recognise hallucination, outdated knowledge, confident wrong answers and bias; cross-check answers with official sites, NCERT and two sources; and keep passwords, OTPs and Aadhaar out of chatbots.

If something looks wrong, please tell us.

Every AI chatbot makes mistakes, and it rarely signals when it does. Knowing the typical kinds of error lets you spot them quickly and decide what to check. Exam questions on AI often ask you to name these limits or choose the safe action in a scenario.

Four limits to know

LimitWhat it meansExample
HallucinationThe model produces false information that sounds plausibleA made-up book title, court case or quotation
Training cut-offIts knowledge stops when its training data was collectedIt does not know about a recent scheme or result
Confident wrong answersThe tone sounds equally sure whether right or wrongAn arithmetic slip stated firmly
BiasIt repeats unfair patterns present in its training dataShowing only men when asked for a picture of an engineer

These limits follow from how the model works. It predicts likely text and has no built-in step that checks a statement against reality. Some chatbots can search the web and show sources, which helps with recent events, but they can still misread or misquote what they find.

How to cross-check an answer

Cross-checking a chatbot's answer

  1. Pick out the checkable claims Names, numbers, dates, article or section numbers, quotations
  2. Government matters: official website Exams, schemes and rules - read the official website or notification
  3. Subject facts: NCERT and textbooks
  4. Confirm in two independent sources Two chatbots agreeing does not count
  5. Open any cited source yourself Confirm it exists and says what was claimed
  6. Recompute any arithmetic
  1. Pick out the checkable claims: names, numbers, dates, article or section numbers, quotations.
  2. For government matters - exams, schemes, rules - go to the official website or notification.
  3. For subject facts, use NCERT and standard textbooks.
  4. Confirm important facts in at least two independent, reliable sources. Two chatbots agreeing does not count: they may share the same error.
  5. If the chatbot gives a source, open it yourself and confirm it exists and says what was claimed.
  6. Recompute any arithmetic yourself.

Fluent is not the same as correct. Check facts in the official source or textbook, confirm in two reliable sources, and never share passwords, OTPs or ID numbers with a chatbot.

Worked example 1: catching an arithmetic slip

A chatbot states: "15% of 240 is 32." Check it yourself: 10% of 240 = 24, and 5% of 240 = 12, so 15% = 24 + 12 = 36. The chatbot is wrong by 4, even though it sounded certain. Lesson: recompute numbers before you use them, and never let a confident tone replace your own working.

Worked example 2: a reference that does not exist

A student asks for three books on the Indian freedom struggle. The chatbot lists titles with authors and years. A library or publisher search finds one real book; the other two do not exist. This is hallucination: the model produced titles that look like real references because they fit the pattern of book lists. The fix: check every reference before quoting it, and prefer reading lists from your syllabus, teachers or textbooks.

Never paste sensitive data into a chatbot

What you type into a chatbot may be stored, reviewed or used to improve the service, and accounts can be hacked. Treat a chat like a message to a stranger. Keep these out:

  • Passwords and PINs of any account
  • OTPs (one-time passwords) - no genuine service asks you to type an OTP into a chatbot
  • Aadhaar, PAN or passport numbers
  • Bank account, debit or credit card numbers and CVV
  • Photos of identity documents, admit cards or signatures

If a chatbot, website or message ever asks for an OTP or password, treat it as a likely scam and stop. When you need help with a document, remove or replace the personal details first (for example write "XXXX" in place of numbers).

Safe to typeNot safe to type
Your own notes on the Mughal empire, asking for a summaryYour email password
A maths question from a practice bookAn OTP received on your phone
A paragraph to translate, with names removedYour Aadhaar or bank account number

Key takeaways: chatbots hallucinate, have a training cut-off, sound confident when wrong and can reflect bias. Cross-check claims in official sources and textbooks, use two independent sources, open any cited source and redo arithmetic. Never paste passwords, OTPs, Aadhaar, bank details or ID documents into a chatbot.

Check your understanding

  1. 1. In the context of AI chatbots, "hallucination" means:

    1. The chatbot refusing to answer a question
    2. The chatbot replying very slowly
    3. The chatbot producing false information that sounds plausible, such as a made-up reference
    4. The chatbot translating text into the wrong language
    Show answer

    Answer: C. Hallucination is the model generating plausible but false content - invented facts, quotations or references - because it predicts likely text rather than checking truth.

  2. 2. A chatbot knows nothing about a government scheme announced last month. The most likely reason is:

    1. It is deliberately hiding the information
    2. The scheme must be fake
    3. Chatbots cannot process Indian names
    4. Its knowledge comes from training data collected up to a cut-off date
    Show answer

    Answer: D. A model learns only from data collected up to its training cut-off; later events are unknown to it unless it can search the web. That says nothing about whether the scheme is real.

  3. 3. A chatbot says "15% of 240 is 32". What is the correct value?

    1. 36, so the chatbot is wrong
    2. 32, so the chatbot is right
    3. 38, so the chatbot is wrong
    4. 30, so the chatbot is wrong
    Show answer

    Answer: A. 10% of 240 = 24 and 5% of 240 = 12, so 15% = 24 + 12 = 36. The chatbot's confident answer is wrong, which is why numbers must be recomputed.

  4. 4. Which of these is safe to type into an AI chatbot?

    1. The OTP your bank just sent you
    2. Your own notes on the Mughal empire, asking for a summary
    3. Your Aadhaar number
    4. Your email password
    Show answer

    Answer: B. Your own study notes contain no secret or personal identifier. OTPs, Aadhaar numbers and passwords must never be shared with a chatbot, as chats may be stored or exposed.

  5. 5. A chatbot supports its answer with a link to a source. What is the best thing to do?

    1. Accept the answer, since a source was given
    2. Assume every link a chatbot gives is fake
    3. Copy the link into your notes without opening it
    4. Open the source, confirm it exists and says what was claimed, and check a second reliable source
    Show answer

    Answer: D. Chatbots can cite sources that do not exist or do not support the claim. Opening the source and confirming in a second reliable source is the only sound check.

Privacy · Terms · Refunds · Delivery