- Practice
- About 15 min
- 5 practice questions
Limits of AI chatbots: hallucination, training cut-off, bias and keeping data safe
Recognise hallucination, outdated knowledge, confident wrong answers and bias; cross-check answers with official sites, NCERT and two sources; and keep passwords, OTPs and Aadhaar out of chatbots.
If something looks wrong, please tell us.
Every AI chatbot makes mistakes, and it rarely signals when it does. Knowing the typical kinds of error lets you spot them quickly and decide what to check. Exam questions on AI often ask you to name these limits or choose the safe action in a scenario.
Four limits to know
| Limit | What it means | Example |
|---|---|---|
| Hallucination | The model produces false information that sounds plausible | A made-up book title, court case or quotation |
| Training cut-off | Its knowledge stops when its training data was collected | It does not know about a recent scheme or result |
| Confident wrong answers | The tone sounds equally sure whether right or wrong | An arithmetic slip stated firmly |
| Bias | It repeats unfair patterns present in its training data | Showing only men when asked for a picture of an engineer |
These limits follow from how the model works. It predicts likely text and has no built-in step that checks a statement against reality. Some chatbots can search the web and show sources, which helps with recent events, but they can still misread or misquote what they find.
How to cross-check an answer
Cross-checking a chatbot's answer
- Pick out the checkable claims Names, numbers, dates, article or section numbers, quotations
- Government matters: official website Exams, schemes and rules - read the official website or notification
- Subject facts: NCERT and textbooks
- Confirm in two independent sources Two chatbots agreeing does not count
- Open any cited source yourself Confirm it exists and says what was claimed
- Recompute any arithmetic
- Pick out the checkable claims: names, numbers, dates, article or section numbers, quotations.
- For government matters - exams, schemes, rules - go to the official website or notification.
- For subject facts, use NCERT and standard textbooks.
- Confirm important facts in at least two independent, reliable sources. Two chatbots agreeing does not count: they may share the same error.
- If the chatbot gives a source, open it yourself and confirm it exists and says what was claimed.
- Recompute any arithmetic yourself.
Fluent is not the same as correct. Check facts in the official source or textbook, confirm in two reliable sources, and never share passwords, OTPs or ID numbers with a chatbot.
Worked example 1: catching an arithmetic slip
A chatbot states: "15% of 240 is 32." Check it yourself: 10% of 240 = 24, and 5% of 240 = 12, so 15% = 24 + 12 = 36. The chatbot is wrong by 4, even though it sounded certain. Lesson: recompute numbers before you use them, and never let a confident tone replace your own working.
Worked example 2: a reference that does not exist
A student asks for three books on the Indian freedom struggle. The chatbot lists titles with authors and years. A library or publisher search finds one real book; the other two do not exist. This is hallucination: the model produced titles that look like real references because they fit the pattern of book lists. The fix: check every reference before quoting it, and prefer reading lists from your syllabus, teachers or textbooks.
Never paste sensitive data into a chatbot
What you type into a chatbot may be stored, reviewed or used to improve the service, and accounts can be hacked. Treat a chat like a message to a stranger. Keep these out:
- Passwords and PINs of any account
- OTPs (one-time passwords) - no genuine service asks you to type an OTP into a chatbot
- Aadhaar, PAN or passport numbers
- Bank account, debit or credit card numbers and CVV
- Photos of identity documents, admit cards or signatures
If a chatbot, website or message ever asks for an OTP or password, treat it as a likely scam and stop. When you need help with a document, remove or replace the personal details first (for example write "XXXX" in place of numbers).
| Safe to type | Not safe to type |
|---|---|
| Your own notes on the Mughal empire, asking for a summary | Your email password |
| A maths question from a practice book | An OTP received on your phone |
| A paragraph to translate, with names removed | Your Aadhaar or bank account number |
Key takeaways: chatbots hallucinate, have a training cut-off, sound confident when wrong and can reflect bias. Cross-check claims in official sources and textbooks, use two independent sources, open any cited source and redo arithmetic. Never paste passwords, OTPs, Aadhaar, bank details or ID documents into a chatbot.
Check your understanding
-
1. In the context of AI chatbots, "hallucination" means:
- The chatbot refusing to answer a question
- The chatbot replying very slowly
- The chatbot producing false information that sounds plausible, such as a made-up reference
- The chatbot translating text into the wrong language
Show answer
Answer: C. Hallucination is the model generating plausible but false content - invented facts, quotations or references - because it predicts likely text rather than checking truth.
-
2. A chatbot knows nothing about a government scheme announced last month. The most likely reason is:
- It is deliberately hiding the information
- The scheme must be fake
- Chatbots cannot process Indian names
- Its knowledge comes from training data collected up to a cut-off date
Show answer
Answer: D. A model learns only from data collected up to its training cut-off; later events are unknown to it unless it can search the web. That says nothing about whether the scheme is real.
-
3. A chatbot says "15% of 240 is 32". What is the correct value?
- 36, so the chatbot is wrong
- 32, so the chatbot is right
- 38, so the chatbot is wrong
- 30, so the chatbot is wrong
Show answer
Answer: A. 10% of 240 = 24 and 5% of 240 = 12, so 15% = 24 + 12 = 36. The chatbot's confident answer is wrong, which is why numbers must be recomputed.
-
4. Which of these is safe to type into an AI chatbot?
- The OTP your bank just sent you
- Your own notes on the Mughal empire, asking for a summary
- Your Aadhaar number
- Your email password
Show answer
Answer: B. Your own study notes contain no secret or personal identifier. OTPs, Aadhaar numbers and passwords must never be shared with a chatbot, as chats may be stored or exposed.
-
5. A chatbot supports its answer with a link to a source. What is the best thing to do?
- Accept the answer, since a source was given
- Assume every link a chatbot gives is fake
- Copy the link into your notes without opening it
- Open the source, confirm it exists and says what was claimed, and check a second reliable source
Show answer
Answer: D. Chatbots can cite sources that do not exist or do not support the claim. Opening the source and confirming in a second reliable source is the only sound check.
Sign in to keep track of the lessons you finish.