- Foundation
- About 14 min
- 5 practice questions
Writing good items: multiple-choice parts, common faults, guessing and penalty
Name the parts of a multiple-choice item, spot common faults in items, and work out what blind guessing earns with and without a penalty for wrong answers.
If something looks wrong, please tell us.
Even a well-planned test fails if its items are poorly written. A multiple-choice item can give away its answer, can confuse the learner, or can reward guessing. This lesson names the parts of an item, lists the common faults, and then looks at the arithmetic of guessing, which explains why some tests use a penalty for wrong answers.
Parts of a multiple-choice item
| Part | Meaning | Example |
|---|---|---|
| Stem | The question or the incomplete statement | The process by which plants make food is called: |
| Key | The one correct option | photosynthesis |
| Distractors | The wrong options, written to look possible | respiration, digestion, evaporation |
| Item | The whole question: stem, key and distractors together | — |
Common faults
- An unclear stem. The learner cannot tell what is being asked.
- Clues to the key. The key is longer or more detailed than the others, or only one option fits the grammar of the stem. For example, a stem that ends with 'an' and only one option that begins with a vowel.
- Weak distractors. Options that no learner would choose, so the item is easier than it looks.
- 'All of the above' or 'none of the above'. A learner who knows that two options are right can pick 'all of the above' without knowing the third.
- More than one right answer, or none. The item is faulty and the key is disputed.
- A pattern in the position of the key. Keys that are always B, or that follow a pattern, can be guessed.
Good distractors are not silly. They are based on mistakes that learners really make. For a question on 12 ÷ 0.5, a good distractor is 6, because many learners divide by 2 instead of multiplying by 2. Distractors built from real errors also help the teacher: if many learners choose the same distractor, that shows the error to correct next, which is the idea of diagnosis.
Short-answer and true-or-false items
A short-answer item should ask for one specific answer, such as a word or a number. A true-or-false statement should be clearly true or clearly false, and should avoid words such as 'always' and 'never', which often signal that a statement is false and so help the guesser.
The arithmetic of guessing
Suppose an item has 4 options and a learner guesses blindly. The chance of being right is 1 in 4. In a test of 24 such items, the expected number right by guessing is 24 × ¼ = 6, so a learner who knows nothing can still earn 6 marks. A penalty for wrong answers is meant to remove this advantage. If a wrong answer loses ⅓ mark, the expected score from guessing is: 6 right answers × 1 mark = 6 marks, minus 18 wrong answers × ⅓ = 6 marks, which gives 0. For four options a penalty of ⅓ exactly cancels the gain from guessing. (Whether a test uses a penalty is a decision of those who set it; the figures here are illustrative.)
Worked example 1
Situation: 'A word that describes a noun is called an: (a) verb (b) adjective (c) noun (d) conjunction.' Step 1: the stem ends with 'an'. Step 2: only option (b) begins with a vowel, so it alone fits the grammar. Answer: the item has a grammatical clue to the key. It can be fixed by ending the stem with 'a/an' or by rewriting the question as 'The word class that describes a noun is called the:'.
Worked example 2
Situation: a test has 40 items of 4 options, 1 mark each, with no penalty. A student guesses every item. Step 1: expected right answers = 40 × ¼ = 10. Step 2: expected score = 10 marks. If a penalty of ⅓ mark applies per wrong answer: wrong answers = 30, loss = 30 × ⅓ = 10, so the expected score = 10 − 10 = 0. Answer: 10 without the penalty and 0 with it.
Stem, key, distractors. Remove clues, write distractors from real errors, avoid 'all of the above'. For four options, guessing earns ¼ of the items; a penalty of ⅓ per wrong answer makes the expected gain zero.
Key takeaways
- An item has a stem, a key and distractors.
- Clues, weak distractors and 'all of the above' are common faults.
- Good distractors come from common errors and help diagnosis.
- Blind guessing on four-option items earns about a quarter of the marks unless a penalty is used.
Check your understanding
-
1. In a multiple-choice item the correct option is called the:
- stem
- distractor
- key
- item
Show answer
Answer: C. The key is the one correct option. The stem is the question, the distractors are the wrong options and the item is the whole question.
-
2. Which is a fault in a multiple-choice item?
- the key is longer and more detailed than the other options
- all options are of about the same length
- distractors are based on mistakes learners make
- the stem is a clear and complete question
Show answer
Answer: A. A key that stands out in length is a clue that helps the guesser. The other three are features of a good item.
-
3. A student guesses all 24 four-option items, 1 mark each, with no penalty. The expected score is:
- 4
- 8
- 12
- 6
Show answer
Answer: D. The chance of being right is ¼, so 24 × ¼ = 6 marks.
-
4. With a penalty of ⅓ mark per wrong answer, the expected score from guessing all 24 four-option items is:
- 6
- 0
- 2
- −6
Show answer
Answer: B. Expected right answers are 6, which earn 6 marks. Expected wrong answers are 18, which lose 18 × ⅓ = 6 marks. So 6 − 6 = 0.
-
5. Which is the best way to write distractors?
- Based on mistakes that learners commonly make
- As obviously wrong as possible
- As funny sentences to relax learners
- By copying the key and adding 'not'
Show answer
Answer: A. Distractors from real errors look possible and also show the teacher which errors are common. Obviously wrong options make the item easy.
Sign in to keep track of the lessons you finish.