- Practice
- About 16 min
- 5 practice questions
Types of reliability and validity in scenarios
Identify test-retest, parallel-forms, split-half and inter-rater reliability, and content, construct, criterion-related and face validity, in exam-style scenarios.
If something looks wrong, please tell us.
You already know that reliability means consistency and validity means measuring the right thing. Practice-level questions go further: they describe how a test was checked and ask which type of reliability or validity is involved. Each type matches a simple procedure, so learning the procedures is the fastest route.
Types of reliability
| Type | Procedure | What it shows |
|---|---|---|
| Test-retest | Same test given twice to the same group, after a gap; scores compared | Stability over time |
| Parallel (equivalent) forms | Two comparable versions given to the same group; scores compared | Equivalence of forms |
| Split-half | One test given once, divided into two halves (for example odd and even items); halves compared | Internal consistency |
| Inter-rater (inter-scorer) | Two or more markers score the same answers; their scores compared | Agreement between markers |
Reliability is usually reported as a coefficient between 0 and 1; the closer to 1, the more consistent. Split-half compares two half-length tests, and because shorter tests are less reliable, the value is usually corrected upward using the Spearman-Brown formula. Keywords help: 'same test twice' means test-retest, 'two versions' means parallel forms, 'one sitting, two halves' means split-half, 'two examiners' means inter-rater.
Types of validity
| Type | Question | Typical evidence |
|---|---|---|
| Content validity | Do the items fairly cover the syllabus and objectives? | Comparing items with a blueprint |
| Construct validity | Does the test measure the intended trait, such as creativity or reasoning? | Evidence that scores behave as the trait should |
| Criterion-related: concurrent | Do scores agree with another measure taken now? | Comparing with teachers' current ratings |
| Criterion-related: predictive | Do scores forecast later performance? | Comparing entrance scores with later results |
| Face validity | Does the test look suitable to those taking it? | Opinions of test-takers; weakest evidence |
Content validity is the main concern for classroom achievement tests, because a test should sample the whole syllabus in proportion to its importance. Face validity is only appearance: a test can look right and still measure the wrong thing.
Factors that affect each
Reliability falls with very short tests, ambiguous items, guessing, unclear instructions and subjective marking. Validity falls when items cover only part of the syllabus, when language is harder than the skill being tested, or when items test memory while the objective was application. Because consistent scores are needed before a test can measure anything accurately, a test with very low reliability cannot have high validity.
Worked example 1
A teacher prepares two versions of a vocabulary test with different but matched words, gives both to the same class in one week, and finds the scores closely agree. Step 1: procedure: two versions, same group. Step 2: keyword: 'two versions'. Answer: parallel-forms reliability. It would be test-retest only if the same version were repeated.
Worked example 2
A science test for Class 8 covers eight chapters, but 18 of its 20 items come from two chapters. Step 1: does it sample the syllabus fairly? No. Step 2: the problem is coverage. Answer: content validity is poor. The proportion from those two chapters is 18/20 = 90%, while they are 2 of 8 chapters, or 25% of the syllabus by count. A blueprint would fix this.
Worked example 3
A school wants to know whether its new essay marking scheme helps teachers agree. Three teachers mark the same 20 essays with the scheme, and their scores are compared. Step 1: what varies? The marker, not the test or the occasion. Step 2: keyword: several markers, same answers. Answer: inter-rater reliability. If the scores agree closely, the scheme has improved objectivity, and with it reliability. Notice that this says nothing yet about validity: the teachers could agree perfectly while all rewarding neat handwriting instead of the quality of ideas.
A quick way to remember the difference between the two families: reliability questions ask whether you would get the same result again (same test later, another version, another half, another marker), while validity questions ask whether the result means what it claims to mean (right content, right trait, right link to other measures).
Same test twice: test-retest. Two versions: parallel forms. One sitting split in two: split-half. Two markers: inter-rater. Coverage of syllabus: content validity. Forecasting later results: predictive validity.
Key takeaways
- Name reliability types from the procedure used.
- Content validity is the key concern for classroom achievement tests.
- Predictive validity forecasts later performance; concurrent validity agrees with a present measure.
- Face validity is appearance only and is the weakest evidence.
- Very low reliability rules out high validity.
Check your understanding
-
1. A test is divided into odd-numbered and even-numbered items after one sitting, and scores on the two halves are compared. This estimates:
- Test-retest reliability
- Split-half reliability
- Predictive validity
- Inter-rater reliability
Show answer
Answer: B. One sitting, two halves compared is split-half reliability.
-
2. Two examiners independently mark the same set of essays and their marks are compared. This checks:
- Content validity
- Parallel-forms reliability
- Inter-rater reliability
- Face validity
Show answer
Answer: C. Agreement between markers is inter-rater (inter-scorer) reliability.
-
3. Entrance test scores are compared with the same students' results at the end of the course. This examines:
- Concurrent validity
- Split-half reliability
- Predictive validity
- Face validity
Show answer
Answer: C. Checking whether scores forecast later performance is predictive validity.
-
4. A history test has most items from one chapter of a ten-chapter syllabus. Which quality is most weakened?
- Content validity
- Inter-rater reliability
- Usability
- Face validity only
Show answer
Answer: A. The items do not sample the syllabus fairly, so content validity is poor.
-
5. Which statement is correct?
- Face validity is the strongest kind of validity
- Reliability coefficients range from 1 to 10
- Test-retest reliability needs two different versions of the test
- A test with very low reliability cannot have high validity
Show answer
Answer: D. Inconsistent scores cannot measure anything accurately. Face validity is weakest, coefficients run from 0 to 1, and test-retest repeats the same test.
Sign in to keep track of the lessons you finish.