Measurement error in e-asTTle
A score of any kind is always an estimate of true performance. Usually, if you were to re-test the same student again, their score would be at least slightly different.
Even raw scores, such as 8/10 are often treated as if they are 100% precise, however even these are subject to error.
How can we know how precise a score is?
We can estimate how widely spread out a student’s scores would be, if they were re-tested again and again. This calculation is called the standard error of measurement, or the measurement error.
A common analogy for the SEM is shooting arrows at a target. The SEM is how closely the arrows cluster together. According to the Rasch Model used in e-asTTle, on a given test, it is the very high and very low scores that have the highest SEM, and the scores in the middle with the lower SEM. The length of the test is also a factor – longer tests tend to have smaller SEM.
How precise are e-asTTle scores?
On average, the measurement error of measurement is +/-22 points for e-asTTle Maths and Reading, and +/-40 points in e-asTTle Writing.
One way of interpreting this is:
- for reading and maths, scores that differ by less than 44 points (22 x 2) tend not to be meaningfully different – they could essentially be the same score, given chance factors involved in assessment.
- for writing, scores that differ by less than 80 points tend not to be meaningfully different – they could essentially be the same score, given chance factors involved in assessment.
However, of course, this is just an average. The measurement error of measurement can be calculated for each instance of a student sitting a test.
For reading, maths, and writing, you can find the SEM for specific students by looking at their Individual Learning Pathway.
The SEM displays in two ways:
- As a +/- value next to the student’s score
- The circle on the score arrow represents this graphically. There is a good probability that the student’s true score falls within the red circle.
Measurement error differences across subjects
The measurement error is calculated slightly differently depending on the subject.
Maths and reading
The measurement error is calculated based on the specific student’s score and test.
– Longer tests will generally have smaller measurement errors than shorter tests.
– Students achieving in the middle of the test (i.e. ~ 50% correct) will have smaller measurement errors than students who score high or low.
Writing
Similarly to reading and maths, there is an estimate of the error associated with inconsistencies in how students respond to the writing test. The measurement error will be higher for students with very low or high rubric scores.
An additional error component is added to this estimate to represent the variability in how different markers apply the marking rubrics.
Māori-Medium subjects (Pāngarau, Pānui, Tuhituhi)
An average measurement error of measurement (22 points) is used.
- Related information