Guidelines for creating effective Custom and Adaptive Tests
e-asTTle testing is about getting useful information, not about correctly answering all questions. To get the most useful information from e-asTTle, set a test where you expect students to answer about 50% of the test correctly – 100% correct is not a desirable outcome.
Understanding the principles behind e-asTTle scoring will help you understand why the 50% benchmark is optimum.
Imagine the questions in the test ordered from easiest to hardest. e-asTTle is trying to find the point where the student is transitioning from answering questions mostly correctly to mostly incorrectly – the “tipping point” where those taking the test have a 50/50 chance of answering correctly. The difficulty level associated with the “tipping point” question becomes the student’s score.
To find this “tipping point”, the test should include some questions that are easy and some that are challenging for the student. Choosing the appropriate curriculum level is crucial.
Simply put, a student at 3A is not expected to get 100% of 3A questions correct. Instead, a 3A student is considered to have a 50% chance of answering a Level 3A question correctly.
The Rasch Model in e-asTTle has in-depth information about the underlying test theory used in e-asTTle.
Test Creation Advice
Use these four steps to guide your test creation.
1. Target your test to the students’ current ability
A common misconception is that no matter where you set the curriculum levels, your students can be accurately assessed across curriculum levels 2-6.
However, when students are at a much higher or lower curriculum level than the questions in the test, e-asTTle scores are much less precise. The results will lack formative information from the Individual Learning Pathways.
Without questions at the student’s level in the test, e-asTTle must extrapolate to work out the “tipping point”.
- When students answer everything correctly, that tells you the student is operating above the level of the test – not how far above.
- When students answer few questions correctly, that tells you the student is operating below the test – not how far below.
If you have new students check the student result summary
Student result summary
If you have new students, this is where you can check any e-asTTle results they have from previous schools.
2. Consider creating multiple tests for a class or cohort
e-asTTle is not a one-size-fits-all approach – it is often advisable to create multiple tests and assign them to students of different abilities. This will give you higher-quality formative information across all students.
For example, if your students are achieving from Levels 2-4, create tests for levels 2/3, 3/4 and 4/5.
Planning templates for download
3. Test across two levels
Creating each test across two levels is a good starting point.
Tests that cover only one level often lack scope, particularly for adaptive tests.
Tests covering three levels can also be useful. If the test you make covers three levels, choose “Most” for at least one of the sliders. This prevents the test from becoming too broad.
4. Review and edit the test
Always review and edit the test to ensure it meets your expectations.
You are able to edit the levels selected and/or replace some questions if needed.
Replace questions
Use this option if you are mostly happy with the test you have created but there are one or two questions that you would like replaced.
Creating, reviewing, and accepting a Custom or Adaptive test
Step-by-step information for creating, reviewing and accepting the test.
Choosing Curriculum Levels for an Adaptive Test
This process is similar to the Custom Tests process; however, for an Adaptive Test, you simply set Yes/No for a level.
“Yes” indicates the test may adapt to that level. Select at least two curriculum levels.
A common misconception about e-asTTle Adaptive tests is that no matter where you set the curriculum levels, the test can adapt from Levels 2-6. This is not the case.
When you create Adaptive tests, you are in charge of the boundaries of the test. The levels you select set the lower and upper limits of the test. Within those levels, e-asTTle will attempt to present the student with questions most appropriate to them.
For example, if you create a Level 3 and 4 test, the lowest level question will be at 3B and the highest at 4A.
As with Custom tests, consider creating multiple Adaptive tests and targeting them to the appropriate students.
- Related information