Quantifying Test Item Difficulty through ALEKS-generated Reports: A Teacher’s Item Bank
Jen Dela TorreWas the test too easy or too difficult for certain groups of students? Which of the items or topics or learning outcomes were difficult? While we have ensured appropriate distribution of item difficulty through content analysis, item difficulty may also depend on the level of ability of certain set of learners.
We often use the ALEKS assignment reports to generate the students’ scores and provide feedback based on their individual responses. With this, we were able to discuss with our students the items that they answered incorrectly, thus, determine their common mistakes and weaknesses. But I also use this ALEKS feature to explore on the difficulty of each of the items and the overall test. Item difficulty is simply the percentage of students taking the test who answered the item correctly. The larger the percentage getting an item right, the easier the item. Item difficulty indices is readily available from the “per question results” of the ALEKS assignment.
Analysis of the difficulty of each of the items can help teachers identify specific areas of course content which need greater emphasis or clarity. For example, the difficulty indices of the items in the mid-cycle exam give us information on which of the topics from the first three learning outcomes should be clarified or reinforced to ensure students’ mastery which is especially useful in preparation for their final exam. Similarly, previous cycle analyses results will guide us in planning for strategies or activities that may be employed for specific difficult topics that need to be prioritized from among the range of topics or learning outcomes that should be covered within the cycle.
Likewise, if we would like to check on the discrimination power of the items, we may also explore the “detailed student results”. Other psychometric item qualities, e.g. reliability or validity may also be delved into from these ALEKS-generated reports. From here we will be able to build our own item bank with corresponding item descriptions. It is hoped that all these information will help us plan and design learning activities that will build a stronger conceptual and procedural mathematical skills of students.
