MESA8220Psychometrics I

Identification of problematic items:1.Item Difficulty:Determined by calculating the p-value. It’s the percentage of people who answeredthe items correctlyThe Higher the p-value the easier the item and the lower the p-value, the moredifficult the item.The Criteria that is used for determining difficulty:if the p-value is less than 0.3, the item isconsidered to be difficult and if the p-value is above 0.9, the item is too easy. For example, a pvalue of 0.19 means that the item is not that easy and it is adifficult item. Therefore, a p-valuebetween 0.3 and 0.9 is considered desirable.2.Item Discrimination:·Determined by calculating the correlation between the item score and the total score;how people do on the item vs. how they do on the overall test.·Positive correlation is necessary because a negative correlation coefficient meansthat the item is unrelated to the overall test (the higher you perform on a specific item, thelower you are performing on the overall test or people who tend to get the item right tendto perform bad on the overall test and vice versa).·Higher thediscvalue, higher the correlation and lower thediscvalue, lower thecorrelationCriteria used for determining discrimination:A disc value between 0 to 0.2 signals concerningdiscrimination (low correlation), a disc value between 0.2 to 0.4 signals acceptable discrimination(relatively better correlation) and a disc value greater than 0.4 signals desirable discrimination(correlation is high)2.Item Bias:·Determined by calculating theDifferential Item Functioning (DIF) valuewhichcalculates that given the total test score, is there a difference in p-value for a given item?·A bias is indicated if the item functions differently across various groups. The focalgroup is the group of concern (often females, non-white, poor, ELL students, etc.) and thereference group is the group against which the comparison is being made (often thedominant/most socially advantaged group)Criteria used for determining bias:a DIF value greater than±0.1 (above 0.1 and below -0.1)signals probable bias in the item, a DIF value between 0.05 to 0.1 signals some level of possiblebias and a DIF value between -0.05 to 0.05 signals negligible bias