Date of Award
1-8-2021
Degree Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Educational Policy Studies
First Advisor
Hongli Li
Second Advisor
T. Chris Oshima
Third Advisor
Sarah Carlson
Fourth Advisor
Randy W. Kamphaus
Abstract
Achievement motivation is a well-documented predictor of a variety of positive student outcomes. However, researchers have also found threats to fairness and measurement scale comparability in motivation items, including group differences in response scale use and response styles. As such, the measurement comparability of achievement motivation items was evaluated before and after using anchoring vignettes to account for the effect of group-specific response scale use as a source of differential item functioning (DIF) across gender and ethnicity. Within a combined item response theory/ordinal logistic regression DIF framework, gender DIF was assessed using pairwise comparisons and ethnicity DIF was tested using both multiple-group DIF with a common base group as the reference group and all possible pairwise comparisons.
Overall, using the vignettes changed both the form of DIF within items and the pattern of DIF between groups across items. Results indicated the presence of DIF between genders, but the DIF was unrelated to group differences in response scale use. Across ethnic groups, Black/African American students and Asian students demonstrated group-specific response scale use. When groups showed response tendencies, accounting for such scale use with the vignettes had a greater effect on reducing DIF in base group comparisons than in pairwise comparisons. Despite that DIF was identified in multiple items, the magnitude of all DIF was negligible and had little practical implication. Therefore, achievement motivation items appeared to demonstrate measurement comparability. As sources of DIF often go unidentified, a contribution of this study was the novel use of anchoring vignettes to account for group differences in response scale use as the source of DIF and to clarify the effect of those differences on measurement scale comparability and DIF.
DOI
https://doi.org/10.57709/20506213
Recommended Citation
Bialo, Jacquelyn, "Using Differential Item Functioning and Anchoring Vignettes to Examine the Fairness of Achievement Motivation Items." Dissertation, Georgia State University, 2021.
doi: https://doi.org/10.57709/20506213
File Upload Confirmation
1