The Influence of the Method Effect Associated with Item Wording on the Internal Consistency Reliability of Balanced Scale
-
摘要: 通过蒙特卡洛模拟研究考察平衡量表中源自条目不同计分方向的表述效应及量表内因素对测量结果内部一致性信度系数α系数和组合信度(CR)系数的影响.方差分析结果验证了研究假设:表述效应使平衡量表的α系数与CR系数被高估,高估程度随方法因子负荷升高(F(2,392)=345.10,344.87,p<0.001),不同计分条目比例失衡(F(2,392)=8.03,8.32,p<0.001),方法因子间相关(F(4,392)=7.14,7.02,p<0.001) 和特质因子负荷的降低(F(2,392)=411.96,410.49,p<0.001) 而加剧.表明编制量表时,须提升条目质量并尽量使不同计分条目数相等.Abstract: Monte Carlo simulation was made to investigate the laws and characteristics of how wording effect (or the Method Effect Associated with Item Wording) and its relevant endogenous factors affected the internal consistency reliability coefficient, Cronbach's α and composite reliability coefficient of the outcomes of measurement. Then multifactor variance analysis was made, in which five independent variables were manipulated: method factor loading of item, trait factor loading of item, ratio of different scoring items, correlation between different method factors and sample size. The results showed that both Cronbach's α and composite reliability coefficient in the balance scale would be overestimated due to the item wording effect and that the extent of overestimation increased with the increase in the item's method factor loading (F(2, 392)=345.10 and 344.87, p < 0.001) and the decrease in imbalance of different scoring items (F(2, 392)=8.03 and 8.32, p < 0.001), correlation between different method factors (F(4, 392)=7.14 and 7.02, p < 0.001), or item's trait factor loading (F(2, 392)=411.96 and 410.49, p < 0.001). The results suggested that the completely balanced scale was better under the condition of improving the psychometric properties of items.
-
表 1 10条目单维平衡量表不同计分条目比例表述效应关系与平均相关系数的高估量
条目比a 相关系数比b 平均相关系数改变/nc φMxMy=0.2 φMxMy=0.4 φMxMy=0.6 φMxMy=0.8 φMxMy=1 5:5 40:50 50/90 60/90 70/90 80/90 1 6:4(4:6) 42:48 51.6/90 61.2/90 70.8/90 80.4/90 1 7:3(3:7) 48:42 56.4/90 64.8/90 73.2/90 81.6/90 1 8:2(2:8) 58:32 64.4/90 70.8/90 77.2/90 83.6/90 1 9:1(1:9) 81:9 82.8/90 84.6/90 86.4/90 88.2/90 1 注:a:直接计分条目数:反向计分条目数;b:计分方向相同的条目相关系数个数:计分方向相反的条目相关系数个数. c:n为常数=λMx2=λMy2=λMxλMy. 表 2 有表述效应与无表述效应α系数差异的组间效应方差分析表摘要表
变异来源 平方和(type Ⅲ) 自由度 均方 F partial η2 方法因子负荷 0.185 2 0.092 345.095*** 0.638 特质因子负荷 0.221 2 0.110 411.963*** 0.678 不同方法条目比例 0.004 2 0.002 8.029*** 0.039 方法因子相关 0.008 4 0.002 7.140*** 0.068 样本量 0 a 2 0 b 0 0 误差 0.105 392 总变异 0.522 404 注:a:实际数值为5.842e-008;b:实际数值为2.921e-008;***:p<0.001. 表 3 有表述效应与无表述效应CR系数差异的组间效应方差分析摘要表
变异来源 平方和(type Ⅲ) 自由度 均方 F partial η2 方法因子负荷 0.184 2 0.092 344.865*** 0.638 特质因子负荷 0.219 2 0.109 410.493*** 0.677 不同方法条目比例 0.004 2 0.002 8.320*** 0.041 方法因子相关 0.007 4 0.002 7.016*** 0.067 样本量 0 a 2 0 b 0.004 0 误差 0.104 392 总变异 0.518 404 注:a:实际数值为1.914e-006;b:实际数值为9.568e-007;***:p<0.001. -
[1] 邱皓政.量化研究法(三):测验原理与量表发展技术[M].台北:双叶书廊, 2012. [2] 关守义.克龙巴赫α系数研究述评[J].心理科学, 2009, 32(3): 685-687. doi: http://www.cnki.com.cn/Article/CJFDTOTAL-XLKX200903049.htm [3] 肖坤月.信度计算公式初探[J].西南师范大学学报(自然科学版), 1989, 14(4): 161-166. doi: http://www.cnki.com.cn/Article/CJFDTOTAL-XNZK198904018.htm [4] CRONBACH L J. Coefficient Alpha and the Internal Structure of Tests [J]. Psychometrika, 1951, 16(3): 297-334. doi: 10.1007/BF02310555 [5] VAUTIER S, POHL S. Do Balanced Scales Assess Bipolar Constructs? The Case of the STAI Scales [J]. Psychological Assessment, 2009, 21(2): 187-193. doi: 10.1037/a0015312 [6] 温忠麟, 叶宝娟.测验信度估计:从α系数到内部一致性信度[J].心理学报, 2011, 43(7): 821-829. doi: http://www.cnki.com.cn/Article/CJFDTOTAL-XLXB201107013.htm [7] FORNELL C, LARCKER D F. Evaluating Structural Equation Models with Unobservable Variables and Measurement Error [J]. Journal of Marketing Research, 1981, 18(1): 39-50. doi: 10.2307/3151312 [8] 韦嘉. 基于单维平衡量表的表述效应研究——以人格测验为例[D]. 重庆: 西南大学, 2016. [9] LANCE C E, DAWSON B, BIRKELBACH D, et al. Method Effects, Measurement Error, and Substantive Conclusions [J]. Organizational Research Methods, 2010, 13(3): 435-455. doi: 10.1177/1094428109352528 [10] CAMPBELL D T, FISKE D W. Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix [J]. Psychological Bulletin, 1959, 56(2): 81-105. doi: 10.1037/h0046016 [11] 顾红磊, 温忠麟.项目表述效应对自陈量表信效度的影响——以核心自我评价量表为例[J].心理科学, 2014, 37(5): 1245-1252. doi: http://www.cnki.com.cn/Article/CJFDTOTAL-XLKX201405036.htm [12] REISE S P, MORIZOT J, HAYS R D. The Role of the Bifactor Model in Resolving Dimensionality Issues in Health Outcomes Measures [J]. Quality of Life Research, 2007, 16(1): 19-31. [13] WANG, CHEN, JIN. Item Response Theory Models for Wording Effects in Mixed-Format Scales [J]. Educational and Psychological Measurement, 2015, 75(1): 157-178. doi: 10.1177/0013164414528209 [14] 詹沛达, 王文中, 王立君, 等.多维题组效应Rasch模型[J].心理学报, 2014, 46(8): 1208-1222. doi: http://cpfd.cnki.com.cn/Article/CPFDTOTAL-ZGXG201311001220.htm [15] 温忠麟, 刘红云, 侯杰泰.调节效应和中介效应分析[M].北京:教育科学出版社, 2012. [16] 韦嘉, 张春雨, 赵永萍, 等. 随机截距因子分析模型在控制条目表述效应中的应用[J]. 心理科学, 39(4): 1005-1010. [17] 胡竹菁.心理统计学[M].北京:高等教育出版社, 2010. [18] COHEN J. Statistical Power Analysis for the Behavioral Sciences [M]. Mahwah, NJ: Lawrence Erlbaum Associates, 1988. [19] PODSAKOFF P M, MACKENZIE S B, PODSAKOFF N P. Sources of Method Bias in Social Science Research and Recommendations on How to Control It [J]. Annual Review of Psychology, 2012, 63(1): 539-569. doi: 10.1146/annurev-psych-120710-100452 [20] RODEBAUGH T L, HEIMBERG R G, BROWN P J, et al. More Reasons to be Straightforward: Findings and Norms for Two Scales Relevant to Social Anxiety [J]. Journal of Anxiety Disorders, 2011, 25(5): 623-630. doi: 10.1016/j.janxdis.2011.02.002
计量
- 文章访问数: 1277
- HTML全文浏览数: 758
- PDF下载数: 186
- 施引文献: 0