Reliability of Self-reported Data in College Student Engagement Surveys: Social Desirability Bias in Self-reported Survey

GUO Fei; ZHAO Lin; LIAN Zhixin

doi:10.16382/j.cnki.1000-5560.2018.04.005

Volume 36 Issue 4

Jul. 2018

Turn off MathJax

Article Contents

Article Navigation > Journal of East China Normal University (Educational Sciences) > 2018 > 36(4): 53-61+163

GUO Fei, ZHAO Lin, LIAN Zhixin. Reliability of Self-reported Data in College Student Engagement Surveys: Social Desirability Bias in Self-reported Survey[J]. Journal of East China Normal University (Educational Sciences), 2018, 36(4): 53-61+163. doi: 10.16382/j.cnki.1000-5560.2018.04.005

Citation:

GUO Fei, ZHAO Lin, LIAN Zhixin. Reliability of Self-reported Data in College Student Engagement Surveys: Social Desirability Bias in Self-reported Survey[J]. Journal of East China Normal University (Educational Sciences), 2018, 36(4): 53-61+163. doi: 10.16382/j.cnki.1000-5560.2018.04.005

Citation:

PDF( 648 KB)

Reliability of Self-reported Data in College Student Engagement Surveys: Social Desirability Bias in Self-reported Survey

doi: 10.16382/j.cnki.1000-5560.2018.04.005

Tsinghua University, Institute of Education, Beijing 100084, China

Publish Date: 2018-08-20

Abstract

Abstract

Inspired by learner-centered and evidence-based evaluation in higher education quality assur-ance, investigations on college student experience and engagement become increasingly popular in China. However, with self-reported survey as a major instrument, the quality of data collected in such investigations is vulnerable to bias caused by socially desirable responses. Using data from the China College Student Survey (CCSS) in 2014-2016, the authors found that indicators constructed from student self-reported data on learn-ing activities and perceptions are subject to a social desirability response bias of 5-10%, while the bias in ob-jective information reported in the survey (such as GPA) is negligible. What's more, the higher the students' social desirability level, the larger the bias. It's also found that the size of social desirability bias shows its correlation with the content of the question items and indicators. These findings have methodological signifi-cance for quantitative investigations in social sciences. Further, implications for survey instrument develop-ment are discussed.
- self-reported survey,
- social desirability response bias,
- college students

FullText(HTML)

References(29)

References

[1]	韩振华, 任剑峰. (2002).社会调查研究中的社会称许性偏见效应.华中科技大学学报(社会科学版), 16(3), 47-50. http://www.cqvip.com/QK/82843A/200203/6521419.html
[2]	蒋华林, 邢全超, 吴芳, 朱晓华. (2014). "社会称许性"影响大学生学情调查的实证研究.大学教育科学, 6(6), 105-111. http://www.cqvip.com/QK/81233A/201406/662848634.html
[3]	李强, 徐晟, 李凌, 黄艳, 徐旻. (2012).心理健康素质测评系统·中国成年人多维社会赞许性量表的编制.心理与行为研究, 10(4), 255-261. http://edu.wanfangdata.com.cn/Periodical/Detail/xlyxwyj201204003
[4]	刘萃侠. (2001).马洛-克罗恩社会赞许性量表对中国被试适用性之初步验证.社会学研究, (2), 49-57. http://cdmd.cnki.com.cn/Article/CDMD-10345-1012259106.htm
[5]	吴燕. (2008). 人格测验中社会赞许性反应的测定与控制. 陕西师范大学. http://cdmd.cnki.com.cn/article/cdmd-10718-2009045938.htm
[6]	徐晟, 李强, 李凌, 尹艺璇, 梁栋. (2014).心理健康素质测评系统·中国成年人社会赞许性全国常模的制定.心理与行为研究, 12(6), 748-755. http://mall.cnki.net/magazine/Article/CLXW201406006.htm
[7]	赵丹. (2011). 马洛-克罗恩社会赞许性量表的修订及相关研究. 浙江师范大学. http://cdmd.cnki.com.cn/Article/CDMD-10345-1012259106.htm
[8]	赵琳, 王传毅. (2015).以"学"为中心:研究生教育质量评价与保障的新趋势.学位与研究生教育, (3), 11-14. http://www.cnki.com.cn/Article/CJFDTotal-XWYY201503004.htm
[9]	赵志裕, 邹智敏, 林升栋. (2010).文化与社会赞许反应:社会个人互动的观点.心理学报, 42(1), 48-55. http://mall.cnki.net/magazine/Article/XLXB201001009.htm
[10]	甄育玲. (2011). 社会赞许性反应量表的修订. 湖南师范大学. http://cdmd.cnki.com.cn/Article/CDMD-10542-1011167207.htm
[11]	Bowman, N. A., & Hill, P. L. (2011). Measuring how college affects students:Social desirability and other potential biases in college student self-reported gains. New Directions for Institutional Research, 2011(150), 73-85. doi: 10.1002/ir.v2011.150
[12]	Cowen, E. L., & Tongas, P. N. (1959). The social desirability of trait descriptive terms:Applications to a self-concept inventory. Journal of Consulting Psychology, 23(4), 361-365. doi: 10.1037/h0046699
[13]	Crowne, D. P., & Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24(4), 349-354. doi: 10.1037/h0047358
[14]	Damarin, F., & Messick, S. (1965). Response styles as personality variables:A theoretical integration of multivariate research. ETS Research Bulletin Series, 1965(1), 1-116. http://cn.bing.com/academic/profile?id=42401e84e47e645ffc0ee09f598ddbb2&encoded=0&v=paper_preview&mkt=zh-cn
[15]	Edwards, A. L. (1953). The relationship between the judged desirability of a trait and the probability that the trait will be endorsed. Journal of Applied Psychology, 37(2), 90-93. doi: 10.1037/h0058073
[16]	Edwards, A. L., & Horst, P. (1953). Social Desirability as a Variable in 2 Technique Studies. Educational and Psychological Measurement, 13(4), 620-625. doi: 10.1177/001316445301300409
[17]	Fordyce, W. E. (1956). Social desirability in the MMPI. Journal of Consulting Psychology, 20(3), 171-175. doi: 10.1037/h0048547
[18]	Marlow, D., & Crowne, D. P. (1961). Social desirability and response to perceived situational demands. Journal of Consulting Psychology, 25(2), 109-115. doi: 10.1037/h0041627
[19]	Miller, A. L. (2012). Investigating social desirability bias in student self-report surveys. Association for Institutional Research, 36(1), 30-47. http://cn.bing.com/academic/profile?id=c47a9221e7410c8e095411aefafd21ee&encoded=0&v=paper_preview&mkt=zh-cn
[20]	NSSE. (2012). Are NSSE scores influenced by a desire to respond in a socially desirable manner? Retrieved August 23, 2017, from http://nsse.indiana.edu/pdf/psychometric_portfolio/2010socialdesirability_formated.pdf.
[21]	Paulhus, D. L. (1984). Two-component models of socially desirable responding. Journal of Personality and Social Psychology, 46(3), 598-609. doi: 10.1037/0022-3514.46.3.598
[22]	Paulhus, D. L. (1986). Self-Deception and Impression Management in Test Responses. In A. Angleitner & J. S. Wiggins (Eds. ), Personality Assessment via Questionnaires (pp. 143-165). Berlin, Heidelberg: Springer Berlin Heidelberg. doi: 10.1007%2F978-3-642-70751-3_8
[23]	Paulhus, D. L. (2002). Socially desirable responding: The evolution of a construct. In I. Brown, D. N. Jackson, & D. E. Wiley (Eds. ), The role of constructs in psychological and education measurement (pp. 49-69). Mahwah, NJ: Erlbaum.
[24]	Sackeim, H. A., & Gur, R. C. (1978). Self-Deception, Self-Confrontation, and Consciousness. Springer US. http://cn.bing.com/academic/profile?id=a441032a288be8867066e1b247325317&encoded=0&v=paper_preview&mkt=zh-cn
[25]	Saunders, D. G. (1991). Procedures for adjusting self-reports of violence for social desirability bias. Journal of Interpersonal Violence, 6(3), 336-344. doi: 10.1177/088626091006003006
[26]	Stark, S., Chernyshenko, O.-S., Chan, K.-Y., Lee, W.-C., & Drasgow, F. (2001). Effects of the testing situation on item responding:Cause for concern. Journal of Applied Psychology, 86(5), 943-953. doi: 10.1037/0021-9010.86.5.943
[27]	Steenkamp, J.-B. E. M., de Jong, M. G., & Baumgartner, H. (2010). Socially Desirable Response Tendencies in Survey Research. Journal of Marketing Research, 47(2), 199-214. doi: 10.1509/jmkr.47.2.199
[28]	Wiggins, J. S. (1959). Interrelationships among MMPI measures of dissimulation under standard and social desirability instruction. Journal of Consulting Psychology, 23(5), 419-427. doi: 10.1037/h0047823
[29]	Zerbe, W. J., & Paulhus, D. L. (1987). Socially Desirable Responding in Organizational Behavior:A Reconception. The Academy of Management Review, 12(2), 250-264. doi: 10.5465/amr.1987.4307820