多级评价的四步协调模式
-
摘要: 本文以W.K.凯洛格基金会的释放资源倡议(Unleashing resources initiative)的评估工作为例, 阐述了适用于协调多级评价的一般四步模式。我们检阅了相关文献, 提出了协调对于多级评价的必要性, 解释了四步模式, 讨论了这一模式的优点, 并提出了运用该协调模式时应注意的问题。1) ① URI开发的次级评价问题的完整清单,见沈和Warfield(2003)。
-
表 1 URI基本评价问题体系
表 2 一个URI评价问题的评价子问题
-
[1] Babor, T. F., Steinberg, k L, Mcree, B., Vendetti.J., & Carroll, K. M. (2002). Treating marijuana dependence in adults: A muhisite, randomized clinical trial. In J. M. Herrell & R. B. Straw(Eds.).Conducting multiple site evaluations in real—world settings. New directions for evaluation, no. 94(PP. 17—30). San Francisco: Jossey—Bass. [2] Banks, S., McHugo, G.J., Williams, V., Drake, R.E., & Shinn, M. (2002). A perspectiv emeta-analyticapproach in a multisite study of homeless prevention. In J. M. Herrell & R. B. Straw (Eds.), Conducting multiple site evaluations in real-world settings. New directions for evaluation, no.94 (pp.45-60). San Francisco: Jossey-Bass. [3] Bryk, A. S., & Raudenbush, S. (1992). Hierarchical linear models: Applications and data analysis.Newbury Park, CA: Sage. [4] Chen, H. T. (2001). Development of a national evaluation system to evaluate CDC-funded health department HIV prevention programs. American Journal of Evaluation, 22, 55-77. [5] Cook, J. A., Carey, M. A., Razzano, L.A., Burke, J., & Blyler, C.R. (2002). The pioneer: The employment intervention demonstration program. In J. M. Herrell & R. B. Straw (Eds.), Conducting multiple site evaluations in real-world settings. New directions for evaluation, no.94 (pp.31-44). SanF rancisco: Jossey-Bass. [6] Dehejia, R. (1999). Evaluation in multi-site programs. Working paper, Columbia University and NBER (retrieved May 20th, 2004, from http://emlab.berkeley.edu/symposia/nsf99/papers/dehejia.pdf). Hamilton, L.S., McCaffrey, D.F., Stecher, B.M., Klein, S.P., Robyn, A., & Bugliari, D. (2003). [7] Studying large-scale reforms of instructional practice: An example from mathematics and science. Educational Evaluation and Policy Analysis, 25(1), 1-29. [8] Hanssen, C. E. (2004). Structural equation modeling as a tool for multisite evaluation. Unpublished doctoral dissertation, Western Michigan University, Kalamazoo. [9] Hanushek, E. A. (1986). The economics of schooling: Production and efficiency in public schools. Journal of Economic Literature, 24, 1141-1177. [10] Herrell, J. M., & Straw, R.B. (2002). Conducting multiple site evaluations in real-world settings. New directions for evaluation, no.94.San Francisco: Jossey-Bass. [11] Howard, B. C., McGee, S., Shia, R., & Hong, N.S. (2001). Computer-based science inquiry: How components of metacognitive self-regulation affect problem-solving. Paper presented at the annual meeting of the American Educational Research Association, Seattle, WA (retrieved May 20th, 2004, from http://www.cet.edu/research/papers/inquiry/AERA2001BHcbsi.pdf). [12] Kalaian, S.A. (2003). Meta-analysis methods for synthesizing treatment effects in multisite studies: Hierarchical linear modeling (HLM) perspective. Practical Assessment, Research and Evaluation, 8 (15) (retrievedMay20th, 2004, from http://pareonline.net/getvn.asp?v=8&n=15). [13] Kellogg Foundation, W.K. (2001). Logic model development guide. Battle Creek, MI: Kellogg Foundation(retrieved May 20th, 2004, from http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf). [14] Kellogg Foundation, W. K. (2003a). Philanthropy and volunteerism: Unleashing resources for the common good. Battle Creek, MI: Kellogg Foundation(retrieved May 20th, 2004, from http://www.wkkf.org/Pubs/PhilVol/Pub644.pdf). [15] Kellogg Foundation, W.K. (2003b). Evaluation guidebook for the unleashing resources initiative, phase Ⅱ. Battle Creek, MI: WS & Associates. [16] Konu, A., Lintonen, T., & Autio, V.(2002). Evaluation of well-being in schools: A multilevel analysis of general subjective well-being. School Effectiveness and School Improvement, 13, 187-200. [17] Lawrenz, F., & Huffman, D. (2003). How can multi-site evaluation be participatory? American Journal of Evaluation, 24, 471-482. [18] Livert, D., Rindskopf, D., Saxe, L., & Stirratt, M.(2000). Using multilevel modeling in the evaluation of community-based treatment programs. Multivariate Behavior Research, 36, 155-184. McDonald, D.(1999). Developing guidelines to enhance the evaluation of overseas development projects. Evaluation and Program Planning, 22, 163-174.