ITP: Interdisciplinary Training Program in the Education Sciences ITP: Interdisciplinary Training Program in the Education Sciences
at the University of Wisconsin-Madison

Fall 2014 - Interdisciplinary Training Seminar in Education Sciences

Geoffrey Borman
SOC 875 / ELPA 940
Fridays, 12 - 1:30 pm
Room 259 Ed Sciences

Educational Program and Policy Implementation

Researchers exploring the effects of educational interventions are increasingly focusing on conceptualizing and measuring the implementation of the programs and policies they study.  IES grants and other federally funded research efforts typically encourage or require researchers to articulate a program’s theory of action, the program components or mechanisms hypothesized to produce the program’s impacts, a plan for measuring the extent to which the key components or mechanisms were implemented with fidelity, and an analytical approach to link measures of implementation fidelity to the observed program effects.

The theme for the fall ITP seminar is program and policy implementation.  In exploring this theme, we will consider four interrelated areas:

  • The conceptualization and theory of program and policy implementation.
  • Designing implementation studies and measuring implementation fidelity.
  • Linking implementation fidelity measures to program outcomes.
  • Case examples of implementation studies from across the disciplines

Through participation in this seminar, students will come to understand current thinking about, and evolving standards for, implementation studies.  Ultimately, students will learn how implementation studies can illuminate the mechanisms of change and impact underlying educational policies and programs.  


September 12:
Geoffrey Borman, ITP Director & ELPA, "Program and Policy Implementation: An Introduction"

September 19:
Greg Duncan, University California-Irvine, "Replication and Robustness in Education Research"

Background reading:
Duncan, Engel, Claessens, & Dowsett, "Replication and robustness in developmental research" (circulated to seminar participants)

September 26:
David Weimer, La Follette School of Public Affairs, "Anticipating Implementation in Policy Design and Evaluation"

October 3:
Chris Taber, Economics, "The Econometrics of Randomized Control Trials"

October 10:
Barbara Goodson & Catherine Darrow, Abt Associates,"Studying Implementation in Impact Evaluations"

Background readings:
Kellogg 2004
O'Donnell 2008
Century et al. 2010
Nelson et al. 2012

October 17:
Chris Hulleman, University of Virginia, "Intervention Fidelity: Models, Methods, and Applications"

October 24:
Mitchell Nathan, Educational Psychology, "Implementation and Scale Up: A Design Sciences Perspective"

Background reading:
Nathan & Alibali (2010)

October 31:
Jeremy Fiel, ITP & Sociology, "Closing Ranks: Closure, Status Competition, and School Segregation"

Background reading:
Fiel 2014 (circulated to seminar participants)

November 7: NO MEETING

Background reading: Paper by Katherine Magnuson (Social Work) will be distributed to seminar participants

November 14:
Eric Camburn, ELPA, "Improving Literacy Instruction In America’s High Poverty Schools: Resources And Routines For Learning"

Background reading:
Barnes et al. 2014 (circulated to seminar participants)

November 21:
Peter Steiner, Educational Psychology, "Designing Valid and Reliable Vignette Experiments for Survey Research: A Case Study on the Just Gender Income Gap"

Background reading (circulated to seminar participants):
Steiner, Atzmuller, & Su 2014
Atzmuller & Steiner 2010


December 5:
Michelle Robinson, ITP & Sociology, "A Fidelity Analysis of a Multi-Year Implementation of the FAST Program."

Background readings:
McDonald & Frey (1999)
UNODC publication

December 12:
Dominique Bradley (ELPA) & Evan Crawford (ITP & Political Science), "Identifying the Critical Components of Values Affirmation: Fidelity of Implementation in a Large-Scale Randomized Controlled Trial."



Atzmuller, C., & Steiner, P. M. (2010). Experimental vignette studies in survey research. Methodology 6 (3): 128-138.

Barnes, C., Camburn, E., Han, S. W., & Holmstrom, K. (2014). Improving literacy instruction in America's high poverty scools: Resources and routines for learning. Unpublished MS.

Century, J., Rudnick, M., & Freeman, C. (2010). A framework for measuring fidelity of implementation: A foundation for shared language and accumulation of knowledge. American Journal of Education 31 (2): 199-218. DOI 10.1007/sl1414-012-9295-x

Compilation of Evidence-Based Family Skills Training Programmes. United Nations Office on Drugs and Crime. United Nations Publication. Sales No. E.09.XL8.

Duncan, G. J., Engel, M., Claessens, A., & Dowsett, C. (2014). Replication and robustness in developmental research. Unpublished MS.

Fiel, J. E. (2014). Closing ranks: Closure, status competition, and school segregation. Unpublished MS.

McDonald, L., & Frey, H. E. (1999). Families and schools together: Building relationships. Juvenile Justice Bulletin, November 1999, 1-19.

Nathan, M. J., & Alibali, M. W. (2010). Learning sciences. Wiley Interdisciplinary Reviews--Cognitive Science, 1 (May/June), 329-345. DOI: 10.1002/wcs.54.

Nelson, M. C., Cordray, D. S., Hulleman, C. S., Darrow, C. L., & Sommer, E. C. (2012). A procedure for assessing intervention fidelity in experiments testing educational and behavioral interventions. Journal of Behavioral Health Services & Research, 1-22. Published online: 31 August 2012.

O'Donnell, C. L. (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research 78 (1): 33-84.

Steiner, P. M., Atzmuller, C., & Su, D. (2014). Designing valid and reliable vignette experiments for survey research: A case study on the fair gender income gap. Unpublished MS.

Using Logic Models to Bring Together Planning, Evaluations, and Action: Logic Model Development Guide (2004). W. K. Kellogg Foundation, Battle Creek, Michigan.