The Enthusiasm Effect, Reality Effect and Things to Know When Evaluating Professional Development

Home » Research » The Enthusiasm Effect, Reality Effect and Things to Know When Evaluating Professional Development

Jason Ravitz

This paper provides a number of ideas that are important for professional developers and evaluators to consider with an overview of steps that can be taken to improve confidence in findings. Evidence is provided concerning the usefulness of:

judging impacts based on a theory of what was supposed to happen
establishing baseline measures to see where people started
controlling for where people start before judging gains
identifying and disaggregating data to account for systematic sources of error that will bias your findings, like school grade and size, and,
seeing how initial reactions to workshops change over time.

This paper discusses how to think about changes in teachers' technology skills and teaching beliefs and whether these sustained themselves in the face of school year and classroom realities. It also shows the importance of disaggregating data to see for whom and under what conditions the changes were most substantial.

The results provide guidance for evaluating the impacts of professional development. Teachers' beliefs about the efficacy of teaching reforms and their own technology skills varied at three different times: prior to summer workshops, after the summer workshops, and after ten months of subsequent teaching. Initial changes reported at the conclusion of a workshop represent an "enthusiasm effect" and changes after teachers have tried new ideas and new technologies in the classroom represent a "reality effect."

Suggested citation: Ravitz, J. (2005). Things to Know When Evaluating Professional Development Impacts. Paper presented at The National Educational Computing Conference (NECC). July 1, 2003. Seattle, WA.

You might also be interested in the full report from this study.