Concept Papers
We frequently like to reflect on the work we have done and respond to the challenges inherent in understanding the impacts of technology. At the same time, it is important not to run ahead, "blindfolded" about what the impacts might be in reality. Below are some of our concept papers. If you have questions or need additional information, please contact Jason Ravitz.
-
The Enthusiasm Effect, the Reality Effect and Other Things to Know When Evaluating Professional Development Impacts (2005)
Jason RavitzSummary [close]Ravitz, J. (2005). Paper presented at The National Educational Computing Conference (NECC). July 1, 2003. Seattle, WA.
This paper provides a number of ideas that are important for professional developers and evaluators to consider with an overview of steps that can be taken to improve confidence in findings.
Evidence is provided concerning the usefulness of: 1) judging impacts based on a theory of what was supposed to happen; 2) establishing baseline measures to see where people started; 3) controlling for where people start before judging gains; 4) identifying and disaggregating data to account for systematic sources of error that will bias your findings, like school grade and size; and, 5) seeing how initial reactions to workshops change over time.
This paper discusses how to think about changes in teachers' technology skills and teaching beliefs and whether these sustained themselves in the face of school year and classroom realities. It also shows the importance of disaggregating data to see for whom and under what conditions the changes were most substantial.
The results provide guidance for evaluating the impacts of professional development. Teachers' beliefs about the efficacy of teaching reforms and their own technology skills varied at three different times: prior to summer workshops, after the summer workshops, and after ten months of subsequent teaching. Initial changes reported at the conclusion of a workshop represent an "enthusiasm effect" and changes after teachers have tried new ideas and new technologies in the classroom represent a "reality effect."
-
Supporting Change and Scholarship Through Review of Online Resources in Professional Development Settings (2005)
Jason Ravitz and Christopher HoadleySummary [close]Ravitz, J. & Hoadley, C. (2005). Published in British Journal of Educational Technology, 36(6), pp. 957-974.
How can we accelerate innovation and ensure effective dissemination of knowledge about online learning resources? This paper advocates strategies that systematically link online professional development with the research, development and diffusion cycle. The systemic approach we describe can accelerate knowledge advancement and help manage change by improving communication among teachers, trainers, developers and researchers. The examples that are provided are set within two funded projects in the United States that led to the development of two distinct but related strategies�the Online Site Evaluation Form for educators (a web-based review form used in workshops) and a six-week online course on technology-supported assessments. Both strategies make it easier to give feedback to developers and offer incentives to do so in ways that help teachers to learn about online resources individually and with colleagues. The examples are discussed with analysis of their strengths and weaknesses in supporting different modes of interaction. We highlight implications for instructional development, professional development, research and knowledge management in online communities. -
A Doorway to New Tools and Practices: Supporting Teacher Education, Research, and Development with an Online Netcourse (2004)
Jason RavitzSummary [close]Ravitz, J. (2004). A doorway to new tools and practices: Supporting teacher education, research, and development with an online netcourse. In N.H. Sabelli and R. Pea (2004). Six years of knowledge networking in learning sciences and technologies. Center for Innovative Learning Technologies (CILT) final report. Menlo Park, CA: SRI International.
This chapter describes an online "netcourse" that introduced educators to technology supported assessments in an innovative way that combined the interests of educators, researchers and developers. It appears in the Final Report from the the Center for Innovative Learning Technologies (CILT).
Netcourses like this can serve as portals, providing multiple entry paths, to new tools and practices. They can make it easier for educators to learn to use online tools and easier for developers and researchers to study this use.
The course emphasized that providing timely feedback to learners is an essential instructional strategy and that technology offers exciting opportunities for improving formative assessment for teaching and learning. Technology-supported assessments that were reviewed by the class via the Internet included IMMEX (Underdahl, Palacio-Cayetano, & Stevens, 2001), Intelligent Essay Assessor (Foltz, Laham, & Landauer, 1999), CRESST's Knowledge Mapper (Baker, 1998; Ruiz-Primo, 1999), and the Analysis Toolkit of Knowledge Forum (Lamon, Reeve, & Scardamalia, 2001).
In addition to being given instructions for trying a new tool each week, participants also read about the learning theories and research that supported use of the tool.
Download the Ravitz paper (PDF file, below) or see the complete SRI report
-
Balancing Teachers’ Willingness to Change with Classroom Realities: Moving Towards an Error Model in Professional Development Research (2003)
Jason RavitzSummary [close]Ravtiz, J. (2003). Paper presented at annual meetings of the Society for Information Technology in Teacher Education (SITE). March 26, 2003. Albuquerque, NM.
A statewide evaluation in Idaho addressed whether teachers changed in teaching philosophy and technology skills following summer in-service workshops and whether these changes sustained themselves in the face of school year and classroom realities. The timing of data collection is an important factor to consider when interpreting program outcomes. Teachers' beliefs about teaching and their sense of competency with technologies varied at three different times: prior to summer workshops, after the summer workshops, and after 10 months of subsequent teaching. It is critical to distinguish changes that are temporary and due to enthusiasm immediately after a workshop from changes that sustain themselves or emerge over time as the initial impact of the workshop fades and the reality of the classroom comes to the fore. Guided by Katzer's (1981) "error model" of research, we address time of data collection and other sources of error to improve our knowledge of program impacts.
-
Using Technology to Support Ongoing Formative Assessment in the Classroom (2002)
Jason RavitzSummary [close]Ravitz, J. (2002). CILT2000: Using Technology to support ongoing formative assessment in the classroom, journal of science education and technology, 11(3): 293-296, September 2002.
BIE and similar organizations such as CILT have been developing studies around the use of technology to identify under what conditions-who, what, where, when, and how-technologies can be used to support learning. One way that technology can be a substantial help to teachers and learners is by improving the ability to offer formative assessments of a learner's knowledge and skills, assessments that can support teachers and learners in the classroom.
-
Demystifying Data About Technology Impacts in Schools (2002)
Jason RavitzSummary [close]Ravitz, J. (2002). Presented at The National Educational Computing Conference (NECC). June 28, 2002. San Antonio, TX (revised 9/10/02).
This paper is for educators who wish to measure the impacts of technology in schools. It should be of interest to:
* principals, administrators, and technology-coordinators;
* technology-using educators;
* researchers, policy makers, funders; and
* those who are held accountable for technology integration.
The purpose of this document is to help educators communicate effectively about the impacts of their technology use. It presents Jeffrey Katzer's "error model" of research as a way to approach evidence about technology impacts and provides examples from several studies.
-
A Distance Scholarship Model for Teaching and Learning About Technology Supported Assessment (2002)
Jason RavitzSummary [close]Ravitz, J. (2002). Paper presented at the annual Society for Information Technology In Education (SITE) Conference. Nashville, TN. March, 2002.
This paper advances the notion of "scholarship" as a primary goal of distance education. It describes an online course taught in Blackboard™ that attempted to be accountable not only to the learning of individual students in the class, but to learning of various stakeholder groups including developers of new assessments for learning.
With the rapid pace of development in educational technology it is unacceptable to focus on producing individual knowledge among users with no sense accountability to the broader educational technology community (Shulman, 1999). It is important for pioneering technology-using educators to make clear to others where well-worn paths are leading and where new possibilities for teaching and learning are emerging. It is also important to improve communication between educators and developers about what types of tools and services are most valuable in the classroom. Online courses offer unique opportunities to support the advancement of scholarship within the educational technology community. -
Will Technology Pass The Test? (2001)
Jason RavitzSummary [close]Ravitz, J. (2001). Will Technology Pass the Test? PT3 Vision Quest paper commissioned by the U.S. Department of Education. Washington, DC.
The goal of this paper is to focus attention on emerging uses of technology to improve formative assessments of learners and to increase the cumulative contribution to knowledge about teaching and learning arising from technology-supported assessments and evaluations.
-
Technology and Learning: A Critical Assessment (2000)
John R. MergendollerSummary [close]Mergendoller, J.R. (2000). Published in Principal (Reston, Va.) Vol.79 No.3, January 2000. pp. 5-9
This paper outlines the progress and the challenges on the road to integrating technology and education. Tremendous progress has been made over the past two decades to increase the ratio of computers per student in schools. However, for schools to successfully adapt educational technology to support and extend student learning, three key factors must come into play. This paper discusses:
-
the importance of committing appropriate technological resources to the goal of integrating technology;
-
making a cultural commitment to the use of technology in teaching; and
- using effective instructional practices to produce substantial improvement in learning.
-
the importance of committing appropriate technological resources to the goal of integrating technology;
Download PDF (50 kB)
