Hello and welcome!

Web Based InstructionThis page is supplemental to a chapter in Web Based Instruction, Badrul Khan (Ed.). published by Educational Technologies.

 In the chapter, I listed a number of data that could be used to evaluate collaborative online learning environments. This list includes the following:
 
 

In future work, I hope to clarify the nature of these activities, and their relative strengths and weaknesses.

 This page is intended to demonstrate the usefulness of having a list of strategies that can be used for evaluating online learning experiments. When working in an online environment, no less in any type of instruction, it is necessary to consider evaluation as a design issue. For an update -- see presentations atNECC'98 on embedded assessmemnt and CSCL'99 paper on the"Interactive Portfolio" idea.

>Three projects are given below as examples. Based on the design of the projects, some likely evaluation strategies are given. Then, there is some indication of the strengths and weakness of the different evaluation strategies.

Potential aspects of evaluaton from three different online projects.*
(*not based on actual evaluation plans, experiences or results)
Examples
 Aspects 
that may 
support 
evaluation		 Possible Strong Points Possible Issues of concern
Discovery Learning
Community: Focus on Promised Land
- Classroom proposals, plans, and finished projects are posted.
-Peer & expert review.
- Listserv discussions are archived.
 -Stage analysis of classroom projects over time (toe-wade-dive) 
-Online portfolio with opportunities for review 
-Consistent structure for all projects		 -Motivating and facilitating contributions of off-line work. 
-Fostering discussion about projects on listserv or elsewhere. 
National School Network
- Baseline and follow-up surveys used to identify needs, issues, and change over time.
- Face-to-face meetings and phone interviews
- Members contribute to monthly newsletters.
 -Quantitative data. Key issues identified from survey. 
-Rolling design based on needs assessment. 
- Face-to-face and phone interviews provide vicarious experience. 
-Hard to determine benefits to distant, "quiet" users		 -Fostering open discussion, surveys are not interactive 
-Limited scope for face-to-face meetings and phone interviews 
-Very large group size
Online Internet Institute
- Threaded discussion archives on the Web
- "Roving reporter" tells stories and explains activities
- Participants state personal goals when joining and provide information in order to improve their own experience (and help the project organizers)
- Teacher projects organized and placed for viewing on the Web 
-Permanent, ongoing record.
-Vicarious experience from stories
-Listserv encourages and supports discussion of issues
 - Lack of structure and organization at participant sites. 
- Severe information overload 
-Multiple understandings reflected
Internet Addresses (World Wide Web) of projects in above Table-- 
 
 
Cite this page: Ravitz, J. (1997). Supplementary page to chapter in B. Khan (Ed.). Web-based Instruction. Englewood Cliffs, NJ: Educational Technology. [WWW Document]. URL http://nsn.bbn.com/Ravitz/ipv.html 

Summary of First Year Evaluation Report for the Online Internet Institute now available: http://www.bie.org/Ravitz/oii_summary.html

Evaluation Primers and Resources
Program Evaluation Standards, Designing Evaluations
Educator's Guide to Evaluating the Use of Technology in Schools and Classrooms
http://www.ed.gov/offices/OUS/eval/primer1.html
http://www.ed.gov/pubs/EdTechGuide/
http://www.ombwatch.org/www/ombw/gpra/gpra1.html
http://www.wmich.edu/evalctr/
 

Please send comments to Dr. Jason Ravitz -- jason@bie.org