Tech-Education : Evaluation of E-Learning Environments part 1

Overview

Evaluation is systematic acquisition of feedback on the use, worth and impact of some object, program or process in relation to its intended outcomes. The goal of that is to influence decision−making of some sort through the provision of feedback. Types of Evaluation are Formative, Summative, and Monitoring of integrative.

Formative Evaluation 

Process of gathering data as part of the design and development process. The goal is to ensure checks and balances and to enable improvements to be made as the project unfolds. Formative is Indicates that data is being gathered during the formation of the project so that revisions to it can be made cost−effective.

A Part of Front-end analysis - need assessment :

• Who needs the program
• How great the need is
• What might work to meet that need

Comprises :

• Design - based
• Expert - based
• User - based

Design Based - Formative Evaluation

Involves

1. Designer or evaluator 
2. Ascertaining the match between: 
• Learning 
• Instructional goals 
• Design specifications 

No real target users involved

Methods

• Formal modeling (conceptual, learning and instructional design)

For example :

• An architect evaluating the match between functionality of a building and its design specifications

Expert Based - Formative Evaluation

Evaluator using the system to decide whether the innovation matches predefined design criteria.

For example

• A building inspector assessing a building against the architect’s plan of the building 

Referred to as

• “Construct” or “content” evaluation 
• Carried out by design and/or content experts 

Methods

• Walk-through (with think-aloud) 
• Observation (combined with structured responses) 
• Interview (structured and/or semi-structured)

User Based - Formative Evaluation

Involves

• Gathering data from a larger sample of users in a full-scale use of the materials in a real teaching and learning situation 

Methods

• Observation 
• Video-based recall of user interactions (e.g. querying, think-aloud) 
• User’s self-reporting (e.g. critical reflections, student diaries, learning logs) 
• Semi-structured questionnaires 
• Audit trail/user log data (automatic collection of details on user login/use)