Get documents about "Formative Evaluation and Assessment
Document Sample
Formative Evaluation and
Assessment
Stanley Varnhagen (xLS)
Myrna Sears (xLS)
Dr. Samira ElAtia
(English Language Program)
Faculty of Extension
Presentation Download
• www.xls.ualberta.ca
• Select Resources
22 March 2006 Formative Evaluation and 2
Assessment
Overview
• Background
• Summative Bias
• Formative Assessment Case
• Formative Evaluation
• Potential Directions
22 March 2006 Formative Evaluation and 3
Assessment
Evaluation and Assessment
• Used synonymously by many
• We refer to Assessment as occurring
with individuals
– Grades
– Performance
• Evaluation as occurring with programs
or larger entities
– Course evaluations
22 March 2006 Formative Evaluation and 4
Assessment
Examples
• What are some examples of how you
have seen assessment and evaluation
used around instruction and learning at
the University of Alberta?
22 March 2006 Formative Evaluation and 5
Assessment
How Evaluation/Assessment
Are Typically Used
• Promotions and tenure
• Judging which course to take
• Determining grades
22 March 2006 Formative Evaluation and 6
Assessment
Summative Bias
• Both Assessment and Evaluation
emphasize a summative approach
– By that, assessment and evaluation and
the judgment that occurs at the end of the
process being examined
• Mid-terms and Finals that complete a unit
• End-of-term course evaluations
22 March 2006 Formative Evaluation and 7
Assessment
Other Characteristics
• Judgment by others
– Criteria are often not known
– Consequences for the judgment
• Late in the process
22 March 2006 Formative Evaluation and 8
Assessment
What’s Missing
• An opportunity to judge for oneself
• An opportunity to learn from the
experience while it can still affect what
is currently being evaluated.
22 March 2006 Formative Evaluation and 9
Assessment
Alternatives
• The remainder of this workshop will
present an alternate perspective of
assessment and evaluation.
• First assessment
• Then evaluation
22 March 2006 Formative Evaluation and 10
Assessment
Practical Considerations in
Formative Assessment:
A Case Study of A Team Work
Research Project
Dr. Samira ElAtia
The Assessment Tool
• In a Research Methods course, a
semester-long project was used as a
tool to assess the students’ learning in a
formative way.
• For each project, students were
supposed to work in groups of three
members minimum—five maximum
22 March 2006 Formative Evaluation and 12
Assessment
Objective of the Course Itself
• Think critically
• Read research material with a critical mind
• Make a synthesis of academic texts
• Cooperate and collaborate in group work
• Evaluate academic work of oneself and other
peers
• Identify and analyze a problem
• Apply the knowledge of the class to new
contexts --- namely to solve the problem of
the research project
22 March 2006 Formative Evaluation and 13
Assessment
The Project Assessment
• The project constituted 50% of the final
grade. It was broken down as such:
– 20% for the procedure and steps
– 20% for the final product
– 10% for the project portfolio
22 March 2006 Formative Evaluation and 14
Assessment
Project Description
• The goal of the project was to provide
opportunity to students to apply the
theories and methods learned in the
class to a real-life problem that they
investigated throughout the semester.
22 March 2006 Formative Evaluation and 15
Assessment
Steps of the Research Project
Forming groups
Identifying a problem
Investigating the problem solution
Interpreting the results
22 March 2006 Formative Evaluation and 16
Assessment
Work Required for the Project
• Choosing group members, and distributing tasks
• Choosing a problem to investigate and
formulating research question
• Learning more about the problem (Literature
Review)
• Design and methodology for research (subjects,
tools, running tests and surveys…)
• Calculating data and presenting it in graphs
• Interpreting and discussing the results.
• Answering the research questions
22 March 2006 Formative Evaluation and 17
Assessment
Continuous
Assessment of Each
Step of the Project
Teacher’s Work of the Feedback
instruction and students’ teams from the
demonstration of outside of class teacher in
previous project for each step of class
the project
Teacher Team does the
Corrections correction and
feedback and
and work of the integrates the step
grade on the
students’ teams in the final draft of
specific step
outside of class project
START ANTOHER STEP IN
THE PROJECT
22 March 2006 Formative Evaluation and 18
Assessment
Issues and Perspectives on
Continuous Assessment
• Discuss the benefits to students
• Discuss the challenges to students
22 March 2006 Formative Evaluation and 19
Assessment
Issues and Perspectives on
Continuous Assessment
• Students benefit from working on each step
separately because :
– they fully covered and understand the scope and
the important elements of each step
– they receive feedback on it twice
– they are graded on the process of their work
– they receive another grade for the final product
which is the complete project (along with the teacher,
another independent rater grades the project)
– their final grade on the project is a combination of
work they did throughout the semester and not only
at the end
22 March 2006 Formative Evaluation and 20
Assessment
Issues and Perspectives on
Continuous Evaluation
• In some cases, working in teams is
challenging. Some students feel that
they are working more than others, and
hence the process of continuous
evaluation ensures that each student
gets involved, because the instructor is
present in the process.
22 March 2006 Formative Evaluation and 21
Assessment
Group Work
• How do you assess group work so that
it fairly reflects the individual’s
contribution to the group?
22 March 2006 Formative Evaluation and 22
Assessment
Issues and Perspectives
on Continuous
Assessment
• Continuous assessment serves to inform
students if they are not doing well in their
course work.
• Through continuous feedback on the steps,
students are made aware of their status and
can catch up and do a better job in
subsequent steps.
• So in a way, the process pushes students
to be aware of their learning and to modify
their studying/learning habits and methods.
22 March 2006 Formative Evaluation and 23
Assessment
Considerations for
the Teachers
• Teacher becomes a facilitator and a guide
for the project.
• Lack of class time to do other work
• Higher workload given the amount of
marking and feedback that needs to be
given to each group on each step
• Dealing with logistical issues (mainly
students’ complaints about others)
• The grade of the final product may
sometimes be lower than the grade for the
process/steps
22 March 2006 Formative Evaluation and 24
Assessment
Discussion
• Based upon your own experiences as a
teacher or a learner, what additional
insights, suggestions or comments on
the case study can you provide?
22 March 2006 Formative Evaluation and 25
Assessment
Formative Evaluation
• Offers valuable tools that allows quick
adjustment during the course
• Can find problems quickly
• Can empower both faculty and students
22 March 2006 Formative Evaluation and 26
Assessment
Instructional Development & Evaluation
Model
Instructional Evaluation
Plan Planning Planning
Analysis Needs Assessment
Develop
Formative
Design
Reflection
Evaluation
Offer Support
Summative
Evaluation
Review Discovery Dissemination
Planning
• Planning should take place EARLY
– Early planning allows` more options
– Early is often at the same time that the
planning of the project itself is occurring
22 March 2006 Formative Evaluation and 28
Assessment
Planning
• What: There is a need to work out a
plan for evaluation
• When: Should occur as early as
possible so potential options are not
eliminated
• How: Considering what should be
evaluated and how it can best be
integrate with the design
• Why: To allow the maximum flexibility in
planning
22 March 2006 Formative Evaluation and 29
Assessment
So What is Formative
Evaluation Anyway?
• Evaluation that is performed
during the development and
implementation
• Principal focus is on improvement
- not judging success
• Tends to be less formal
– More likely to use internal evaluators
22 March 2006 Formative Evaluation and 30
Assessment
Formative Evaluation
Continued
• Process Oriented
• More flexible
• Evaluation results should allow time
for implementation of findings
22 March 2006 Formative Evaluation and 31
Assessment
Formative Evaluation
• What: An evaluation focused on
making improvements
• When: During the development and
implementation of a project
• How: Using surveys and focus groups
• Why: Provide empirical information to
catch problems early and help make
improvements
22 March 2006 Formative Evaluation and 32
Assessment
Discussion
• From your own experience, describe a
formative evaluation activity.
• What were some of the benefits and
consequences?
• Where can you see it being useful?
22 March 2006 Formative Evaluation and 33
Assessment
Other Ways Formative
Evaluation Can Be Used
• Better understand environment/students
• Make judgment more transparent
22 March 2006 Formative Evaluation and 34
Assessment
Not All Courses Are Created
Equally
• Some are safer – some take risks
• Some are more conventional and some
are innovative
• Some are being taught for the first time,
some have been taught (mostly
unchanged) several times
22 March 2006 Formative Evaluation and 35
Assessment
New Delivery Methods
• New Instructional Technology often
means new software and procedures
need to be learned
– E.g., WebCT – Conferencing, Elluminate
• How to use these programs must often
be learned
22 March 2006 Formative Evaluation and 36
Assessment
New Pedagogical Methods
• The instructional methods often change
(Samira’s example)
– A course delivered in a conventional
didactic manner is now taught
– using a learner centred approach
• The changed learning assumptions are
often not explicitly discussed
22 March 2006 Formative Evaluation and 37
Assessment
How Formative Evaluation
Can Be Used
• Formative Evaluation can be used to
help improve the course
– Questions can be answered and different
methods compared during development
– Feedback can be collected early in the
course to see how it is going
22 March 2006 Formative Evaluation and 38
Assessment
Evidence-based Evaluation
• Use of Formative Evaluation allows
feedback that can allow specific
methods to be examined
• To teach this particular concept, which
instructional method is the most
effective?
22 March 2006 Formative Evaluation and 39
Assessment
Discussion
• How might formative evaluation and
assessment be incorporated into your
courses?
22 March 2006 Formative Evaluation and 40
Assessment
Assessment and Evaluation
Support on Campus
• xLS
• AICT – Test Scoring and Questionnaire
Services (TSQS)
• UTS
• Centre for Research on Applied
Measurement and Evaluation (CRAME)
22 March 2006 Formative Evaluation and 41
Assessment
Contacting Us
• Stanley Varnhagen, xLS
– stanley.varnhagn@ualberta.ca
– (780) 492-3641
• Myrna Sears, xLS
– myrna.sears@ualberta.ca
– (780) 492-6332
• Dr. Samira ElAtia, English Language Program
– samira.elatia@ualberta.ca
– (780) 492-9224
22 March 2006 Formative Evaluation and 42
Assessment
What is Innovation?
• For this discussion, innovation is any
teaching method or teaching activity
that is new to you.
22 March 2006 Formative Evaluation and 43
Assessment
Considerations When Using
Innovation
• Added complexity
• Added risk
• Added learning
– Delivery methods
– Pedagogical methods
22 March 2006 Formative Evaluation and 44
Assessment
Added Complexity
• Instructional Technology seldom
simplifies things
• New procedures that need to be learned
• More things that can go wrong
• Often students come with uneven
experiences
22 March 2006 Formative Evaluation and 45
Assessment
Added Risk
• Problems are not always obvious
• Students may not always be
appreciative of the changes
• The evaluation methods/material may
not always properly suit the changes
22 March 2006 Formative Evaluation and 46
Assessment
Methods
• Surveys
• Focus Groups
• Other
– Logs
– Observations
22 March 2006 Formative Evaluation and 47
Assessment
Surveys
• Good for collecting quantitative
information
• Are generally, easily understood by the
population
• Can be quickly administered
• Basic analysis can be straightforward
22 March 2006 Formative Evaluation and 48
Assessment
Online Surveys
• Access
• Automatic Data Collection
• Different Assumptions
22 March 2006 Formative Evaluation and 49
Assessment
Focus Groups
• Good for collecting qualitative
information
• Can be relatively informal and easy to
put together
• Can sometimes learn unanticipated
results
22 March 2006 Formative Evaluation and 50
Assessment
Focus Groups (2)
• Good for understanding if assumptions
are being met
• Often consensus oriented
• Good for understanding the reasons
behind respondents beliefs
22 March 2006 Formative Evaluation and 51
Assessment
Related docs
Other docs by Hs934L7n
