BlackandWiliam - Learning Centred Feedback in Practice

Document Sample
BlackandWiliam - Learning Centred Feedback in Practice Powered By Docstoc
					Black and Wiliam 1998

4 year study by 2 leading Professors at King‟s College

Trawled world wide for effective classroom based research
Considered 700 studies, but chose only those with large
effect sizes, classroom environment, and good design.

Test before                              Test after

Test before      Control Group           Test after

The effect size is how much better the Experimental Group
was than the Control Group

The strategies they found:

 had the greatest effect on the weakest learners
 could yield an effect size equivalent to a two grade leap at

Their study also found that established formative
assessment practice is weak, tending to ape summative
assessment practice.

We have weak practice in a key variable... a real
Bill or Sid?
Which builder would you choose to build your new extension?

Sid’s High Grade Grafters
Quality Control System:

 Sid grades each worker’s efforts at the end of each day
  informing them of this grade
 He praises work of above average standard
 He draws attention to errors and deficiencies

 He constructively criticises work which is not of an acceptable
 He moves on to the next day’s work to guarantee speedy

Bill’s Trouble Shooters
Quality Control System:

 Bill asks each worker to inspect their own work and fix errors
  and deficiencies as they go
 He inspects work at the end of each day, praising work of an
  acceptable standard
 He gets workers to put right any errors and deficiencies and
  checks these corrections have been made

   He constructively criticises work which is not of an
    acceptable standard
   Moves on to the next day’s work to guarantee speedy
Medal and Mission feedback



1. Make the goals very clear: criteria are explained and illustrated
   with examples
2. Ask the student for a self-assessment
   (they will be helped if you give them criteria to
   self-assess against, and exemplars.)

3. Give non-judgmental feedback:
 accept the student’s present standard
 avoid competition or comparison with others, instead let them
   compete with:
     the task, and
     with themselves, (i.e. with previous work)

Feedback should be
 forward looking, positive, & constructive
 task centred not ego centred
 medal and mission
Feedback exercise (graph)
1. “Well done John, that’s brilliant! 9/10

2. “Good graph Martha. Nice and neat. All the
points are well plotted.”

3. “Better than last week. Rather thick lines
though! Keep improvement up.”

4. “Better standard. A sharper pencil improves
accuracy. Keep improvement up!”

5. “Good scales, good line, but some points missing!!
Nice and neat.”

6. “Good standard. No title. neat writing.”
7. “Good axes, points and line.

8. “Well done, you handed it in! Please finish it
now. Keep this improvement up.”
Learning Centred Feedback in Practice.                     Geoff Petty

Professors Ian Black and Dylan Wiliam of Kings College London reviewed many
hundreds of research studies and showed that formative assessment has more effect on
learning than any other single factor (including prior learning).*

The following summarises the advice suggested by Black and Wiliam, and then goes
further to add some concrete suggestions for implementing their ideas.
Effective formative assessment has its most positive effect on low attainers, and few teachers
adopt good practice.

General Advice.
 Avoid grading. Grades are consistently found to demotivate low attainers. They also
fail to challenge high attainers, often making them complacent. So avoid giving a grade or
mark except where absolutely necessary. This is not easy to do on some courses.
However it is rarely necessary, and almost never desirable, to grade every piece of work.

 Use self-assessment: Ask students to self-assess, providing them with self-assessment
criteria or helping them to develop their own. See examples below. Self-assessment has
been shown to double attainment if it is used very frequently. It encourages the reflective
habit of mind essential for improvement, ensures students take responsibility for their
own learning, focusses attention on criteria for success, and increases effort and
persistence. It should be followed by action planning and the action plan points should be
followed up, it then generates excellent evidence for the Key Skill „Improve Own Learning
and Performance‟.

  Give learning-centred feedback “Give a medal and a mission”
Accept the student‟s present attainment however low, without blame or disapproval.
Set about improving this by giving a:
Medal for what the student can do or has done well. Effort persistence and other good
study habits can be included in the criteria.
Mission: what the student needs to do to improve. This can be an improvement to the
existing work, or a target (feed-forward task) for the next piece of work.

Focus your feedback on the following:

        tasks: e.g. provide positive comments on the completion of tasks, strengths,
       criterion-referenced achievement etc. If teachers set mastery tasks** this provides
       opportunities to give positive feedback to the very lowest attainers.

         Meeting personal targets. If students are encouraged to self-assess and to set
       themselves targets for improvement, then the teacher can comment on a student‟s
       progress towards these targets.

         improvements     effected by the student can be positively commented on.

           opportunities for improvement and constructive criticism can be given.
         Feedback proformas can help teachers give such feedback in practice, examples are
         shown below. In each case they need to be larger than shown to allow space for

   Use the „praise sandwich‟ That is: praise; constructive criticism; then praise again.

Use Mastery Learning. This is a series of easy tests on key material set every four or six
weeks, with retests for those students who do not pass. It takes time to set this up, but it
works very well. See the „Mastery Learning‟ chapter in Teaching Today.

Use   formative teaching methods that „find faults, fix, and and follow up‟ see the
handout called „formative teaching methods‟ which you can download from These ideas were subsequent to the research review but
are clearly linked with it.

Feedback Proformas
Why not give assignments, homeworks, classwork etc with a feedback proforma like
these, (only with more space). This helps the teacher give learning-focussed rather than
grading-focussed feedback. Such proformas are not new, but are under-used. Make sure
they are bigger than shown to allow space for writing.

Generic assessment criteria
Here, as an example, is a set of negotiated criteria used to assess essays. Any generic skill
could be developed in the same way, e.g. electronic circuit design, painting, lab report
writing, etc.

Ask students for criteria first. They will come up with most of them, and will then really
„own‟ them. Follow this with a discussion on which criteria really count and why. This is
very helpful to clarify good practice and your expectations. Each criterion needs to be
discussed, explained, and justified to the class. (I do not pretend that the criteria below are
the only, or the best criteria for essay writing, you must decide your own!)

Criteria are then used repeatedly, perhaps for every essay written on the course. Students
hand in the work already self-assessed, then the teacher assesses against the same criteria.
Ideally no grade is awarded, or if it is, it is given some weeks after this informative
feedback. Black and Wiliam‟s research review shows that if you grade students pay
attention only to this, and don‟t read your feedback.

Ruth Beard in “Teaching in Higher Education” claims that such generic criteria greatly
improve importance over a course even without self-assessment. Black and Wiliam
showed that self-assessment and informative feedback were amongst the most important
things a teacher did.

The following grids should all be much bigger in practice, to allow more space for
* Black and Wiliam (1998) “Assessment and Classroom Learning” in the
journal Assessment in Education.
**See separate handout for explanation on mastery tasks

read also “Inside the black box” at:
Essay writing assessment proforma
Title:                                                            Name:
Criteria                             Self-assessment         Teacher assessment
Did you relate each of your
arguments to the essay

Did you give arguments both
„for‟ and „against‟ both:
 The proposition in the essay
 Any major points or
     conclusions you made?

Did you give enough evidence,
examples, and illustrations for
each of your arguments?

Did you prioritise the arguments
for and against, and evaluate

Did you draw a justified
conclusion related directly to the
essay title?

Main strengths

Improvements needed for this

Targets for the next essay           Self assessed target:
                              Maths Assessment
Exercise:                             Name:

 Assessment criteria        grade               Teacher, peer, or self-assessment
Methods: aim to
make these
appropriate, and as
simple or elegant as
Methods justified
The principles or
formulae used are
made clear
Working: aim to
make working clear;
complete; easy to
follow; stating
principles or formulae
used where necessary.
Care taken: aim to
check your work for
errors, and present
work neatly.
Main strengths

      It’s okay if you don’t fully understand a concept first time, learning takes time.
      If this work is graded, aim to beat your own record, not someone else’s
      what counts is whether you understand the problem and solution, not whether you
       made any silly slips
      If you got something wrong that’s fine. It’s how we learn.
      You will learn from mistakes if you find out how to do it without mistakes next time,
       and really understand this.

Corrective work on this exercise

(Find someone with an A for …….. and ask them to show and explain their work.)
Target for your next piece of work

Freely based on ‘Using Assessment to Raise Achievement in Mathematics’ QCA Nov 2001

Opportunities for Development

General Comments

Below are examples of assignment-specific assessment proformas:

Assessment Criteria       Strengths and development

Plan for improving        I like your ideas on diet exercise and
Health       and          entertainment. Most points well covered.
well-being                A well designed table! Some rest would
                          help. Read assignment brief carefully!

Purpose for this plan     You explain this well referring to
                          evidence. Quite the best bit of your
General Comments.         I notice some of your work is neater, keep
                          this up.

Student‟s goals:          Comments:

Be better at checking     You have definitely achieved an
my spelling               improvement here Simon.

Criteria                     Student assessment   Teacher assessment
A diagram of the heart:
accurate, neat, and
correctly labelled

Explanation of how the
heart works:
valve sequence, and
General comments
                                        Vocational Skills – Care

                             Self-assessment of key criteria for written work

Below is a list of some of the most important skills needed when completing any written tasks.
These skills will help you achieve your Care Modules and will also help you in the future whenever
you need to find out information for yourself and present it well.

Please think about each skill carefully and assess how well you think you do. Then score each one
as follows

                         Red:           Needs improving
                         Amber:         Average/okay
                         Green:         good.

         Skill                 Red     Amber     Green             Teacher assessment
Neat writing

Correct spelling

Using paragraphs

Using only relevant

Writing in your own

Using the library

Using computers and
the internet

Keeping a record of
sources of information
eg. books, internet
sites, etc.
Finding pictures,
articles, leaflets, etc to
add interest to your
Handing work in on

Which of these skills do you need to improve most?

Learning Target for next piece of work:
                             Assessment of Writing Skills

Please think about each skill carefully and assess how well you think you do. Then score
each one as follows. Please hand in your plan with your finished work.

                                    Self Assessment
              Skill             Didn’t I think I did          Teacher Assessment
                                          I did

Used sentences well

Used paragraphs well

Used verbs well

Proof read

Used capital letters well

Used full stops and commas
Appropriate style
Answered the question
Good conclusion


 Corrections needed for this piece of work:

 Targets for next written assignment:
English Literature AS

             Unit 3 Assessment: 20th Century
Name:                                             Tutor:
AO                  Criteria                 Comment
            Grasps significance of
             differing critical
             opinions in title
            Uses more than one
            Confident judgement of
            Personal response
            Conceptual approach
            Overview
            Clear, accurate
            Relevant response well-
 1           focused on task
 2          Methodical, structured
 3           approach
            Develops an argument
            Analyses detail
            Focuses on presentation
Mark:                      MEG:          Target mark:

     A        Targets



David Rowbottom
                                 Self Assessment of Language Skills
For each substantial piece of work mark the number of times each error has occurred and use this
to action plan your improvement.

Type of error                        Homework date and title/ Number of errors of
                                     that type

past part. agreement
gender of noun
use of le, la, un, une
adjective agreement
Adjective in wrong place
Not a French word!
use of negative
order of pronouns
Pronoun in the wrong place
ue of qui, que etc
Failure to spot ce qui etc
use of subjunctive
use of imperative
Write other error types below!
Butler 1988:                                                             Able

Experimental group: 1/2 in the top quartile
                              1/2 in the bottom quartile


Were given tasks:    (tasks were new to students)
            some tasks were divergent some convergent
            tasks were given over three separate sessions
            grading criteria were given beforehand
       Tas k 1   feedback 1      Tas k 2    feedback 2      Tas k 3     feedback 3

And feedback:
   1/3 of students were given comments only

     1/3 of students were given only grades

     1/3 of students were given grades + comments.

Who learned best?

comments only:     performance improved by 33%
grades only:       performance declined significantly
comments + grades: performance declined, particularly on
                   the convergent task

Who maintained interest?
low achievers: interest was undermined by grading (with or
                without comments)
high achievers: interest was sustained at a high level
Moral: grading degrades learning. But students need grades! So: withhold grades and
give them alltogether every 6-12 weeks? Give students their grades four weeks ‘late’?
Discussion of Black and Wiliam‟s review
How do Black and Wiliam‟s findings differ from conventional practice?

Conventional practice: “teach, test, grade, and move on” (assessment is summative)
Too often the teacher teaches a topic, sets some work, grades it, criticises it in a more or less
constructive way, but does not check that the student has made good any deficiencies. Then the
teacher moves on to the next topic.

A common assumption behind this approach is that learning quality and quantity depend on
talent or ability, and that the role of assessment is to measure this ability. If learning is wanting,
this is attributed to a lack of ability, flair or intelligence.

Best Practice: “find faults and fix”         (assessment is diagnostic)
Black and Wiliam‟s review suggests a different approach. A topic is taught, and some work is set.
The student and the teacher use this work to diagnose deficiencies and set targets for
improvement. This improvement is monitored.

The assumption is that learning quality and quantity depends on time and effort spent on
improvement, and that the role of assessment is to diagnose deficiencies so that time and effort can
be focussed on improvement. Indeed weaknesses are the very areas where greatest improvement
can be made with least effort. If learning is wanting, you need to try harder for longer.

Common Practice: teach, test, grade, and move on:
    100 %
                                             Criticise this        “You can‟t fatten a pig
                Can‟t do
                                                                   by weighing it”

                Can do                        Grade that


Best Practice: “Find faults and fix”

    100 %      Can‟t do                                                 Self-assessment
                                                Find this and
                                                                        Medals and missions
                                                get the student         Corrections
                                                to fix it               Targets for next
               Can do                                                    piece of work
                                                Praise this             Mastery learning
                                                                        Avoid grading
                                                                        Assessment
    0%                                                                   proformas.

Research is very much in favour of the „time and effort‟ assumption rather than the talent
assumption, even in areas like music where you might expect talent to be important. „Talent‟ is
often a by-product of how much time and effort has been spent on learning in the past, due
perhaps to intense interest.
Teachers in the East, for example in Pacific Rim countries, are puzzled by the Western obsession
with talent. Their very successful education systems are built firmly on the „find faults and fix‟

Find a summary by Black and Wiliam on their research, with full references at:
Geoff Petty 2001
      Judgmental Feedback                                            Informative Feedback
        “Here is my measurement”                                “These are your goals, this is what you
                                                                 do well, and this is how to get better”
                                   Characteristics of this Feedback

The feedback compares students with each                     There are clear assessment criteria and
other, and encourages them to compete. It is                 goals. Feedback consists of information about
„norm referenced‟.                                           the extent to which these have been met.
The teacher gives grades, marks, and                         There are:
comments that make conscious or                              Medals: for what they have done well
unconscious comparisons with others                          Missions: showing how to improve

                                          Effect on self-esteem

Judgement makes students nervous and                         The student feels accepted, and that their
protective of their self-esteem. So students                 efforts are being recognised and valued
avoid risks and challenges.                                  Self esteem and commitment tends to rise
The self-esteem of high achieving students                   and there is increased emotional
                                                             involvement in tasks

                                 Consequent learning strategies
                                                                     Effective learning strategies
       Maladaptive learning strategies
                                                             Deep learning. Their eyes are on the goals,
Surface learning is likely. Their eye is on the
                                                             assessment criteria, tasks, and their
grade, not understanding, learning or the
                                                             „missions‟. High quality learning aimed at
task. The student memorises, seeks short
                                                             understanding and improvement.
cuts, copies etc.
                                                             As esteem comes from effort, not
Right answer syndrome
                                                             comparative attainment, students are
                                                             prepared to take risks and accept challenges.

                                     Student's‟ learning theory

 Maladaptive and blaming learning theory                        Adaptive and blame free learning theory
„mistakes are shameful‟                                      „Effort is the key and it‟s up to me‟
„effort shows you must be stupid‟                            mistakes are informative feedback
„ability is the key and it is inborn‟                        Instrinsic motivation: Learning is an end in
Extrinsic motivation: It‟s only worth working if             itself
you get something out of it

                                          Effect on low achievers

There is reduced: interest, effort,                        There is increased:interest, effort, persistence,
persistence, self-esteem and self-belief, and              self-esteem and self-belief.
less emotional investment in learning
                                                           In time: Learned resourcefulness: “There
In some cases: “Learned helplessness”                      must be a way round my difficulties and if I find
“No matter what I do I‟m bound to fail”                    it I will succeed.”
The student withdraws and retires hurt,                    “Learning depends on time, effort, corrected
rejecting the teachers, college, etc                       practice, and using the right strategies.”
Hostility towards learning                                 Identification with the aims of the course

Learning is seen as something for others.                  Learning is seen as an end in itself

                    This document summarises some key ideas in the Black and Wiliam review 1998
                    “Assessment and classroom learning‟ in the journal „Assessment in Education‟

Shared By:
Jun Wang Jun Wang Dr
About Some of Those documents come from internet for research purpose,if you have the copyrights of one of them,tell me by mail you!