Docstoc

Evaluation tools

Document Sample
Evaluation tools Powered By Docstoc
					                                                                 EVALUATION TOOLS
                                                                                                                                Examples and more
Evaluation tool         Advantages              Disadvantages                          General advice                              information



Individual         In depth data can be      Skills needed to         Practicalities: decide where you will carry out the   E.g. to a young participant: ‘if
interviews         gathered                  conduct interview        interviews, be clear about timing (so your            your best friend said they
                                                                      interviewees know how long they are needed)           wanted to get involved in a
                                                                      and provide refreshments.                             similar project what advice
                                                                                                                            would you give them? or ‘what
(in person or by   Responsive to             Time consuming           Different target audiences need different             would you tell a friend about
phone)             unexpected outcomes                                approaches e.g. asking a young participant            this project?’
                   (can clarify and probe)                            questions vs asking staff from your partner
                                                                      organisation questions                                E.g. to a member of staff from
                                             Requires participants                                                          your partner organisation you
                                             to be articulate                                                               might ask ‘what do you feel
                   Private                                                                                                  your organisation got (is
                                                                                                                            getting) out of the project’ or
                                                                                                                            ‘what in your view were (are)
                                                                                                                            the strengths and
                                                                                                                            weaknesses of the project?’

                                                                                                                            See Interview Questions
                                                                                                                            Family (Milestones)

Focus Groups       In depth data can be      Skills needed to run     Practicalities: decide where you will carry out the   See Focus Groups (Kate
                   gathered                                           focus group, be clear about timing (so your           Pontin)
                                             Requires objectivity     interviewees know how long they are needed)
                   Responsive to             by facilitator           and provide refreshments.
(small group       unexpected outcomes
discussions)       (can clarify and probe)   Generally include only   Group discussion;
                                             a small number of
                                             people                   http://www.evaluationforall.org.uk/toolkit/disc
                                                                      ussion.html (the Scottish Arts Council
                                             (8-10 adults,            Evaluation Toolkit)

                                             4-6 children)            It is good practice to have two people running a
                                                                      focus group, one to ask the questions and

                                                                                                                                                        1
                                                               EVALUATION TOOLS
                                                                                                                                   Examples and more
Evaluation tool        Advantages              Disadvantages                            General advice                                information



                                            Requires participants     facilitate discussion, the other to take detailed
                                            to be articulate          notes and to operate recording equipment

Personal          Provides very rich data   Skills needed to          Used to establish the knowledge and attitudes            Personal meaning mapping
meaning                                     conduct this and          visitors have about a topic before and/or after          was used at Bodiam Castle
mapping                                     analyse data              participation in an activity or visiting an exhibition   with children who were asked
                                                                      for example. It is a very visitor-centred approach.      to respond to the word ‘castle’
                                                                      Participants are presented with a sheet of paper         before and after a visit.
                                                                      with a word or image in the centre and asked to
                                            May affect learning       note down (or draw) anything they can think of to        Alison James /Museum
                                            outcomes by               do with this. The evaluator then clarifies these         Consultant (Audiences) often
                                            participants if asked     words/phrases /drawings and makes additional             uses this tool
                                            before as well as after   notes on the sheet using a different coloured            http://www.alison-
                                            an activity.              pen.                                                     james.co.uk/evaluation.htm.

                                                                      As the participant finishes the activity or leaves       See Report on Personal
                                                                      the display they are approached again and                Meaning Mapping (Sarah
                                                                      asked if there is anything they would like to add        Lewis, Cultural Entitlement
                                                                      or amend. The evaluator again clarifies these            Officer, Oxfordshire County
                                                                      additions and makes their own notes on the               Council)
                                                                      sheet. Different colours of ink are used for each
                                                                      stage.




Photography,      Directly includes         Can be expensive          Needs to be structured to ensure useful evidence
video and audio   participants – can                                  is collected
                  become part of the
                  activity                                            i.e. photographs taken to illustrate the ‘before’
                                            Requires skills           and ‘after’ which relate to what you are aiming to
                  Not dependent on                                    evaluate.

                                                                                                                                                          2
                                                            EVALUATION TOOLS
                                                                                                                         Examples and more
Evaluation tool            Advantages       Disadvantages                           General advice                          information


                     literacy skills

                     Provides a visual
                     record

Written diaries /    Simple and          Needs to be               It is a good idea to provide participants with
scrapbooks or        inexpensive         structured to ensure      templates with headings or key questions that
logs                                     useful evidence is        will enable participants to reflect on what you are
                                         collected in order to     aiming to evaluate e.g.
                                         capture participants’
                                         changing thoughts
                                         and feelings
                                         throughout the
                                         project.

(written             Can include         Requires literacy skills What have I learnt/found out today?
individually or as   photographs and
part of a group      pictures.           Participants may be       What did I enjoy most today? Why?
by participants                          discouraged if their
and project team)                        diaries are going to be   What did I enjoy least today? Why?
                                         public. You may
                                         consider putting          Remember – diaries shouldn’t be made public
                                         headings on a large       without permission if specifically for evaluation
                                         sheet of paper and        purposes.
                                         participants could
                                         copy extracts from        It is good practice for members of the project
                                         their diary onto sticky   team to keep diary/notes about the process of
                                         notes to add under        their project and to include what they think they
                                         the appropriate           could do better next time.
                                         headings.




                                                                                                                                             3
                                                           EVALUATION TOOLS
                                                                                                                           Examples and more
Evaluation tool         Advantages         Disadvantages                           General advice                             information



Comment boxes       Simple and          Requires literacy skills Practicalities: think about the location for books,
/ books / board     inexpensive         (only if asking for       boards etc., make sure you enough pencils.
and verbal                              written comments)
comments to         Comment boxes can                             You can also ask people to consider drawing
front of house      be private          You have to be aware instead of writing (but this may be harder to
members of                              of bias as this tool      analyse as you will not be there to ask them to
staff                                   allows the participants explain their drawings).
                                        to self-select to fill in
                                        comments

                                        Prompt questions are
                                        needed (e.g. asking
                                        people to comment on
                                        something in
                                        particular that relates
                                        to what you want to
                                        evaluate)

Graffiti walls      Simple and          May rely on literacy                                                           See Graffiti Walls (Kate
                    inexpensive         skills                                                                         Pontin)
(Participants are
invited to write    Can be anonymous    Prompt questions are
comments or                             needed (e.g. asking
draw on a big                           people to comment on
piece of blank                          something in
paper on the wall                       particular that relates
or write or draw                        to what you want to
on post-it notes                        evaluate)
to stick on the
wall)




                                                                                                                                                  4
                                                                  EVALUATION TOOLS
                                                                                                                                 Examples and more
Evaluation tool        Advantages                 Disadvantages                         General advice                              information



Drawings or       Simple and                   Skills needed to        Needs to be structured to ensure useful evidence
other artistic    inexpensive                  analyse data            is collected e.g. by using prompt questions
responses (e.g.                                                        relating to what you are aiming to evaluate
modelling)        Do not rely on literacy      Some people are not     or/and by using drawings to illustrate the ‘before’
                  skills                       confident about         and ‘after’.
                                               drawing (particularly
                  Good for children            adults!)

                                               May need artists to
                                               explain their work.

Observation /     Can give in-depth            Time consuming          Observing visitors                                    http://www.swmlac.org.uk/
tracking /        insight if structured well                                                                                 mli/eval/obs_vis.htm with
accompanied                                    Needs to be             Accompanied visits are when a staff member            two examples of observation
visits            Can be carried out by        structured to ensure    accompanies a group of visitors around a              forms (South West Museum
                  stewards present in a        useful evidence is      museum as a fully integrated member of the            Library and Archive Council)
                  gallery/room.                collected.              group.

                                               Be clear about what     To avoid just describing what your participants
                                               you are aiming to       are doing make specific headings on a checklist
                                               evaluate                relating to what you are aiming to evaluate e.g.
                                                                       confidence, inter-generational interaction to
                                               Can be intrusive        focus your attention. You can track change by
                                                                       undertaking observation with the same group of
                                                                       participants at different times.




                                                                                                                                                       5
                                                              EVALUATION TOOLS
                                                                                                                            Examples and more
Evaluation tool        Advantages              Disadvantages                         General advice                            information



                  Fun way of finding out    Needs to be              Think very carefully about the questions you are   Playback theatre: a
                  views, feelings,          structured to ensure     going to ask, they have to be very simple          technique where people act
                  attitudes                 useful evidence is       especially when you are using participatory        out responses to questions
                                            collected i.e. must      techniques with a group                            (e.g. ‘what was the worst thing
Participatory     Do not rely on literacy   relate to what you are                                                      about the workshop or
 techniques       skills                    aiming to evaluate                                                          project?’) and other group
                                                                                                                        members can step in to act
                                                                                                                        out alternative outcomes...
                                                                                                                        thereby suggesting solutions

                                                                                                                        Spotty frogs: For primary
                                                                                                                        children taking part in a
                                                                                                                        workshop on extinction and
                                                                                                                        conservation – a poster of 3
                                                                                                                        endangered tree frogs
                                                                                                                        climbing a twig – each child
                                                                                                                        was given a sticky dot and
                                                                                                                        asked to put their dot on the
                                                                                                                        poster on the top frogs if they
                                                                                                                        had had an excellent time, on
                                                                                                                        the middle frog if they had had
                                                                                                                        a good time and on the
                                                                                                                        bottom frog if they had not
                                                                                                                        enjoyed the workshop (NT)

                                                                                                                        Evaluation Targets: for
                                                                                                                        primary and secondary

                                                                                                                        Sticky webs: for secondary
                                                                                                                        pupils/adults to measure how
                                                                                                                        well the project has met
                                                                                                                        different aims.


                                                                                                                                                   6
                                                                    EVALUATION TOOLS
                                                                                                                             Examples and more
Evaluation tool         Advantages               Disadvantages                         General advice                           information



                                                                                                                         Timelines: for primary,
                                                                                                                         secondary. Adult to plot the
Participatory                                                                                                            highs and lows of the project
techniques
                                                                                                                         Evaluation scale: for primary
                                                                                                                         and secondary (NT / Youth
                                                                                                                         Involvement)

                                                                                                                         Hearts and Minds: for
                                                                                                                         primary and secondary to
                                                                                                                         explore the impact of the)
                                                                                                                         project on participants’
                                                                                                                         attitudes and feelings

                                                                                                                         Smiley faces: for primary

                                                                                                                         Smiley face / weather /
                                                                                                                         traffic light symbols (all can
                                                                                                                         be used when asking a group
                                                                                                                         a question to get their
                                                                                                                         response)




Questionnaires     Questionnaires can         If not filled in at       Practicalities: think about the location for     Summative questionnaires
                   gather qualitative data    project often not         questionnaires, make sure you enough pencils,    that combine a mixture of tick
                   if the questions are       returned                  decide whether you are asking people to          box and open ended
                   open-ended but                                       complete there or take away (if so make sure     questions for general visitors
(can be sent out   quantitative data if the   You have to be aware      you have your address on the questionnaire and   and schools (teachers and
                                                                                                                                                      7
                                                                 EVALUATION TOOLS
                                                                                                                            Examples and more
Evaluation tool         Advantages             Disadvantages                           General advice                          information


by post or email   questions are closed     of bias if you allow the   possibly attach an SAE), think carefully about   pupils) See Summative
or handed out at   (e.g. did you like the   participants to self-      accessibility of font and text size etc.         Questionnaire Key stage 2
events)            activity?).              select to fill in                                                           (Oxfordshire County
                                            questionnaires             How to design a questionnaire;                   Council),)
                   Can target a large
                   number of people         Rely on literacy skills    http://www.evaluationforall.org.uk/toolkit/que
                                                                       stions.html?tkrmid=32d2d25dc5241124f882fd        Example of summative
                   Generates standard       Have to be very            (the Scottish Arts Council Evaluation Toolkit)   questionnaires that use
                   results which can be     carefully written (it is                                                    mainly tick box questions to
                   compared                 easy to write loaded                                                        evaluate school museum
                                            questions)                                                                  visits
                   Can be anonymous                                    A step by step recipe for a successful
                                            Do not allow               questionnaire;                                   Summative Questionnaire
                                            measurement of                                                              Key Stage 2 (Brighton
                                            complex aspects of         http://swmlac.org.uk/mli/eval/q_recipe.htm       Museums and Galleries)
                                            learning                   (Ben Gammon for SWMLAC - the South West
                                                                       Museums, Libraries and Archives Council)         Example of a front-end and
                                            Less responsive to                                                          mainly summative
                                            unexpected outcomes        Examples of open-ended questions in              questionnaires that use open
                                            (cannot clarify or         questionnaires include:                          ended questions
                                            probe)
                                                                       I didn’t know that……It made me think that……I     Report on Front-end
                                                                       found out that……I was surprised that…….          Evaluation for the
                                                                                                                        Development of Loans
                                                                                                                        (Brighton Museums and
                                                                                                                        Galleries)

                                                                                                                        Teacher Feedback Form
                                                                                                                        (Brighton Museums and
                                                                                                                        Galleries)
Questionnaires
                                                                                                                        Young People Survey (Kate
                                                                                                                        Pontin)

                                                                                                                                                    8
                                            EVALUATION TOOLS
                                                                                Examples and more
Evaluation tool    Advantages   Disadvantages              General advice          information



(can be sent out
by post or email
or handed out at                                                            Example of summative
events)                                                                     questionnaires that use open
                                                                            ended questions and smiley
                                                                            face ratings for teachers and
                                                                            pupils Summative
                                                                            Questionnaire for Pupils
                                                                            (Chatham Historic Dockyard)

                                                                            Pupil questionnaire (can be
                                                                            used by either asking a class
                                                                            questions and scribing their
                                                                            responses or printed and
                                                                            allowing them to complete
                                                                            form.




                                                                                                      9

				
DOCUMENT INFO
Jun Wang Jun Wang Dr
About Some of Those documents come from internet for research purpose,if you have the copyrights of one of them,tell me by mail vixychina@gmail.com.Thank you!