YiA EVS Quality Assessment Guidance Notes - Draft

Document Sample
YiA EVS Quality Assessment Guidance Notes - Draft Powered By Docstoc
					                                    2011 v1




Youth in Action
     Youth Exchanges
       (Action 1.1)



Application Assessment Guidelines

               for

   External Quality Assessors




                                       1/17
                                                                  2011 v1


CONTENTS



Assessments and Evaluations

The Quality Assessment form

How to complete the form

Youth Exchanges - specific information for assessors

Youth Exchanges involving young people with fewer opportunities



ANNEX I : Youth Exchanges - Specific information for assessors

ANNEX II : YiA Objectives and Prioritities




                                                                     2/17
                                                                                                 2011 v1


Assessments and Evaluations

The life cycle for all Youth in Action applications involves the following assessment and evaluation
stages:

 1. Eligibility checks
    All applications submitted on time to an official YiA deadline will go through an official eligibility
    check, through which basic programme criteria are confirmed. These checks are carried out
    by the UK National Agency.

 2. Quality assessments
    Those applications that successfully pass the initial eligibility check must undergo a further
    quality assessment. This is carried out on the basis of award criteria published in the YiA
    programme guide and on the UK National Agency website.

    Official quality assessment forms (with a predefined points-based system) are used by all
    Youth in Action National Agencies and minimum programme requirements specify that at least
    one quality assessment must be used for each application. In the UK each application will
    undergo at least two quality assessments.

 3. Proposed funds and financial capacity checks
    The grant requested for each application undergoes a formal assessment in relation to the
    activities proposed. This assessment is carried out by the UK NA and takes into account
    proposed activities, application quality and specific grant allocation rules.

    In addition, financial capacity checks may be carried out by the UK NA on first time applicants,
    and any grant requests exceeding €25,000 (excluding public bodies).

 4. Evaluation
    Following the quality assessment exercise, the UK NA prepares a list where the results are
    sorted in order of individual merit. The results are derived from the average total score of two
    assessments per application. Third internal assessments are carried out when the difference
    between two final scores is greater than 20 points and when a project is separately assessed
    as ‘approved’ and ‘rejected’ by the two original assessors.

    An Evaluation Committee will convene within 2 months of the official application deadline. The
    Committee will consist of at least three members representing at least two organisational
    entities with no hierarchical link between them. The UK Youth in Action Evaluation Committee
    will include key stakeholders and representatives from the following:
        The Devolved Administration Governments in Scotland, Wales and Northern Ireland.
        The statutory and non-governmental youth sector in the UK.
        Other British Council-managed youth programmes.
        The British Council’s UK Directorate.
        The UK National Agency(ies) for the Lifelong Learning Programme.

    In addition to the above, National Agency staff who have been involved in the assessment
    process will attend in an advisory capacity (but will not participate in any decision-making
    processes)

 5. Final Decision
    Members of the Evaluation Committee sign a final grant award proposal which is considered
    by the Director of the National Agency, who makes the final grant award decision.



                                                                                                     3/17
                                                                                              2011 v1


The Quality Assessment form
The official quality assessment form consists of 5 parts, of which 3 refer directly to the assessment
of application quality based on predefined Award Criteria. For each of these three parts a score
and accompanying narrative justification is required. It is important that each narrative clearly
demonstrates and justifies the points awarded for each section.

The key quality related sections and their respective weighting (as a percentage) can be seen
below:

                Relevance to the objectives of the programme            30%
                Quality of the project and methods applied              50%
                Profile of participants                                 20%


The predefined score categories awarded for the majority of sections are:

                   Points      Descriptor
                  20 or 10     Excellent
                  16 or 8      Very Good
                  12 or 6      Good
                   8 or 4      Adequate
                   4 or 2      Poor


In addition, the following score categories exist for the assessment of objectives, specific sub
action aims and the profile of participants:

                  10 or 5      YES
                     0         NO




                                                                                                   4/17
                                                                                             2011 v1


How to complete the form
The quality assessment form consists of 5 separate parts, all of which need to be completed by the
appointed Quality Assessor. It is important to be as clear as possible throughout the assessment
as both the project description and assessment summary will be presented to the Evaluation
Committee. In addition, the entire assessment will be used as feedback to unsuccessful applicants
who may wish to reapply. Any criticism must be constructive and polite.

Please note: before you begin to complete the assessment form, it is very important that you read
the entire application first.

For minimum standards in terms of length and quality of justifying narrative, please see Annex III:
Sample – EVS Quality Assessment.

The table below provides an overview of the quality assessment form:

            Part I       Identification of the application
            Part II      Assessment Conclusions
            Part III     Brief description of the project
            Part IV      Assessment of Award Criteria
            Part V       Summary of Quality Assessment


Part I – Identification of the Application

This section contains basic data which identifies the applicant with a unique reference number
(located on the first page of the application starting with ‘GB’ followed by the relevant Action, a
unique identifying number, the year and finally the relevant selection round – e.g. GB-11-36-2011-
R1).



Part II – Assessment Conclusions

This section names the relevant Quality Assessor and is verified by a date and signature. It also
provides the overall final scoring once all relevant sections have been completed and scored. The
UK National Agency requires an electronic version and a hard copy in the post.



Part III – Brief Description of the Project

This section provides a brief summary of the entire project, including basic information about the
applicant, relevant partners, length and nature of the project – aims and activities.

This information can usually be copied directly from the original application form although at times,
assessors might have to add additional information in order to portray an accurate project
summary.




                                                                                                 5/17
                                                                                                   2011 v1


Part IV – Assessment of Award Criteria



1. Relevance to the Objectives and Priorities of the Programme

     In this section, assessors should list the key objectives, specific Action aims and priorities of the
     project as indicated by the applicant and comment on how well these are reflected throughout
     the project. Applicants are likely to have ticked more than one box on the application form; but
     please note: - it is not the quantity of priorities or objectives that is important but how well they
     are reflected throughout the project.

     Questions to ask: How well does the entire application meet the aims and objectives that they
     have selected/ticked on the application form? And are they clearly described?

     It is important that the applicant does not treat themes, priorities and objectives in isolation.
     They should form an integral part of the whole project.

     Questions to ask: Is the link clear between themes, priorities and objectives and is it
     demonstrated in the activities? Does this also follow through to things such as dissemination of
     results and visibility?

     This section can be awarded a total of 30 points (out of 100) which are broken down as follows:

         1. General objectives of the programme 1– 5 points

         2. Specific aims of the sub-Action2 – 5 points

         3. Permanent and annual priorities – 10 points

         4. Demonstrated relevance to a national priority – 10 points




1
    See Annexe II for programme objectives, permanent and annual priorities
2
    See Annexe II for specific aims of Action 1.1


                                                                                                         6/17
                                                                                           2011 v1


The table below should act as an aide when deciding which scoring level to choose for point 3 -
permanent and annual priorities:

Permanent and Annual Priorities:

 Points           Descriptor
 10 – Excellent   A clear and very strong demonstration of how the project meets a number
                  of permanent and annual priorities in the application, with detailed evidence
                  throughout the application, from support and community impact to tasks,
                  learning outcomes and visibility .The quality of the applicant response is
                  very high and detailed.
 8 – Very Good    A clear and good demonstration of how the project meets a number of
                  priorities in the application, from support and community impact to tasks,
                  learning outcomes and visibility. The quality of the applicant response is
                  very good but could be enhanced by more details.
 6 – Good         The applicant has identified permanent and annual priorities, and they are
                  present in most application sections but not all. The applicant response is
                  relatively clear but could be more detailed
 4 – Adequate     The applicant has identified permanent priorities and there is some
                  mention/link made in one or two other sections of the application form.
                  However, the applicant response is not very clear or detailed.
 2 – Poor         Priorities have been identified but are not reflected or indeed referred to in
                  any other part of the application.


Relevance to a UK National priority (relevance to UK youth public policy):

No specific UK National Priorities have been set for 2011. It is possible for applicants to gain
extra points by evidencing a clear link between their project idea and a relevant UK youth-
related public policy or strategy, whether at national, regional or local level. This might be a UK
government policy (e.g. a link to the Big Society agenda), a policy area specific to one of the
UK nations, or a local government initiative. Examples of this are available on our website
(http://www.britishcouncil.org/youthinaction-objectives-and-priorities.htm).

In order to be awarded the extra points available for a project’s relevance to UK youth public
policy, the application should clearly include verifiable evidence of each policy area you
reference, e.g. the relevant Northern Ireland, Welsh, Scottish or English government agency
web address and/or a link to the relevant policy document. Each specific national policy area
should then be reflected in the nature and design of the project as described below.

 Points           Descriptor
 10 – Excellent   A clear and very strong demonstration of how the project meets a national
                  priority in the application; the applicant has sourced and quoted relevant
                  publications or documents to evidence this as a priority and then drawn a
                  link to their own project’s aims in the project summary. There is further
                  evidence throughout the application of how the project will address this
                  priority, with a clear and obvious harmony between the activities planned
                  and the results expected as a contribution to the priority .The quality of the
                  applicant response is very high and detailed.


                                                                                               7/17
                                                                                                2011 v1


    8 – Very Good     A clearly identified priority is linked well to the aims of the project. The link
                      between this context and the activities is clear but the outcomes or impact
                      could be greater or linked more clearly to the priority.
    6 – Good          The applicant has identified a national priority and linked it to the aims of
                      the project. The activities and outcomes from the project could be better
                      linked to the priority or to show how they are contributing to the priority. The
                      applicant response is relatively clear but could be more detailed.
    4 – Adequate      The applicant has identified a national priority and linked it to the aims of
                      the project. The activities and outcomes from the project don’t always link
                      clearly to the priority or show how they are contributing to the priority. The
                      applicant response is lacking in detail or more detailed planning.
    2 – Poor          Priorities have been identified but are not reflected or indeed referred to in
                      any other part of the application.




2. Quality of the Project and Methods applied

  This section is the most important part of the award criteria assessment, as it covers everything
  from support, partnerships, tasks and training, to development, European dimension and
  visibility. It carries 50% of the overall score awarded to any 1.1 application.

  This section can be awarded a total of 50 points (out of 100) which are broken down as follows:

        a) The quality of project design – 20 points

        b) The quality of project content and methodology – 20 points

        c) The quality of project reach – 10 points



  a) The quality of project design

  This section asks Assessors to reflect on the quality of partnerships; the preparation phase;
  activity programme; evaluation phase; and risks regarding the protection and safety of
  participants.

  Questions to ask:

  Partnerships: Is the relationship between promoters clearly described throughout the project?
  Is there evidence of a solid partnership between all groups, with clear roles/ responsibilities?

  Preparation: Are participants and leaders going to be adequately prepared for the exchange?

  Activity Programme: Does the programme of activities link to the objectives, themes and
  priorities chosen by the applicant at the beginning of the application form? Does the programme
  use a variety of non-formal learning methods?

  Evaluation phase: Has adequate evaluation and reflection time been built into the programme
  content? Is there adequate evaluation/follow-up taking place following activity completion?


                                                                                                    8/17
                                                                                          2011 v1



Protection and safety: Have adequate health & safety/risk prevention measures been
considered for all participants?

The table below should act as an aide when deciding which scoring level to choose:

 Points          Descriptor
 20 –            There is very strong and detailed evidence of all partner groups involved in
 Excellent       project preparation, design and implementation, with roles and
                 responsibilities clearly defined at all stages. Programme activities clearly
                 illustrate how objectives, themes and priorities are explored on a daily basis
                 using a wide range of creative non-formal learning methods with clearly
                 defined learning outcomes. Health and safety is addressed rigorously and in
                 great detail. Evaluation and feedback is fully integrated at every project
                 stage. This is the ideal youth exchange – may be used as a case study for
                 good practice.
 16 – Very       There is clear evidence of a solid partnership between promoters throughout
 Good            the project with clear roles/responsibilities outlined. The programme reflects
                 themes, objectives and priorities and uses non-formal learning methods with
                 time allocated for evaluation/follow-up. Health & safety has been addressed
                 in detail.
                 More creativity and varied forms of non-formal learning methods in the
                 programme, linked to more detailed young people participation approach
                 would make this an excellent project.
 12 – Good       There is evidence of a good partnership between promoters which
                 addresses preparation, programme implementation, non formal learning
                 methods, evaluation and H & S. There is also evidence of
                 objectives/priorities reflected throughout. A more detailed approach,
                 innovation/creativity and thorough exploration/understanding and
                 implementation of the above would increase the score.
 8–              There is some evidence to suggest that all promoters have been involved in
 Adequate        all project stages and that objectives/priorities are reflected throughout the
                 programme alongside evaluation. There are statements referring to
                 evaluation, the use of non-formal learning methods and H&S, but not
                 consistently or in great detail

  4 – Poor       There is little or no evidence of a solid partnership, an activity programme
                 reflecting objectives or the use of non-formal learning methods. Promoter
                 roles are unclear and H &S has not been adequately addressed.



b) The quality of project content and methodology

Again this score reflects multiple elements, reflecting on whether the themes are of common
interest and relevance to the group of participants; using non-formal learning methods; with
active involvement and social/personal development of participants; as well as the intercultural
and European dimension of the project.



                                                                                                9/17
                                                                                          2011 v1


Questions to ask:

Interests and relevance to the group: Is there evidence that the project is of common interest
and relevance to the participants involved in the project?

Non-formal learning methods: Has a variety of non-formal learning methods been applied to
activities? Are they adequately reflected in the activity programme and elsewhere? Are they
relevant to the young people involved?

Active involvement of participants: Are all young people actively involved in the project from
preparation to evaluation? For those groups or individuals facing barriers to participation:- Have
these barriers been identified and addressed?

Promotion of social and personal development: Does the project aim to promote the social
and personal development of participants involved? Is there adequate evidence to that effect?

Intercultural dimension: Does the project encourage and promote intercultural dialogue
between participating groups? Will participants learn from each other?

European Dimension: Does the project offer participants the opportunity to identify common
values with other young people from different countries?

The table below should act as an aide when deciding which scoring level to choose:

 Points         Descriptor
 20 –           There is strong and detailed evidence of both programme subject and
 Excellent      methodology directly relating to and affecting young people involved. The
                project clearly promotes intercultural learning by actively encouraging
                participants to explore common topics/issues and identifying common values
                across Europe. Young people participate extensively and are involved
                throughout, and detailed confirmation of personal and social development as
                a result. An extensive variety and range of methods which clearly propagate
                non-formal learning aimed at young people is
                present throughout the activity programme.
 16 – Very      There is strong evidence that the project relates to young people and
 Good           explores intercultural learning by tackling common topics/issues across
                Europe. Young people are involved throughout the project and distinct
                learning outcomes are identified as a result. A variety of non-formal learning
                methods is implemented as part of the programme. A higher score would
                require more detail, innovation and a clearer path of learning outcomes for
                young people covering personal and social development.

 12 – Good      There is a decent amount of evidence for all key areas such as relevance of
                project to young people, active participation, learning outcomes, non-formal
                methods, and the existence of an intercultural and European dimension. A
                higher score would provide more and clearer evidence on most areas in order
                to raise the overall project quality. Greater innovation and synergy between all
                project areas is also needed.

  8–            There is some evidence to suggest that all key areas are implemented by


                                                                                             10/17
                                                                                           2011 v1


 Adequate         promoters. There are individual statements that for example highlight active
                  participation of young people and learning outcomes. However, evidence is
                  not always detailed (at times patchy or inconsistent) and does not easily
                  present a coherent picture of the entire project.

  4 – Poor        There is little or no evidence to show key areas are consistently implemented
                  by promoters. Young people involvement is not evident and intercultural and
                  European dimension minimal.



c) The quality of project reach

This section asks Assessors to reflect on wider project implications related to impact and
multiplying effect; visibility of the project and the YiA programme and dissemination and
exploitation of results (DEOR).

Questions to ask: Does the project have an impact on the local community during the
implementation of the exchange, and will it continue after the project has ended? Is the project
visible at a local /regional/national level? Has the visibility of the YiA programme increased as a
result of this exchange? Is there a DEOR strategy?

The table below should act as an aide when deciding which scoring level to choose:

 Points               Descriptor
 10 – Excellent       The application addresses impact, multiplying effect, visibility and DEOR
                      in great detail. There is a coherent and creative plan implemented
                      throughout the project and beyond, which reaches large and new
                      audiences and uses a variety of creative multimedia tools. Project results
                      continue long after the project has ended and young people’s
                      participation is at the core.

 8 – Very Good        The application addresses impact, multiplying effect, visibility and DEOR.
                      There is a long term plan which permeates the project and beyond. It
                      reaches a wider audience than the immediate environment and uses
                      multimedia tools. Greater detail and more creativity would make this an
                      excellent project.
 6 – Good             The project details short term impact, benefits and visibility in the
                      immediate environment and project results are disseminated through e.g.
                      an already existing website, local newspaper or newsletter. Throughout,
                      the YiA programme is also promoted and clearly visible. Long term
                      impact and DEOR activities could be further elaborated upon.
 4 – Adequate         There is some interaction with the local community but no evidence of
                      real impact, integration or indeed benefits. An isolated activity such as an
                      information booth at a seminar is briefly mentioned, but it is not part of a
                      wider plan or strategy.
 2 – Poor             There is very little impact in the local community and no visibility for the
                      YiA programme or the project. There is no evidence of DEOR activities.



                                                                                              11/17
                                                                                                   2011 v1


3. Profile of Participants

     One of the permanent priorities of the YiA programme is the inclusion of young people with
     fewer opportunities3 and this is also reflected in the application assessment. How many times
     an individual promoter has applied for funding and the need to balance the number of promoters
     and participants are additional factors in the assessment of participants’ profiles.

     This section can be awarded a total of 20 points, which are broken down as follows:

       Award criteria         Points       Descriptor
                              Score
       Involvement of          5 points    Award 5 points if the application involves young people with
       young people with          or       fewer opportunities as defined on page 5 of the 2011 YiA
       fewer opportunities                 programme guide.
                               0 points    Award 0 points if the application does not involve young
                                           people with fewer opportunities as defined on page 5 of the
                                           2011 YiA programme guide.
       Multilateral -          5 points    Award 5 points if:
       number of partner
       promoters involved                      o   for a multilateral project, the number of partner
       exceeds four OR                             promoters exceeds four
       Bilateral - the            or           o   for a bilateral project, the promoter is a first-time
       promoter is a first-                        applicant or a small-scale actor (if this is clear from
       time applicant or                           the application information)
       small-scale actor

                               0 points    Award 0 points if neither of the above statements are true
       Number of               5 points    Award 5 points if for the total number of participants there is
       participants per                    a variance in group size of no more than five participants
       promoter - variance                 per partner group.
       of no more than            or
       five


                               0 points    Award 0 points if for the total number of participants there is
                                           a variance in group size of more than five participants per
                                           partner group.
       Gender balance –        5 points    Award 5 points if for the total number of participants there is
       variance of no                      a variance between the numbers of males and females of
       more than five                      no more than five per group.
                                  or
                                           Award 0 points if for the total number of participants there is
                                           a variance between the numbers of males and females of
                               0 points    more than five per group or where the group is single sex.



3
    Please see 2011 programme guide page 5 for a definition


                                                                                                     12/17
                                                                                                      2011 v1


Part V - Summary of Quality Assessment

The final section to complete is an overall assessment of the project, summarising previous
sections, and highlighting the main strengths and weaknesses of the entire application. Assessors
should avoid empty statements such as ‘good support’ and instead explain why it is good.

It is important to ensure that the assessment summary accurately justifies the overall score (the
total points score can be found in part II).

For example: a project with an overall score of 95 (out of 100), should clearly reflect what makes
this project outstanding and near perfect. And similarly, a project with an overall score of 35 (out of
100) should provide a clear list of weaknesses which need to be addressed should the applicant
wish to reapply in the future.




        Important - welfare of young people: Support and welfare of young people is
        essential in any YiA project. Based on application information – an assessor should
        be confident that the applicant has addressed relevant welfare issues in order to
        ensure that all young people involved are safe. Any concerns that these have not
        been addressed adequately must be clearly communicated in the summary of the
        assessment. Welfare concerns will be raised in the Evaluation Committee and
        followed up by the UK NA.



Finally, it is important to remember that constructive feedback and suggestions for quality
improvements are welcome for all projects, regardless of the final score. The UK National Agency
would also urge assessors, if appropriate, to recommend projects for potential monitoring visits 4.

In addition please bear in mind that the summary assessment will be made available to the
Evaluation Committee and as such should be clearly and well written.




4
  Monitoring visits (site visits during project implementation) are used to support the beneficiary, gather and
disseminate examples of good practice, and establish and maintain good relations between the beneficiary
and the NA. Projects may be chosen for a number of reasons such as being a first time applicant (may need
additional support) or being a very innovative/creative project (used as a good practice example). In addition,
the NA will ensure there is a balanced spread of the sample of projects to be monitored, by selecting
applicant organisations of varying size, type and geographical location.


                                                                                                         13/17
                                                                2011 v1


ANNEX I: Youth Exchanges - Specific Information for Assessors



1. A Youth Exchange does not include:
   academic study trips

   exchange activities which can be classed as tourism

   festivals

   holiday travel

   sports competitions

   school class exchanges

   language courses

   performance tours

   exchange activities which aim to make financial profit

   statutory meetings of organisations

   political gatherings

   work camps




                                                                  14/17
                                                                                             2011 v1


Youth Exchanges involving young people with fewer opportunities

Participants taking part in youth exchanges may be disadvantaged geographically, socially,
culturally or financially.

Any youth exchange involving young people with fewer opportunities needs to pay particular
attention to the following:

      Preparation: How will the participants be prepared for the project? Would an Advanced
       Planning Visit be useful?
      Support: The applicant may have considered applying for Exceptional Costs to further
       support members of the groups.
      Risk: Are there any particular risks identified for particular groups (e.g. disabilities).
      Programme of activities: The activity duration should offer all participants valuable
       learning experiences. Learning outcomes such as team work and confidence building must
       be clearly defined.




                                                                                               15/17
                                                                                                2011 v1


Youth Exchanges – The Basics


A Youth Exchange is a project which brings together groups of young people from two or more
eligible countries, providing them with an opportunity to discuss and confront various themes whilst
learning about each other’s countries and cultures.

Youth Exchanges aim to contribute to the personal development of the young people taking part.
Exchanges are open to all young people from the UK, regardless of their background, education or
socio-economic status.

Promoters: A bilateral project has 2 promoters, a trilateral 3 and a multilateral 4 or more. Promoters
must be from an eligible programme country and can be either a non-profit/non governmental
organisation, local/regional public body or an informal group of young people.

Participants: Young people aged 13-25 who are legally resident in a programme country. The
Youth in Action Programme encourages good quality projects actively involving young people with
fewer opportunities.

Group size: The minimum group size per promoter is 8 for bilaterals, 6 for trilaterals and 4 for
multilaterals. There can be no more than 60 participants in total, not including group leaders.
Groups should be balanced in terms of the number of participants and consideration given to
gender balance.

Duration: 6-21 days, excluding travel.

Group leaders: The UK promoter must nominate two (no more, no less) group leaders (one male
one female) whose task is to oversee the smooth running of the project. For promoters from other
countries there is a minimum of one group leader and maximum of two.


Youth Exchanges – Action specific objectives

Youth Exchanges allow one or more groups of young people to be hosted by a group from another
country in order to participate together in a joint programme of activities. These projects involve the
active participation of young people and are designed to allow them to discover and become aware
of different social and cultural realities, to learn from each other and reinforce their feeling of being
European citizens.




                                                                                                   16/17
                                                                                            2011 v1


ANNEX II: YiA Objectives and Priorities

Youth in Action Objectives:
   promote young people’s active citizenship in general and their European citizenship in
    particular
   develop solidarity and promote tolerance among young people, in particular in order to foster
    social cohesion in the European Union
   foster mutual understanding between young people in different countries
   contribute to developing the quality of support systems for youth activities and the capabilities
    of civil society organisations in the youth field
   promote European cooperation in the youth field.


Youth in Action Permanent Priorities:
   European Citizenship
   Participation of Young People
   Cultural Diversity
   Inclusion of young people with fewer opportunities


2011 Annual Priorities:
   European Year of Volunteering
   Youth unemployment
   Inclusive growth
   Global environment challenges and climate change
   Creativity and entrepreneurship
   EU-China Year of Youth (Executive Agency or EACEA projects, for Action 2 and sub-Action
    3.2 only)

For more details please see the 2011 Youth in Action Programme Guide page 6, available for
download at http://www.britishcouncil.org/youthinaction.




                                                                                               17/17

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:6
posted:3/7/2012
language:English
pages:17