Mid-term Evaluation Report by keralaguest

VIEWS: 3 PAGES: 103

									Mid Term Evaluation Report of the European
EQUAL Community Initiative Programme for
       Northern Ireland 2000-2006




                December 2003
                               TABLE OF CONTENTS

                                                                         Page No

1.         Introduction                                                     1
1.1        Structure of the Report                                          1
2.         Context: Overview of the EQUAL CIP in NI and ToR                 2
2.1        Equal Principles                                                 2
2.2        The key stages or actions of EQUAL                               4
3.         The Approach to the Evaluation                                   6
3.1        The evaluation approach                                          6
3.2        Evaluation Activities and Tools                                  8
4.         EQUAL within the Labour Market                                  12
4.1        Overview of the Section                                         12
4.2        Key Labour Market Trends                                        12
4.3        Regional Policy Context                                         13
4.4        Policy Links of Individual Projects                             18
4.5        Policy Proofing Conclusions                                     33
5.         Evaluation of the Key Principles of EQUAL                       34
5.1        Partnership                                                     34
5.2        Innovation and Partnership Learning                             46
5.3        Empowerment                                                     54
5.4        Mainstreaming                                                   57
5.5        Equal Opportunities                                             63
5.6        Transnational Cooperation                                       70
5.7        Thematic Approach                                               75
6.         Implementation and Delivery of EQUAL                            75
6.1        The Actions of EQUAL                                            75
6.2        Evaluation of the promotion and selection process               77
6.3        Role of the NSS – Proteus                                       90
7.         Achieving Objectives                                            94
8.         Recommendations                                                 95

Annex 1      The Thematic Priorities of EQUAL
Annex 2      Details of the Recruitment and Selection Process
Annex 3      Evaluation of the Key Eligibility Questions
Annex 4      The Partnership Health Check
Annex 5      Active Refection Session Methodology
Annex 6      The Partnership Health Check Results
Annex 7      Thematic Studies
Annex 8      Measuring Soft Outcomes – Additional Resources
Annex 9      Transnational Cooperation Healthcheck
Annex 10     Transnational Cooperation Healthcheck - Results
Annex 11     DP surveys
Annex 12     Questionnaire to Unsuccessful Applicants
Annex 13     Project Details - DISCRIM C
Annex 14     Terms of Reference
Annex 15     Bibliography
Annex 16     Glossary of Terms

(Note all Annexes are bound separately to the main body of the report)
                                      List of Figures

Figure (1)    Policy Learning Cycle

Figure (2)    Partnership Member‟s Position in Organisation

Figure (3)    Clarity of Purpose

Figure (4)    Role Clarity

Figure (5)    Partnership process

Figure (6)    Formation of Social Capital

Figure (7)    Understanding of Innovation

Figure (8)    Monitoring Qualitative Outcomes

Figure (9)    Arrangements for Empowerment

Figure (10)   Awareness of Need to Inform Policy and Practice

Figure (11)   Mainstreaming Outcomes and Processes

Figure (12)   Equal Opportunities Membership Profile

Figure (13)   Effective Communication
                                      List of Tables


Table 1    Innovation Assessment/Policy Map – ACCORD

Table 2    Innovation Assessment/Policy Map – Digital Step

Table 3    Innovation Assessment/Policy Map – Young Women

Table 4    Innovation Assessment/Policy Map – Futures

Table 5    Innovation Assessment/Policy Map – Diversity Matters

Table 6    Innovation Assessment/Policy Map – PPS

Table 7    Innovation Types

Table 8    Views on Innovation During Action 2

Table 9    Benefits of Partnership

Table 10   Contacts cited by DPs as part of networking

Table 11   Progress Summary of Beneficiary Recruitment

Table 12   Detailed Breakdown of Beneficiaries

Table 13   Progress against Targets

Table 14   Progress in Implementing key actions

Table 15   Transnational Activity

Table 16   Distribution of Applicant Organisations

Table 17   Target Groups and Geographic Areas

Table 18   Approaches to Assessment – A comparison

Table 19   Key „knowledge‟/ Project development gaps
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




Mid Term Evaluation Report of the European EQUAL
Community Initiative Programme for Northern Ireland
                     2000-2006



1.      INTRODUCTION

1.      This is the mid term evaluation report of the EQUAL Community Initiative
        Programme (CIP) for Northern Ireland. The evaluation is intended to be
        formative, that is, to provide information which will enable ongoing adjustment
        and refinement of programme and project activities. The evaluation is also
        intended to inform the second call of the EQUAL Community Initiative
        Programme. It is focused on process in addition to products; the impacts and
        outputs of the programme will be monitored by projects themselves and by
        programme support structures. This evaluation is focused on the way the
        programme works to deliver those outputs, not least, the effective mainstreaming
        of policy lessons emerging from the work.


1.1     Structure of the Report
1.      Section 2 provides the context for the evaluation results. It outlines the distinctive
        features of the EQUAL programme, in particular, the key Principles by which
        EQUAL operates. The purpose of the four stages or actions of EQUAL is also
        outlined.

2.      Section 3 outlines the evaluation approach and methodologies used to evaluate the
        EQUAL programme in Northern Ireland.

3.      Section 4 addresses the Labour Market and policy environment of EQUAL and
        assesses the potential contribution of EQUAL Development Partnerships (DPs) to
        policy and practice.

4.      Section 5 of the report focuses of the findings of the evaluation of the key
        principles of EQUAL (Partnership, Innovation, Empowerment, Mainstreaming,
        Transnationality, and Equal Opportunities).



                                                                                                 1

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




5.      Section 6 summarises the implementation and delivery of EQUAL in terms of the
        selection and setting up of DPs, the role of NSS in monitoring and support and the
        effectiveness of the monitoring system.

6.      Section 7 provides a summary of the progress that EQUAL has made on
        achieving objectives described in the monitoring indicators laid out in the
        Northern Ireland EQUAL CIP Annex 8.

7.      Section 8 summarises the recommendations for the ongoing implementation of
        the First Call and recommendations for the Second Call.



2.      CONTEXT:   OVERVIEW    OF   THE EQUAL
        COMMUNITY   INITIATIVE PROGRAMME   IN
        NORTHERN IRELAND AND THE TERMS OF
        REFERENCE FOR THIS EVALUATION
1.      The key features of the EQUAL programme are outlined here and provide the
        framework for this evaluation articulated in the Terms of Reference. (The Terms
        of Reference are located in Annex 14).

2.     Funded through the European Social Fund (ESF), EQUAL is intended to
       contribute to the European Employment Strategy and Social Inclusion Process by
       testing and promoting new means to combat all forms of discrimination and
       inequalities in the labour market both for those seeking work and those in work.


2.1     Equal Principles
1.      EQUAL is built upon a set of key principles which include:

2.1.1 A Thematic Approach

1.      EQUAL is based on nine thematic priorities drawn from the four pillars of the
        European Employment Strategy (see Annex 1). In Northern Ireland the EQUAL
        programme focuses on two of these - Employability and Equal Opportunities.
        Within each of these pillars EQUAL places emphasis on ensuring projects are
        capable of supporting a range of target groups. They should identify and address
        problems of discrimination and inequality common across social groups, rather
        than focus on one particular social group.


                                                                                                 2

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




2.1.2 Innovation

1.      EQUAL is intended to test and promote new ways to combat all forms of
        discrimination and inequalities by those groups most disadvantaged in the labour
        market, and in this regard must seek to be distinctive from the range of other
        policies and programmes aimed at improving levels of employment or equality. It
        seeks to address policy gaps or weaknesses and to avoid overlap with current
        policy initiatives.

2.1.3 Partnership

1.      EQUAL operates by bringing together key players in a geographic area or sector
        in Development Partnerships (DPs). DPs select one of the thematic priorities to
        work under. In this way EQUAL brings different kinds of knowledge and
        expertise together towards integrated strategic approaches.

2.1.4 Empowerment

1.      Projects must provide for the involvement of those targeted for support. This
        includes involvement of intended beneficiaries in influencing the design of the
        project and in the evaluation of project effectiveness.

2.1.5 Mainstreaming

1.      Emphasis is placed in EQUAL on ensuring that the lessons learned from DP
        projects inform policy. A mainstreaming approach has been arranged for each
        thematic priority. Key to this are Thematic Networking Groups (TNGs) – now
        referred to as National Thematic Networks (NTNs). NTNs are made up of
        representatives from EQUAL DPs, relevant Government Departments, the
        Equality Organisations and Social Partners.

2.1.6 Transnational Cooperation

1.      A central feature of EQUAL is that each DP is obliged to have at least one
        transnational partner. This emphasis is intended to ensure lessons learned across
        member states are exchanged, and examples of good practice more effectively
        channeled into the European Employment Strategy.




                                                                                                 3

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


2.1.7 Equal Opportunities

1.     In addition to the Equal Opportunity thematic priority, Equal Opportunities is a
       cross-cutting principle of the programme and its approach. Each DP must have an
       equal opportunities policy detailing how it will ensure equality between men and
       women and across the Section 75 categories.

2.      The implementation of each of these principles through the course of the first call
        of the EQUAL programme represents a key focus for this evaluation.


2.2     The Key Stages or Actions of EQUAL
1.      The EQUAL programme is designed around four „Actions‟.

2.      Action 1 represents the first stage for DPs and lasts 6 months. For this first call
        the six month period was from the 15th of November 2001 to the 14th of May
        2002. During Action 1 DPs finalized their membership and constitution, project
        aims, objectives and associated programme of activities. DPs also prepared an
        equal opportunities policy. Each of these outputs is contained in a Development
        Partnership Agreement (DPA). The DPA forms the basis for obtaining funding
        for Action 2. During Action 1 DPs also formulated a Transnational Cooperation
        Agreement (TCA) detailing arrangements for a transnational partnership and
        programme of work. At the end of Action 1 six DPs were in place ready to
        progress into Action 2.

3.      Action 2 is the main implementation stage of work programmes approved at the
        end of Action 1. For the first call this stage began on the 15th of May 2002.

4.      Action 3 is concerned with networking, dissemination and mainstreaming
        activities and transferring good practice lessons learned during Action 2 into
        policy and practice. Actions 1 and 2 follow on from one another, but Action 3
        will begin as soon as possible once results become available to disseminate.

5.     Action 4 funds the technical assistance to provide support to the DPs. This
       technical assistance includes publicity and evaluation, as well as providing support
       to a monitoring committee for EQUAL in Northern Ireland and other coordinating
       activities.

6.     Support arrangements to assist the DPs to realize the full potential of the
       programme centres around a National Support Structure (NSS). The NSS in
       Northern Ireland is Proteus. The role of Proteus is to provide a range of ongoing
       support and guidance to DPs as they implement their projects. This includes
       (i) advice and assistance to applicants in completing applications, (ii) support for
       DPs in completing DPAs and TCAs, (iii) initial assessment of applications,
       (iv) design of data collection for programme monitoring in conjunction with the

                                                                                                 4

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


       Managing Authority, (v) coordination with NSSs in other member states,
       (vi) assistance with mainstreaming and dissemination and (vii) the provision of
       robust financial systems. An additional role of Proteus is to support both
       individual DPs and the European Unit of the Department for Employment and
       Learning (DEL). DEL has overall responsibility for the implementation and
       delivery of the EQUAL programme in Northern Ireland.

7.      In addition to the support provided by Proteus there are communication and
        dissemination arrangements to assist individual DPs to transfer learning achieved
        within their projects into policy formulation at Northern Ireland, all Ireland or EU
        levels.    This centres on the establishment of NTNs and an associated
        mainstreaming and dissemination strategy. NTNs bring together DPs and policy
        experts to explore and take forward key policy lessons. One NTN was established
        in Northern Ireland.

8.      DEL and Proteus have been involved in work to develop two other coordinating
        and mainstreaming arrangements. These include a North/South Steering Group,
        reflecting a commitment on behalf of the two Managing Authorities North and
        South and the Special European Programmes Body (SEUPB) to facilitating cross-
        border cooperation on policy issues. The group is chaired by SEUPB and
        involves both Managing Authorities and NSS bodies North and South. In
        addition, a Joint Support Structure has been established between EQUAL
        Managing Authorities and NSS bodies representing Northern Ireland, Scotland,
        England, Wales and Ireland. This works to assist the coordination and
        standardisation of processes and procedures for EQUAL, for example, in relation
        to DP application forms and assessment procedures.

9.      The mid term evaluation will look at developments as each of the four Actions are
        implemented, and at the contribution each Action makes to programme
        effectiveness. To date, the first call of EQUAL has progressed to the end of
        Action 1 and into implementation of Action 2 and receipt of applications for
        Action 3. The focus of the evaluation for this mid term report is therefore
        primarily on the contribution of Action 1 and Action 2.




                                                                                                 5

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland



3.      THE APPROACH TO THE EVALUATION AND AN
        OVERVIEW OF EVALUATION ACTIVITIES
1.      The evaluation of EQUAL has a number of distinct features. This section
        provides an overview of evaluation activities and signposts more detailed
        information about methodology in the accompanying annexes.

3.1     The Evaluation Approach
1.      The Commission, in a paper entitled „Key Issues for the Mid Term Evaluation of
        EQUAL CIP in the Member States‟ (3.09.01) states that the evaluation of EQUAL
        “needs to reflect (the EQUAL programmes‟) experimental approach”, particularly
        in attending to process as well as outcomes.

2.      An approach is required which can follow the different innovative features of
        EQUAL at programme level; that is, not just the goals and impacts of individual
        DP projects (goal innovation), but also the process by which these goals are
        pursued (process innovation). The collaborative way in which the process is
        conducted amongst key players (context innovation) is also a key focus, from the
        partnership approach itself as a mechanism for achieving effective outcomes, to
        the quality and effectiveness of the relationships that are developed with
        beneficiaries and with transnational partners.

3.      Additionally important at programme level was to look at the extent to which
        EQUAL was able to support a Policy Learning Cycle. A key concept of EQUAL
        is that gaps or weaknesses for the knowledge base underpinning policy at National
        or European level identified through consultation are taken up by DP projects, and
        the practical learning achieved is then fed back into policy thinking. Figure 1
        illustrates the key elements contributing to the Policy Learning Cycle.




                                                                                                 6

TL Associates
         Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


                                       Figure (1) The Policy Learning Cycle




                                       European Employment Strategy




                                         National Labour Market Policies
Identification of gaps in                  and Northern Ireland policy                    Key lessons transmitted to
   policy know-how                                  initiatives                                 policy targets




            Development Partnerships and                                   Mainstreaming Activities,
              Projects : Monitoring, self                                   in particular through
            assessment and active learning                                    National Thematic
                                                                                  Networks



            Empowerment: Involvement                                              Transnational
           and learning from Beneficiary                                        Cooperation Work
            Groups – those targeted for
                                                                                  Programmes
                 support by projects




                                          Exchange of individual learning
                                          and identification of common
                                          factors / elements of best practice




                                                                                                          7

         TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


4.      For the Policy Learning Cycle to be effective it is important to facilitate the key
        links and activities which produce and convey learning at the level of individual
        DPs and into National and European policy.

5.      This evaluation was designed to be able to complement the achievement of this by
        adopting a formative approach, working with DPs, Proteus and DEL to build on
        the learning which emerges from implementing the EQUAL programme with the
        overall aim of improving its overall effectiveness.


3.2     Evaluation Activities and Tools
1.      The evaluation combined a number of key activities and associated tools.

3.2.1 Liaison with the Evaluation Steering Group, Proteus and DEL

1.      An Evaluation Steering Group manages the evaluation of EQUAL. In addition to
        reporting to and agreeing the evaluation approach with the steering group, liaison
        with Proteus and DEL reflected the formative approach of this evaluation, to
        inform and support the effective delivery of the programme and its innovative
        approach.

3.2.2 Desk Review of EQUAL Documentation

1.      This included:

                    Examination of the original applications by prospective DPs, their
                     subsequent DPAs, and their Transnational Cooperation Agreements
                     (TCAs). These provided a key baseline for a range of evaluation
                     activities including policy proofing, partnership development and
                     transnational cooperation.

                    Examination of documentation relating to the assessment and
                     selection process (Reported in Section 6.2)

                    Quantitative analysis of DP monitoring returns to assess the extent to
                     which the principle of Equal Opportunities was being met.

3.2.3 Policy Proofing

1.      A policy proofing exercise was designed in order to assess the portfolio of DP
        projects established by the selection process for their potential contribution to
        policy. The exercise included establishing the policy context for EQUAL in
        Northern Ireland, and a more specific examination of sectoral policy and
        programme initiatives relevant to the individual DP project proposals. Policy
        proofing included two key activities:

                                                                                                 8

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




2.      A Literature Review: key documents reporting or evaluating policy initiatives
        relevant to Northern Ireland were reviewed noted in the bibliography in Annex 15.

3.      A Web Survey: To assess more specific policy and programme activity, a survey
        was conducted of organisational and programme web sites and their linkages.
        Web sites were used to be able to cover as much of the complexity as possible of
        the policy and practice areas touched on by the six DPs. In contrast to telephone
        interviews, web sites provide a quick and ready overview of the linkages and
        activities of an initiative that can be assessed by the evaluator. The approach
        assumes that most reasonably well organized programmes, organizations or
        projects today are likely to have established a web site. This means some smaller
        or more discrete initiatives may not have been identified. However, as a sampling
        frame of initiatives which were liable to be contributing to policy and practice,
        web sites represent a ready testbed.

4.      Web sites were sampled using key search terms drawn from the DPAs, conducted
        through the Google search engine. In addition, web sites typically provided
        detailed links to other organizations and initiatives which it was possible to follow
        to build the picture of players and activities.

5.      A total of 140 sites were visited.


3.2.4 A Partnership Health Check

1.      The DPs in EQUAL are of key significance. They are crucial to the production of
        key policy messages and best practice learning. They are also a key vehicle for
        „horizontal mainstreaming‟; that is, the translation of learning into the future
        activities and approaches of partner organizations. Emphasis was therefore placed
        on building a picture of how DPs were progressing through the course of this
        evaluation.

2.      Progress was assessed through the design of a „Partnership Health Check‟, which
        defined and then monitored a number of partnership performance areas using an
        associated set of key indicators. The approach to constructing the Health Check is
        described in detail in Annex 4 and summarized in reporting the findings in
        Section 5.1.

3.      Measurement of progress against each indicator was done using a „descriptor
        scale‟ also detailed in Annex 4. Descriptor scales are made up of four different
        descriptions of partnership working, each one representing different levels of
        development for a particular performance indicator. The appropriate description
        for each DP was identified on three separate occasions during the evaluation
        period; at the end of Action 1, mid way through the first year of Action 2 and at
        the end of the first year of Action 2. Descriptions were determined either through
                                                                                                 9

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


        participant observation during Active Reflection Sessions (described below),
        through an examination of DPAs, or through the results of two postal surveys of
        DP members.

4.      The profiles produced from the Partnership Health Check can be found in
        Annex 6. It is important to note that this was not intended to be a test of the
        fitness of each DP for the programme, rather, a means to assist and inform both
        DPs, Proteus and DEL on areas where more work may be helpful to ensure overall
        effectiveness. This kind of support tool for partnerships is not typically available
        to partnerships and represents a way in which the evaluation was able to inform
        and assist partnership development during the programme.

3.2.5 Active Reflection Sessions

1.      To implement parts of the Partnership Health Check it was necessary to see the
        DP members at work. To provide a means by which this could be done quickly
        but effectively within the timescale of the evaluation, and at the same time be
        complementary to the work of the DPs themselves, specific Active Reflection
        Sessions were arranged. There were three Active Reflection Sessions for each DP
        during the mid term review. The Active Reflection Sessions were designed
        around specific development areas pertinent to the DPs. This evaluation activity
        created the opportunity for members to discuss and agree actions for the
        progression of their projects. The approach to Active Reflection Sessions is
        detailed in Annex 5.

2.      The Active Reflection Sessions provided a means to observe and probe a range of
        Partnership Health Check areas to determine the position of the DP on descriptor
        scales. Feedback at the session and in subsequent feedback sheets provided the
        DP with ideas about where more attention could be paid or where things are going
        well. DPs found the process positive and beneficial.

3.2.6 Transnational Cooperation Health Check

1.     In a similar way to the Partnership Health Check, a Transnational Cooperation
       (TNC) Health Check was also designed to monitor TNC processes and dynamics,
       and the nature and extent of the exchange of learning during the evaluation period.
       The approach to the Health Check is detailed in Annex 9. Positions along
       descriptor scales for key indicators were determined through DP visits, active
       reflection sessions and associated interviews.




                                                                                                 10

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


3.2.7 Member Surveys and Case Studies

1.      Two written surveys of all DP members were undertaken. These were separately
        designed and distributed to address parts of the Partnership and TNC Health
        Checks not covered in the Active Reflection Sessions. The first survey
        questionnaire was also designed to explore the profile of organisational
        involvement in the programme. The questionnaires are reproduced in Annex 11.
        The response rate to the first survey was 82%, with all but one of the DPs
        returning questionnaires for all, or all but one, of their members. 44% of DP
        members returned questionnaires for the second survey, the lower response rate
        reflecting the timing of the survey at the start of the summer break.

2.      Detailed interviews were held with DP managers to highlight development areas
        emerging from the work to date. The results were compiled into a set of four
        thematic studies which can be found in Annex 7.

3.2.8 Survey of Unsuccessful Applicants

1.      As part of the evaluation of the selection process all unsuccessful applicants were
        invited to complete a written questionnaire to explore their experiences of the
        process. Following telephone contact with applicant organisations, 7 responses
        were eventually received, representing a 35% response rate. The questionnaire is
        reproduced in Annex 12.

3.2.9 System Reviews

1.      Meetings were held with each DP to review development of monitoring systems,
        particularly „soft monitoring systems‟ concerning changes in, for example, the
        attitudes, self esteem, or quality of life of target beneficiaries, or the „distance
        travelled‟ by beneficiaries along measures of employability. Sessions also
        explored readiness of DPs in relation to mainstreaming strategies, their
        understanding of the provision at programme level for mainstreaming support,
        their experience of establishing transnational partners, and the quality of support
        received from Proteus.

2.     The evaluators undertook a review session with Proteus to discuss and assess
       systems for programme monitoring and support.




                                                                                                 11

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland



4      EQUAL WITHIN THE LABOUR MARKET AND
       BROADER POLICY CONTEXT: FINDINGS FROM
       THE POLICY MAPPING EXERCISE

4.1     Overview of the Section
1.      A key objective for EQUAL N.I. is to address policy gaps within the Thematic
        Priorities identified for Northern Ireland, and to contribute to best practice. This
        section begins by providing a brief review of key labour market trends in Northern
        Ireland, providing a context for the evaluation of EQUAL at the end of the first
        year of Action 2.

2.      As part of the evaluation of Action 1 in June 2002 a Policy Proofing exercise was
        conducted to explore the potential contribution and level of innovation in EQUAL
        Northern Ireland. The exercise included an overview of regional policy initiatives
        relevant to EQUAL in Northern Ireland, and a more specific examination of
        sectoral policy and programme initiatives relevant to the individual DP project
        proposals. Policy trends and the activities of EQUAL DPs were reviewed again in
        July 2003, at the end of the first full year of Action 2. The second review enabled
        new regional policy developments affecting the mainstreaming potential of the
        work to be taken into account. The results are also reported in this section in
        Tables 1 to 6 (page 20 onwards)


4.2     Key Labour Market Trends In Northern Ireland For July 2003

1.      July figures from the Department of Enterprise Trade and Investment (DETI)
        report that 5.3% of the workforce was unemployed in the period March to May
        2003. 1 The unemployment rate decreased slightly from the previous quarter,
        December 2002 to February 2003 by 0.2 percentage points to historically low
        levels. The percentage unemployed was lower than 5 other UK regions including
        Scotland (5.6%) but higher than the UK average (5.0%). It was 2.8 percentage
        points lower than the seasonally adjusted EU 15 average of 8.1%.

2.      Catholics however are almost twice as likely to be unemployed than Protestants,
        according to a breakdown of figures contained in the Labour Force Survey's
        religion report in February of 2003. The figures put the unemployment rate for
        Catholics at 8.3% and the rate for Protestants at 4.3%.




1
  According to the seasonally adjusted official unemployment measure as estimated by
the Labour Force Survey. Source : DETI 23rd July 2003
                                                                                                 12

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


3.      The unemployment rate for 18-24 year olds in Northern Ireland was reported by
        DETI to be 9.8% (the number unemployed aged 18-24 as a percentage of those
        economically active aged 18-24), 4.5 percentage points higher than the rate for all
        ages (5.3%). The corresponding UK figure was 10.7%.

4.      Unemployment levels were uneven across the Province, with higher levels of
        unemployment in the West than the East with the exception of Belfast 2

5.      505,000 persons were classified as economically inactive in Northern Ireland in
        July 2003 (that is, neither employed nor unemployed due to retirement, domestic
        circumstances and so on). Of these, 11% (55,000 people) wanted to work but
        were not actively seeking employment, because, for example, of family
        commitments or long-term sickness3.

6.      Long term claimants (duration of 1 year or over) decreased by 20.4% (1,929) over
        the year 2002 / 34

7.      The Northern Ireland Executive reports that the Northern Ireland economy has
        been performing relatively well within the context of a global slow down.
        Economic forecasts suggests the economy in Northern Ireland grew by 2% in
        2002 and will grow by 2.5% in 20035. In common with trends across Europe the
        main sector to show significant growth was the Services sector, growing by 15%
        in NI over the past 5 years in contrast to 10% in the UK as a whole over the same
        period. In contrast manufacturing has continued to decline by 12.2% over the past
        5 years and 4.3% over the past year. The exception has been in the Engineering
        sector, which is expected to continue to grow over the next year.


4.3     The Regional Policy Context for EQUAL in Northern Ireland

1.      The consultation and policy review process carried out in formulating the EQUAL
        CIP for Northern Ireland took place in 2000, at a time when some policy areas,
        particularly in relation to the devolved Northern Ireland Administration, were not
        yet fully realized. Since then, some policy areas have continued to change and
        move forward. The policy mapping exercise sought to update on key policy
        trends current in June 2002 and any significant changes have been included as part
        of the review exercise in July 2003. In addition, broader policy initiatives were
        identified of relevance in the light of the work of DPs to date. These are outlined
        here.



2
  Rural Development Council Baseline Report „A Picture of Rural Change‟ May 2003
3
  DETI Section 2 Labour Force Survey June – August 2003 Table 2.4
4
  Findings reported by the EQUAL Futures project however highlight that this may hide a percentage of
people „recycled‟ within the New Deal system so that they only appear as claimants for 6 monthly periods.
5
  First Trust Bank June 2003 and Price Waterhouse Coopers June 2002
                                                                                                        13

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


4.3.1 EU Policy

1.      The European Commission‟s plans for implementation of a six year Action
        Programme associated with Article 13 of the EU Amsterdam Treaty have
        progressed through a one year pilot programme, evaluation of which was
        completed in March 2001, and into a second phase of funded projects. The
        Action Programme is entitled DISCRIMIN C, or the „Community Action
        Programme to Combat discrimination 2001-2006‟. It is concerned to „elaborate
        the understanding of issues related to discrimination through improved knowledge
        and measurement and through evaluation of the effectiveness of policies and
        practice‟. The programme focuses on legislative and statistical studies and
        networking rather than action-based research.           The programme will be
        administered from Europe and funded initiatives must be conducted within a
        transnational framework.

2.      Organisations from Northern Ireland are involved as partners in three of these
        transnational projects. Details of the projects can be found in Annex 13.

3.      The EQUAL CIP identifies the potential for EQUAL to play a role in
        complementing this programme, and likewise, DISCRIMIN C specifies that the
        Commission and the member states shall ensure consistency and complementarity
        between actions undertaken and other relevant actions such as EQUAL. The
        nature of the DISCRIMIN C projects detailed in Annex 13 does not suggest
        overlap with EQUAL.

4.3.2 UK Policy

1.      The arrangement for National Action Plans in each member state to implement the
        Employment Guidelines of the European Employment Strategy has provided an
        impetus for a dynamic range of policy initiatives. The trend within UK
        employment policy initiatives has continued to focus on integration between
        services toward a more joined up support and pathway infrastructure. Changes in
        the relationships amongst institutions in delivering support is an example of
        „Context Innovation‟ and is an area of innovation that EQUAL in Northern
        Ireland should be able to contribute to. Specific opportunities are detailed in
        Section 4.4 in looking at each DPs work to date.

2.      In the UK and Northern Ireland a key area of policy development has been in the
        refinement and extension of approaches under New Deal. Amongst those are the
        New Deal Innovation Funds which support pilot projects to test and develop
        improved services. Evaluation of six Innovation Fund projects carried out in
        Northern Ireland was published in 2003. The evaluation suggests that innovation
        was not a strong feature of the projects; “rather funding was used to enhance New
        Deal and provide more intensive support to difficult clients, working within the
        New Deal structures” (Evaluation Report page 56 1.1.1).


                                                                                                 14

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


3.        A wide ranging evaluation of New Deal for Communities was published in 2001 6
          which is very significant to the approach of a number of the DP projects in this
          round of EQUAL in Northern Ireland. Specific findings are related to individual
          DP activities in Section 4.4.

4.        The final evaluations of ADAPT and EMPLOYMENT, the previous CIPs for
          human resource development, were completed in June 2001. In common with the
          New Deal Innovation Fund evaluation they also highlight the difficulties of
          identifying with certainty the extent to which projects were innovative.


4.3.3 Northern Ireland Specific Policy Developments Relevant to EQUAL

1.        The suspension of the Northern Ireland Assembly on the 14th October 2002
          resulted in immediate changes to government in Northern Ireland. Some
          functions moved from the control of one Department to another, but there were no
          immediate changes in the way services were delivered. For EQUAL in Northern
          Ireland there are implications for the prospects and strategies for local
          mainstreaming in the loss of Statutory Departmental Committees, which had in
          particular a role in policy development and could call for people and papers which
          could advise ministers in formulating policy. The restrictions of only four
          caretaker Ministers covering three Departments each also lessen the attention that
          might be paid to policy submissions and the action which might be taken. It
          remains uncertain when and in what for an Assembly might be re-established.

2.        As part of the implementation of the Programme for Government for the Northern
          Ireland Assembly, Departments prepared and published Public Service
          Agreements (PSAs), which set out each Departments overall aim, associated
          objectives, targets, actions and budget allocations. These are updated annually.
          Of these, DELs PSA is most relevant to EQUAL and in particular Objective 2.2
          (helping people from welfare to work) and 2.5 and 2.6 (introduction of a new co-
          located „jobs and benefits offices and a combined approach to jobseeker and
          benefit assessment processes). This latter entails a partnership approach with the
          Social Security Agency of the Department of Social Development has piloted and
          18 jobs and benefits offices are scheduled to be in place by March 2004.

3.        Three additional areas of policy development are relevant to the policy context of
          EQUAL; (i) New Targeting Social Need (NTSN) and Equality Legislation under
          Section 75 of the Northern Ireland Act for Government, (ii) the Employability
          Task Force Action Plan, and (iii) the current round of European Structural Fund
          Programmes „Building sustainable Prosperity‟ 2000 – 2006, and PEACE II, the
          special EU programme for Peace and Reconciliation in Northern Ireland and the
          border region of Ireland, 2000-2004.



6
    New Deal for Communities National Evaluation Scoping Phase Report. DTLGR 2001
                                                                                                 15

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


4.        A planned review of NTSN has recently been completed and will include an
          evaluation of the implementation and impact of NTSN. Better understanding of
          the tools and approaches by which to reach and assist marginalized social groups
          achieved within EQUAL should be of value in helping Government Departments
          to improve the effectiveness of their TSN Action Plans.

5.        Objective 2 of the OFMDFM (Office of the First Minister and Deputy First
          Minister) PSA provides a key focus for outputs from EQUAL. The objective
          seeks to “promote equality of opportunity, human rights and improved community
          relations, tackle poverty and social disadvantage, and meet the needs of victims”7
          and is closely tied to the development of a single equality bill. It has been a
          particular success for EQUAL in Northern Ireland that DEL have successfully bid
          to the Executive Programme Funds of the Programme for Government for the
          match funding of all DPs. EQUAL will have the potential to feed into annual
          reviews of the Programme, particularly in relation to the implementation and
          impact of Section 75.

6.        As part of this PSA Objective, a Steering Group has been established; the Equality
          and Social Need Research and Information Group. It consists of statisticians from
          across Government Departments and representatives from the Equality
          Commission and Northern Ireland Council for Voluntary Action. The group is
          completing a strategy paper for the establishment of a research and information
          agenda in relation to gaps in knowledge on Section 75 categories, and on social
          need, deprivation and poverty. The agenda is intended to underpin a cross-
          departmental research budget and seek to coordinate research and information
          activity across Departments. The insights gained from EQUAL DPs such as
          ACCORD, the Futures Project, and YWMW (described in detail on pages 24 and
          32) might prove important in further illuminating gaps in knowledge.

7.        The Task Force for Employability and Long Term Unemployment recently
          launched its Strategy for Employability for Northern Ireland in December 2002.
          The plan draws on a scoping study completed in 2001. The Task Force was
          established as part of the Programme for Government under the chairmanship of
          the Minister of the DEL and DEL have lead responsibility for taking forward the
          strategy. The aim of the Task Force was to prepare an action plan which
          integrates actions across Government Departments and Agencies to address
          problems of employability.

8.        The Task Force strategy presents a number of key attributes making up the basis
          for employability in the modern work environment based on a broad consultation
          of key players and the literature. The attributes of employability that are adopted
          are highly relevant as a key part of soft monitoring systems and distance traveled
          indicators by DPs, and to the preparation of reports from DP work as part of
          mainstreaming. This is discussed further in Section 5.5.2 concerning the
          evaluation of EQUAL in Northern Ireland.
7
    NI Executive Programme for Government www.pfgni.gov.uk/dec2001pfg/ofmdfm.htm
                                                                                                 16

TL Associates
    Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


    9.      Additional local Task Forces have been established to address high levels of
            unemployment in Belfast and the North West. They include the West Belfast and
            Greater Shankill Task Forces which submitted recommendations to DEL for
            approaches to tackle unemployment in these localities. These are being taken
            forward by DEL as part of its Targeted Initiatives Programme which is taking
            place in each Task Force area. The Targeted Initiatives Programme will work
            closely with the community and voluntary sectors to customize approaches to the
            particular needs of each area. The programme will examine new approaches and
            services; such as Job Assist Centres hosted by a consortium of community based
            organisations, and help with product development – for example innovations in
            New Deal.

    10.     Building Sustainable Prosperity (BSP) and PEACE II each include measures to
            address unemployment, and in particular, employment opportunities for women
            (Measure 2.8 in BSP, Measures 1.5 , 2.5, 3.1 and 3.2 under PEACE II), and the
            improvement of basic skills for employment (Measure 1.3). In approving
            PEACE II, the EU has demonstrated an additional interest in intervening in
            particular forms of discrimination in Northern Ireland (beyond the initiatives for
            Article 13, for example) which has some relevance to EQUAL, particularly where
            PEACE II deals with issues of employability, overcoming barriers to employment,
            and expressions of discrimination. The involvement of the Special European
            Union Programmes Body (SEUPB) on the Monitoring Committee of EQUAL
            should assist linkages to be made between areas of learning emerging from
            EQUAL and PEACE II.


    4.3.4 Additional Policy Developments in Northern Ireland

    1.      A key feature of EQUAL is its focus on people most marginal to the labour
            market and for whom a wider range of social, psychological and material barriers
            act against progress in employability. They include health, housing and social
            connectivity essential to the wellbeing of individuals and their capacity to engage
            in society. Integrated approaches are highlighted by the Employability Task
            Force but this theme is also echoed in other key strategies such as:-

                 DSD‟s Neighbourhood Renewal, Strategy; „People and Place‟ formally
                  launched in June 2003. The strategy is described as a strategic and co-
                  ordinated way to close the gap between the quality of life for people in the
                  most deprived neighbourhoods and the rest of society.

                 The models and approaches being developed for the DHSSPS „Sure Start‟
                  and „Investment for Health‟ programmes
.




                                                                                                     17

    TL Associates
    Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


    2.      Of additional significance to a number of the EQUAL DPs is the DSD‟s Partners
            for Change Strategy which seeks to encourage government to build stronger
            partnership relations with the community and voluntary sector in delivering policy
            goals. The strategy includes a best practice guide which is currently under review.
            It has also resulted in a number of initiatives to build examples and case studies of
            partnership working. At the heart of the strategy is the concept of social capital,
            citizenship and community infrastructure as mechanisms for enhancing social
            inclusion and individual wellbeing.
.
    3.      Work within EQUAL necessarily overlaps with these different areas of policy and
            practice thinking and there are significant opportunities for cross-fertilisation
            between them.


    4.3.5 Conclusions

    1.      Despite the absence of a devolved Assembly and local Ministers there has
            continued to be gradual progress in taking forward policy initiatives which target
            different aspects of employability, discrimination and social inclusion in Northern
            Ireland. For EQUAL this has benefits, but it also creates challenges. It has
            benefits because it provides a number of complementary areas of development in
            which the work of DPs can be readily recognised and perhaps incorporated. It
            creates challenges in requiring DPs to be precise in distinguishing how their work
            adds to what is already being tried. These benefits and challenges are looked at in
            more specific detail in Section 4.4 below.

    2.      Addressing the challenges will require individual DPs to draw on the knowledge
            and links of their partnership members (examined further in Section 5.1). DPs
            will also need to formulate and communicate well articulated policy and practice
            messages. The National Thematic Network of Northern Ireland will be a key
            means to make additional connections between EQUAL and other related
            initiatives to assist communication. This is reviewed again in reporting on the
            evaluation of mainstreaming in Section 5.3.


    4.4     Policy Proofing Individual DP Projects
    1.      The potential contribution of EQUAL DP activities to policy and practice was
            examined through a detailed policy proofing exercise. The exercise consisted of
            assessing the key project innovations proposed by each of the DPs against a cross-
            section of other policy initiatives in Northern Ireland. Emphasis was placed on
            assessing the extent to which projects appeared distinctive and progressive, and
            how well they suggested complementarity or added value for different policy
            areas. The methodology for the policy proofing exercise is described in Section
            3.2.3.


                                                                                                     18

    TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


2.         Because of the small number of DPs in contrast to most other European member
           states, it was possible to look at each on a case-by-case basis8. The exercise was
           first carried out in June 2002 as part of the evaluation of the effectiveness of the
           promotion and selection process for Action 1. It was updated and revised in July
           2003 to take account of DP activities and any new policy initiatives during the
           first year of Action 2.

3.         A summary of the key points from the analysis for each DP is provided in Tables
           1 to 6. The Table headings represent the following basic assessments:

           A brief description of the DP project.

           The key innovative features of each project including contribution to
            empowerment.

           Links to policy and potential for mainstreaming.

           Relationship with the practice environment of the project; that is, key projects,
            initiatives and actors which share the same general target groups, project goals
            or types of intervention to the DP projects.

           Key challenges in realizing the potential of each project.


4.         The links identified for each DP to policy, or the strengths of DP approaches
           relative to a cross section of existing practice, were used to allocate a benchmark
           score for each project. These scores provide some comparative perspective on the
           policy and practice significance of the approaches of each DP. Links and
           strengths were scored according to the extent to which the project approach is
           liable to offer insights distinct from existing policy or practice thinking. The
           scoring approach includes the following:

                    ++ High potential contribution
                    + Moderate potential for contribution
                     Uncertain contribution (pros and cons balance)
                    - Moderate difficulty possible in achieving contribution
                    -- Significant difficulties possible in achieving contribution




8
    Common features of DP activities are brought together within four thematic studies in Annex 7
                                                                                                    19

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


                   Table 1 - Innovation Assessment/Policy Map - ACCORD

DP Title: ACCORD

Project Summary
The establishment of seven community hubs, each piloting „bespoke‟ projects to
integrate target groups into training for basic and key skills. As a result,
beneficiaries might be better able to access job vacancies, further education or
training opportunities. Use is made of novel and creative project work with
beneficiaries to provide a meaningful application for core skills.

Innovation and Empowerment in the Project Approach
The approach includes strong emphasis on context and process innovation each of
which contribute to empowerment.

Context innovation occurs in the considerable number of community based
partnership members, many of whom (as hubs) also have their own
partnerships and member groups. The approach as such seeks to make good
use of the extensive community infrastructure of the area established through
previous rounds of programmes. (++)


The local nature of delivery, bringing training and education services out to
beneficiaries in their own communities is another key context innovation. The
combination of community outreach and delivery can provide a strong
supportive environment for those unhappy with institutional settings
associated with the idea and experience of ‘learning’ (++)


Process innovation arises in the user led and applied delivery of key skills.
The integration of a practical and often creative or satisfying project with the
communication and application of key skills is a significant feature shared
with the Digital Step approach (see ‘Pathways to Learning’ thematic study).
Beneficiaries are able to influence the fine tuning of delivery in shaping the
creative / applied project and training arrangements so that skills are
introduced into a situation of ownership and empowerment. (+)


Overall Policy Relevance - ACCORD
The use of community infrastructure and associated social capital for outreach is of
common concern as a means to address long term unemployment. It is also a
concern of the Voluntary and Community Unit (VCU)9 of the Department for
Social Development (DSD) in considering the future role of the voluntary and
community sectors post 2006 as part of its „Partners for Change‟ Strategy. There

9
    (previously, Voluntary Action Unit)
                                                                                                 20

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


should be opportunities to provide useful case studies and good practice guidelines
to these initiatives from ACCORD (++).

The work may complement existing practice within New Deal in providing for pre-
employment skills and competencies and will need to take into account new
developments in the UK and Northern Ireland stemming from New Deal
Innovation Fund projects. ()

The evaluation of New Deal for Communities emphasises that „how services are
delivered may be as important as the services themselves‟, and highlights as a
critical success factor, active outreach and local flexibility in reaching the most
disadvantaged (+).

The work is well placed to inform DEL‟s own Targeted Initiatives Programme,
seeking to test and develop new approaches and services in relation to
employability. (+)

Relative Strengths within the Existing Activity Environment

The Hub concept is becoming more prevalent (see, for example the Gasworks
Employment Matching Service (GEMS)), but its function as an outreach tool for
long-term unemployment is still under-explored. (+)

The range of different „vehicles‟ for training in basic skills provides an opportunity
to conduct a comprehensive review of approaches on a comparable footing, but
this will need good associated guidelines to go with it to enable others to choose
the best vehicle. (+)

The approach needs to be compared with innovative training and learning
approaches being developed under Learn Direct (-).

Challenges in Influencing Policy or Practice - ACCORD

Many of the training vehicles are well known. Similar initiatives took place under
PEACE I (for example analyses of long-term unemployment, labour market
barriers and the supply and demand side factors in the Derry City Council Area)
but the learning from these has not been captured and taken forward. (-)

Improving basic skills across men and women in areas affected by the conflict is
an objective of Measure 1.3 (New Skills and New Opportunities) under PEACE II
(-)

The range of target groups is considerable and, in common with the experience of
the Futures Project, many are invisible and extremely hard to make contact with.
(-)


                                                                                                 21

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


The approach depends on good community infrastructure. Areas of weak
community infrastructure (often Protestant communities) may not therefore be able
to utilise it (-).

The hub concept has a coordinated monitoring system in place for tracking
beneficiaries for process information but evaluation may need to be enhanced to
fully capture context innovation for policy relevant information on best practice for
the model itself. The „bespoke‟ and „pilot‟ basis of the approach means the need to
have a strong and effective shared vision of the approach in continuing to develop
a best practice model if partner providers are not to act too autonomously. ()


                              Assessment Summary: 11+ / 5 -



                Table 2 - Innovation Assessment/Policy Map - Digital Step

DP Title: Digital Step
Project Summary
The project seeks to address unequal access to the labour market for marginalised
young people by developing new models for learning, training and employment
based on the use of digital ICT media and popular culture, particularly in music,
animation, video, graphic design.
Innovation and Empowerment in the Project Approach

Emphasis here is on both process and context innovation.

Process innovation arises in the use of novel creative media to entice disenchanted
young people who have left school back into the learning process. The approach
has been linked to a wider programme of learning with two participating schools.
The media work leads to an accreditation through the youth achievement awards
scheme, bridging the „distance gap‟ faced by marginalised young people from
other forms of accreditation. A degree of empowerment is provided through
opportunities for beneficiaries to develop their own multimedia project content
reflecting issues of concern to them.

Context innovation is represented in the location of the project work within
schools, in youth centres or in „Bytes‟ drop in centres (advanced technology
centres open to young people in disadvantaged areas), wherein tutors drawn from
the multimedia industry and with experience in working with young people work
alongside teachers or youth workers, transferring multimedia training skills.

With one or two exceptions noted below, there was little which matched the
approach being taken by Digital Step in the web review and this project has
                                                                                                 22

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


significant potential for innovation.

Overall Policy Relevance – Digital Step

In addition to Employability this project could have relevance beyond Northern
Ireland to the Adaptability Thematic Priority of EQUAL, particularly in relation to
Lifelong Learning. This is helped by the close involvement of the Further
Education and Education and Training Inspectorate in the partnership. (+)

Specific links exist to concerns about the „digital divide‟ and the EU Social
Agenda recognition of Information and Communication Technology as a means to
address social exclusion, „e-inclusion‟ being a core objective of the new social
inclusion strategy. (++)

There are strong links to the Departments for Culture And Leisure, Education,
Enterprise Trade and Investment and for Employment and Learning who together
have published the „Unlocking Creativity‟ document which makes clear links to
economic and social development, career paths and accreditation objectives for
multimedia and creative enterprise. (++)

Relative Strengths within the Existing Activity Environment

Courses for multi-media skills are typically university based. This focus on
marginalized young people is therefore relatively unique, although arts-based
initiatives for excluded people using film and video are common (Belfast
Community Arts Foundation, Belfast Community Theatre). (++)

Use of the arts to re-engage disadvantaged young people has begun to receive
considerable attention, and has been used successfully in pilot schemes in the UK
concerned with low performing schools in disadvantaged areas. ()

The approach has already resulted in some mainstreaming; the Curriculum and
Education Authority (CEA) has accepted and will be piloting a new occupational
qualification in schools using the same model in September 2003. Early results
suggest the approach has galvanised some beneficiaries to take a proactive
approach to deciding and pursuing further learning needs. (++)

The range of pilot locations and user involvement in evaluating the model is
enabling the approach to be fine tuned and adapted to different applications. (+)

Challenges in Informing Policy or Practice – Digital Step

There is a need to distinguish and build on the Belfast and Derry Bytes Projects as
there are strong similarities between the DP project and these.(-)

Likewise, there are other similar initiatives such as the Shankill Spectrum Centre,
                                                                                                 23

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


the Nerve Centre, Derry, and QUB Armagh, which is developing new innovative
multi-media initiatives for a range of excluded groups drawing from Leonardo
pilot projects. (--)

                              Assessment Summary: 10+ / 3 -



                Table 3 - Innovation Assessment/Policy Map - YWMW

DP Title: YWMW (YWMW)
Project Summary
Developing and delivering a number of tailored training and personal development
programmes for marginalised young women to enable access to further education
and employment. The project also seeks to address problems of identify and sense
of belonging acting to disempower young women from marginalised social groups.
Innovation and Empowerment in the Project Approach

Emphasis here is on process and goal innovations.

„Holistic training programmes‟ are, within the women‟s sector in particular, not in
themselves wholly innovative. The key area of innovation here is in the emphasis
placed on identifying cultural as well as physical barriers to employment which
might be associated with, for example, young women travellers, lesbians or ethnic
groups. Cultural barriers are not well understood or much recognised in
employability strategies and studies to date10 indicate that positive self identity lies
at the heart of personal or group empowerment.

Overall Policy Relevance

The UK and EU place considerable emphasis on young people, with cross-
departmental approaches under the Policy Action Teams of the Social Exclusion
Unit in DSD. Likewise, women‟s unequal involvement in the labour market is of
obvious concern and there was little web-based evidence of programmes linking
gender and other social identities such as disability, ethnicity and other cultural
aspects together (with the exception of the „multiple identity‟ approach of
Diversity Matters). (++)



10
    see for example Ullman, C. (1997). Social identity and the adult ESL classroom. ERIC Digest.
Washington, DC: National Center for ESL Literacy Education. (EDRS No. ED 413 795)
or see Enhancing Employability:National Institute On Disability And Rehabilitation Research
Office Of Special Education And Rehabilitative Services Department Of Education Washington, D.C.
20202 Vol. XV, No. 5 (1993)
For an Ireland example see www.iol.ie/EMPLOYMENT/integra/5empowerment.html
                                                                                              24

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


OFMDFM include a PSA objective (2.6) on tackling gender inequalities which can
be addressed and enhanced by this work. The DP‟s consultation work with
marginalised women and subsequent training programmes may also provide scope
for findings feeding into community relations thinking. It may, for example, help
in tackling perceptions of difference and cultural stereotyping as part of the
emerging „Shared Future‟ cross-departmental strategy lead by the Northern Ireland
Office. (+)

Studies across employability initiatives highlight the need to „take account of
multiple disadvantages‟ 11 (+).

Article 13 of the Amsterdam Treaty and the action programme associated with it,
which addresses combating all forms of discrimination, is also relevant here. The
terms of reference for the evaluation of EQUAL (Annex 14) highlights the integral
part to be played in the European strategy to combat discrimination within and
beyond the labour market area with particular reference to Article 13. It will be
important to monitor any projects and developments flowing from the 6 year action
programme in Northern Ireland which involvement of OFMDFM on the NTN
should help to facilitate. (++)

Relative Strengths within the Existing Activity Environment – Young Women

The youth service do not currently provide services suited to young women over
13 and there is a significant knowledge gap in working with the social groups
targeted by this programme. „Young Citizens in Action‟ are active in developing
capacity amongst young people to engage in social and political issues on a broad
footing, but there was no specific reference to employability. (++)

The web sites visited during the mapping exercise addressing women did not
distinguish between younger and older women. (+)

Work to address women in employment is widespread, but often focused on self
employment initiatives (for example, DETI‟s Investing in Women strategy
associated with the Accelerating Entrepreneurship Strategy). PEACE II contains a
specific measure for Positive Action for Women (1.5) which includes provision for
projects identifying barriers for accessing the labour market, or building self
confidence, but it does not place a specific focus on the marginalized groups
amongst young women addressed here. (+)

The detailed contact with women and attention to particular (often hidden) cultural
differences has the potential to enhance the effectiveness of training and outreach
programmes in the future and to develop important approaches to self identity as a

11
  See for example Michael Campbell, Ian Sanderson and Fiona Walton, Local Responses to Long-term
Unemployment, Joseph Rowntree Foundation, 1998, or New Deal for Communities National Evaluation
Scoping Phase Report. 2001
                                                                                                   25

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


foundation stone for empowerment (++)

Challenges in influencing Policy and Practice

The broad range of excluded groups within the overall heading of young women
may prove challenging, particularly in relation to recruitment. This has already
proven to be a difficulty reported by the DP particularly in relation to Lesbian
women for whom there is no clear representative organisation or social scene. (- -)

Demonstrating the influence of cultural barriers on employability will be difficult
and represents a key challenge for the approach to evaluation. (- -)

Collaboration with the work of Diversity Matters would seem to offer considerable
potential in increasing policy and practice influence and should be encouraged. (+)


                              Assessment Summary: 13+ / 4 -



                 Table 4 - Innovation Assessment/Policy Map – Futures

DP Title: The Futures Project
Project Summary
To provide, through a team of support workers, holistic support to unemployed
individuals and their households to develop and take forward personal action plans
for employment. The outreach work is linked to the development of a recruitment
charter with employers. The combination seeks to address both demand and supply
side barriers to employment.

Innovation and Empowerment in the Project Approach
Innovation arises primarily in both context and process
In common with ACCORD, a key feature of the approach is its use of community
infrastructure and the development of social capital to enhance and extend outreach to
unemployed households. Community groups are also a key source of advice and local
knowledge informing the projects understanding of unemployment in the different
communities it covers. A key finding to date is the distinctiveness of unemployment in
each community and the need for local responses.
Exploration of different complementary working relationships with other outreach
initiatives, in particular the „Sure Start‟ programmes in each area are another potential
means to enhance context innovation.
Process innovation lies in the home-based approach to support, provided by support
workers.

                                                                                                 26

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland



The evaluation plans for the project include emphasis on an empowerment approach,
placing beneficiaries at the centre of the approach. It includes plans to create an
ongoing participant focus group drawing on beneficiaries of the programme and a
community sub group to strengthen inputs to project development from the community.

Overall Policy Relevance - Futures

Tackling long term unemployment is of primary concern at EU, UK and NI levels.
However, some of the elements proposed in this project are already in existence.
The programme‟s focus on research, evaluation and learning however provides an
important opportunity to inform best practice (+).

Key policy initiatives elsewhere in the UK are also relevant in providing
complementary or potentially overlapping lessons which should be explored. These
include the range of New Deal initiatives which are being expanded and developed
in England, which presently do not apply in Northern Ireland. Examples of these
are Employment Action Zones and Action Teams which provide detailed individual
support and mentoring services. (+)

Because of the household approach, the research may be able to inform issues
relating to benefit transfer and hidden unemployment amongst women, for example,
providing a more holistic picture of employability barriers and feeding into New
Deal for Partners of the Unemployed. (++)

The work is well placed to inform DEL‟s own Targeted Initiatives Programme
which seeks to test and develop new approaches and services in the same locations
as the Futures DP (+)

The involvement of DEL and the DSD in the partnership provide potential for good
vertical linkages into policy and practice. (+)

Relative Strengths within the Existing Activity Environment
The Policy Mapping found no other initiatives that focus on the household with the
exception of „Family Farm Development‟ (FFD). FFD worked with farm
households under the LEADER II programme, where a similar process of agreeing
household-wide action plans linked to a combination of farm and off-farm
employment and benefits actions was a core part of the process. (++)
Mentoring is now a well recognised method of working with a range of social
groups (see for example the New Deal), but less so on the basis of „multi-agency
training‟ and what it is assumed is a „one-stop‟ integrated information and guidance
support service. Similar approaches occur more frequently in England with the use
of Action Teams and Employment Action Zones. „One-stop-shop‟ provision was
seen as an effective approach under EMPLOYMENT and ADAPT. (+)

                                                                                                 27

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




Recent studies have identified some critical success factors associated with
employment zones common to the Futures approach; for example, a focus on the
individual and individual solutions was found to motivate and empower individuals.
Local flexibility was another key factor. The Futures project has also identified this
as important, however this suggests the innovation Futures can provide may be
limited to a Northern Ireland context (+) .
The links to Sure Start are particularly interesting given the common key elements
shared by the Futures and Sure Start approach. Each set out to provide a support
service in the home set within a wider local community support environment.
Evaluation of Sure Start schemes has highlighted the value placed amongst
disadvantaged parents on the home service, relationships built with Sure Start
workers and ongoing and readily accessible local support. There is substantial
potential for cross-learning between these two programmes which the Futures
project is well placed to capitalize on (++)
New Deal Innovation funds have funded 12 pilot projects in Northern Ireland which
have now been evaluated and the lessons learned from projects in Omagh, Clanrye
and Upper Springfield would seem particularly important to take into account for
this initiative. ()
Challenges in Influencing Policy and Practice - Futures

Greater clarity on how both the demand and supply side of the labour market will be
addressed would be useful. Demand side interventions, such as local labour
initiatives where job take-up by local unemployed people is encouraged have been
previously used, but nevertheless the involvement of employers is generally seen to
be an important success factor to include. (-)
The concept of „integrated working‟ emphasised by the project is now common across
a range of partnerships, indeed, partnership approaches to delivery are increasingly
used for a range of services and it will be important to distinguish the additional best
practice identified by Futures in approaching mainstreaming. (-)

In common with a number of LSPs, the West Belfast Partnership Board in
conjunction with other partnership boards in the city recently completed an
employability survey which may well provide overlapping information for this
initiative ().

This project is complex in the inter-relations of partnership member organisations to
one another either through past co-operation or through overlapping personnel with
more than one body, many of whom (e.g. the partnership boards) have their own
wide ranging project activities. If coordination can be achieved through this project
this will be a strength, but equally, experience shows that members sitting on many
partnerships can lose focus on any one of them. (- -)

The project has experienced significant difficulty in recruiting beneficiaries and
more time may be required to build trust, however some barriers such as benefit
                                                                                                 28

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


traps and cross-generational unemployment may prove hard to broach (- -).

In common with YWMW, there is scope for linkage with the training work being
developed by Diversity Matters which places emphasis on the demand (employer)
side in addressing barriers based on discrimination.(+)

                              Assessment Summary: 13+ / 6 -



           Table 5 - Innovation Assessment/Policy Map - Diversity Matters

DP Title: Diversity Matters
Project Summary

To develop and pilot training materials and programmes to encourage cooperation
and a partnership approach to diversity issues in the workplace, including a cross-
sectoral capacity building programme across disadvantaged groups, a unified
employer training programme and a cross-party Diversity Forum

Innovation and Empowerment in the Project Approach

Innovation is primarily context and process.
The project focuses on employers and providers / practitioners as much as it does
on marginalised groups. It aims to build a positive attitude and principles of
tolerance toward diversity in general rather than on a case-by-case basis for
individual social groups.
The project is facilitating a broad-ranging exploration of the concept and influence
of diversity to encourage broad institutional change in attitudes toward marginal
groups, and seeks to enable personal change and awareness raising (capacity
building) amongst „included‟ as well as excluded people.
The emphasis is on multiple disadvantage and perceiving individuals as
„multifaceted‟
There has been an extensive process of consultation in developing programme
content as a result across a cross section of interests, to enable the programme and
materials to be as adaptable as possible.
Overall Policy Relevance

There is a wealth of policy activity around disability; a disability rights task force
has made broad recommendations included within the PSA of OFMDFM
(Objective 2.2). Within HORIZON, work in relation to disability was noted for
having strong consortia with „strong links into the system (ie the policy
framework)‟, which is also shown in this DP. There has also been a growth of
activity around ethnic minorities, although much of this tends to be around social
                                                                                                 29

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


issues rather than employability. ()

However within a crowded political debate the aspiration to look at cross-social
(multi identity) discrimination issues is more distinct. Of particular relevance is
Article 13 of the Amsterdam Treaty and the action programme associated with it,
which addresses how to combat all forms of discrimination. There is involvement
of a representative from the NI Equality Commission and OFMDFM in this
project, and the involvement of both of these bodies on the EQUAL Monitoring
Committee should also facilitate the transfer of learning into this policy area. (++)

More laterally, the work has relevance to the development agenda for community
relations work and policy in Northern Ireland. As part of a trend toward greater
outreach encouraged by the partners for change strategy, it is also relevant to
recognition for the need for capacity building within statutory bodies and their staff
in building better partnership relations with communities. (++)

Reviews of employment initiatives in the UK (New Deal, Employment Zones)
emphasise the need to recognise and work with „multiple disadvantages‟ and
cultural factors. (+)

Evaluation of EMPLOYMENT also highlighted lessons around the need for
„changing culture‟ within organisations toward disadvantaged groups. (+)

Relative Strengths within the Existing Activity Environment – Diversity
Matters

There are evident strong links into policy and an associated clarity of purpose
within the DP (see also Section 5.1.3) because of a closeness of fit between DP
objectives and member interests. There is considerable potential benefit from the
partnership bringing together such diverse members (+).

The move to focus on multiple identities has not been well developed elsewhere
but progress toward a single equality bill favours this sort of approach. (++)

There is considerable potential for collaborative links to be developed between this
DP and YWMW. In effect, each is seeking to address opposite sides of the
problem and each has complementary interest in multiple identities and identity
barriers. (+)

Challenges in Influencing Policy and Practice

The wide range of different kinds of disability behind the term means this is a very
ambitious project, encompassing both disability and a range of other social groups.
The experience represented by Disability Action and other members means the DP
is well placed to rise to the challenge that this project presents, however it will be
equally challenging to encourage the cultural and attitudinal shifts required to
                                                                                                 30

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


adopt an overall awareness and non-discriminatory attitude amongst employers
and providers. (-)

Additional resistance may arise from the representative organisations of different
communities of interest and the extent to which they feel such an approach can
properly safeguard these interests in a competitive policy and funding
environment. (- - )

Evaluating the approach and the extent of positive change resulting from the
programmes will also pose challenges (-).


                              Assessment Summary: 10+ / 4 –



    Table 6 - Innovation Assessment/Policy Map - Personal Progression Systems

DP Title: Personal Progression Systems (PPS)
Project Summary

The project aims to provide pathways for employability and resettlement for those
who have served a period of custody. It will establish an assessment tool,
beneficiary portfolios and a referral mechanism or protocols with helping agencies
and public services to connect to prisoners before release.

Innovation and Empowerment in the Project Approach
In common with other DPs context and process innovation are prominent in this
project.

The project seeks to establish a wide range of links between providers external to
prisons and prisoners, enabling prisoners to „build social capital‟ and information and
support networks prior to release. This acts to reduce the isolation prisoners have
with the outside and the subsequent „shock‟ those who lack the support of family or
friends may have on release. The project approach seeks to provide a focus and
objectives for prisoners during and after time in prison
A key innovation of the project is the introduction of opportunities to explore self
employment and business start in conjunction with Invest NI. Beneficiaries have
also been able to secure and take up jobs whilst in the Foyleview open prison at
Magilligan.
Overall Policy Relevance

Measures to help the resettlement and the employability of ex-prisoners are highly
relevant following the 1998 Belfast Agreement (++).

                                                                                                 31

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




Links being made with a wide range of appropriate actors in the policy arena could
help build a more cohesive overall picture of the process barriers affecting ex-
offenders re-integrating into society, of which employment is a key part. (+)

Finding ways to empower and motivate prisoners prior to release contributes to a
wide-ranging debate on policy toward prison sentencing in tackling crime.
Likewise, enabling prisoners to make arrangements before leaving prison may also
make a significant impact on rates of re-offending and as such has relevance to the
Northern Ireland Office objective to reduce the re-offending risk (++).

Relative Strengths within the Existing Activity Environment - PPS

Essential core players are in the DP and there is evidence of engagement with a
number of other appropriate official bodies. (+)

The approach builds on and works with existing services within prisons. It has
begun to provide links to a wider range or providers, particularly in relation to
employment and business start. (+)

A pilot project under the New Deal Innovation funds explored running the
Gateway within prison for one year. The project has the potential to inform the
approach to New Deal for prisoners. (+)

Challenges in Influencing Policy and Practice

There is scope for greater links with other existing bodies (e.g. the West Belfast
Partnership and the Accord DP) that have voiced an interest in the resettlement and
employability of former prisoners, and engagement with other active organisations
such as Tar Isteach, An Loiste Uir, Amach Agus Isteach and the Marrowbone Ex-
Prisoners Group in North Belfast to compare approaches and support needs. (-)

The project is seeking to develop its approach to monitoring and evaluation in
order to encompass the wide range of circumstances and associated interventions
required across different prisoners. ()

The close involvement of the prison service in the DP provides a direct link to the
specialist interests associated with mainstreaming the approach however there may
be more work required to influence other providers to invest additional resources
in outreach activities within prisons. ()



                              Assessment Summary: 18+ / 1 -



                                                                                                 32

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland



4.5     Policy Proofing Conclusions
1.      The policy proofing exercise has shown that the policy fit of the current round of
        DP projects is constructive. All projects were judged to have a medium to high or
        a high relevance in addressing areas where there is changing policy thinking. The
        portfolio of projects seems well placed to contribute to the Thematic Priorities for
        Northern Ireland.

2.      Projects also demonstrated a range of innovative features. Context innovation is a
        key characteristic of all the DPs, however it was more difficult to judge process
        innovation with certainty at this stage and it will be important for this to be drawn
        out clearly in DP evaluation and mainstreaming activities.

3.      Whilst coverage of the key social groups is wide in this call, it could equally be
        said that the range of target groups in many of the DPs is considerable. There is
        evidence to suggest that progress is uneven across all targeted groups in some of
        the DPs (Section 5.5, Equal Opportunities) and continued monitoring of progress
        should be used to inform an assessment of how wide DPs should be encouraged to
        be in their approaches.

4.      Maximising the potential to inform policy places an emphasis on DPs making
        good use of the knowledge and links of their partnership members and
        formulating well articulated policy and practice messages. It will also important
        to make full use of the National Thematic Network of Northern Ireland to make
        the connections between EQUAL and other related initiatives.




                                                                                                 33

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland



5.      EVALUATION OF THE KEY PRINCIPLES OF
        EQUAL
1.      This section of the report details the analysis and findings of the evaluation of the
        key principles of EQUAL. The evaluation of the EQUAL principles has involved
        the tools and techniques described in Section 3.1.

5.1     Partnership

1.      EQUAL operates by bringing together key players in a geographic area or sector
        in Development Partnerships (DPs). DPs select one of the thematic priorities to
        work under. In this way EQUAL seeks to bring together different stakeholders
        with different kinds of knowledge and expertise towards integrated approaches.

2.      Effective partnership working offers a number of key potential outcomes. For
        example, helping to forge new working links between institutions and sectors
        concerned with employability and discrimination (context innovation). The
        experience of partnership working will also have an impact on whether member
        organisations take back findings into their own approaches (horizontal
        mainstreaming). To the extent that relationships between stakeholders formed
        during the lifetime of the DP continue, there are longer term implications for
        efficiency and effectiveness gains in addressing key employment and wider social
        issues. A strong partnership may also be better able to influence mainstream
        thinking.

3.      DPs are central to the policy learning cycle. Success in realizing the potential of
        EQUAL then, depends in no small measure on the effectiveness of partnership
        working. A key requirement of the evaluation was therefore to examine the
        partnership approach itself as a mechanism for achieving effective outcomes

4.      The partnership principle is one of the guiding principles of the Structural Funds
        and is increasingly prevalent as a result of public sector reform. As such, the
        factors influencing successful partnership working have become a significant
        focus for improving best practice and the subject of a growing number of studies.
        A list of references is included in the bibliography in Annex 15.

5.      To evaluate the Partnership Principle a literature review was conducted drawing
        together the findings of a cross-section of these studies and experience in the
        evaluation team of monitoring partnership development elsewhere12. The review
        identified key performance indicators describing effective partnership working
        which were then used to draw up Partnership Health Check Tool. Design of the
        tool also took into account the particular needs of partnership within the EQUAL

12
  For example TL Associates have designed and implemented a health check methodology for use in
assessing 9 newly formed partnerships as part of the Baltic States Rural Partnerships Programme funded by
the UK Department for International Development from 2000 - 2003.
                                                                                                        34

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


         programme. The review findings are described in detail in Annex 4 along with
         compilation of the Health Check.

6.       The review exercise organized performance indicators under 6 key areas:

         1. Representativeness - the nature of partnership membership and its
         representativeness of the key organizations and actors in the chosen area of work.

         2. Clarity of Purpose - the extent to which all members have a clear
         understanding about partnership objectives and approaches and their own roles.

         3. Collaboration - both between organizations who are members of the
         partnership, and between the members themselves in progressing partnership
         activities.

         4. Learning partnerships - the extent to which partnerships actively seek, share
         and use information, from each other, and from their monitoring arrangements for
         the project.

         5. Mainstreaming - the extent to which partnerships have planned strategies and
         prepared materials which enable the effective transmission of learning into policy
         and practice.

         The results of the first three of these areas of partnership working are reported
         here. Findings in relation to Area 4 (Learning Partnerships) are presented as part
         of the evaluation of the Principle of Innovation in Section 5.2, and for Area 5
         (Mainstreaming) as part of the broader examination of Mainstreaming in
         Section 5.3

7.       Measuring change in performance against the indicators under each heading was
         conducted using Descriptor Scales prepared by the evaluators for each indicator.
         These are reproduced in full in Annex 4 13 Each descriptor scale consists of four
         different descriptions, each representing an incremental improvement in
         performance from a situation of little or no effective working on the indicator
         (a 0 rating) to a situation of full achievement of the performance indicator
         (a rating of 3).

8.       DPs were matched against the set of descriptions for performance on each
         indicator either through participant observation (conducted during Active
         Reflection sessions; see Annex 5 for a description of these), through DP visits,
         examination of the DPAs or from findings of two surveys of DP members (see


13
   The reader is encouraged to refer to these descriptions as background in understanding the presentation
of findings which follow, however where findings are presented graphically the appropriate descriptor scale
is also provided alongside. The code number following the title of each graph refers the reader to the full
description of the performance indicator and associated descriptor scales in Annex 4
                                                                                                            35

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


        Annex 11 for survey questionnaires). Use of these different approaches is noted
        in presenting findings.

9.      Active Reflection or DP visits took place on three separate occasions during the
        evaluation. Session 1 took place toward the end of Action 1. Sessions 2 and 3
        took place mid way and toward the end of the first year of Action 2.

10.     In this report results are presented comparatively for all DPs for each performance
        indicator to discuss the situation at programme level. Alternatively, the
        performance profile across all performance indicators for an individual DP could
        also be examined to identify areas where additional support might be useful.

5.1.1 Representativeness;
      DP Membership Composition

1.      Representativeness is a key foundation for achieving the potential of EQUAL
        DPs. The development of integrated approaches to improving employability and
        combating discrimination across social groups requires DPs to have a cross
        sectoral membership. A good representation of the key stakeholders has an
        impact on the extent of context innovation and associated effectiveness and
        efficiency gains. It also has implications for the wider acceptance and uptake of
        findings in the practice community as part of horizontal mainstreaming. The
        challenges of a complex and changing policy environment outlined in Section 4
        also place emphasis on effective communication of the added value of DP
        activities in relation to what is already being tried. This might be achieved by
        individual DPs building on the knowledge and links of their partnership members.

2.      The first Partnership Survey results showed that a total of fifty organisations are
        involved in the six DPs. An examination of the DPAs and the member survey
        shows that 34% of DP members are from the public sector, whilst 54% describe
        themselves as belonging to the community or voluntary sector. The blend of
        public and community sector organisations is a positive outcome for the first call
        of EQUAL.

3.      Two DP members describe themselves as from the private sector. Discussion
        with DP representatives during Active Reflection sessions suggested that the non-
        profit making nature of the programme, and difficulties in finding individuals able
        to represent the sector and to allocate time and resources to partnership activities
        contribute to this low representation. The Private sector may, however, be more
        appropriately involved as participants in DP project initiatives where there are
        immediate practical benefits to involvement. The potential for this can be seen in
        the training activities of Diversity Matters, the research work of the Futures
        Project, for example.

4.      Only one DP member described themselves as belonging to the „social partners‟.
        EU recommendations in relation to the National Action Plan for the UK

                                                                                                 36

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


        encourage greater consultation and involvement of the „social partners‟ in
        achieving modernisation of workplace practice. In addition, the NI EQUAL CIP
        states that; „The involvement of social partners is expected in all DPs in EQUAL
        and this should involve them in specific actions‟.


          Recommendation:
          Rather than as DP members, the involvement of the Private Sector might be
          alternatively assessed as part of the DPA targets and monitoring returns.

          Measures to increase the participation of social partners in EQUAL DPs for
          the second call should be explored, for example, targeted promotional material
          and presentations to key forums in Northern Ireland such as Concordia.



5.1.2 Representativeness:
      Seniority of Membership

1.      The potential benefits of good representation across key stakeholders lie in
        maximising the potential to inform policy through the knowledge and links of
        more senior partnership members, sustaining the benefits of context innovation,
        and enabling horizontal mainstreaming. The seniority of partner organisation
        representatives will be strongly related to their capacity to influence the thinking
        of their own organisation and that of others.

2.      Figure 2 shows the Health Check results for this aspect of representation. Scores
        were determined from attendance at the three Active Reflection Sessions
        (Sessions 1, 2 and 3 in Figure 2), and from DP visits carried out shortly after each
        session, during which DP membership information was reviewed.

3.      Scores have remained unchanged over the period of Action 1 and 2. Three of the
        six DPs have sustained good representation at a senior level, a further 2 have
        maintained some representation, whilst only 1 (Accord) was found to have more
        limited representation at senior level. This largely reflected the extensive
        involvement in Accord of community sector organisations.

4.      Seniority was felt by Digital Step to be a key benefit in the early take-up of their
        training model by the Council for the Curriculum and Examinations and
        assessment (CEA) for piloting a new occupational qualification in schools in
        September 2003. Likewise, the extensive links into the policy framework shown
        by Diversity Matters is also enabled by the seniority of its members. The wider
        potential impact on mainstreaming is illustrated by the fact that it is the same
        five DPs with good representation at senior level that score more highly in Area 6
        of the health check, related to effective mainstreaming (reported under 5.3
        figures 9 and 10).

                                                                                                 37

TL Associates
  Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


                                                 Fig 2 – Partnership Member Position in
                                                           Organisation



       Key to Vertical Scale                     Partnership - Members Position in Organisation (1.2
                                                                        a)
0 All or most members are at a junior
position in their organisation
                                             3
1 A significant proportion of                                                                   Session 1
members are at a junior position in
their organisation
                                             2
                                                                                                Session 2

2 Most members are at a senior               1
position in their organisation                                                                  Session 3
                                             0
3 All members are at a senior
                                                    Di it al

                                                     Fu t y
                                                             rd




                                                              s


                                                           en
                                                              S
position in their organisation



                                                          re
                                                         PP
                                                            i
                                             co



                                                         rs




                                                       om
                                                         g


                                                       tu
                                                      Di

                                                      ve
                                           Ac




                                                     W
                                                   g
                                                un
                                              Yo




    Recommendation:
    Seniority amongst DP members is not of course, the only factor influencing overall
    success. However given the importance placed in EQUAL on mainstreaming as an
    indicator of success, we would suggest that applicants for the second call are guided
    towards securing representation at a senior level from partner organisations.



  5.1.3 Clarity of Purpose

  1.      The literature examining factors affecting successful partnerships place
          considerable emphasis on clarity of purpose. (See Annex 15 - Bibliography).
          Three performance indicators explored the level of shared understanding of the
          strategic aims and objectives of the partnership; the consistency with which
          different DP member described aims and how they are met, the extent to which
          DP members referred to the need to inform policy and practice in describing aims
          and objectives, and the degree to which a common language was evident or was
          being developed in discussing the partnerships mission. These were explored
          during three Active Reflection Sessions and through the members surveys at the
          end of Action 1 and Action 2. Figure 2 below shows the combined descriptor
          scores for these three indicators for each DP over the three Active Reflection
          Sessions.
                                                                                                            38

  TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




                                                Fig (3) – Clarity of Purpose



                                                   Clarity of purpose (2.1 a,b,c)

Key to Vertical Axis

Scores represent the result of              8                                       Session 1
combining individual scores
from the three performance                  6
indicators. The maximum score                                                       Session 2
on any one indicator is 3,                  4
therefore the maximum score
possible is 9                               2                                       Session 3

                                            0


                                                 Fu it y
                                                  Di rd




                                                        s
                                                        S
                                                  ve l




                                                      en
                                                Di ita


                                                      re
                                                    PP
                                                    co



                                                    rs




                                                  om
                                                    g


                                                   tu
                                                 Ac




                                                 W
                                               g
                                            un
                                         Yo




2.     At a programme level at the end of Action 1 there was some variation across the
       DPs as to whether there were shared aims, however scores were on the whole
       quite high. Scores improved in the early stages of Action 2, and by the end of the
       evaluation period all of the partnerships displayed clarity of purpose. Whilst this
       is a generally positive outcome after the first year of Action 2, nevertheless it has
       required a period considerably longer than provided by Action 1 to achieve in
       some cases. As such there would be merit in providing support to DPs to check
       and strengthen clarity of purpose during Action 1 so that Action 2 focuses on
       implementation issues. An example of a support intervention might be Clarity of
       Purpose sessions to help crystallise a shared vision.


        Recommendation:
        Intervention during Action 1 to support DPs in achieving Clarity of Purpose




                                                                                                 39

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


5.1.4 Role Clarity

1.      Alongside Clarity of collective purpose, clear individual roles for DP member
        organisations are important as part of the „joined-up approach‟ enabled by
        forming a DP. Developing these complementary roles lays the foundations for the
        sustainability of context innovations and future collaborative working across
        member organisations after the EQUAL project has finished, as a key element of
        horizontal mainstreaming. Role Clarity is also important for reconciling a range
        of diverse values and interests amongst partners from different organisations and
        sectors. A lack of role clarity may lead to duplication of organizational effort,
        conflict (or competition) and burnout of DP members. This aspect was explored
        during Active Reflection Sessions and in the first DP member survey. The results
        are shown below.

                                                 Figure (4) – Role Clarity

      Key to Vertical Scale
                                                         Role Clarity (4.3 a)
 0 Partnership members totally
 lack role clarity
                                            3
 1 The role of the lead partner
                                                                                    Session 1
 member is apparent, however
 the contribution of less than              2
 half of other DP members is                                                        Session 2
 clear
                                            1
 2 More than half but not all of                                                    Session 3
 members are clear about their
 own and other roles within the             0
 DP
                                                 Fu it y
                                                  Di rd




                                                        s
                                                        S
                                                  ve l




                                                      en
                                                Di ita



                                                      re
                                                    PP
                                                    co



                                                    rs




                                                  om
                                                    g


                                                   tu
                                                 Ac




                                                 W




 3 There is full agreement
                                               g
                                            un




 within the DP about individual
                                         Yo




 roles and responsibilities



2.      The results show that individual organizational roles were slow to become clear in
        the majority of DPs and only two had established clear roles for more than half of
        their members by the end of Action 1, one of which (PPS) was a small partnership
        of just three member organisations. As with overall clarity of purpose, role clarity
        improved in most cases during the early stages of Action 2 and was fully clarified
        for two DPs by the end of the first year of Action 2. There remains some residual
        lack of clarity of roles amongst members in the other four DPs, however,
        reflecting the complexity of these partnerships, which on the whole are larger, and
        of their projects.

                                                                                                 40

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


3.      One of the potential benefits of Action 1 is in enabling DPs to achieve greater
        clarity around the individual roles and responsibilities of members. However DPs
        were working largely in abstract terms before Action 2. Action 2 enabled more
        concrete practical actions which could better inform members of the nature of
        their roles and responsibilities.

4.      It is important partnerships under EQUAL are formed on the basis of clear
        functional linkages between organisational members and not as a marriage of
        convenience. It is therefore important that role clarity is clearly established during
        Action 1 even if this is in largely „theoretical‟ terms. It would also be realistic,
        however, to recognise that full clarity of purpose and organizational role clarity
        will only develop as concrete actions are taken forward, and to consolidate these
        roles, a review session a short period (perhaps three to six months) into Action 2
        would be advisable. Sustainable working relations represent a key potential
        outcome of EQUAL and it is recommended that regular reflection on
        organizational roles is encouraged throughout Action 2.

           Recommendation:
           Role Clarity should be clear at least in theory in Action 1. It should then be
           regularly reviewed and developed throughout Action 2 as a key focus of the
           EQUAL programme and its potential outcomes.


5.1.5 Collaboration and the Partnership Process

1.      In complement to Clarity of Purpose and Role Clarity, an effective partnership
        needs to manage the way in which members work together as well as the
        implementation of the project14. To explore this aspect three performance
        indicators were examined during the Active Reflection Sessions; Balance of Task
        and Process, Managing Challenge and Conflict and the Effective Range of Team
        Roles. The tools used to assist in assessing these indicators during the active
        reflection sessions are described in Annex 5.

2.      Task/process looks at how a partnership balances the activities required to
        achieve the task and those required to ensure that group process is maintained (for
        example interpersonal issues). A framework of indicative behaviors was
        employed drawn from recognised management training resources 15.

3.      Conflict Management. Research has demonstrated that there are broadly five
        approaches that a partnership can use to deal with conflict. The framework 16 used
        for this aspect (See Annex 5) captures the approach that the DPs used.

14
   Unlike Role Clarity, which refers to the contribution each member organisation is making to the
partnership, working together places focus on group dynamics and inter-personal relations in discussing
and developing agreed actions
15
   Handy with contribution from Deutsch and Hawthorne
16
   Thomas – Kilmann
                                                                                                          41

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




4.     Team Roles. An effective partnership utilises a broad range of Belbin‟s team
       roles. Belbin is a well researched management writer on teams and what makes
       them effective. The framework used for the Active Reflection Sessions
       (reproduced in Annex 5) looked at the range of roles utilised by each DP. The
       Scores indicated the extent to which different roles were observed by the
       evaluators in the way DPs addressed problem solving discussions within the
       Active Reflection Sessions.

5.     The results of the scores for each of these performance indicators were combined
       to give an overall score for partnership process. Individual scores for these
       indicators for each DP can be found in Annex 6 (indicators 4.5 and 4.6a) and b))


          Key to Vertical Scale                Partnership Process (4.5 a), 4.6a), b))

       The maximum score for
       each of the three
       performance indicators              9
       examined in this area of                                                     Session 1
                                           7
       the health check was three,
       making a maximum                    5
                                                                                    Session 2
       possible for the total
       combined score 9.                   3
                                           1                                        Session 3

                                          -1
                                                        en
                                                 D d

                                                  er l




                                                W S
                                                          s
                                                Fu ity
                                              D gita
                                                       or




                                                       re
                                                      PP
                                                   om
                                                      s
                                                   cc




                                                   tu
                                                    i
                                       A


                                                iv




                                              g
                                           un
                                         Yo




                                               Figure (5) – Partnership Process



6.      The overall picture shows that there was steady progress in developing
        partnership working across all DPs during Action 1 and 2. In one case, Accord,
        progress was made primarily during Action 2, whilst there appeared to be a
        reduction in partnership working at the start of Action 2 for Digital Step. One
        partner left the DP at this time which may have required a re-adjustment in roles
        and responsibilities.




                                                                                                 42

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


7.      Two of the DPs, Diversity Matters and YWMW demonstrated high scores across
        all three performance indicators. These were also the partnerships with a strong
        history of partnership activity amongst members and an associated belief in
        partnership working (Indicators 1.4 and 2.2, Annex 6).

8.      Accord has perhaps struggled most to make progress. During case study work in
        preparing the thematic studies presented in Annex 7, however, the Accord DP
        manager reported that understanding of the potential of partnership working has
        developed considerably over the course of Action 2. As Indicator 1.4 shows
        (Annex 6) this has been built from a low base, from a situation where “it was
        difficult to get the partners into the same room” to one in which a level of
        familiarity and comfort has been established and new project ideas are being
        discussed by partners.

9.      The patterns of progress shown across DPs reflect both the history of partnership
        working (Diversity Matters, YWMW) and progress with project implementation.
        Action 2 again assisted in the partnership development process. Where concrete
        actions were slower to get of the ground (for example, the Futures Project and
        Accord) partnership development has also been more gradual.

10.     The second survey of DP members at the end of the first year of Action 2 showed
        that 100% of respondents felt they had been able to contribute their views and
        know-how effectively during DP meetings. In most DPs the value and meaning
        of partnership working has become apparent to members and as a result has
        developed considerably during the past year. Examples of how partnerships have
        developed in specific cases are provided in the Thematic Study Sheets (Annex 7).

11.     Generally, then, partnership working has developed reasonably often despite
        difficult political and cultural barriers (Accord). The main recommendation to
        make here is that the value and benefits of partnership working are a specific point
        of focus for DPs during early support and where appropriate, a needs analysis to
        help address gaps or problems in partnership working would be of value for those
        DPs experiencing difficulties.

          Recommendation:
          Highlight the value and potential of partnership working during Action One,
          perhaps drawing on examples from the current round.

          Monitor and where necessary provide additional support to DPs to develop
          effective partnership process skills.




                                                                                                 43

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


5.1.6 Partnership policies, principles and procedures

1.      It is important that when a group of individuals representing a diverse range of
        organisations (with a diverse range of principles and values) come together that
        they agree a set of Principles and procedures which are common but still
        accommodate the individual differences. These were looked for during DP visits.

2.      At the end of Action 1 there were a significant number of DPs without written
        Principles and Procedures. At the end of this phase of the evaluation without
        exception each DP has significantly improved their procedures resulting in DPs
        with effective operating systems.

          Recommendation:
          DPs are facilitated during Action 1 to finalise principles, policies and
          procedures


5.1.7 Social Capital

1.      Social capital has become recognised as an important but often hidden ingredient
        of effective development, economic growth and governance. The potential for
        social capital to be a more explicit focus of development policy has been
        discussed by the OECD17 and, more locally, by the Northern Ireland Economic
        Council18. It is defined by the British Cabinet Office as all those institutional
        arrangements, networks and relationships which promote understanding, trust and
        mutual respect, and allow (partnerships) to pursue shared goals more effectively,
        improve information flows and generally improve quality of life.19

2.      The formation of social capital in the EQUAL context represents a possible
        overall indicator of success for partnership working and the potential for longer
        term working relationships amongst DP members with the associated potential for
        efficiency and effectiveness gains in address employability and discrimination
        issues. CENI has conducted a programme of research scoping the development of
        social capital indicators for the Voluntary and Community Unit of DSD which
        have informed this aspect of the evaluation.

3.      The Partnership Health Check offers some capacity for assessing the
        accumulation of social capital in DPs by combining key indicators across the
        Health Check which represent the operational „ingredients‟ of social capital. The
        following indicators from the Partnership Health Check are indicative of the
        formation of social capital (1.1a, 1.4a, 2.2a, 4.3a, 4.4a, 4.6a, b and 5.3a - details

17
   The Contribution of Human and Social Capital to Sustained Economic Growth and Well-Being
International Symposium Report Human Resources Development Canada (HRDC) and Organisation for
Economic Co-operation and Development (OECD)September 2001
18
   Local Development: A Turning Point. NIEC. Occasional Paper 13. December 2000
19
   Cabinet Office Performance and Innovation Unit, Report on Rural Economies, 1999 Annex 3.
                                                                                            44

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


        can be found in the Partnership Health Check in Annex 8). The maximum „social
        capital‟ score is 24. The formation of social capital across the DPs is illustrated in
        figure (6) below:-

                          Figure (6) – Formation of Social Capital



                                    Formation of Social Capital




                   24
                   22
                   20
                   18                                                         Session 1
                   16
                   14
                                                                              Session 2
                   12
                   10
                    8                                                         Session 3
                    6
                    4
                    2
                    0
                                                 l

                                    Fu y
                        rd




                                                s



                                            en
                                              S
                                            ita




                                           re
                                              t



                                          PP
                                           si
                      o




                                       om
                                         ig



                                        tu
                   cc




                                        er
                          D

                                     iv
                  A




                                    W
                                   D




                                  g
                               un
                             Yo




4.      Figure (6) illustrates the positive impact that EQUAL has had overall on the
        development of social capital amongst all of the DPs. There has been an upward
        trend since Action 1.

5.1.8 Conclusion

        As an outcome of EQUAL in Northern Ireland, partnership working and the
        underlying potential for long term joined up working amongst partner
        organisations has for the most part developed well. The main recommendation for
        enhancing this, in conjunction with the implementation of projects, is to place a
        stronger focus on partnership working throughout, from promotional roadshows to
        the advice and support offered during Action 1 and Action 2. Success in this area


                                                                                                 45

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


          has the potential benefit of providing a strong foundation for horizontal
          mainstreaming of partnership working in future.


5.2       Innovation and Learning
1.        EQUAL is intended to test and promote new ways to combat all forms of
          discrimination and inequalities by those groups most disadvantaged in the labour
          market, and seeks to address policy gaps or weaknesses. The Policy Proofing
          exercise reported in Section 4.4.2 suggests that DP projects are well placed to
          make a positive contribution to policy and practice gaps. In this section the
          operational issues of taking forward Innovation and learning as a prelude to
          mainstreaming are examined.

5.2.1 The Overall Pattern of Innovation

1.        Three types of innovation are recognised within EQUAL:

2.        Goal Oriented: the development, for example, of new vocational qualifications,
          new areas of employment, or new models for working with disadvantaged groups.
          A specific „product‟ or outcome forms the key focus of the innovation.

3.        Process-Oriented: improvements in methods, use of technology or other
          development tools, new training materials or delivery methods.

4.        Context Oriented: changes to the institutional relationships or structural
          frameworks within which labour market interventions take place, for example the
          formation of new networks or linkages between actors in delivery of support.
          This is particularly important in relation to Partnership working within the DPs
          themselves discussed in Section 5.1

5.        Evaluations of EMPLOYMENT and of the New Deal Innovation Funds have
          highlighted the difficulties in judging how a project is innovative20. This is also
          reflected in the EQUAL DPAs. Often, Section 8 of the DPAs which relates to
          innovation had not been completed in all the detail requested by the guidance
          notes. This was particularly so in relation to the nature of the innovation (process,
          goal or context) for which three of the six DPs had not been specific, but also in
          describing the level of innovation (local, regional, national or transnational),
          which none of the DPs addressed.




20
     Page 60 Final Evaluation of the EMPLOYMENT Community Initiative. June 2001
     New Deal Evaluation Report No 8: New Deal Innovation Funds June 2003

                                                                                                 46

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


6.      To assess the pattern of innovation therefore, three perspectives were compared:

       a) Results from a postal survey of DP members and their individual assessments
           of the innovative features of their project, thereby drawing on partner
           organisations perspectives

       b) The scoring by lead partners included in Section 8 of the DPA.

       c) An independent assessment by the evaluators of the types of innovation,
           suggested by the details provided in Section 8.

        The results are shown in Table 7 below:-

                                 Table 7 - Innovation Types

                              Member                Applicant self           Independent
       Innovation Type        Questionnaire         assessment               assessment (Section
                              (members own          (Section 8 of DPA)       8 DPA)
                              description of
                              innovation)
       Goal                   19%                   31%                      23%
       Process                42%                   42%                      48%
       Context                39%                   27%                      29%

7.      The analysis suggests the following;

                    Process innovation was rated the main form of innovation in all three
                     assessment exercises.

                    Context innovation was more likely to be identified by DP members
                     than lead partners, reflecting the significance to member organisations
                     of joining a partnership approach

                    Goal innovations were emphasized more by lead partners.

                    Individual member survey results showed that in general, views on
                     innovative content were quite diverse.

8.      Thinking around innovation was re-visited as part of the second survey of DP
        members at the end of the first year of Action 2. Respondents were asked for
        examples of how their project or the DP had been innovative to date (See Survey
        Question 2.4 Second Survey questionnaire Annex 11). The comments were then
        categorized into the different forms of innovation. The results are shown in
        Table 8



                                                                                                 47

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


                  Table 8 Views on Innovation during Action 2

                 Type of Innovation              % responses
                 Goal                            8
                 Process                         42
                 Context                         42
                 Don‟t Know                      8


9.      The results continue to place significant emphasis on relationship building
        (context innovation) and process. For example; collaborative working was
        regularly highlighted;

       “More collaborative thinking from strands of organisations that previously ran
       parallel rather than working together” (Futures DP Member)

       “I think the DP has been innovative in the way it works as a partnership” (Accord
       DP Member)

       Process was also linked to context in one or two cases;

       “The level of cooperation and joint working (and) operation of the project in the
       Young Offenders Centre creates new opportunities to develop release plans for a
       target group of inmates whose needs would not otherwise be addressed in a
       coordinated way” (PPS Lead Partner)

       20% of responses linked process or context innovations to examples of
       empowerment;

       “On hand evaluation techniques are helping hubs and their target groups to have
       more of a say in how projects roll out” (Accord DP member)

10.     The emphasis on process and context innovation was re-emphasised during
        interviews with DP project managers. The more detailed pictures and analysis of
        project interventions and activities produced from these interviews are presented
        as a series of four thematic studies in Annex 7. A „pathway‟ concept was
        common across DP actions combining process and social capital building;
        Pathways for Outreach‟; (ACCORD and the Futures Project), the creation of
        Learning Pathways (Digital Step, ACCORD, YWMW) Integration Pathways
        (Personal Progression Systems) and Pathways to Acceptance (Diversity Matters,
        YWMW).

11.     The focus within EQUAL on the most marginal social groups and individuals
        suggests process will be a key area of importance in contrast to Goal. Innovation
        often lies in reaching and guiding marginal groups into existing methods of
        support. Likewise, it is clear that partnership working is also proving to be an
                                                                                                 48

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


          important source of innovation for many at this time. This profile may change
          toward the end of each DP project as attention turns to product, however it is
          important its significance at this time is recognised and communicated as part of
          mainstreaming and dissemination.

5.2.2     Understanding of Innovation

1.        The different start points in understanding innovation between DP members and
          lead partners noted in 5.2.1 highlights a potential for the different benefits to be
          lost unless a common understanding of the innovative content of the DP‟s work in
          all its aspects is established and maintained. The understanding of innovation
          amongst DPs and DP members was individually examined as part of the
          Partnership Health Check. The Health Check is described in Section 5.1, and the
          reader is also referred to indicator 2.3a in the Descriptor Sets provided in
          Annex 4. It was assessed as part of the three Active Reflection Sessions held
          during the evaluation and by the two surveys of DP members at the end of Action
          1 and 2. The results are shown in the figure below.

                                                 Figure (7) – Understanding of Innovation
                Key to Vertical Scale
                                                                   Innovation
        0 The innovative content of the
        project has not been discussed / there          Understanding of Innovation (2.3a)
        is poor awareness of this as a key
        element of the project                     3
                                                                                            Session 1
        1 Description of innovative content
        of the project is poorly articulated       2
        and / or varies considerably between                                                Session 2
        members                                    1
        2 Description of the innovative                                                     Session 3
        content of the project is clearly          0
        described by some members but not
                                                         Fu it y
                                                          Di rd




                                                                s
                                                                S




        by others
                                                          ve l




                                                              en
                                                        Di ita


                                                              re
                                                       g PP
                                                            co



                                                            rs




                                                          om
                                                            g


                                                           tu
                                                         Ac




        3 Description of the innovative
                                                        W




        content of the project is clearly
                                                    un
                                                 Yo




        described and understood by all or
        most members

2.        The results show that some DP members in most of the DPs are able to provide a
          clear description of the innovative approach of their project, but others are less
          clear. This has not changed during Action 2. The challenge for Accord lies in the
          size and complexity of the partnership, encompassing a large number of
          community groups representing the „hubs‟ each of which have their own specific
          training and outreach concerns. Building a common understanding of innovation
          will take time.

                                                                                                 49

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


3.      The variable results contrast with a consistently strong shared clarity of purpose
        amongst DP members (Section 5.1 Fig 3), where the focus is agreement around
        aims and objectives. The results for innovation are more similar to (but not
        always coincident with) the variation in clarity amongst some DP members about
        their individual organizational roles (Section 5.1 Fig 4).

5.2.3 Conclusions: innovation

1.     Taken as a whole, the picture of understanding around innovation amongst DP
       members presented here suggests some differences between DP members and lead
       partners, particularly around goal innovation. In some ways this may not be a
       problem. Key members of DPs may be able to articulate the innovative nature of
       the project on behalf of the DP and as such assist in vertical mainstreaming (see in
       particular observations in relation to the importance of DP member representatives
       being from senior positions in their organisations in Section 5.1.2). Reviewing the
       links between context, process and goal innovations and relating these to the
       operational roles „on the ground‟ of individual member organisations will however
       be important in consolidating the potential for horizontal mainstreaming and
       future joined up working beyond the lifetime of the DP.

2.      As such it is recommended that DPs are encouraged to take time out to reflect on
        how the various areas of DP and project work are coming together to form a new
        intervention – innovation may be easier to apprehend after rather than before
        practical work has taken place.

5.2.4 Partnership Learning

1.      As a crucial part of the Policy Learning Cycle described in Section 3.1.3, and of
        the learning environment within each DP it is important that DPs adopt an active
        approach to monitoring and evaluation, not only in developing systems, but in
        working with findings and applying them to the development of best practice. An
        indication of how far partnership working may be converting into changed
        attitudes and approaches on the part of partner organisations comes in the extent
        to which they acknowledge learning from their involvement in the project and
        from each other.

2.      The Partnership Health Check included a number of performance indicators under
        Area 5 (A Learning Partnership) which assessed the extent to which DP members
        were able to learn from one another, the development of monitoring and
        evaluation systems and procedures and the extent to which there was DP member
        involvement in interpreting results. (see Annex 4, and indicators 5.1 and 5.2 for
        details of the indicators and scales used).




                                                                                                 50

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


3.      A postal survey of DP members at the end of Action 1 explored member
        expectations for possible benefits from being in their DP. Responses were post-
        coded into the headings shown in Table 9:-

4.      The picture of expectations illustrates that the highest percentage of responses
        related to learning, along with networking and partnership, particularly if the two
        different forms of learning (new skills, and new approaches) are combined.

           Table 9 - Benefits of Partnership - Responses from DP members

 Benefit Type                                                    Proportion of responses
 Learning: for example, learning what works / doesn‟t 25%
 work, learning from others

 Development: for example, learning new                  skills, 14%
 improving organisational practice

 Networking / partnership building                               21%

 Impacting on policy and practice                                18%

 Achieving organisational goals                                  13%

 No answer given                                                 8%



5.      The second survey of DP members re-visited the question of DP learning. It
        asked respondents if they felt they had been able to learn from other DP members,
        or from implementation of the project. In each case, they were also asked to give
        examples. 79% of respondents felt they had been able to learn from other DP
        members. Of these, 73% were able to cite clear examples of what they had learnt.
        89% said they had been able to learn something from implementation of their DP
        project, of which a lower percentage (65%) were able to provide an example.

6.      Learning reported by respondents was a roughly equal mix of new skills and
        learning of what works / doesn‟t work. Examples of learning highlighted more
        understanding of target groups and issues of engagement with beneficiaries,
        training techniques, and better familiarity with other organisations with similar
        goals, including recognition of the value of the community sector role. Other
        examples given, particularly in relation to project implementation, were less
        substantial; for example “patience with bureaucracy” or “shared information”.
        There was a tendency for responses here to be generalized and a little vague.

7.      The generalized nature of the examples given of learning from implementation,
        and the less convincing responses around learning from project implementation
        (in contrast to learning identified from other DP members), may reflect the
                                                                                                 51

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


        relatively early stage of Action 2 and project implementation. Whilst there is
        evidence to suggest that members are for the most part engaging with DP
        development work and learning from it, there might have been more evidence of
        early lessons being learnt for ongoing refinement of approaches. The extent of
        this aspect of partnership learning will be influenced by the approach to
        monitoring and evaluation, which is now discussed.

5.2.5 Monitoring ‘Soft’ Outcomes

1.      As part of the approach to effective policy mainstreaming, DPs will need to
        demonstrate how gaps in support can be reliably addressed and with what
        resource requirements. This requires that in implementing projects, partnerships
        adopt an active approach to monitoring outcomes and process as well as outputs.
        Evaluating process will additionally benefit from a successful approach to
        Empowerment, discussed in Section 5.3.

2.      DPs generally all have members who appreciate the significance of innovation,
        however the emphasis on process and context innovation also places emphasis on
        the need for appropriate systems able to gather meaningful information on
        qualitative impacts and the processes.

3.      Indicator 5.2 in the Health Check explored the development of monitoring
        systems and procedures. Assessments of this indicator took place as part of the DP
        visits on three occasions (end of Action 1, mid point and end of the first year of
        Action 2). The interim evaluation report after Action 1 recommended more
        support to DPs to develop systems to monitor „soft outcomes‟ and that this be
        progressed quickly during the early part of Action 2. The results of subsequent
        assessments during Action 2 are shown in Figure 8 below.




                                                                                                 52

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




                                           Figure (8) – Monitoring Qualitative Outcomes


            Key to Vertical Scale                                  Innovation
                                                     Monitoring Qualitative Outcomes (5.2a)
     There are no qualitative monitoring
     systems in place
                                                 3
     Qualitative monitoring systems are                                                  Session 1
     poorly developed with little
                                                 2
     involvement of DP members
                                                                                         Session 2
     Qualitative systems are fairly or           1
     well developed, but involvement of                                                  Session 3
     DP members is low
                                                 0
     Qualitative systems are well




                                                      Fu it y
                                                       Di rd




                                                             s
                                                             S
                                                       ve l




                                                           en
                                                     Di ita



                                                           re
                                                    g PP
     developed and DP member                             co



                                                         rs




                                                       om
                                                         g


                                                        tu
                                                      Ac

     involvement has been significant




                                                     W
                                                 un
                                              Yo
5.     Figure (8) illustrates that for the most part, DPs have made slow progress in
       establishing monitoring systems and procedures capable of supporting effective
       partnership learning. DPs have, however each explored ideas and possible
       solutions which could be shared as a significant programme level resource in
       arriving at the building blocks of an evaluation. For example:-

               ACCORD have been using a system able to record qualitative information
                about beneficiaries moving through training programmes at hubs.

               Futures have noted the value of having a dedicated evaluator as part of the
                DP facilitating reflection and discussion on learning points, and have been
                exploring the key principles underpinning „empowerment evaluation‟.

               PPS have been developing approaches to profiling beneficiaries as a baseline
                against which to understand the efficacy of different interventions.

               Diversity Matters and YWMW have developed clearer pictures of cultural
                barriers that could be incorporated into soft monitoring Systems.

               Digital Step have considerable capacity to develop and provide computer
                based assessment tools as part of their work with young people.

6.       Development of robust approaches to soft outcome measurement is a recognised
         challenge across organisations and programmes addressing employability.
                                                                                                 53

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


        Further broad research is due to be completed soon by the ESF Evaluation Team
        based within the Department of Work and Pensions, which aims to develop a
        more systematic framework to guide projects. Recent work by CENI for DSD on
        the measurement of social capital should also be examined and developed, given
        the significant emphasis across DPs on relationship and trust building as key
        building blocks of their approach (see Annex 7). Likewise, descriptor scales
        linked to an appropriate logical framework, illustrated as part of this evaluation,
        can be developed to catch qualitative change. Descriptor scales are being used,
        for example, in evaluating impacts within the Rural Development Programme in
        Northern Ireland.

5.2.6 Conclusion

1.      Developing effective and innovative monitoring and evaluation systems, and best
        practice in their use as part of partnership learning should be a key focus for
        EQUAL. It represents a substantial contribution to realising project innovation
        and the achievement of horizontal and vertical mainstreaming. Soft monitoring
        systems represent a key innovation at programme level for which there are a
        number of initial ideas and approaches emerging across DPs which can be built
        upon. The formation of a sub group of the NTN to look at Soft Outcomes and
        Distance Traveled is a positive step, the momentum of which needs to be
        proactively sustained with a clear action plan including training and development
        events.

       Recommendations:
       Current DPs need input and facilitation in the development of appropriate
       systems for monitoring „soft‟ outcomes.

       An action plan and programme of training and development events should be
       implemented

       A process of providing DPs with input and support to develop monitoring
       systems for „soft outcomes‟ should be introduced during Action One.



5.3     Empowerment

1.      DP projects must provide for the involvement of those targeted for support. This
        includes involvement of intended beneficiaries in influencing the design of the
        project and in the evaluation of project effectiveness. There is also scope for
        dialogue with beneficiaries to highlight barriers in the regulatory environment
        which prevent progress toward employment.

2.      A key reference distributed by the NSS to NI EQUAL DPs early in Action 1 and
        followed up during Action 2 in a workshop with DPs is the „Guide to Measuring

                                                                                                 54

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


           Soft Outcomes and Distance Traveled‟ based on a review of current practice in
           2000 (Dewson et al 2000) 21. This provides a range of suggestions for soft
           outcome indicators and data collection. Key to the approach is the involvement of
           beneficiaries in assessment exercises before, during and after DP support.

5.3.1 Arrangements for Empowerment

1.         Arrangements by DPs for Empowerment were assessed during DP visits. The
           second DP member survey also explored the understanding and benefits perceived
           by members of empowerment at the end of the first year of Action 2. The results
           are illustrated in Figure (9) below


                                            Figure (9) – Arrangements for Empowerment


                                                               Empowerment
          Key to Vertical Scale                      Arrangements for Empowerment (5.3a)

     0 There are no arrangements for
     beneficiary involvement                     3
                                                                                    Session 1
     1 Arrangements are poor and
     ineffective at involving                    2
     beneficiaries                                                                  Session 2

                                                 1
     2 Arrangements are fairly well                                                 Session 3
     thought out but beneficiary
     involvement is inconsistent or              0
     limited in scope
                                                      Fu it y
                                                       Di rd




                                                             s
                                                             S
                                                       ve l




                                                           en
                                                     Di ita



                                                           re
                                                         PP
                                                         co



                                                         rs




                                                       om
                                                         g


                                                        tu
                                                      Ac




     3 Arrangements are well thought
                                                      W
                                                    g




     out and enable a high degree of
                                                 un
                                              Yo




     involvement by beneficiaries



2.         The aspirations underpinning the Principle of Empowerment in EQUAL are
           challenging given the social groups which are to be addressed. Given this context
           the majority of DPs have fairly well thought out arrangements for involving
           beneficiaries. In the case of Personal Progression Systems, which shows a low
           score relative to other DPs in the Figure, the scope for empowerment is limited at
           this time due to the nature of the beneficiary group and the needs being met,
           however the approach taken by PPS acts to empower individual beneficiaries to
           prepare more effectively for life post release. This individual empowerment is
           embedded in a number of the DP projects (See Annex 7).

21
     Produced by the Institute for Employment Studies for the DfEE UK
                                                                                                 55

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




3.      Diversity Matters and YWMW focused specifically on piloting or developing
        training course and content during the first year of Action 2 which placed
        emphasis on gathering detailed information from potential beneficiaries. For
        YWMW the findings should enable training course content and planning which
        builds on the particular cultural needs of different minority groups. YWMW are
        also experimenting with a „Shadow‟ DP composed of beneficiary representatives.

4.      The Futures Project recently discussed approaches to communication and mutual
        awareness raising with community groups in their target area and as a result
        developed a community forum. The forum will enable the DP to update
        community groups on developments and test ideas as key contacts within the
        communities of target beneficiaries. Likewise, community groups will be able to
        advise on other initiatives they are involved with or planning. Direct contact with
        beneficiaries was not yet possible as very few were involved in the project at the
        end of the first year of Action 2. The Futures Project is, however, developing
        plans for what it calls „Empowerment Evaluation‟, one in which beneficiaries are
        at the centre of and benefit from, the evaluation process.

5.      Both Digital Step and Accord have focused on providing beneficiaries attending
        training courses with some scope to influence the timing and content of course.
        Training hubs within Accord, for example, use an arts based project as a vehicle
        for imparting vocational skills in a practical and applied manner (See Thematic
        Study 2, Annex 7). Beneficiaries discuss and develop ideas for this project. As a
        result there is a degree of ownership of the project, and in this context vocational
        skills are imparted as support to taking forward aspects of the project work. The
        result is contributing to a more equal relationship between beneficiaries and
        trainers.

4.      The second survey of DP members showed that 68% had noticed examples of
        empowerment in relation to any beneficiaries of their project of which 60% could
        provide a clear example. These mainly concerned consultation with beneficiaries
        participating in pilot exercises, or some examples of increased individual self
        esteem. Fewer respondents however (38%) felt they had gained any useful
        insights from beneficiaries to date.

5.3.2 Conclusions

1.      Overall, the evaluation found that ideas for empowerment were being developed,
        but for many DPs they were not yet fully operationalised. This again reflects the
        relatively early stage of project development in some DPs after the first year of
        Action 2




                                                                                                 56

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland



        Recommendation:
        DPs should be encouraged to continue to develop and take forward the
        involvement of beneficiaries, by sharing practice and where necessary being
        given specialist support.

        Beneficiary involvement should be included as part of the action plan to
        develop approaches to monitoring and evaluation noted under 5.2.6 above



5.4     Mainstreaming
1.      A measure of the success of EQUAL is the extent to which the lessons learned
        from DP projects inform mainstream policy. To support DPs in this ambition, a
        mainstreaming approach and strategy has been arranged by the NSS and DEL for
        each thematic priority. Key to the mainstreaming approach is the Thematic
        Networking Group (NTN). This group brings together representatives from DPs,
        relevant Government Departments, the Equality Organisations and Social Partners
        to provide an opportunity for DPs to present findings and for these to be assessed
        for their potential to inform mainstream thinking. The evaluation of the NTN
        process is reported is Section 6.3.3.

2.      Equally important to the success of mainstreaming is the preparation and strategic
        thinking at individual DP level, drawing on effective monitoring and evaluation
        and associated partnership learning. Indicator 2.1b under Area 2 of the Partnership
        Health Check (see Annex 4 Descriptor sets) explored the extent to which DP
        members refer to the need to inform policy and practice in describing the aims and
        objectives of their DP. This was explored through the three Active Reflection
        sessions held during the evaluation. In addition, indicator 6.1 under Area 6 of the
        Partnership Health Check assessed preparation for mainstreaming among DPs.
        This was explored as the key theme for the first Active Reflection Session, and
        subsequently through DP visits. It was also assessed through an examination of
        Action 3 applications (where available) and through questions to individual DP
        members on mainstreaming in the second DP members survey (Annex 11
        Questions 4.7 – 4.11).

5.4.1 Awareness of the Need to Inform Policy and Practice

1.      Figure 10 below shows the results of the evaluation of 2.1b over the three Active
        Reflection Sessions.




                                                                                                 57

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


                                       Figure (10) – Awareness of Need to Inform Policy
                                                      and Practice



     Key to Vertical Scale                               Mainstreaming
                                               Awareness of need to inform policy and
0 Strategic aims and objectives
                                                          practice (2.1b)
have not been discussed
                                           3
1 Neither informing policy or                                                       Session 1
practice is mentioned in aims and
objectives                                 2
                                                                                    Session 2
2 Informing either policy or
practice is mentioned (but not             1
                                                                                    Session 3
both), and the aim is generally
shared.                                    0


                                                 Fu it y
                                                  Di rd




                                                        s
                                                        S
3 Informing both policy and                       ve l




                                                      en
                                                Di ita



                                                      re
                                               g PP
                                                    co



                                                    rs




                                                  om
                                                    g


                                                   tu
practice is clearly mentioned and
                                                 Ac




                                                W
this objective is widely shared
                                            un
                                         Yo




2.      Awareness of mainstreaming aims and objectives in EQUAL was recorded as
        high in the first Active Reflection Session at the end of Action 1 across all DPs
        and this was largely consolidated during the first year of Action 2. The
        mainstreaming Principle is well understood by the majority of DPs.

5.4.2   Identification of Outcomes and Processes for Mainstreaming

1.      There is strong recognition, then, of the aim of mainstreaming findings as a key
        principle of EQUAL. Figure 11 below shows that there has also been some
        progress toward operationalising this aim during the first year of Action 2.




                                                                                                 58

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


                                                      Figure (11) – Mainstreaming of Outcomes
                                                                 and Processes

             Key to Vertical Scale
                                                                       Mainstreaming
     0 The partnership is unable to identify                    Outcomes and Processes (6.1a)
     any key outcomes or processes to
     communicate
                                                          3
     1 The partnership is able to identify                                                         Session 1
     only one or two key outcomes to
                                                          2
     communicate and does not also
     identify associated process learning                                                          Session 2
     (alternatively, it may identify some                 1
     process learning but with indeterminate                                                       Session 3
     outcomes)
                                                          0
     2 The partnership is able to identify a




                                                               Fu it y
                                                                Di rd




                                                                      s
                                                                      S
                                                                ve l




                                                                    en
                                                              Di ita



                                                                    re
                                                             g PP
     number of outcomes, for which it is                          co



                                                                  rs




                                                                om
                                                                  g


                                                                 tu
                                                               Ac

     sometimes able to include associated




                                                              W
     process learning. (Alternatively, it may



                                                          un
                                                       Yo
     be able to identify a range of process
     learning but not always relate these to
     determinate outcomes)

     3 The partnership is able to identify a
     range of outcomes and in each case the
     associated process learning


2.         The results shown in Figure 11 include a focus on process learning, given the
           significance of this as a source of innovation in all EQUAL projects.
           Consideration of DP visits and examination of Action 3 applications 22
           demonstrated a number of ideas and plans or current actions by which process
           learning could be communicated in the achievement of policy and practice impacts

3.         The Futures project, for example, identifies New Deal, DEL‟s Targeted Initiatives
           Programme, and Belfast Local Strategic Partnerships work on Intermediary
           Labour Markets as part of its funding of the social economy as important links for
           sharing and dissemination of learning. It also plans a specific forum for
           discussing and transmitting findings, forum membership was not yet decided at
           the time of writing but allows for targeting of key policy and practice
           stakeholders.

4.         Digital Step has been able to progress process learning through liaison with the
           Council for the Curriculum Examinations and Assessment pilot of multimedia
           technology and vocational training in 8 schools during Autumn 2003.

22
     At the time of writing, 4 Applications were available for examination with two outstanding.
                                                                                                      59

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


5.      Diversity Matters is seeking to link promotion of its Employers and Diversity
        training programmes to the business case for diversity policies and their
        relationship to business success, drawing on research conducted by the Cabinet
        Office. Similarly, it plans to link the content developed for its Diversity Training
        Programmes to key policy debates on social exclusion and marginalization in the
        workplace. Each of these approaches can make systematic links between process
        findings and key policy or practice issues.

6.      The potential for policy impacts depends in DPs each developing similar
        mechanisms by which to communicate process learning as well as outcomes, as
        well as attending to the issues raised under Section 5.2 (innovation) around
        monitoring and capturing process learning. Both monitoring and dissemination
        approaches represent key programme level areas for innovation. As noted in the
        conclusion to that section, DPs are demonstrating examples which can be
        communicated across EQUAL DPs as a whole to assist in individual development

5.4.3 Involving DP members; Horizontal Mainstreaming

1.     Ensuring DP members are involved in or otherwise made fully aware of project
       findings and their significance represents an additional focus for effective
       mainstreaming. It ensures consistent communication of findings across the
       different networks DP members belong to. It also helps improve the chances that
       findings will be translated into adjustments in the practice of member
       organisations, for the future sustainability of the DP approach.

2.      The second survey of DP members explored views on progress in mainstreaming
        with the following results.

                 Just over half of those who responded had been involved in developing
                  strategies for mainstreaming findings

                 37% saw modest potential for their project to inform policy, 21% saw
                  high potential and only 10% saw low potential. Around half of
                  respondents were able to identify specific policy areas they felt could be
                  targeted.

                 Respondents were split almost 50/50 between those who felt their
                  projects had been able to document suggestions for best practice and
                  those who didn‟t or were not sure. There was no particular association
                  between these two groups and particular DPs, rather, the variation was
                  within DP members.

                 Almost all respondents said they reported back findings and updates
                  from the DP to their parent organisation. 26% did so through written
                  reports, the rest verbally. 60% felt their reports had some influence on
                  thinking within their organisation.

                                                                                                 60

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


3.       The results suggest there is more awareness and confidence amongst members in
         relation to influencing policy than practice. Improvements in partnership learning
         may help to enhance this aspect of member awareness. Building common
         awareness of process as well as product outcomes will be important in
         communicating the significance of EQUAL interventions and learning, in turn
         assisting in the overall effectiveness of mainstreaming activity.

4.       The results also show that for the most part there are formal lines of
         communication for findings into partner organisations, and that members feel this
         is having an influence on organisational thinking.

5.4.4 Networking for Mainstreaming

         Analysis of the monitoring returns for each DP showed that in some cases a wide
         range of contacts were made during Action 2 to date. Table 10 illustrates this by
         presenting the number of contact or contact events 23 cited by DPs in their last two
         monitoring returns 24.


     Table 10: Contacts cited by DPs as part of networking activity during Action 2


                                Final Return           Additional contacts,           Total cumulative
                               December 2002           First quarter return           contacts over the
                                                          March 2003                      first year
DP
ACCORD                     5                           9                          14
Digital Step               11                          7                          18
Diversity Matters          7                           9                          16
Futures                    Not yet available
Personal            0                                  Not yet available
Progression Systems
Young Women                1                           Not yet available          1
Making it Happen
Totals                     24                          25                         49




23
   Contact events are events such as conferences, seminars and so forth held outside of EQUAL at which
contact with other organisations is noted by DPs or presentations are made
24
   In some cases parts of the monitoring return had not been completed or returned by DPs to enable this
exercise to be done
                                                                                                           61

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


2.      The table illustrates the extensiveness of contacts made during the early stages of
        Action 2. Whilst the quality of the contacts in terms of information exchange and
        influence cannot be commented on, the quantity demonstrates that effort is being
        made to build the links to enable dissemination of findings as part of
        mainstreaming.

5.4.5 Conclusions

1.      The overall picture resulting from the evaluation is one of well developed thinking
        and planning around mainstreaming amongst individual DPs. As part of the work
        to further develop soft systems and process learning as part of monitoring and
        evaluation commented upon under Section 5.2.6, mechanisms by which process
        information can be mapped onto policy or practice thinking might also be usefully
        included to form clear functional links between evaluation and mainstreaming
        activities. In each case DPs have ideas to bring to the table on which to build.

2.      It will be important to continue to ensure DPs seek to involve all DP members in
        or make them fully aware of mainstreaming activity, and to lace a focus on
        process as well as output and outcome findings and the links between these to
        make best use of the policy and practice networks and organisational roles
        represented by members.


       Recommendations:
       Mechanisms being developed by some DPs to disseminate findings into policy and
       practice should be reviewed and shared to enable individual DPs to enhance
       capacity, particularly in communicating process learning. This might usefully be
       linked to the outputs of the DP working group examining Monitoring and
       Evaluation approaches

       DPs should be encouraged to maximize the involvement of all DP members in
       considering findings and the approach to mainstreaming to build on their
       individual networks.




                                                                                                 62

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland



5.5      Equal Opportunities
1.       In addition to the Equal Opportunity thematic priority, Equal Opportunities is a
         cross-cutting principle of the programme and its approach. This was examined in
         two ways in the evaluation, firstly, in relation to the DP itself, and secondly in the
         pattern of beneficiaries to date.

2.       Each DP must have an equal opportunities policy detailing how it will ensure
         equality between men and women and across the Section 75 categories. This was
         evaluated as part of DP visits and through an analysis of monitoring returns.

5.5.1 Equal opportunities Policies and Procedures

1.       DP visits and an examination of DPAs showed that Equal Opportunities Policies
         have been well documented by all the DPs with comprehensive policies and
         procedures.

         In the majority of cases it was the Equality Policy of the Lead Partner which was
         adopted and any further policies e.g. grievance, volunteer, child protection would
         be taken from the lead partner.

5.5.2 Membership Profile

1.       Examination of the DPAs showed that the equality balance of the Partnerships is
         positive. They are representative of age and gender in addition to broadly having
         good representation of (rather than from) the target groupings. This is illustrated
         in figure (12) below:-

                                                       Figure (12) – Membership Profile
                Key to Vertical Scale
                                                                  Equal Opportunities
     0 There is a poor gender balance and no                    Membership Profile (1.3a)
     member is representative of the intended
     beneficiary group                                          3


     1 The gender balance is satisfactory but                   2
     no member is representative of the
     intended beneficiary group                                 1

     2 The gender balance is satisfactory,                      0
     one or more members are able to
                                                                      iv tal




                                                                        om S
                                                                        D d


                                                                     Fu ity

                                                                              s


                                                                            en
                                                                          PP




     represent the intended beneficiary group
                                                                           or




                                                                           re
                                                                            i
                                                                           s
                                                                         ig
                                                                        cc




                                                                        tu
                                                                        er




     but are not from the group itself
                                                            A




                                                                     W
                                                                    D




                                                                   g
                                                                un
                                                             Yo




     3 The gender balance is good, members
     representing the intended beneficiary
     group include one or more drawn
     directly from the group
                                                                                                 63

TL Associates
        Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




        5.5.3 Section 75 Review: Patterns in NI EQUAL DP Beneficiary Uptake

        5.5.3.1 Analysis

        1.      The tables which follow present progress to date across NI EQUAL DPs during
                the first year in recruiting beneficiaries. They are drawn from an analysis of
                available monitoring returns up to March 2003. Table 11 provides a summary of
                overall progress broken down by Gender over the project year. Progress is
                compared to outcomes anticipated by each DP as part of their DPAs. The Table
                provides a review of Equal Opportunities between men and women in the support
                provided by DPs.

                        Table 11 – Progress Summary of Beneficiary Recruitment

                  Number         Breakdown of            Numbers reported in        Cumulative Numbers        Stage of
                  of             Numbers Predicted       Final Claim                including those           Project
                  beneficiari    within DPA for          monitoring Returns         reported in First         (first
                  es Planned     project lifetime        Dec 2002                   Quarter monitoring        year over
                  for 2002                                                          returns 200325            total
                                                                                                              years of
DP                               Male       Female       Male          Female       Male           Female     project )
ACCORD            50             100        100          17            10           17             10         1/2
Digital Step      35                 Not recorded        26            4            29             4          1/2
Diversity         Not applicable ; no unemployed beneficiaries are targeted by this DP,                       1/3
Matters           rather, training resources are being developed for providers and employers
Futures           Not            230        0            0             n/a          Awaiting data             1 /3
Project           recorded
PPS               Not            465        15           34            1            Awaiting data             1/3
                  recorded

YWMW              60             0          174          n/a           24           n/a        24             1/3
Total             -              795        289          77            39           -          -


        2.      The table shows that progress in recruiting beneficiaries has generally been patchy
                up until March 2003, the latest monitoring returns available for this analysis.
                Within this, there has been a significant tendency toward more male beneficiaries
                taking up support than female, although at programme level this is balanced a
                little by the specialist support provided by YWMW.


        25
           These figures report the combined total of those benefiting from each DP project when new beneficiaries
        in Quarter 1 2003 are added to those previously reported
                                                                                                                64

        TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


3.      The first year of Action 2 has been characterized by a broad range of piloting and
        feedback exercises (ACCORD, Digital Step, Diversity Matters, and Personal
        Progression Systems), consultation exercises (Young Women Making it Happen)
        and research (Futures Project). This limited the numbers of beneficiaries during
        the period, partly as a reflection of piloting and partly as a reflection of the
        learning exercises in finding and recruiting target groups. In some cases full
        programme design based on consultation and piloting will lead to main
        beneficiary activity in 2004 (Young Women Making it Happen, Diversity
        Matters), whilst for all DPs learning from the first year is intended to be employed
        to improve outreach and recruitment processes.

4       The higher numbers of male beneficiaries can in part be explained by the focus
        within PPS on male prisoners, however the low numbers across other DPs, and the
        scope to also look at female prisoners means that this imbalance should be
        carefully monitored during the remainder of Action 2.

5.     Table 12 shows progress against targets across a broader range of target groups.
       The breakdown is based on the main social groups identified by DPs as target
       beneficiaries in their DPAs. Table 13 summarises progress by presenting the
       relative proportions of each group amongst beneficiaries to date. The table shows
       for each DP in turn both the percentage achieved to date of numbers of
       beneficiaries intended for each target group (% IA), and the overall percentage
       represented by each social group of all beneficiaries (% total). The figures provide
       an initial indication of the level of inclusion across different social groups to date.
       Again, the overall trend for low numbers of beneficiaries limits the extent to which
       progress has been made towards targets for any particular social group.

6.      Of those DPs for which there is adequate data, most have achieved recruitment of
        a significantly higher percentage of Catholics than Protestants to date. For
        example, 82% of ACCORD beneficiaries were Catholic, compared to 18%
        Protestant. Likewise Catholics represented 77% of beneficiaries supported by
        Digital Step, or 60% of those supported by PPS. The balance was more even for
        YWMW. In all but one case the achieved against target percentages (%IA) are
        more evenly balanced. This reflects a stronger underlying emphasis on Catholics
        within planned targets. The exception is Accord, where there is also an imbalance
        in the achieved against target percentages of 49% Catholic to 11% Protestant.
        The difference in percentages achieved is likely to reflect how well Accord has
        been able to recruit Protestant compared to Catholic beneficiaries.

7.      Beneficiaries with disabilities are reasonably well represented overall. Those in
        the 16 to 24 age group are well represented and in keeping with the higher
        unemployment rates for this age group overall.




                                                                                                 65

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


8.      Those unemployed represent just over half of all beneficiaries to date. The figure
        is lower than might be expected because of the inclusion on this occasion of a
        significant number of those still at school. These largely result from Digital
        Steps‟ project work encouraging those who have dropped out from school to
        return to a learning programme, also a feature of the approach of ACCORD (see
        Thematic Study sheet 2 „Pathways to Learning‟, Annex 7 ).




                                                                                                 66

TL Associates
            Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




                                                              Table 12 – Detailed Breakdown of Beneficiaries

                 In the columns below, „I‟ represents the number of beneficiaries each DP Intended to recruit for each target groups, and „A‟
                 records the Actual number according to the most recent DP monitoring returns (March 2003)

                                               RELIGION                         DISABIL-    SEXUAL          RACE         EMPLOYMENT STATUS                       GEOGRAPHY
                 Age 16-24
                                                                                ITY         ORIEN-
                                 Catholic       Protestant      Other                                                    Un-            Em-           At         Metro-       Non
                                                                                            TATION
                                                                                                                         employed       ployed        School     politan      Metro
DP               I       A       I       A      I        A      I           A I       A     I26      A      I     A      I      A       I    A        I A        I27 A        I A
ACCORD           100     30      75      37     75       8      50          0   160   528   -        1      20    0      200      21    0      0      0   11     -     33     -   12
Digital          -       33      -       27     -        2      -           6   -     1     -        -      -     -      -        7     -      5      -   23     -     25     -   10
Step
Diversity        Beneficiary numbers not recorded due to the focus of the project on training resources for practitioners / providers and employers
Matters
Futures          0       0       140      0     90       0      -       -       -     -     -        -      2     0      230      0     -      -      -   -      -     0      -   0
Project
Personal    290          13      300      21    180      13     1       1       50    9     -        -      10    1      480      35    0      0      0   0      NI wide
Progression
Systems
Young            130     24      78       10    78       12     18      3       10    0     -        7      44    10     130      15    44     5      -   3      -     17     -   8
Women
Making it
Happen
Total            520     100     593      95    423      35     69      10      220   15    -        8      76    11     1040     78    44     10     0   37     -     75     -   20



            26
               DPAs do not ask for details on numbers of target beneficiaries under sexual orientation
            27
               DPAs do not request numbers of beneficiaries by geographic area, however final monitoring returns do enable this information to be recorded
            28
               ACCORD place significant emphasis on people with a learning disability (160 target) however the monitoring returns do not include the same extensive list of
            possible disabilities as the DPA and there may need to be clarification as to how these target beneficiaries are recorded
                                                                                                                                                                         67

            TL Associates
            Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


                                              Table 13 – Progress Against Targets

            In the table below, % IA represents the percentage of the Intended or target numbers of
            each type of beneficiary achieved by DPs by March 2003. % Total indicates for each DP
            the proportion each particular category of beneficiary represents for all beneficiaries. So
            for example, for ACCORD 11% of all beneficiaries were disabled, and 67% were
            between 16 and 24 years old.

             Catholic                  Protestant           Disability           Race               Unemployed      Age 16-24

DP           % IA         %total29     % IA      %total     % IA      %total     % IA      %total   % IA   %total   % IA   %total
ACCORD       49           82           11        18         3         11         0         0        10.5   47       30     67
Digital      -            77           -         6          -         3          -         -        -      20       -      100
Diversity    -            -            -         -          -         -          -         -        -      -        -      -
Matters
Futures      0                         0                    -         -          0                  0               0
PPS          7            60           7         37         18        26         10        3        7      100      4      37
YWMIH        13           43           15        48         0         0          23        40       12     60       18     100
Total        -            68           -         25         -         11         -         8        -      56       -      71


            5.5.3.2 Strategic Drift

            1.         The beneficiary information provided in the tables above provides one way to
                       assess how well DPs are managing to keep to original target outcomes and
                       objectives.

            2.         As noted, the nature of activities during the first year has limited the number of
                       beneficiaries participating in projects and as a result the figures do not enable
                       robust statements to be made on any trends away from the intended profile of
                       target beneficiaries. In general, the beneficiary tables show that

                        3 of the 6 DPs have made most progress overall in relation to their primary
                         target groups (Youth Action and race / ethnicity, for example, Digital step and
                         young people, PPS and unemployed prisoners) and would appear as such to be
                         on track.

                        Whilst lower than planned, the trend in recruitment for ACCORD would not
                         suggest a shift away from strategic objectives.

                        Progress by Diversity Matters has been in keeping with its aims and objectives
                         in developing training materials for employers and providers / practioners in
                         relation to multiple discrimination.



            29
                 % refers to those benefiting to date as a proportion of total beneficiaries
                                                                                                                    68

            TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


4.      Table 14 below complements the beneficiary information by presenting progress
        toward implementing intended key actions. The information is drawn from table
        6.16 in the quarterly monitoring returns and from 13.9 in the DPAs. It focuses on
        those actions given a score of „4‟ (the maximum available) in describing the
        relative importance of the action within the DP.

         Table 14: Progress in Implementing Key Actions Within Each DP


                       No key actions      No of these        % delivered to      Stage of
 DP                    (scored „4‟) in     delivered to       date                Project
                       DPA                 beneficiaries
 ACCORD                9                   12                 100+                1/2
 Digital Step          8                   8                  100                 1/2
 Diversity Matters     -                   -                  -                   1/3
 Futures Project       15                  0                  0                   1 /3
 Personal              10                  2                  20                  1/3
 Progression
 Systems
 Young Women      12                       5                  42                  1/3
 Making it Happen


5.      Accord‟s returns show an increase in the number of key actions to include
        additional ones in relation to promoting and training in the use of ICT.

6.      Digital Step, also with a relatively lower number of key actions to start with, has
        been able to deliver all of them. Those with higher numbers of key actions have
        had more variable success but this might be expected to change in the next year as
        new programme designs are implemented.

5.5.3.3 Questionnaire Survey Returns

1.      To complete the assessment of „strategic shift‟ a question was included in the
        survey of DP members exploring any need for adjustments in aims or objectives.
        Only 3 of the 21 respondents felt there had been a need for adjustments. These
        were, however, felt to be refinements in approaches and processes resulting from
        feedback from beneficiaries and not overall strategic changes.

5.5.4   Conclusions

1.      There are evident challenges ahead in meeting targets for some DPs and these
        may have to be reviewed against other possible benefits. The Futures project, for
        example, (whose strap line reads; „understanding employability issues in
                                                                                                 69

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


        North and West Belfast‟) has highlighted the value of the detailed research it has
        been able to conduct in order to build a clearer picture of the nature of
        unemployment and the barriers to employment in its different target areas.
        Feedback from wide-ranging discussions with members of unemployed
        households carried out by the support workers, or with community
        representatives, is proving to be a valuable asset for re-thinking current policy and
        practice for employability, however this would be strengthened by action research
        with target beneficiaries testing the Model proposed by Futures.

       Recommendation:
       Progress in recruiting beneficiaries is cause for careful monitoring. However, to
       recognise some of the groundwork undertaken by DPs information on difficulties
       and barriers in reaching and recruiting beneficiaries onto DP projects should also
       be recorded and discussed as a means to inform practice, even where beneficiary
       numbers has been lower than planned.


5.6     Transnational Cooperation
1.      Transnational cooperation is a core aspect of the EQUAL programme, enabling
        policy lessons at a European level to be identified and communicated. Evaluation
        of ADAPT and EMPLOYMENT highlighted the need for transnational
        cooperation to be addressed early in project development to ensure meaningful
        activities were agreed with clear linkages into individual project objectives, rather
        than be „bolted on‟ later.

2.      A central feature of EQUAL is that each DP is obliged to have at least one
        transnational partner. This emphasis is intended to ensure lessons learned across
        member states are exchanged, and examples of good practice more effectively
        channeled into the European Employment Strategy.

3.      The use of the Equal Common Database (ECDB) for identifying transnational
        partners is discussed in Section 6.3.2.

5.6.1 Profile of Transnational Cooperation

1.      Review of Transnational Cooperation Agreements (TCAs) shows that the profile
        of Transnational Cooperation established at the end of Action 1 includes the
        following:

               Between them, the six Northern Ireland DPs are involved with a total of
                11 other European Countries including Ireland.

               Numbers of partners in TCAs range from 3 to 8.

               2 of the 25 transnational partners are from the Republic of Ireland.
                                                                                                 70

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


5.6.2 Emerging Findings

1.      At this stage in the evaluation it is premature to assess the outcomes of
        transnational activity and the added value to EQUAL. However, during active
        reflection sessions, DP visits and use of the Transnational Cooperation
        Healthcheck (see Annex 9 for details on the formulation of the Healthcheck) the
        following findings have emerged:-

5.6.2.1 Expectations of Transnational Activity

1.      Most TCAs place emphasis on information exchange as a key activity, however
        across the TCAs emphasis is also variously placed on the other four categories of
        activities:-

            1) Parallel development of innovative approaches
            2) Import or export of approaches and their adaptation
            3) Joint development of tools or approaches
            4) Exchange of trainees

2.      The guidance notes for transnational cooperation encourage DPs to be „ambitious‟
        in their approach to transnationality and this is reflected here. There is potentially
        a risk that the extent of cooperative activity is too ambitious, for example, in two
        DPs significant emphasis is placed on joint development, parallel development
        and import / export. These are three quite different activities each requiring not
        insignificant resources.

3.      Demands faced by DPs include:-

               Managing the Development Partnership
               Implementing an innovative work programme
               Ensuring detailed monitoring and self assessment activities including
                empowerment
               Delivering quality mainstreaming outputs to the NTNs and other target
                audiences

       As such there is a need to carefully monitor and support the additional challenges
       of transnational activities of this calibre, particularly if the learning from these
       activities is to be articulated into policy and practice as part of mainstreaming.

5.      Evaluation during Action 2 using the Transnational Health Check developed for
        EQUAL NI (Annex 7) has confirmed that most DPs have found their TCA
        objectives to be ambitious. In most cases activity has centred around one or two
        key activities, as illustrated in Table 15 below.

                                                                                                 71

TL Associates
          Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


                                       Table 15– Transnational Activity

DP                Main Category      Comments / examples
                    of Activity
ACCORD                  3            To progress an ambitious set of TNC objectives different aspects of work
                 (Joint Working)     have been allocated to different TNC DPs as lead agents, depending on their
                                     areas of expertise.
                                     The tasks provide vehicles for exchange of learning and may also provide
                                     some additional tools suited to the work of ACCORD in tracking
                                     beneficiaries through training and in evaluation.
Digital Step            1            Considerable attention has been paid in this TNC to establishing a common
                     (Parallel       language, Digital Step report the collective preparation of a glossary of
                     Working)        terms by TNC partners to establish a clear common basis for future
                                     exchange of learning.
Diversity               3            After an uncertain start in finding an appropriate „fit‟ to the DPs work,
Matters          (Joint working)     activity has begun to find its feet.
                                     The focus is on a cross-EU research exercise concerning government and
                                     EU employment strategies in relation to African Immigrants and is expected
                                     to have some value in application to wider issues in Northern Ireland of
                                     asylum seekers, migrant and seasonal workers
Futures                 1            TNC work is at the early stages in this case and is based on a series of
                     (Parallel       research exercises. At present these are separate exercises with different
                     working)        points of focus in each transnational DP, but may help build a picture of
                                     unemployment issues and barriers across Europe.
                                     The DP noted difficulties in making a clear link between this work and the
                                     DP project.
Personal                2            PPS are both importing and exporting tools and approaches.
Progression          (Import /       Importing activity for use in their own model includes motivation
Systems               Export)        programmes to help encourage those reluctant to take steps toward re-
                                     integration, or approaches to mentoring for those without family support
                                     post release)
                                     Exporting concerns assistance to Greek partners to explore „within prison‟
                                     support services not yet available in Greek institutions.
                                     There is also joint working on instruments to measure employability
YWMW                     4           Whilst not strictly a long term exchange of trainees, a key feature of
                   (Exchange of      YWMW has been to take beneficiaries across to other partner countries as
                     Trainees)       part of the exchange of learning; for example, young mothers to Austria,
                                     Lesbian and ethnic minority women to Spain.
                                     Exchange of learning has not been restricted to beneficiaries but has also
                                     included transnational bodies concerned with training people to work with
                                     marginalized groups – in Spain for example, Lesbians were not thought of
                                     as marginalized; discussion with lesbian women from Northern Ireland has
                                     helped provide more understanding of why this might be so.
                                                                                                           72

          TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




6.      The Table illustrates the range of activity which is beginning to emerge after some
        initial difficulties in DPs such as Diversity Matters or the Futures Project. In the
        majority of cases this is based on joint or parallel working.

7.      3 DPs reported initial difficulties in identifying and developing meaningful
        transnational work and in 2 cases the results are not well integrated to project
        work, reducing (but not removing) scope for exchange learning. In other DPs
        however transnational work has been better integrated and has good
        complementarity.


          Recommendation:
          Advise DPs that the expectations of TCA objectives need to be realistic



5.6.2.2 DP involvement in Transnational Activity

1.      The second survey of DP members explored member involvement (Annex 11
        Questions 3.4 and 3.5) Involvement amongst DP members in transnational
        activity was found to be broad-based. Around half said they attended some TNC
        meetings (often those in Northern Ireland), whilst the other half attended most or
        all meetings.

2.      78% of DP members responding to the second member survey said they felt TNC
        work had benefited their DP and its project. However only around a third were
        able to provide a clear example. Most examples highlighted the opportunity for
        shared learning and the comparison of approaches. Some, however, also
        highlighted additional benefits, for example, validation of the partnership
        principle;

        “you realize from the beginning that there is a partnership – different countries
        taking part give that understanding and its then easier to work in partnership
        locally” Accord Member

        Broadening the horizons of participants and DP members was another example;

       “gives the added „European‟ dimension – many of the participants on the Austrian
       trip had never been outside of the country before” Youth Action member




                                                                                                 73

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


5.6.2.3 Clarity of Communication

1.      A key challenge in working effectively within a transnational arrangement is the
        extent to which DPs can communicate with one another effectively across
        language barriers and have a common and clear understanding of technical terms.
        This was explored as part of the Cooperation Health Checks conducted as part of
        DP visits. Figure (13) illustrates the results

                                                    Figure (13) – Effective Communication


          Key to Vertical Scale                                Effective Communication

 0 TNC Partners have difficulties
 with communication due to                                 3
 language barriers.
 1 TNC partners are able to                                2
 communicate without difficulty for
 information exchange but not to an                        1
 extent sufficient to support
 innovative joint development                              0
 activities

                                                                it y
                                                                 rd




                                                                            S
                                                                          PP
                                                             co


 2 TNC partners are able to                                   rs
                                                           ve
                                                         Ac


                                                        Di

 communicate without difficulty to
 coordinate a wide range of
 exchange activities but have some
 difficulties with different
 definitions of key technical terms
 3 TNC partners have no language
 difficulties and have established a
 common set of technical definitions
 toward a fully „shared language‟



2.      While the level of effective communication has evolved to a reasonable standard
        across all of the DPs, Digital Step have the optimum score. A contributing factor
        in achieving this high level of communication has been due to the time spent by
        transnational partners early in the process in devising a „technical glossary of
        terms‟.

        Recommendation:
        The good practice of devising a „transnational technical glossary of terms‟ be
        disseminated to all DPs in the next call.

                                                                                                 74

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


5.6.3 Conclusions

       Whilst there have been teething problems in integrating TCN work into DP project
       work, most DPs report benefits from TCN. However the nature of these benefits
       is not always clear. As DP projects mature, exchange of learning might also be
       expected to become more specific. The predominance of joint or parallel working
       provides a foundation for mutual learning and joint products during the remainder
       of Action 2.

5.7     Thematic Approach
1.      EQUAL is based on nine thematic priorities drawn from the four pillars of the
        European Employment Strategy (see Annex 1). In Northern Ireland the EQUAL
        programme focuses on two of these - Employability and Equal Opportunities.
        Within each of these pillars EQUAL places emphasis on ensuring projects are
        capable of supporting a range of target groups. They should identify and address
        problems of discrimination and inequality common across social groups, rather
        than focus on one particular social group.

2.      The selection process and fall out during Action 1 resulted in 6 DPs focusing on a
        very diverse range of social groups. The potential therefore for identifying and
        addressing the common problems of discrimination and inequality is very high.

3.      Fully realising the thematic potential of EQUAL will be influenced by support to
        enhance monitoring for evaluating the process lessons of interventions (see
        Section 5.2). In addition added value would be achieved by proactive facilitation
        of the sharing of best practice across the 6 DPs. The active reflection sessions and
        DP visits have identified that this facilitation of sharing of practice requires
        development.

          Recommendation:
          Formal events should be held to facilitate DPs to share best practice and distill
          the common factors across social groups (the thematic approach).



6.      SECTION 6 – IMPLEMENTATION AND DELIVERY
        OF EQUAL
6.1     The Actions of EQUAL
1.      The EQUAL programme is designed around four „Actions‟:-

2.      Action 1 represents the first stage for DPs and lasts 6 months. During Action 1
        DPs finalise their DPA and TCA

                                                                                                 75

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


3.      Action 2 is the main implementation stage of work programmes approved at the
        end of Action 1

4.      Action 3 is concerned with networking, dissemination and mainstreaming
        activities and transferring good practice lessons learned during Action 2 into
        policy and practice

5.      Action 4 funds the technical assistance to provide support to the DPs.

6.      The evaluation highlighted that the potential benefits of Action One could be
        further maximised if support interventions were being provided to DPs in line
        with their progress in developing the following key areas:-

               Effective partnership working
               Clarity of purpose
               The formulation of policies, principles and procedures
               Role clarity

7.      In addition to the above all DPs in the second call will require more intensive
        support during Action 1 to develop systems for monitoring qualitative outcomes.

8.      Analysis of mainstreaming amongst current DPs suggests that an effective
        mainstreaming process will be enhanced if DPs can not only identify potential
        innovative outcomes but also complementary locations of policy and practice
        development (see for example, the illustrations provided in Section 5.4.2 for the
        Futures project and Diversity Matters). It may be of benefit if this combination of
        innovation and complementarity is emphasised during Action 1. Incorporating the
        application process for Action 3 activities into Action 1 will encourage this
        analysis by DPs. During the third round of active reflection sessions the
        evaluation identified a negative response from DPs at the necessity to apply for
        Action 3 funding at a time when implementation was their focus.



       Recommendation:
       Action 1 be lengthened to allow for the following:-
       Interventions as required to support DP formation and effective working
       Support in the development of systems for measuring soft outcomes
       Mainstreaming application




                                                                                                 76

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland



6.2     Evaluation of the promotion and selection process
6.2.1 Stages of the Selection Process

1.      There were four main stages in the process of calling for and selecting applicant
        DPs consisting of a number of key activities:-

        1. Promotion

                   Promotional
                   Support for preparing applications

        2. Assessment

                   An eligibility check of applications
                   Detailed assessment and scoring of applications
                          A first round assessment and policy screening of applications
                          A second round assessment of applications
                   Reconciliation of scores from rounds 1 and 2

        3. Consideration of borderline cases

       4. Treatment of Appeals

2.      Technical details of each of these stages are provided in Annex 2. Here the key
        findings are reported emerging from the evaluation of each stage and any
        opportunities for adjusting or refining the process for the second call of EQUAL.

6.2.2 Promotion

1.      The timing of the first call was unfortunate in coinciding with the summer break.
        The timetable was in line with Commission timescales (in order to synchronize
        the programme across member states) and hindered by a slight delay in agreeing
        the EQUAL programme priorities with the Commission. This preventing
        progression to implementation to the timetable envisaged in the EQUAL CIP
        (launch in April 2001 for submissions end of June).

2.      Over 200 people attended roadshow events and a further 100 indicated an interest
        through registering. Least well attended was Fermanagh; the 20 people attending
        represented 8 organisations (6 representatives were from Fermanagh College and
        4 from DEL). This compares with 34 organisations represented in Londonderry,
        19 in Newry, 25 in Dungannon, 29 in Ballymena and 54 in Belfast.



                                                                                                 77

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


3.      For the most part, however, and again with the exception of Fermanagh, the mix
        of community, voluntary, private and statutory representatives was good. Least
        well represented overall were District Councils and Local Strategy Partnerships
        (LSPs) for the PEACE II programme. The latter would at this time have been in
        the process of formation. Given the interest noted in the Terms of Reference for
        the extent to which DPs „embed in‟ or take account of area based strategies (page
        7 paragraph 23) more information to LSPs may be important to raise awareness of
        the programme as part of the dissemination strategy.

6.2.3 Spread of Applications and Successful DPs

6.2.3.1 Geographic Spread

1.      Table 16 below shows the distribution of applicant organisations based on their
        contact address.

                     Table 16 - Distribution of Applicant Organisations

           Geographic Area                                            Number of
                                                                      applications
           The North West and Londonderry (Limavady, 5
           Strabane, Magherafelt)
           The North East (Larne, Ballymena, Ballymoney, 0
           Moyle, Coleraine, Carrickfergus, Newtownabby,
           Castlereagh)
           The South West (Fermanagh, Omagh Dungannon, 4
           Cookstown )
           The South East (Armagh, Newry, Banbridge, 5
           Lisburn, Down, North Down, Ards)
           Belfast                                                    15
           Total                                                      29


2.      There is a predominance of bids from Belfast based organisations as lead partners,
        reflecting the greater density of organisational activity there, and to an extent
        reflective of the level of need. The absence of proposals from the North East is
        also worth noting.

3.      Table 17 summarises the key target groups and geographic areas intend by
        successful DPs to address as described in their DPAs.




                                                                                                 78

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


                    Table 17 - Target Groups and Geographic Areas

DP                     Location of Lead        Area / Focus          Target Groups
                       Partner
Diversity Matters      Belfast                 Belfast               Disabled, race, sexual
                                               Metropolitan          orientation
                                               Area and
                                               Londonderry
Digital Step           Belfast                 NI-wide               Young people
Personal               Belfast                 NI-wide               Ex prisoners, male and
                                                                     female
Progression
YWMW                   Belfast                 A cross-section       Young women with
                                               of Urban and          disabilities, from
                                               Rural NI              different ethnic
                                                                     backgrounds, with a
                                                                     lesbian orientation,
                                                                     young mothers, young
                                                                     women travellers
ACCORD                 Londonderry             Londonderry /         Status O, women, long
                                               Strabane              term unemployed,
                                                                     young mothers, hidden
                                                                     unemployed
The Futures Project Belfast                    Belfast               LTU households


4.      The pattern continues the strong presence in Belfast based DPs. Whilst this is so,
        three out of the six have a regional project focus, one addresses both Belfast and
        L‟Derry and one a focus on the Derry / Strabane area. Only one has a Belfast
        only focus.

5.      An examination of the profile of partnership member organisations shows that:-

                      Twenty eight of the forty seven partner organisations listed are
                       based in Belfast, fourteen are based in Derry.

                      There were three cases where a partner organisation was from a
                       different town in Northern Ireland from the lead partner, I other
                       words, for the most part partner organisations were based close to
                       one another in Belfast or Derry

6.      The pattern is not surprising given the practical problems of arranging and
        attending regular meetings. It also reinforces an issue regarding the concentration

                                                                                                 79

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


        of bids from Belfast and the need to be aware of the consequent potential for
        activities and networking to also be concentrated there, leading to the possibility
        of a partial picture of policy and practice issues elsewhere.

7.      Given the Belfast location for most lead partners, and whilst acknowledging that
        the main concentrations of many disadvantaged groups will be in Belfast and
        Londonderry, it will nevertheless be important during Action 2 to monitor how far
        beneficiaries are reached and involved outside of the main urban centres,
        particularly in relation to Empowerment.

8.     For two reasons there is merit in working to promote a more „decentralised‟ spread
       of potential lead partners in the next call, for example through roadshows and
       other promotional work:-

        It is likely that there will be differences in the types of barriers to (re)entering
         the labour market affecting social groups living outside the main metropolitan
         areas, particularly around access to support services. A stronger input from
         organisations outside of Belfast may reflect these differences more fully in the
         shape and focus of projects.

        An analysis of the profile of successful DPs show that partner organisations are
         typically from the same area as the lead organisation and applications from
         Belfast are liable to have a predominance of Belfast organisations in them. This
         is offset to a degree by the fact that most regional organisations are also based in
         Belfast and may be able to utilise regional offices in other parts of the province.

6.2.3.2 Spread Across Key Themes and Pillars

1.      Of the two pillars for EQUAL in Northern Ireland there was a low response to
        Equal Opportunities. This is unfortunate given the policy emphasis on gender and
        labour market access. To a large extent however, all DPs are obliged to address
        equal opportunities aspects in their approach. In comparison to other EU
        countries, the share of projects under this pillar is quite typical. For Northern
        Ireland the proportion provided by the single project works out as 17%. The
        countries with the largest share in this pillar are Spain (34% of projects funded)
        and the Netherlands (21%), but the average is around 10-15%.




                                                                                                 80

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


2.      The EQUAL CIP and application guidelines identify a range of social groups
        whose needs might be addressed through EQUAL. These include:

               Long term unemployed men

               Women who face difficulties being (re) integrated into the labour market,
                especially those with caring responsibilities and those with low
                qualifications

               Disabled people, those experiencing mental health problems and those with
                learning difficulties

               Status „O‟ young people

3.      Within these broader groups, the EQUAL CIP and application guidelines identify
        a number of other social groups facing particular difficulties, including:

               Minority ethnic groups, in particular, travelers

               Older workers

               Ex-prisoners and ex-offenders

               Those experiencing drug and alcohol dependency

               Homeless people

4.      Of these, the DP projects show most groups have been targeted, with the
        exception of:-

5.      Older workers. The NI EQUAL CIP notes the opportunity for EQUAL to
        address Guideline 4 of the European Employment Strategy (EES) Guidelines
        which calls for member states to develop policies on active ageing so that older
        workers can remain and participate in working life, and Guideline 9 which calls
        for an open labour market.

6.      Homeless people. The CIP highlights the challenges associated with
        demonstrating progress over a short project timescale for people who are
        homeless and places emphasis on transnational solutions. The Simon Community
        was funded under the New Deal Innovation Funds to address this issue
        demonstrated how, although difficult, positive progress can be achieved.

7.      People with mental illnesses. Concerns were communicated during the survey of
        unsuccessful applicants on the inclusion of this social group.


                                                                                                 81

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


8.      Farmers and farm households as a target group are not mentioned by the CIP,
        but did appear as target beneficiaries in three of the unsuccessful applications.
        Changes in farming mean increasing numbers will be obliged to find off-farm
        work and this group could be considered an appropriate one for EQUAL to target
        in the next round. However this would best be considered as part of a wider
        consideration of any changes in the thematic priorities for EQUAL in Northern
        Ireland in the next round, and would need to complement the objectives under
        Measure 1.6 of PEACE II (Training for Farmers).



       Recommendation :
       The inclusion of one or more rurally-based DP should be considered as an
       objective of the promotion exercise in the next call.

       Recommendation :
       Ensure that the assessment panel included rural policy expertise

       Recommendation :
       More promotion is required to encourage applications under the equal
       opportunities pillar.




                                                                                                 82

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


6.2.4 Evaluation of the Approach to Assessment and Scoring

1.      Details of the assessment approach can be found in Annexes 2 and 3.

2.      Second stage assessments resulted in one application being screened out because
        it had applied into a number of other sources (and was subsequently funded from
        one of them).

3.      In one further case a policy development within DEL had not been confirmed in
        time to influence the progression of an application based on a similar idea. Rather
        than risk the work not being done at all, it was decided to continue to process the
        application. Following confirmation that the initiative was to take place within
        DEL this application has now been withdrawn. Contact with the applicant
        indicates an intention to assist the existing work.

4.      At the end of the first assessments nine proposals received an „A band‟, that is,
        they had achieved at least a minimum score for five key eligibility questions.
        Eight applications achieved an A band in the second round of assessments, but not
        always the same ones. In total, assessments resulted in eight applications showing
        a difference in grade from the first to the second assessment. These were
        discussed in more detail in a meeting of the selection sub group of the EQUAL
        Monitoring Committee. The sub group was able to agree final grades and scores
        for all but two of them. Of these, one was referred to an independent body
        (ECOTEC, the NSS for EQUAL in GB) and another was discussed by a meeting
        of the first and second assessors to agree the final grade. In this case the meeting
        was requested by the second assessor. In each case a final grade and score was
        proposed and agreed by the second assessor.

5.      Of the 8 cases where grades differed between assessments, 4 involved one
        assessor, compared with one difference for each of the other 4 pairs of assessors.

6.      The instances where grades differed illustrate how the pairing of assessors can
        balance different views and awareness in determining a final score. This is
        particularly important when innovations are in the approach to the delivery of a
        service, for example, rather than the service itself. In this case detailed knowledge
        of the current state of practice will be as important and will be more
        comprehensively served by more than one assessor.

6.2.5 Feedback to Unsuccessful Applicants and Appeals

1.      A survey of unsuccessful applicants carried out by this evaluation suggests that in
        some cases feedback was insufficient for applicants to feel they understood where
        they had not achieved the standards required and how they could improve.
        (Annex 12) Such detailed feedback support may prove difficult to provide, but
        could act to build goodwill between DEL and the range of organisations in the
        practice arena in the longer term.
                                                                                                 83

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




6.2.6 Feedback from Unsuccessful Applicants

1.      A telephone survey of unsuccessful applicants was undertaken to explore their
        experiences of the promotion and application process. A total of seven responses
        was obtained, representing a 35% response rate.

2.      Key findings from the survey were:

3.      Most said that they had heard about EQUAL through their own research rather
        than the press.

4.      Four of the seven respondents saw the programme providing funding for projects
        they already had or wanted to enhance. One saw it fitting their „client group‟.

5.      Most found it easy to find partners, often based on existing partner links, but one
        respondent noted the difference between finding „names‟ and finding partners able
        and willing to commit to the workload represented by participation in the
        programme.

6.      The application form came under considerable criticism for its length and
        complexity. Others found the guidance comprehensive, and most noted using
        Proteus at some stage.

7.      Ideas for improvement included use of a simpler „preliminary application‟
        focusing on the core idea. If the core idea was felt suitable, the more detailed
        application could be used for stage 2.

8.      There was general disappointment with the quality of the feedback provided.
        Respondents wanted more information about how they could have improved, and
        what was missing.

     Recommendation:
     The size and complexity of the initial application should be reduced. A shorter
     preselection form could be used prior to a full application, focusing on the innovative
     nature of the project and its relationship to the thematic priorities for Northern Ireland.
     Address of other key EQUAL principles should be in the second stage application for
     selection into Action 1.




                                                                                                 84

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


6.2.7 Evaluating the Assessment Process

1.      For the purposes of informing a second call for EQUAL, the effectiveness and
        appropriateness of the assessment process was analysed in two ways:-

        The use of the „paired assessment‟ approach, which is a relatively novel
        approach to assessment, in terms of its efficiency and effectiveness.

        The impact of the key eligibility questions on success rates amongst applicant
        DPs.

6.2.7.1 The Use Of The Paired Assessment Approach

1.      Table 18 below compares some options for the approach to assessment.

2.      The assessment panel „model‟ is the one typically used by a number of funding
        bodies. Given the tight timescales faced by the EQUAL team for this first call,
        however, and the specialist nature of EQUAL the „paired assessment‟ approach
        provided the best means to ensure efficiency and fairness, and in this case, careful
        use was made of available expertise in assessing applications effectively. Use of a
        broader assessment panel also depends upon the number of assessors who are
        available who have sufficient experience and expertise, and in this case numbers
        were limited.




                                                                                                 85

TL Associates
    Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


                     Table 18 - Approaches to Assessment - A Comparison

Assessment Feature                    Strengths                          Weaknesses

Paired assessors: each assessor in Contrasting expertise and             Strong disagreements may not
turn deals with the same sub set experience can be focused in            easily      be     reconciled
of the total applications          assessing     features     of         objectively without a third
                                   applications within specialist        party
                                   policy and practice areas

                                      The     paired   assessment
                                      process supports efficiency
                                      in processing applications –
                                      assessors do not process all
                                      applications

Assessment panels: all assessors A broad range of expertise The process places more
deal with all applications and then and views are used in burden on each assessor in
compare scores                      assessment              processing all applications

                                      There is less likelihood of        Not all assessors will be
                                      the need for applications to       equally equipped to comment
                                      be referred for additional         on applications in specialist
                                      assessment                         areas

Averaging of Assessment Scores. All views are taken into Averaging becomes more
Scores across assessors are account in determining the meaningful when there are
averaged to give a final score  final score              more than two different
                                                         assessment scores.

                                                                         Averaging could lessen the
                                                                         value of using assessors with
                                                                         high levels of policy or
                                                                         practice expertise.


    3.      A more difficult issue is the relative merits of averaging scores. The approach
            used in this call of EQUAL was not to average. The first assessors views were
            taken into consideration by the Second Assessor sub group in determining the
            final scores where there was a significant difference in scores and grades (+/- 15).
            The mechanism for doing so is based on detailed discussion of individual scores
            within the Monitoring Committee selection sub group, which is in turn minuted.
            This meant that the first assessors were not present for the discussion, but again,
            placing responsibility on the second assessors only ensured that a decision was
            made quickly and as fairly as possible.


                                                                                                     86

    TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


4.      Averaging could provide a more systematic and objective mechanism for dealing
        with differences in score and would also allow for time efficiency. However, the
        value of discussion and adjustment becomes significant the wider the differences
        in views between assessors are. The wider the difference in scores the lower the
        average score will be, and a potential project becomes unlikely to achieve a score
        sufficient for funding. Discussion may result in key features of an application
        becoming better understood (as is evident in the minutes of the selection sub
        group meeting) and that project achieving a score sufficient for funding.

5.      The effect that averaging might have had on the final scores was tested. This
        revealed only one case in which a different result might have emerged.

6.      Having examined the assessment process in detail it was found that on balance the
        „adjustment by discussion‟ approach has the most merit for EQUAL. It enables
        the range of expertise and experience amongst assessors to be fully utilized.

6.2.7.2 The Role of the Key Scoring and Eligibility Questions

1.      Technical details regarding the approach to scoring is also provided in Annex 3.

2.      The Key Eligibility Questions were designed to ensure that projects were selected
        for funding which had the most potential to meet the key Principles of the
        EQUAL programme (innovation, equal opportunity, empowerment, partnership
        and policy relevance).

3.      The overall quality of applications for the first call was good. Of those assessed
        under the Employability theme, seven achieved a grade A after the full assessment
        process, and four a grade B. Given the sensitivity of the scoring system in
        determining grades, evaluation focused on whether or not one or more of the
        questions had proven particularly difficult for applicants to address. In particular,
        applications were checked to see if those with good innovative potential could
        have been lost because of difficulties with another question.

4.      An analysis of the overall success by which applicants as a whole scored across
        each of the twelve questions for both the Employability priority and Equal
        Opportunities is shown in Annex 3.

6.2.8 Results of the Analysis

1.      DP applicants had most difficulty providing an adequate description of the
        innovative content of the proposed approach and activity (5.7, Table A in
        Annex 3). Scores also vary considerably from DP to DP on this question.

2.      For first round assessors, DPs had most difficulty providing an effective rationale
        for the formation of the partnership (5.4a). Overall, most assessments resulted in
        a low score on this question.

                                                                                                 87

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




3.      Low assessment scores were associated with the impact anticipated from proposed
        activities on „all socially excluded groups that the DP intends to support that are
        subject to the main forms of discrimination‟ (5.5b). This question relates to the
        Principle of equal opportunities, but also to the thematic approach within EQUAL
        which asks DPs to move away from a focus on a specific social group to look at
        problems of discrimination common across social groups. The lower success rate
        here contrasts with the high success rate amongst applicants in addressing the
        empowerment of disadvantaged groups and individuals (5.8).

4.      Typically, scores on either 5.7 (innovation) or 5.5b (impact across all socially
        excluded groups) were the main cause for cases where scoring differences
        between first and second assessments changed the grade awarded.

5.      Within the small number of applications for Equal Opportunities, question areas
        scoring consistently well (at least, at or above minimum requirements) were
        primarily associated with project management (Table B in Annex 3). Applicants
        had more difficulty with the rationale for forming the partnership and how they
        would impact across social groups. Particular weaknesses were in the approaches
        to empowerment and in describing how added value might be gained from
        transnational links.

6.      Taken overall, key „knowledge‟ or project development gaps are summarized
        under Table 19 below:-




                                                                                                 88

TL Associates
    Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


                    Table 19 – Key ‘Knowledge’ / Project Development Gaps

Problem areas Highlighted during        Comment
Assessments
accounting for innovation in the The challenges associated with scoring well for
approach or activity                    innovation are to be expected given the considerable
                                        weight attributed to this throughout the assessment
                                        process. The evaluations of similar programmes to
                                        EQUAL, such as ADAPT and EMPLOYMENT, and the
                                        New Deal Innovation Funds, have also highlighted the
                                        difficulties of identifying and articulating innovation
understanding about how to identify and DPs are obliged to look outside their individual
act upon factors of discrimination organisations and expertise to consider how they might
operating across different excluded operationalise equal opportunities across a range of
social groups (part of the principle of groups, and here there is more uncertainty than there may
the thematic approach)                  be in formulating „on paper‟ an equal opportunities
                                        policy. Secondly, DPs are now obliged to think in terms
                                        of a „non group specific‟ general form of discrimination
                                        which is more abstract and difficult to envisage at this
                                        stage. Such general principles of discrimination should
                                        be identified as a result of project work but are more
                                        difficult to imagine at this stage.
Rationale for organisations coming Guidance on the value and potential on partnership
together as a partnership               working was suggested in Section 5 in evaluating
                                        partnership
Describing the added value to be gained Transnational partnership work is often likely to be
from transnational partnership within   unknown territory for many applicants and examples
Equal Opportunities Applications        from this round of EQUAL may be helpful


    7.      This analysis suggests additional support to applicants in the second call should be
            provided in relation to innovation and the thematic approach. To guide thinking
            on potential projects it would be useful to provide examples during promotion
            roadshows of innovation drawn from previous programmes. Alternatively this
            might be provided on the Proteus website. This would provide the opportunity for
            the next round of DPs to build on work done previously under, for example,
            ADAPT, EMPLOYMENT, the New Deal Innovation Fund and the present call of
            EQUAL.

    8.      The application form or the DPA should ask for more consideration from DPs of
            the sorts of underlying factors of discrimination that might be shared across their
            target social groups and how they intend to address these, or ask how they might
            set out to identify underlying factors of discrimination.

    9.      Ideas for innovation or views on discriminatory factors should, however,
            ultimately come from the expertise and experience of the applicant.
                                                                                                     89

    TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




     Recommendation:
     Examples should be provided during promotion roadshows of innovation drawn from
     previous programmes.

     Recommendation:
     Guidance and examples on the value and scope of partnership and transnational
     working should be provided in the application guidelines and during Action 1



6.2.9 Conclusions in Relation to the Selection Exercise

1.      The speed in which the three assessment steps from receiving applications were
        carried out and successful and unsuccessful applicants notified is commendable.
        The process was completed within a little over four months from start to finish,
        contrasting markedly with application processes in other programmes.

2.      The conclusions from this part of the analysis are that overall, the process was fair
        and effective.

3.      Selection had to be strict, given the disparity between what funding was asked for
        (£11 million) and what was available on this round (£3.3 million). However there
        is also a sense of how challenging it is for DP organisations to identify where
        precisely policy gaps could be addressed within the busy area of employability,
        particularly as this is, effectively, a moving target, with new policy initiatives
        appearing even during the selection process.


6.3      Role of the NSS - Proteus
1.      The European Unit of the Department of Employment and Learning has the
        overall responsibility for the implementation and delivery of the EQUAL
        programme in Northern Ireland. Proteus, the National Support Structure, has been
        contracted to assist with the provision of some of the more specialist DP support
        roles. Proteus has been contracted to monitor DP activity, assist with
        transnational cooperation and offer ongoing advice and guidance to DPs.

2.      The salient findings following the analysis of the role of Proteus are given below:-




                                                                                                 90

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


6.3.1   Monitoring Systems

1.      The systems developed to gather quantitative monitoring information follow
        standard audit practice requirements and the format used has been based on
        existing formats from (i) DFP (ii) Commission (iii) DEL (iv) Proteus.

                 The quantitative information provided for this evaluation was found to be
                  patchy in places. For example: DPAs do not ask for details on numbers
                  of target beneficiaries under sexual orientation, whilst DPAs do not
                  request numbers of beneficiaries by geographic area however final
                  monitoring returns do enable this information to be recorded

                 Two DPs had not completed beneficiary data in the DPAs

                 Narrative sections of monitoring returns were still to be returned to
                  Proteus in some cases

2.      A greater challenge is the support of DPs to monitor qualitative outcomes. DP
        visits and active reflection sessions have identified that the emphasis of Proteus is
        on the quantitative monitoring role. This has resulted in a reduced focus on the
        development support role for which there is a significant need.

          Recommendation :
          An appraisal of the support structure role against the needs of DPs to refine the
          NSS role for the second call (i.e. the balance between monitoring role and
          development support role)


6.3.2   Support for the transnational cooperation process

6.3.2.1 The EQUAL Common Database and TCA

1.      Support for transnational partnership formation is provided in EQUAL by a
        dedicated database; „ECDB‟ (EQUAL Common Database), which enables DPs to
        identify potential partners and helps to coordinate the progression of the
        Transnational Co-operation Agreement (TCA). Progression to an agreement takes
        the following key steps:-

2.      The details of all funded DPs are entered onto the database by the Support
        Structure (Proteus). The database can be searched by a DP using a variety of
        criteria to identify the best „partnership fit‟ across all EU DPs.

3.      DPs are encouraged and supported to meet once potential partners have been
        identified in this way. One partner undertakes the secretariat role. This DP takes
        the lead in writing the cooperation agreement.


                                                                                                 91

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


4.      Once complete the proposed agreement is sent out simultaneously to all the DPs
        in the agreement through the ECDB site. Each DP must then validate the
        agreement. This is also done electronically.

5.      Once all have validated the proposed agreement it is automatically sent to the
        Managing Authorities in each of the countries of participating DPs. These,
        typically in conjunction with the NSS in each country, test the agreement against
        EQUAL quality assessment criteria and then either also validate the agreement or
        reject it.

6.      Should any Member State‟s Managing Authority reject the agreement it
        electronically triggers the cycle again – i.e., the secretariat DP must make
        appropriate changes, send the result out to partner DPs, who validate, triggering
        re-submission to each Member State‟s Managing Authority.

6.3.2.2 Experience of the Process of Formulating a Transnational Cooperation
        Agreement

       DP Level

1.     The process of formulating a TCA formed part of the agenda for the DP visit. DPs
       reported using a variety of approaches. Most DPs used the database in some form.
       Some DPs reported that their computer hardware had difficulty with the high PC
       specification required for use of the database. Although all DPs reported it to be
       user friendly, this was not the primary mechanism used in Northern Ireland to
       identify suitable partners.

2.     The most common starting point in the partner search was existing partners
       although this did not always lead to Transnational Cooperation Agreements.
       Some used suggestions from Proteus whilst for others the process began through
       proactive contacts i.e. (e-mails) from other member states who had identified
       details through the database.

3.     Two DPs reported that they could not find a match on the database but were
       subsequently e-mailed by potential partners. The most positive way of
       formulating transnational partnerships was reported by DPs to be face to face
       meetings with potential partners.

4.      A number of DPs reported that this was a difficult task and they would have been
        receptive to more support.

        Programme level

5.      In Northern Ireland validation checks were carried out by Proteus, the NSS, at the
        request of the Managing Authority. Proteus reports that the largest challenge in


                                                                                                 92

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


        the process was in achieving validation on the budgets presented in Transnational
        Cooperation Agreements.

6.      For EQUAL, the budget for the transnational element is separate from the budgets
        for individual DP projects, and has a separate control mechanism. A template for
        the presentation of agreement budgets was provided by the European Commission
        for use across member states. Proteus report that a number of differences in the
        interpretation of this template have arisen across the 17 Managing Authorities and
        NSS bodies involved in validation.

7.      Northern Ireland had additional difficulties associated with the exchange rate
        between Sterling and the Euro. A key issue has been the lack of a standard
        exchange rate for transnational costs. In some cases, choice of too low an
        exchange rate could mean projects experiencing resource difficulties later in the
        programme.

8.      As a result of the difficulties associated with agreeing budgets, just one TCA had
        been successfully validated by the start of Action 2.

9.      Proteus report that Joint meetings of the NSSs do not currently take place and
        there is no European level support structure to assist NSS coordination. A
        meeting of member state NSS members would be an important means to improve
        coordination in time for the next call. A possible coordination model suggested
        by Proteus is the Joint Support Structure (JSS) formed by Managing Authorities
        and NSS bodies representing EQUAL in Northern Ireland, England Scotland
        Wales and Ireland. The JSS has been used, for example, for drawing up common
        approaches to the application and assessment process for EQUAL.

6.3.3 Support for       Mainstreaming and Dissemination

1.      It is early in the EQUAL cycle to evaluate mainstreaming itself, however the
        arrangements for it are innovative and much stands to be learnt from their
        implementation.

2.      Proactive support for mainstreaming of project lessons has not in the past been as
        much a focus of attention as it is in EQUAL. Individual DPs are required to
        address both horizontal and vertical mainstreaming in their DPAs, but in addition,
        a programme level mainstreaming strategy is being developed which will
        coordinate mainstreaming activity across the DPs to improve overall
        effectiveness.

        During Action One two Thematic Networking Groups (TNG) were established to
        facilitate the mainstreaming process, one each for Employability and Equal
        Opportunities. A core aim of the TNGs is to facilitate access to policy makers for
        the DPs.


                                                                                                 93

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


        A recommendation of the interim report was to merge the two TNGs due to the
        fact that the Equal Opportunities TNG had a DP membership of one. This
        recommendation has been effectively implemented and currently one National
        Thematic Network (NTN) exists.

4.      A full mapping of wider potential policy targets has commenced with all DPs
        being issued a questionnaire to identify their policy targets. This will inform the
        ongoing development of the membership of the NTN. The Policy Mapping
        exercise conducted as part of this evaluation also provides examples of potential
        policy and practice targets for each DP.

5.     In addition, the NTN will submit potential policy lessons to the EQUAL
       Monitoring Committee. The Monitoring Committee has a much broader
       membership than the NTN covering a range of Government Departments and
       some representation from the community and voluntary sector, the social partners
       and the private sector.

6.      The approach to mainstreaming is innovative and much stands to be learnt from
        its implementation. The active involvement and inter-change of ideas with the
        DPs in developing these components of the programme is recommended.

7.      The DP visits and active reflection sessions have identified the need for NTNs to
        focus on facilitated exchange between DPs and the other NTN members rather
        than a formal presentation format.

          Recommendation:
          The process of the NTN needs to allow for facilitated debate and exchange of
          practice rather than formal presentations only, with specific actions agreed on.



7.      ACHIEVING OBJECTIVES

1.      This section summarises the progress of EQUAL against the monitoring
        indicators contained in the Northern Ireland EQUAL CIP Annex 8.

2.      The recruitment and selection of DPs resulted in 8 DPs being successful and
        represented just over 41% of total budget. The 8 DPs, 6 under Employability and
        2 under Equal Opportunities, was 2/3 of the forecasted 12. Subsequently 2 DPs
        withdrew from the process during the early stages of Action One.

3.      The forecasted average number of partners in DP/TC was 5 -10. The actual
        average number of partners per DP/TC was 6-7.



                                                                                                 94

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


4.     It was forecasted that 100% of DPs would attempt to measure soft outcomes. At
       the time of this evaluation 100% of DPs are aware of the necessity to measure soft
       outcomes, however no DP has yet developed a system for doing so. The
       formation of a sub group of the NTN to look at Soft Outcomes and Distance
       Traveled is a positive step, the momentum of which needs to be proactively
       sustained.

5.      It was forecasted under Equal Opportunities that between 50 and 70% women
        would receive support. The actual figure, based on monitoring returns at the end
        of December 2002, was 100%.

6.      The forecasted figure for women receiving support under Employability was 50%.
        The actual figure at December 2002 according to monitoring returns was 20%.

        Recommendation:
        The outreach process for the second call should include a strategy for
        encouraging applications with a higher percentage of female beneficiaries

7.      The forecasted average number of transnational meetings attended per DP under
        both Employability and Equal Opportunities was 6. At this stage of the evaluation
        the indications are that this forecast will be met.


8.      RECOMMENDATIONS

1.      EQUAL is a new and ambitious programme whose prime aim is to inform
        European and National Policy. As a programme EQUAL represents a concerted
        drive for new and improved approaches to addressing of social exclusion and
        discrimination. In addition, EQUAL encourages innovation and improvements in
        best practice at a programme level in the approach to evaluation, mainstreaming
        and transnational cooperation. A considerable amount of work has been done to
        date. The recommendations presented here, as part of the formative approach to
        this evaluation, aim to assist in continuing to take forward this work.

2.      The following section reviews the key findings from this stage of the evaluation,
        and provides recommendations to Proteus and the Managing Authority.

8.1     Recruitment and Selection of DPs

       Recommendation: The inclusion of one or more rurally-based DP should be
       considered as an objective of the promotion exercise in the next call. (6.2.3.2 –
       Spread Across Key Themes and Pillars)

       Recommendation: Ensure that the assessment panel included rural policy
       expertise. (6.2.3.2 – Spread Across Key Themes and Pillars)
                                                                                                 95

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland




       Recommendation: More promotion is required to encourage applications under
       the equal opportunities pillar. (6.2.3.2 – Spread Across Key Themes and Pillars)

       Recommendation: The size and complexity of the initial application should be
       reduced. A shorter preselection form could be used prior to a full application,
       focusing on the innovative nature of the project and its relationship to the thematic
       priorities for Northern Ireland. Address of other key EQUAL principles should be
       in the second stage application for selection into Action 1. (6.2.6 – Feedback
       from Unsuccessful Applicants).

        Recommendation: Examples should be provided during promotion roadshows of
        innovation drawn from previous programmes. (6.2.8 – Results of the Analysis)

        Recommendation: Guidance and examples on the value and scope of partnership
        and transnational working should be provided in the application guidelines and
        during Action 1. (6.2.8 – Results of the Analysis)

8.2     Partnership

1.      EQUAL operates by bringing together key players in a geographic area or sector
        in Development Partnerships (DPs). DPs select one of the thematic priorities to
        work under. In this way EQUAL brings different kinds of knowledge and
        expertise together towards integrated strategic approaches.

        Recommendation: Rather than as DP members, the involvement of the Private
        Sector might be alternatively assessed as part of the DPA targets and monitoring
        returns.

        Measures to increase the participation of social partners in EQUAL DPs for the
        second call should be explored, for example, targeted promotional material and
        presentations to key forums in Northern Ireland such as Concordia. (5.1.1 – DP
        membership composition)

        Recommendation: Given the importance placed in EQUAL on mainstreaming as
        an indicator of success, we would suggest that applicants for the second call are
        guided towards securing representation at a senior level from partner
        organisations. (5.1.2 – DP membership level)

       Recommendation: Interventions are provided as required during Action One to
       support DPs to develop effective partnership process skills. (5.1.3 - Clarity of
       Purpose)

        Recommendation: Role Clarity should be clear at least in theory in Action 1. It
        should then be regularly reviewed and developed throughout Action 2 as a key


                                                                                                 96

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


        focus of the EQUAL programme and its potential outcomes.                 (5.1.4 – Role
        Clarity)

        Recommendation: Highlight the value and potential of partnership working
        during Action One, perhaps drawing on examples from the current round.

        Monitor and where necessary provide additional support to DPs to develop
        effective partnership process skills. (5.1.5 Collaboration and the Partnership
        Process)

       Recommendation: DPs are facilitated during Action One to finalise principles,
       policies and procedures (5.1.6 – Partnership policies, principles and procedures)

8.3     Innovation

1.      EQUAL is intended to test and promote new ways to combat all forms of
        discrimination and inequalities by those groups most disadvantaged in the labour
        market, and in this regard must seek to be distinctive from the range of other
        policies and programmes aimed at improving levels of employment or equality. It
        seeks to address policy gaps or weaknesses and to avoid overlap with current
        policy initiatives.

        Recommendations: Current DPs need input and facilitation in the development
        of appropriate systems for monitoring „soft‟ outcomes.

        An action plan and programme of training and development events should be
        implemented

        A process of providing DPs with input and support to develop monitoring systems
        for „soft outcomes‟ should be introduced during Action One. (5.2.5 Monitoring
        Soft Outcomes)

8.4     Empowerment

1.      Projects must provide for the involvement of those targeted for support. This
        includes involvement of intended beneficiaries in influencing the design of the
        project and in the evaluation of project effectiveness.

        Recommendations: DPs should be encouraged to continue to develop and take
        forward the involvement of beneficiaries, by sharing practice and where necessary
        being given specialist support.

        Beneficiary involvement should be included as part of the action plan to develop
        approaches to monitoring and evaluation noted under 5.2.6 above (5.3.1 –
        Arrangements for empowerment)


                                                                                                 97

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


8.5     Mainstreaming

1.     A measure of the success of EQUAL is the extent to which the lessons learned
       from DP projects inform mainstream policy

       Recommendations: Mechanisms being developed by some DPs to disseminate
       findings into policy and practice should be reviewed and shared to enable
       individual DPs to enhance capacity, particularly in communicating process
       learning. This might usefully be linked to the outputs of the DP working group
       examining Monitoring and Evaluation approaches

       DPs should be encouraged to maximize the involvement of all DP members in
       considering findings and the approach to mainstreaming to build on their
       individual networks. (5.4 Mainstreaming)

8.6     Equal Opportunities

       Recommendation Progress in recruiting beneficiaries is cause for careful
       monitoring. However, to recognise some of the groundwork undertaken by DPs
       information on difficulties and barriers in reaching and recruiting beneficiaries
       onto DP projects should also be recorded and discussed as a means to inform
       practice, even where beneficiary numbers has been lower than planned. (5.5 Equal
       Opportunities)

8.7     Transnational

1.      A central feature of EQUAL is that each DP is obliged to have at least one
        transnational partner. This emphasis is intended to ensure lessons learned across
        member states are exchanged, and examples of good practice more effectively
        channeled into the European Employment Strategy.

       Recommendation: Advise DPs that the expectations of TCA objectives need to
       be realistic (5.6.2.1 – Expectations of transnational activity)

       Recommendation: The good practice of devising a „transnational technical
       glossary of terms‟ be disseminated to all DPs in the next call. (5.6.2.3 – Clarity of
       communication)

8.8     Thematic Approach

1.      EQUAL is based on nine thematic priorities drawn from the four pillars of the
        European Employment Strategy (see Annex 1). In Northern Ireland the EQUAL
        programme focuses on two of these - Employability and Equal Opportunities.
        Within each of these pillars EQUAL places emphasis on ensuring projects are
        capable of supporting a range of target groups. They should identify and address


                                                                                                 98

TL Associates
Mid Term Evaluation Report: European EQUAL Community Initiative Programme for Northern Ireland


        problems of discrimination and inequality common across social groups, rather
        than focus on one particular social group.

       Recommendation: Formal events should be held to facilitate DPs to share best
       practice and distill the common factors across social groups (the thematic
       approach). (5.7 – Thematic approach) (4.5 Policy Mapping Conclusions)

8.9     The Implementation and Delivery of EQUAL

       Recommendation: Action 1 be lengthened to allow for the following:-

                 Modulated interventions to support DP formation and effective working

                 Support in the development of systems for measuring soft outcomes

                 Mainstreaming application for Action 3

       (6.1 – The actions of EQUAL)

       Recommendation: An appraisal of the support structure role against the needs of
       DPs to refine the NSS role for the second call (i.e. the balance between
       monitoring role and development support role) (6.2.1 – Monitoring systems)

       Recommendation: The process of the NTN needs to allow for facilitated debate
       and exchange of practice rather than formal presentations only, with specific
       actions agreed on. (6.2.3 – Support for mainstreaming and dissemination)

8.10    Achieving Objectives

       Recommendation: The outreach process for the second call should include a
       strategy for encouraging applications with a higher percentage of female
       beneficiaries (7 – Achieving objectives).




                                                                                                 99

TL Associates

								
To top