Executive Summary by 9V1u2q

VIEWS: 6 PAGES: 74

									        Review of ESF Social
         Inclusion Measures


                     Final Report



                           for

                Scottish Executive




                     October 2005



St George's Studios, 93-97 St George's Road, Glasgow, G3 6JA
           Tel: 0141 353 1994 Fax: 0141 353 2660

                  E-mail: ekos@ekos.co.uk

                Glasgow      Inverness
                     Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

CONTENTS
EXECUTIVE SUMMARY ....................................................................................................... 1

1.       INTRODUCTION ....................................................................................................... 7
1.1      STUDY OBJECTIVES ..................................................................................................... 7
1.2      STUDY METHOD ........................................................................................................... 7
1.3      REPORT STRUCTURE ................................................................................................... 8
2.       FINDINGS OF DESK BASED RESEARCH ............................................................... 9
2.1      INTRODUCTION ............................................................................................................ 9
2.2      BENEFICIARY ANALYSIS ................................................................................................ 9
2.3      ANALYSIS OF PROJECT RESULTS ..................................................................................20
2.4      NET IMPACT ANALYSIS .................................................................................................26
2.5      BENCHMARKING PERFORMANCE AND EFFECTIVENESS ..................................................31
3.       FINDINGS FROM STRATEGIC INTERVIEWS .........................................................43
3.1      INTRODUCTION ...........................................................................................................43
3.2      IMPACT AT STRATEGIC LEVEL ......................................................................................43
3.3      PROGRAMME LINKAGES ..............................................................................................46
3.4      EFFECTIVENESS IN REACHING TARGET GROUPS ...........................................................47
3.5      IMPACT ON TARGET GROUPS........................................................................................48
3.6      IMPACT AT PROJECT LEVEL .........................................................................................50
3.7      GEOGRAPHICAL ISSUES ..............................................................................................51
3.8      HORIZONTAL THEMES ..................................................................................................52
3.9      GOOD VALUE .............................................................................................................53
3.10     FUTURE PROSPECTS ..................................................................................................54
4.       PROJECT SURVEY FINDINGS ...............................................................................56
4.1      INTRODUCTION ...........................................................................................................56
4.2      SCOPE OF SOCIAL INCLUSION FOCUSED PROJECTS........................................................56
4.3      REACHING TARGET GROUPS .......................................................................................57
4.4      ISSUES AND CHALLENGES ...........................................................................................59
4.5      GEOGRAPHICAL IMPACT ON PROJECT PERFORMANCE ....................................................60
4.6      ADDRESSING HORIZONTAL THEMES ..............................................................................61
4.7      DEMONSTRATING PROJECT ACHIEVEMENTS .................................................................61
4.8      OUTCOMES AND IMPACTS ............................................................................................62
4.9      KEY AREAS OF SUCCESS AND GOOD PRACTICE ...........................................................63
4.10     ADDED VALUE ............................................................................................................64
4.11     COMPLEMENTARITY .....................................................................................................65
4.12     FUTURE PROSPECTS ..................................................................................................66
4.13     CONCLUSION ..............................................................................................................67

5.       CONCLUSIONS AND RECOMMENDATIONS .........................................................68
5.1      MAKE-UP OF CLIENT GROUP .......................................................................................68
5.2      ADDED VALUE OF ESF .................................................................................................68
5.3      COMPLEMENTARITY ....................................................................................................69
5.4      ACHIEVEMENT OF POSITIVE OUTCOMES ........................................................................70
5.5      IMPACT ANALYSIS .......................................................................................................71
5.6      GOOD VALUE ..............................................................................................................72




EKOS Limited                                                                                                                      ii
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

EXECUTIVE SUMMARY
      CONTEXT

      This study was commissioned by the Scottish Executive with the objective to review the
      effectiveness of social inclusion Measures within the Objective 3, Objective 2 West of
      Scotland, and Highlands and Islands Special Transitional Programme.

      European Structural Funds Programmes must comply with certain monitoring and assessment
      requirements as laid down by the European Community in Articles 40 and 42 of Council
      Regulation No 1260/1999. This study therefore forms part of the Mid Term Evaluation Update
      for the European Social Fund in Scotland.

      METHOD

      A desk-based review of project data was undertaken to examine the types and volumes of
      beneficiaries supported, the extent of positive outcomes, and the net economic impact. In
      addition, a benchmarking exercise reviewed progress against Programme targets, and
      compared the Scottish Objective 3 social inclusion Measure’s performance against that of
      similar measures in England and Wales, as well as other domestic programmes with social
      inclusion objectives.

      A total of 30 mainly face-to-face consultations were undertaken with strategic partners with
      various geographic and thematic interests and involvement in the three Programmes. These
      sought to establish the impact of the Measures at a strategic level; the reach of and impact on
      target groups; areas of good practice and good value. The interviews also looked forward to
      future programming.

      A total of 34 project managers were surveyed, representing all three Programmes and various
      geographical, target group and organisational types. Six case studies were compiled to show
      areas of good practice and impact.

      DATA AND BENCHMARKING ANALSIS

      Chapter 2 provides the findings of the desk-based review. It starts with analysis of the range of
      beneficiaries supported, looking first at the beneficiaries in terms of length of unemployment,
      then by types of target group. As far as data, allows the analysis also assesses the
      proportions of each target and unemployment group which has been helped through ESF. A
      further stage involves assessing which types of project have been most successful in
      achieving employment, education and training outcomes.

      Beneficiary Analysis

      The analysis of beneficiary data has shown that the key unemployed group reached by ESF
      social inclusion Measures is the over three years unemployed, suggesting that the Measures
      are indeed reaching those furthest from the labour market. In terms of target groups, the
      largest supported groups have been those with literacy and numeracy problems (18%) and
      workless households (11%). Between Programmes, certain differences in participation rates
      can be noted: a high proportion of beneficiaries in the Highlands and Islands Programme were
      disabled people (70%), whereas Objective 2 West’s highest share was people with no
      qualifications (27%), and Objective 3’s key target groups included disabled people (30%) and
      people with literacy/numeracy difficulties (23%).

      In terms of achievement of education, training and employment outcomes, over all of the
      Programme areas it has been found that:

              generally, those that tend to support the short term unemployed are most successful,
               whilst those that support the long term unemployed, those who face multiple barriers
               and labour market returners are less so;



EKOS Limited                                                                                         1
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
              projects that perform well tend to do so in terms of employment and in education and
               training;

              the Highlands and Islands Measure underperforms both the Objective 3 and
               Objective 2 West Measures in employment, education and training outcomes. This
               may suggest that rural projects encounter greater external barriers to positive
               outcomes;

              Objective 3 projects perform better on education and training outcomes but
               underperform on employment outcomes in all quintiles against Objective 2 West; and

              it seems that smaller projects have stronger performance than their larger
               counterparts; often these are delivered by voluntary organisations.

      Net Employment Impact

      The social inclusion projects have generated a total of 6,657 gross jobs in Scotland overall.
      More than one-half of all employment outcomes were generated as a result of the West of
      Scotland Objective 2 Programme (51%). A total of 2,590 (39%) were created through the
      Objective 3 Programme and 643 (10%) through the Highlands and Islands Programme.

      Unsurprisingly, Glasgow City accounted for the largest share with 35% of all jobs, whereas the
      remaining 28 Local Authorities individually accounted for less than 10% of the total net
      employment.

      Our net impact analysis found that from this a total of 4,054 net additional jobs were created in
      Scotland as a result of ESF funding.

      Benchmarking exercise

      The benchmarking study examined performance at Measure level against Programme targets;
      at UK level, across comparable social inclusion Measures; and against other funding
      mechanisms with social inclusion objectives.

      The Objective 2 West Measure is forecast to perform well across most indicators with the
      exception of beneficiaries receiving self-employment support. The unit costs for qualifications
      and employment results meets SPD expectations, while education/training outcomes are
      significantly lower.

      The Objective 3 Programme Measure 2.1 funds were significantly increased following
      virements. Again, the Measure is performing well against output targets. Actual unit costs have
      been significantly lower than anticipated.

      The HISTP Measure is forecast to greatly exceed Programme expectations across a range of
      key performance indicators, and indeed, actual achievements have already exceeded targets
      for the numbers of people trained, and entering further education or training. Actual unit costs
      have been considerably lower across all of these indicators indicating that the Measure has
      been more cost-effective than anticipated. Indeed, the Programme makers may have been
      unambitious in their targets.

      We then compared Objective 3 Measure 2.1 with similar Measures in the Wales and England
      Objective 3 Programmes. Although Measures are of comparable size and beneficiary reach,
      the data suggests that England is out-performing Scotland in terms of cost-effectiveness. On
      the other hand, Wales, with a smaller level of resources, has over three times the cost per job
      outcome as Scotland. There are many possible reasons behind variable performance,
      including differences in project structures and delivery organisations, economic conditions and
      geographic barriers. All the same, there may be lessons from England’s experience that could
      benefit from further examination as Scotland prepares for the next funding period.

      Finally, we examined a range of other social inclusion programmes. Although the analysis
      found a range of lessons and experiences that are interesting, it was difficult to draw
      conclusions on comparative cost-effectiveness based on the information available. Our cost-

EKOS Limited                                                                                         2
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      comparison per target group between the Objective 3 Measure and the New Futures Fund
      confirmed the importance of course completion for achievement of positive outcomes, and
      found that in both groups, Lone Parents, Disabled People and Ethnic Minorities achieved the
      highest levels of positive outcomes. Interestingly, the highest spend per head was for Disabled
      People and Lone Parents, whereas Ethnic Minorities were among the groups with the least
      spend per head. ESF recorded higher or equivalent overall positive outcomes across most
      categories, with the exception of Ex-Offenders. In terms of costs per beneficiaries into
      employment, education or training, both Programmes were broadly similar with the exception
      of Lone Parents, where ESF was significantly more cost-effective.

      CONSULTATION FINDINGS

      The study team carried out a total of 30 interviews with organisations with an active
      involvement or interest in the social inclusion Measures of the three Programmes under
      review. These interviews gathered stakeholder views on the effectiveness of the Programmes
      in reaching disadvantaged groups, their ability in adding value to domestic funding, and their
      success in achieving and demonstrating targeted outcomes. Briefly, this found that:


              national and local agendas with regard to the implementation of ESF social inclusion
               Measures seem to be well aligned with the strategic priorities of other partner
               organisations. The key issues in terms of funding streams were highlighted as:
                     a lack of grant funding for projects targeting young people;
                     some difficulties in aligning ESF with Social Inclusion Partnership (SIP)
                      funding and strategies; although, a more concerted effort is being made to
                      align ESF with the Community Regeneration Fund;
                     an increasing administrative burden in managing multiple funding sources,
                      including incompatible reporting requirements and difficulties in the attribution
                      of outcomes; and
                     the need for more cross-departmental integration within the Scottish Executive
                      itself, with more attention paid among policy makers to European funding;
              a broad consensus that the majority of ESF project activity could not survive in the
               absence of public funding. Question marks are raised over the ability of bodies such
               as Jobcentre Plus, Local Authorities and the Scottish Executive to find alternative
               funding to help to sustain projects;

              ESF funding, both in cash and in kind, has provided a buffer to enable risks to be
               taken in order to trial and test various training projects that would not have happened
               otherwise.

              funding has also given projects the initial financial foundation on which to attract
               match funding through, for example, the LECs, the Local Authority Social Work
               Departments, Careers Scotland, Job Centreplus, the National Lottery et al. However,
               the availability of match funding has been an issue for some organisations or for
               some areas not targeted with area-specific programmes;

              despite steps having been taken following the MTE to improve coordination between
               Objective 3 and Objective 2 West, there remains a perception that the social inclusion
               Measures have not been well linked and that mechanisms are inadequate to ensure
               synergy. Opinions were fairly evenly divided as to whether ERDF and ESF had been
               successfully integrated;

              in general, consultees indicate that all targeted groups have benefited from the
               delivery of ESF funded projects. Consultees point to a drop in the participation of
               smaller organisations in this programming period, potentially affecting the
               Programme’s reach to certain niche groups; examples being people with alcohol and
               drug problems and ex-offenders. The dispersed geography of the Highlands and
               Islands area is always challenging in terms of accessibility for such groups in remote
               rural areas to access training support. However, ESF funding has allowed the
               participant training providers to reach more clients by more innovative delivery, e.g.


EKOS Limited                                                                                         3
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
               through the use of open learning, online learning and taking learning in the
               candidate’s own home;

              the introduction of a requirement for evidence of literacy/numeracy needs of a certain
               level may provide evidence which can be comparable with other projects, in order to
               ensure that those most in need are being prioritised;

              it was generally agreed that horizontal themes have been a worthwhile element of the
               Programmes. Most consultees felt that horizontal themes have been effectively
               addressed and some of them are becoming self-evident. However, some consultees
               felt there was still some room for improvement;

              the unit cost of assisting the social inclusion target group is likely to be much higher
               than for mainstream training programmes; the wide range of needs, types of support,
               varying geographical circumstances affecting running costs and “distance to travel”
               means that project-to-project comparisons are meaningless;

              a number of organisations assess the distance travelled by beneficiaries in terms of
               their progression up the learning ladder, with regard to the attainment of specific skills
               sets as well as life skills such confidence building. A range of different techniques
               have been trialled and used by project sponsors, however, it has been argued that
               there is no “one size fits all” technique that could be applied across ESF activities to
               provide a comparable Measure;

              the methodology of comparing the cost to society (i.e. in benefits, allowances, etc) of
               individuals not participating in project activities, against that of providing support and
               moving individuals off benefit and out of other public support services such as day
               care centres, and into work (and paying tax) or training can be a useful indicator of
               value for money; and

              a continued focus on social inclusion support within ESF is widely supported.
               Consultees provided a range of suggestions regarding strategic direction and
               amendments to the operational arrangements to any future funding programmes.

      PROJECT SURVEY FINDINGS

      A key component in assessing the effectiveness of ESF social inclusion activities was the
      project fieldwork carried out by the study team. In total, 34 telephone surveys were completed
      with project managers. The ESF supported projects sample covered a range of activities and
      were delivered by a wide variety of organisations.

      Recruitment of beneficiaries within the project sample was pursued using a number of
      approaches: including referrals Jobcentre Plus was frequently cited as a key organisation,
      along with SIP, social work department or other local economic development agency partners;
      good reputation and positive word of mouth; and active outreach work.

      The recruitment challenges experienced by the project sample included enticing those on
      state benefits to become involved, sourcing suitable candidates that met project criteria e.g.
      the need to specifically target unemployed 18 to 24 years olds from a SIP area. In addition,
      projects also had to work to overcome stigmas, such as admitting to literacy and numeracy
      problems.

      Further issues and challenges in project implementation were defined as the level of
      documentation required, difficulties in recruiting suitable staff, challenges in meeting targeted
      employment outcomes, and working in partnership without losing focus and momentum.

      Evidently the geographical context of projects was perceived by many project managers (47%)
      to have some affect on project performance. This was certainly the case with each of the six
      rural projects supported under the Highlands and Islands Special Transitional Programme, five
      of which stated their project reference area affected project performance “very much”.



EKOS Limited                                                                                           4
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      Project activities were largely considered to fit Measure performance indicators across the
      project sample. However, many wished to see greater emphasis on softer outcomes within
      Programme monitoring, e.g. increased client confidence and improvement in timekeeping as
      Measurements of progress. The above results demonstrate that the vast majority of projects
      believed they had performed very or quite well against ESF targets set.

      From evaluations and discussion with project managers, areas of good or best practice in
      terms of delivering social inclusion orientated support and achieving desirable outcomes were
      identified as:

              development of partnership approaches;
              pro-active linkages with employers;
              exchanging information;
              provision of flexible, client centred support; and
              celebrating success.
      ESF funding also had a number of additional impacts on the projects:

              allowed projects to provide additional support to clients i.e. assistance with transport
               costs, childcare etc - vital in assisting client groups facing multiple barriers to
               employment;

              regular ESF monitoring ensured that project progress was monitored and
               operationally the management of projects was more formalised. This monitoring
               requirement also encouraged regular beneficiary surveys and feedback sessions and
               consequently any issues identified by clients could be addressed effectively;

              some projects used ESF to provide a “waged period” to clients on longer-term work
               placements, this provided a further incentive to beneficiaries to participate and
               complete projects;

              a particularly valuable outcome of the ESF funds was that project managers were
               able to recruit skilled staff, they could allocate a dedicated worker to particular
               projects and this increased the quality of delivery;

              ESF funding further enabled the scale of many projects to increase, especially with
               regards to projects addressing a rural location – funding was used to provide
               additional local services;

          -    marketing of projects – in many cases funding to cover aspects such as advertising
               and marketing of projects to reach key target groups would not have been available,
               ESF funding assisted with this aspect;

          -    attracting additional match funding and project testing – securing ESF funding was
               perceived by some project managers as raising the profile of a project and helping to
               source match funding and delivery partners. In one specific case, ESF funding
               enabled a pilot project to be implemented and later rolled out as a mainstream
               Careers Scotland programme.

      Given the high level of partnership delivery, complementarity with organisations and their
      wider polices such as Jobcentre Plus, local economic development companies, social work
      and health departments and further education establishments was particularly high. Half the
      project sample also identified linkages with ERDF projects.

      The six case studies within Appendix 3 further help to demonstrate a range of the activities
      incorporated by the project sample and provide an insight into current practice for addressing
      social exclusion.




EKOS Limited                                                                                         5
               Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      RECOMMENDATIONS

      The following recommendations were made:

      1. Target group categorisations be linked to standard statistical sources in order to
      enable quantification of the number of individuals within the reference population that
      are being reached by the Programme.

      2. It is recommended that project administration requirements are looked at again in
      relation to key match-funding programmes, seeking to streamline and complement
      where possible.

      3. There continues to be a requirement for improved strategic coordination of ERDF
      and ESF funding streams. It may be appropriate to use structures such as Community
      Planning to facilitate this discussion at a local level.

      4. Longer-term tracking of individuals would be desirable in order to provide a more
      realistic time frame for job seekers to achieve results, as well as to demonstrate
      sustainability of outcomes.

      5. Further examination of the English experience may be valuable, in understanding
      why the data would suggest a significantly better cost-effectiveness achievement than
      Scotland.




EKOS Limited                                                                               6
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

1.    INTRODUCTION
      In March 2005, the Scottish Executive commissioned EKOS Ltd to carry out a review
      of the effectiveness of ESF Social Inclusion Measures within the Objective 3,
      Objective 2 West of Scotland and Highlands and Islands Special Transitional
      Programmes.

1.1   STUDY OBJECTIVES
      The stated objectives of this study are to:

              assess the net impact of ESF at local authority and Scotland levels, e.g. on
               employment, unemployment and benefit claimant rates;

              assess the impact of ESF on social inclusion in Scotland as a whole, taking
               into account the different client groups and different social inclusion issues
               experienced in the Highland and Islands and different parts of Lowland
               Scotland;

              identify the impact of different activities funded under the social inclusion
               Measures of ESF outlined above, including cost comparisons with benchmark
               values;

              identify which types of activities are successful in addressing social inclusion,
               by increasing the employment rate and developing skills for employability;

              assess how effective ESF is in achieving “soft” and “hard” outcomes for
               beneficiaries, and how many have benefited;

              evaluate the value added to domestic policies directed at social inclusion
               (both Scottish and UK-wide), including New Deal;

              assess complementarity between social inclusion support under ESF and
               linked ERDF-funded activities; and

              identify the make-up of the client group, what proportion has been helped
               through ESF, and which disadvantaged groups targeted under social
               inclusion Measures gain most from the ESF projects.



1.2   STUDY METHOD
      The study was undertaken in six stages, as follows:

              Stage 1: Inception Period

              Stage 2: Desk Based Research

              Stage 3: Interim Reporting

              Stage 4: Strategic Interviews

              Stage 5: Project Survey

              Stage 6: Synthesis and Reporting

EKOS Limited                                                                                  7
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      Desk-based research was undertaken to provide a:

              socio-economic baseline analysis, to set the context for ESF social inclusion
               Measure intervention;

              review of project beneficiary information to establish which groups had
               benefited from the Measures;

              review of outcome and results information to establish the factors of success;

              net impact analysis of ESF at local authority and Scotland levels; and

              benchmark of performance against Programme targets,                  other ESF
               Programmes; and other programmes targeting social inclusion.

      Strategic interviews were undertaken on a primarily face-to-face basis with 30
      individuals involved in the three Programmes. We aimed to select organisations that
      could provide a strategic view on Programme performance; organisations with
      national or sub-national remits, with focus on particular target groups and which have
      representatives on Programme Committees and Advisory Groups (see Appendix 1).

      Project survey work consisted of 34 telephone interviews with project managers. The
      projects were identified using criteria, which ensured a fair weighting by total number
      of projects per Programme, by total number of beneficiaries per target group, and by
      rural/urban focus (see Appendix 2).



1.3   REPORT STRUCTURE
      The remainder of this report is structured as follows:

              Chapter 2 presents the results of the desk-based research, providing a
               baseline analysis and presents the findings of the financial and output data
               and benchmarking analysis;

              Chapter 3 presents the findings of the strategic interviews;

              Chapter 4 presents the findings of the project survey; and

              Chapter 5 comments           on    the   findings,   with      conclusions   and
               recommendations.




EKOS Limited                                                                                 8
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

2.    FINDINGS OF DESK BASED RESEARCH

2.1   INTRODUCTION
      The following section provides a:

              review of project beneficiary information to establish which groups benefited
               from the Measures;

              review of outcome and results information to establish the factors of success;

              net impact analysis of ESF at local authority and Scotland levels; and

              benchmark of performance against Programme targets,               other ESF
               Programmes; and other programmes targeting social inclusion.



2.2   BENEFICIARY ANALYSIS
2.2.1 Introduction
      Some analysis of beneficiaries has already been undertaken by the PMEs,
      particularly Objective 3, and this analysis intends to complement the findings already
      presented in other work packages.

      Using beneficiary data provided by the Scottish Executive and the Objective 3 PME,
      the study team analysed the make-up of the client group. It aimed to establish what
      proportion of unemployed has been helped through ESF, and which specific
      disadvantaged groups were most effectively targeted under the social inclusion
      Measures.

      The data provides information on beneficiaries according to two key factors:

              Unemployment
                     Unemployed Up To 6 Months Under 25
                     Unemployed Up To 6 Months Over 25
                     Unemployed 7 To 12 Months Under 25
                     Unemployed 7 To 12 Months Over 25
                     Unemployed 1 to 2 years (all ages)
                     Unemployed 2 to 3 (all ages)
                     Unemployed 3 years + (all ages)
                     Unemployed Other (encompassing            returners,   people     facing
                      redundancy, etc); and
              Target Group
                     Homeless
                     Lone Parents
                     Drug Users
                     Alcoholism Substance User
                     Literacy and Numeracy
                     Young Leaving Care
                     Disabled people
                     Ex Offender


EKOS Limited                                                                                9
               Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
                    Workless Household
                    No Qualifications
                    Other: (for Objective 3 this is further broken down into: at school,
                     asylum seekers, educationally disadvantaged, ethnic minorities, gypsy
                     travellers, long term unemployed, over 50s, people with learning
                     disabilities, pupils in danger of exclusion and returners to the labour
                     market); and
                    Non Applicable



2.2.2 Data Issues
      It should be noted that it was only possible to analyse data from years in which data
      was made available, and this varied by Programme. The HISTP started earlier,
      therefore project data is available from 2000, whereas the other two Programmes
      started projects only in 2001. This meant that when beneficiary data was aggregated
      at Scottish level, it was only possible to do so for projects starting in the years 2001-
      2004.

      There are also issues surrounding the target groupings. In each of the three
      Programmes beneficiaries were placed into only one specific target group despite the
      fact that they may have belonged to more than one. Therefore, when we considered
      what proportion of each target group as a whole was supported, the groups were
      likely to be under-represented as beneficiaries facing multiple barriers are not taken
      into account in the categorisation process.

2.2.3 Overview of Beneficiary Groups
      The following provides an overview of each Programme’s reach in terms of the
      unemployed and target group categories described above.




EKOS Limited                                                                                10
                     Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

      UNEMPLOYED GROUPS

      Figure 2.1 shows how the Programmes’ share of beneficiaries was split across the
      unemployed categories.


                                 Figure 2.1: Proportion Supported by Unemployed Client Group


      60%


      50%                                                                                      50%



      40%
                                                                                                     33% 33%
                                                                                                                            HISTP
                                                                                                                            Objective 2
      30%                                                                                                                   Objective 3
                                                                                                                            Scotland
                        17%
      20%                                                                                                             14%
                  17%            17%                                                                          17%
                              16%                                                                  14%              14%
                                       10%    13%        12%                         7%
                                                                     12%
                                    10%                        6%             9%             8%
                                                                         8%             9%                      8%
      10%                                                           7%             7%
                6%             5%                 5%   5% 5%
                                             4% 4%

       0%
                Under 25 < Over 25 <         7-12 over 7-12 under 1-2 years        2-3 years      3 years +    Other
                   6m        6m                 25         25




      The ‘Other’ category was made up of a number of unemployed groups for each of the
      programmes, which were as follows:

                 for Scotland overall, the unemployed SME self employed category accounted
                  for 8% of unemployed individuals supported, this was followed by 3% for both
                  returners to the labour market and threatened SME self employed,
                  unemployed school and unemployed large enterprise accounted for 1% of
                  those supported;

                 in the case of Objective 3, unemployed school accounted for 10% of
                  beneficiaries and returners to the labour market accounted for 4%;

                 data for the West of Scotland Objective 2 identifies that returners to the
                  labour market accounted for 3%, self employed SME (2%) and the remaining
                  categories individually accounted for only 1%; and

                 for the Highlands and Islands, 8% of those in the other category were
                  unemployed SME self-employed, followed by returners to the labour market
                  and unemployed threatened SME self-employed (3%).

      The analysis of beneficiary data shows that ESF has predominately supported the
      long-term unemployed in Scotland (33%). This is followed by under 25s unemployed
      less than 6 months (17%) The over 25 (7 to 12 months) group has received the
      lowest proportion of support in Scotland (5%).

      There are certain differences between Programmes in participation rates:

                 HISTP has predominately supported the long term unemployed, to a much
                  larger extent than Scotland as a whole (50%). Similar to Scotland as a whole,

EKOS Limited                                                                                                                       11
                 Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
                the over 25 (7 to 12 months) unemployed individuals have also accounted for
                the lowest proportion of beneficiaries (4%);

               the under 25 (under 6 months) unemployed group accounts for the largest
                proportion of beneficiaries in the Objective 2 West Programme (17%); and

               the largest proportion of beneficiaries in the Objective 3 Programme is the
                long term unemployed (3 years +) and similar to HISTP the over 25 (7-12
                months) unemployed (4%) has the lowest proportion.



      TARGET GROUPS

      Target group beneficiary analysis shown in Figure 2.2 below illustrates that 40% of
      beneficiaries were placed in the ‘other’ category. It should be noted that more detailed

                                    Figure 2.2: Proportion Supported by Target Group

     75%                                                            71%

     65%

     55%

     45%                                                                                                      40%
                                                                                                                             HISTP
                                                                                                                             Objective 2
     35%                                                              30%                                                    Objective 3
                                                                                             27%
     25%                                           23%                                                    22%                Scotland
                                                     18%                            17%                  18%
     15%           12%
                     11%                      12%                                      11%              10%
                                8%                                        7% 7%              7%    6%
                                                                                                  5%                 5%
              3%       4%                     3%                     4%        4%                                   4%
      5%     1%   2%         1%       1% 1%                 1% 2%             1% 2% 2% 1%                               2%
            0% 1%           0% 1%    0% 1%                 0% 1%                                                       0%

      -5%
                                               e
                                               y




                                               s
                                               s




                                                                                                        er
                                               d




                                                                                                                    le
                                             rs
                                               s




                                               s




                                             ns
                                               s
                                            ac




                                            er
                                            nt




                                            ar
                                           es




                                            er




                                            ld
                                            le




                                                                                                               ab
                                          se




                                                                                                    th
                                          io
                                         ho
                                        ab
                                         re




                                        nd
                                          C
                                        us


                                         er
                                         el




                                                                                                   O
                                        at




                                                                                                              lic
                                        U
                                    Pa
                                  om




                                       g
                                 um




                                    se
                                      is
                                   ab




                                   f fe




                                    ic




                                                                                                          pp
                                    g




                                  in


                                 D




                                lif
                               ru




                              ou
                             -O
                             ne




                              av
      H




                              N
                               e




                                                                                                         -a
                            ua
                             D


                            nc




                            H
                          Le
                            d
                          Lo




                          Ex




                                                                                                     on
                          Q
                        an
                         ta




                          s
                      es
                        g




                                                                                                    N
                      bs




                       O
                     un
                     cy




                     N
                    kl
                  Su


                  ra


                Yo




                  or
               te




             W
               l/
           ho


            Li
         co
      Al




      information on the make up of the ‘other’ category was provided for the Objective 3
      Programme and this is provided later in the analysis. Following this the next largest
      represented group in Scotland have been individuals with literacy and numeracy
      problems (18%), then workless households (11%). Homeless and alcohol and
      substance abusers have been least represented, both accounting for 1% of
      beneficiaries.

               70% of those supported in the HISTP Programme area were disabled
                individuals, all other beneficiary groups individually account for less than 10%
                of the total supported; over the 5 years analysed there have been no
                supported drug users;

               those with no qualifications represent 27% of total beneficiaries on the
                Objective 2 West Programme followed by workless households (17%); and

               disabled people also accounted for the largest proportion of total beneficiaries
                on the Objective 3 Programme (30%), followed by those with literacy and
                numeracy problems (23%).


EKOS Limited                                                                                                                   12
               Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      We next look at these groups as a proportion of total numbers recorded in each
      category in the population as a whole.

2.2.4 Unemployment
      Due to difficulties in establishing comparable statistical reference points for the
      unemployment groups as categorised by ESF, it was possible only to look at
      unemployment figures against claimant count data for up to 3 years unemployed.
      Thereafter, it is assumed that beneficiaries who are not registered unemployed
      claimants are included in the “more than 3 years” category, including benefit
      claimants. There was no way of identifying the make-up of this target group.

      For groups up to 2 years unemployed, therefore, using average monthly claimant
      unemployment figures, it was possible to calculate for each year, which proportion of
      each unemployed beneficiary group were reached by ESF support. Figure 2.3 shows
      each ESF beneficiary group as a percentage of total unemployment for that particular
      group. In 2002 and 2003 over 40% of individuals that were unemployed 2-3 years
      were reached by ESF supported projects. Over 25s unemployed for fewer than 6
      months were least represented in all years as only 2% of that particular
      disadvantaged group was supported.

                               Figure 2.3: Beneficiaries as % of Unemployment

       60%


       50%


       40%
                                                                                                      2001
                                                                                                      2002
       30%
                                                                                                      2003
                                                                                                      2004
       20%


       10%


        0%
             Under 25 UN < Over 25 UN <   Under 25 UN   Over 25 UN 7-   UN 1-2 years   UN 2-3 Years
                  6M            6M          7-12 M           12M




      HISTP AREA

      Figure 2.4 below demonstrates that those individuals who have been unemployed 2-
      3 years have benefited most proportionately from ESF funded projects in the
      Highlands and Islands. In 2002, 72% of the total unemployed in this group were
      reached by ESF Measures with lower participation rates in other years. Having said
      this, this group has gradually benefited from the largest representation within their
      groups in each year except in 2003 and 2005. The second most represented group
      during the year is the under 25s, unemployed between 7 and 12 months except in the
      year 2003, where this group has accounted for one fifth of all people who have
      consistently benefited from ESF training measures.



EKOS Limited                                                                                          13
               Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      Generally, over the six years ESF funded projects have supported only a very small
      proportion of short-term (<6 months) unemployed under 25 years old.

      The graph also shows that in the year 2002 the Programme had highest participation
      rates in all beneficiary groups.




      LOWLAND SCOTLAND

      It was not possible to obtain claimant unemployment data which accurately reflects
      the West of Scotland Objective 2 area. However, the Objective 2 West and Objective
      3 data can be combined for analysis of beneficiaries against claimant count
      populations in lowland Scotland as a whole. Therefore Figure 2.5, below, illustrates
      which proportion of each unemployed group has been reached in the West of
      Scotland Objective 2 and Objective 3 Programmes combined. It should be noted that
      the final data for projects running in 2004/5 has not yet been submitted.

      The graph shows that 2001 and 2003 have been the most successful years in terms
      of supporting a large proportion of unemployed target groups, except in the
      unemployed 2-3 years category, where a very high proportion was reached in 2002.

      The highest proportion of unemployed 2-3 years have been reached in all years, this
      was almost 50% in 2002 and 2003. The second highest proportion of beneficiaries
      reached is the under 25 group, unemployed between 7 and 12 months, in particular
      almost a third of this group were reached in 2001. The smallest proportion of over
      25s unemployed for more than 6 months were supported in all years (less than 4%).




EKOS Limited                                                                            14
                 Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

                     Figure 2.5: % of Beneficiaries as % of Unemployed – Lowland Scotland



      60%


      50%


      40%
                                                                                                              2001
                                                                                                              2002
      30%                                                                                                     2003
                                                                                                              2004
      20%


      10%


       0%
              Under 25 UN < Over 25 UN >   Under 25 UN   Over 25 UN 7-    UN 1-2 years   UN 2-3 Years
                   6M           6M           7-12 M           12M




2.2.5 Target Groups
      SCOTLAND

      Table 2.1a and 2.1b below demonstrate which proportions of each client group have
      been supported most under the ESF funded projects in Scotland as a whole. It
      includes the total number of beneficiaries that have been supported in the Highlands
      and Islands, Objective 2 West and Objective 3 Programme areas. It can be seen that
      the largest proportion of beneficiaries have been disabled people (29%). The second
      largest proportion of beneficiaries was placed into the ‘other’ category (20%). It
      should be noted that it was possible to identify specific target groups within the ‘other’
      category for the Objective 3 Programme area but not for the Highlands and Islands
      and Objective 2 West areas, therefore this category cannot be broken down further
      when presenting data for Scotland as a whole.

      The next largest beneficiary group supported are those with literacy and numeracy
      problems (19%) and then lone parents (10%). Every other target group accounts for
      less than 9% of the total beneficiaries supported. The young leaving care and
      alcohol/ substance abusers account for the lowest proportion of supported
      beneficiaries (1%).
      TABLE 2.1A: PROPORTION SUPPORTED BY CLIENT GROUP, SCOTLAND
                                                     Alcohol/
                                                    substance  Literacy and    Young
                  Homeless  Lone Parents Drug users  abusers    numeracy    Leaving Care
                    No.     %      No.      %      No.      %       No.       %      No.      %         No.     %
      2001          521     4%    1,626    11%     128      1%      52       0%     2,324    16%        13     0%
      2002          104     1%     695      8%     112      1%       40      0%     1,031    12%        107    1%
      2003           50     1%     774     10%      21      0%      119      2%     2,005    26%        90     1%
      2004           20     0%     362      9%      20      0%       22      1%     1,142    27%        15     0%
      Total         695     2%    3,457    10%     280      1%      234      1%     6,503    19%        225    1%




EKOS Limited                                                                                                  15
                   Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

      TABLE 2.1B: PROPORTION SUPPORTED BY CLIENT GROUP, SCOTLAND
                                          Workless        No                                                Non-
                 Disabled   Ex-offenders households qualifications                            Other       applicable
                    No.         %          No.      %       No.      %      No.      %     No.        %    No.     %
      2001         4,507       31%      678         5%      436      3%     662     5%     3,321    23%    303    2%
      2002         2,936       34%      463         5%      200      2%     802      9%    1,949    23%    102    1%
      2003         1,997       26%       8          0%      635      8%     954     12%     924     12%    86     1%
      2004          864        21%       26         1%       163     4%      840    20%     719     17%    10     0%
      Total        10,305      29%     1,175        3%      1,433    4%     3,258    9%    6,913    20%    491    1%

      Figure 2.6 presents data with regards to the number of beneficiaries supported as a
      percentage of that particular total target group existing in Scotland, drawing on
      baseline statistics that are presented in Appendix 4.

      The total number of beneficiaries supported in 2003 was used in the calculation as
      this provided a best match to the baseline data available and it enabled us to present
      a picture on which target groups were most effectively supported in that particular
      year. Baseline data for 2003/04 was used in all cases other than for the ex-offenders
      target group as the latest baseline information available for this particular target group
      was 2002/03.

      Figure 2.6 suggests that 10% of all alcohol and substance abusers in Scotland
      participated in ESF projects. Almost 7% of young leaving care were supported in
      Scotland and just over 1% of lone parents were successfully targeted. Less than 1%
      of each other target group were reached and no ex-offenders were supported in
      2003. Table 2.19 demonstrated that those with literacy and numeracy problems and
      no qualifications combined accounted for 28% of all beneficiaries, yet 0.50% of that
      particular target group have been supported in Scotland overall. At the same time the
      young leaving care accounted for 1% of total beneficiaries, yet the Programmes
      successfully reached almost 7% of this particular disadvantaged group.

      Extreme caution must be exercised in these calculations, however, as project
      managers were able to attribute only one target group characteristic to each
      beneficiary. We do not know how many of the lone parent beneficiaries also had no
      qualifications, for example.

                                 Figure 2.6: Beneficiaries as % of Target Group, Scotland

       Alcohol/ Substance Abusers                                                                                9.6%

                Young Leaving Care                                                          6.9%

                       Lone Parents               1.1%

                            Disabled       0.3%

              Literacy and Numeracy        0.3%

               Workless Households      0.2%

                         Drug Users    0.2%

                   No Qualifications   0.2%

                           Homeless    0.1%

                       Ex-offenders    0.0%

                                    0.0%     1.0%    2.0%     3.0%   4.0%    5.0%   6.0%   7.0%    8.0%   9.0%   10.0%




EKOS Limited                                                                                                      16
               Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      HISTP AREA

      Almost three quarters of the total beneficiaries supported on the HISTP Programme
      were disabled and in 2005 the same group accounted for over 90% of total
      beneficiaries. Table 2.2A and B shows that in every year 2000-2005 disabled people
      have accounted for the highest proportion of those supported. Every other target
      group accounts for less than 5% of all beneficiaries, except individuals with no
      qualifications (7%) and clients placed in the ‘other’ grouping (10%).
      TABLE 2.2A : PROPORTION SUPPORTED BY CLIENT GROUP, HISTP AREA
                                                                                                         Young
                                                          Alcohol/substance Literacy and                Leaving
                  Homeless    Lone Parents   Drug users        abusers       numeracy                     Care

                 No.     %     No.    %       No.     %     No.           %          No.      %         No.    %
      2000        0     0%     20     2%         0   0%         0         0%         37       4%        0     0%
      2001        15    2%     4      0%         0   0%         0         0%          7       1%        0     0%
      2002        3     0%     48     4%         2   0%         4         0%         58       5%        8     1%
      2003        0     0%     10     2%         0   0%         1         0%          5       1%        2     0%
      2004        0     0%     16     2%         1   0%         9         1%         65       7%        7     1%
      2005        0     0%      3     1%         0   0%       1           0%          0       0%        4     1%
      Total       18    0%     101    2%         3   0%      15           0%         172      3%        21    0%



      TABLE 2.2B: PROPORTION SUPPORTED BY CLIENT GROUP, HISTP AREA
                                         Workless        No                                         Non-
                 Disabled  Ex-offenders households qualifications  Other                          applicable
                No.     %     No.    %       No.     %    No.       %          No.        %       No.         %
      2000      618    62%     0     0%      0       0%    72       7%         189    19%          60         6%
      2001      778    78%     1     0%      14      1%    1        0%          94    9%           83         8%
      2002      803    64%     2     0%      26      2%    98       8%         206    16%          1          0%
      2003      301    59%     0     0%      3       1%    96       19%         9      2%          84         16%
      2004      768    77%     3     0%      30      3%    95       10%         6      1%          0           0%
      2005       471   91%     1     0%      26      5%    8        2%          6      1%          0          0%
      Total     3739   71%     7     0%      99      2%   370       7%         510    10%         228         4%




      Figure 2.6 illustrates which beneficiary groups in the HISTP area have benefited
      most from ESF funded support in the year 2003. Again baseline data for 2003/2004
      has been used for the analysis. It has not been possible to identify the number of ex-
      offenders, disabled people, workless households and individuals with literacy and
      numeracy problems in the Highlands and Islands area, therefore this information is
      excluded from the analysis.

      Similar to the results in Scotland, the young leaving care in the Highlands and Islands
      have been most successfully reached (2%). However, it should be noted that only 2
      of the 100 that exist in the area were supported. Alcohol/ substance abusers in the
      Highlands and Islands area accounted for the second highest most successfully
      supported beneficiaries, yet they made up less than 1% of total beneficiaries
      supported in 2003.

      In 2003, those with no qualifications made up almost a fifth of all those supported.
      The 96 individuals that were reached by the Highlands and Islands Programme
      accounted for 0.25% of the total with no qualifications in the area. The Highlands and
      Islands Programme failed to reach any drug users or homeless in 2003.



EKOS Limited                                                                                                 17
                 Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

                              Figure 2.6: Beneficiaries as % of Target Group, HISTP area


        Young leaving care                                                                                 2.00%


       Alcohol / Substance
                                                                                      1.37%
            Abusers


              Lone Parents              0.27%



          No Qualifications             0.25%



               Drug Users     0.00%



                 Homeless     0.00%


                          0.00%                 0.50%              1.00%               1.50%         2.00%

      OBJECTIVE 2 WEST AREA

      The proportion of target groups that were supported on the Objective 2 West
      Programme is shown in Table 2.3A and B. Over the years 2001-2004, it appears
      that those with no qualifications received most support as they accounted for 27% of
      the total beneficiary group, the second largest group was individuals placed in the
      ‘other’ category (18%) and then workless households (17%). Lone parents and those
      with literacy problems both accounted for 12% of the total. All other groups accounted
      for less than 5%. This pattern is very similar in all years from 2001-2004 except in
      2003 where almost one-quarter of all those supported were individuals with literacy
      and numeracy problems and one-quarter were workless households. There is a
      decline from 2003 in the number of beneficiaries placed into the ‘other’ and ‘non-
      applicable’ categories, possibly due to improvements in definitions and reporting of
      information.
      TABLE 2.3A: PROPORTION SUPPORTED BY CLIENT GROUP, OBJECTIVE 2 WEST AREA
                                                     Alcohol/                  Young
                                                    substance   Literacy and  Leaving
                Homeless   Lone Parents Drug users   abusers     numeracy       Care
                   No.        %       No.       %       No.    %       No.      %        No.    %    No.      %
      2001          21        1%      196       12%      16    1%          14   1%        65    4%    1      0%
      2002          2         0%      160       17%       9    1%          14   2%        25    3%    0      0%
      2003          21        1%      306       11%      15    1%          20   1%       587   22%   80      3%
      2004          10        1%      116        9%      19    1%          12   1%        84    7%    3      0%
      Total         54        1%      778       12%      59    1%          60   1%       761   12%   84      1%



      TABLE 2.3B: PROPORTION SUPPORTED BY CLIENT GROUP, OBJECTIVE 2 WEST AREA
                                          Workless        No                   Non-
                   Disabled Ex-offenders households qualifications   Other   applicable
                 No.      %  No.     %   No.    %    No.       %   No.    %  No.     %
      2001           42       3%       40       3%      245    15%     320      20%      418   26%   220     14%
      2002           44       5%       28       3%      124    13%     211      23%      205   22%   101     11%
      2003          152       6%       0        0%      614    23%    715       26%      190    7%    2       0%
      2004           47       4%       23       2%      123    10%    493       38%      359   28%    0       0%
      Total         285       4%       91       1%      1106   17%    1739      27%     1172   18%   323      5%


EKOS Limited                                                                                                18
                 Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      OBJECTIVE 3 AREA

      It was possible to present a more detailed picture of the percentages of target groups
      supported by the Objective 3 Programme as detailed data on the make- up of the
      ‘other’ category was provided, this is provided in Table 2.4.

      It is apparent from Table 2.4 that over the years 2000-2004 disabled people
      accounted for almost a third of those total beneficiaries supported by the Objective 3
      Programme. The second largest group that were supported was those with literacy
      and numeracy problems (23%), this is followed by the 13% of beneficiaries who were
      classed as long term unemployed and 11% of total clients on Objective 3 were lone
      parents. Beneficiary groups that were least supported over the period (all accounting
      for 0.9% of total beneficiaries) were educationally disadvantaged, gypsy travellers,
      over 50s and people with learning disabilities.

      There was a significant increase in the percentage of literacy and numeracy
      beneficiaries in 2003 and 2004, this group accounted for more than half of all those
      supported in 2004. Table 2.4 shows a considerable increase in the percentage of
      labour market returners in 2004 (to 12% from 0% in all other years). Also in 2004
      there was a decline in the proportion of disabled beneficiaries as they only accounted
      for 3% of the total in 2004 and approximately third in 2001-2003.



      TABLE 2.4: PROPORTION SUPPORTED BY CLIENT GROUP, OBJECTIVE 3 AREA
                             2001           2002               2003          2004            Total
                           No.    %       No.    %        No          %    No.    %        No.     %
      Disabled people     3,687    31%    2,089   33%    1,544    35%       49     3%     7,369   30%
      Literacy and
      Numeracy            2252     19%    948     15%    1413     32%      993     52%    5,606   23%
      LT Unemployed       1767     15%    973     15%    465      10%       0       0%    3,205   13%
      Lone Parents        1426     12%    487      8%    458      10%      230     12%    2,601   11%
      No Qualifications    341     3%     493     8%     143          3%   252     13%    1229    5%
      Ex – Offenders       637     5%     433     7%      8           0%    0       0%    1078    4%
      At School            228     2%     293     5%      0           0%   117     6%      638    3%
      Ethnic Minorities    320     3%     263     4%      43          1%    0      0%      626    3%
      Homeless             485     4%      99     2%      29          1%    10      1%     623    3%
      Returners             53     0%      8      0%      0           0%   237     12%     298    1%
      Workless
      Households           177     1%      50     1%     18           0%    10     1%      255    1%
      Asylum Seekers        92     1%      1      0%     130          3%    0      0%      223    1%
      Drug Users           112     1%     101     2%      6           0%    0      0%      219    1%
      Alcohol/
      Substance
      abusers              38      0%      22     0%      98          2%    1      0%      159    1%
      Pupils in danger
      of exclusion         148     1%      0      0%      0           0%    0      0%      148    1%
      Young Leaving
      Care                 12      0%      99     2%      8           0%    5      0%      124    1%
      Educationally
      Disadvantaged        105     1%      0      0%      0           0%    0      0%      105    0%
      Over 50               2      0%      0      0%      87          2%    0      0%      89     0%
      Learning
      Disabilities         54      0%      0      0%      0           0%    0      0%      54     0%
      Non- Applicable      40      0%      0      0%      0           0%    0      0%      40     0%
      Gypsie
      Travellers            0       0%      0      0%      0       0%        0      0%      0     0%
      Total               11,976   100%   6,359   100%   4,450    100%     1,904   100%   24,689 100%



EKOS Limited                                                                                      19
               Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

2.2.6 Summary
      The analysis of beneficiary data has shown that ESF social inclusion Measures have
      predominately supported the over three years unemployed, suggesting that the
      Measures are indeed reaching those furthest from the labour market. In terms of
      target groups the largest supported groups have been those with literacy and
      numeracy problems (18%) and workless households (11%). Between Programmes,
      certain differences in participation rates can be noted such as in the case of HISTP
      that had a high proportion of beneficiaries who were disabled (70%).

      Compared to the overall numbers of unemployed registered in Scotland and target
      groups which could be identified by statistical references, ESF Measures in Scotland
      were most successful in reaching individuals in target groups such as alcohol and
      substance abusers, the young leaving care and to a certain extent – lone parents. At
      the same time, ESF support reached a small percentage of homeless and ex-offender
      populations. Results for the HISTP area were similar, in that the Measures were
      most successful in reaching alcohol and substance abusers and the young leaving
      care.


2.3   ANALYSIS OF PROJECT RESULTS
2.3.1 Introduction
      Beneficiary data in each Programme area was compared against outcomes in order
      to establish if any relationship exists between the nature of Programme or project
      types and project performance. The analysis aimed to explain why projects performed
      differently to others.

2.3.2 Method
      In order to establish if there was a relationship between the nature of a particular
      project and performance, we chose and analysed data from projects approved in one
      year in each Programme area. The particular years that were examined were those
      approved in 2004 in the HISTP Programme area; 2004 in the Objective 3 Programme
      area; and 2003 in the Objective 2 West Programme area. (Note that the full projects’
      outcomes were analysed, regardless of the duration of the project.)

      Projects set a range of targets from a selection provided by each Programme.
      Reported outcomes included numbers entering employment, numbers entering
      education and numbers entering training. As some of the projects included a range of
      these project indicators and others only one or two, it was decided that it was most
      appropriate to consider the effectiveness of the projects in relation to placing
      beneficiaries into:

              Employment; and

              Employment, education or training

      Projects were organised into quintiles, ranked in relation to which were the most
      effective in meeting these particular targets. The percentage of beneficiaries placed
      into employment and employment, education, training in each quintile was then
      calculated. Following this, client groups within the various projects were analysed in
      order to assess any potential relationships between project category and success.




EKOS Limited                                                                             20
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      HIGHLANDS AND ISLANDS PROGRAMME

      Table 2.5 demonstrates success rates of the top to bottom quintile of projects,
      showing the percentage of beneficiaries that moved into employment, education and
      training in the Highlands and Islands area. It can be seen that those projects in the
      top quintile have placed 51% of those supported into employment, education or
      training; this falls to 21% in the second, 9% in the third and no beneficiaries in the
      bottom quintile.

      Analysis of the data identifies that almost all of the individuals supported through the
      Highlands and Islands Programme are either disabled or long term unemployed,
      therefore, it has been difficult to establish the relation between the nature of the
      projects and success. However, a small number of projects have supported other
      groups and interestingly, those that have done so have performed better. Those
      projects in the top 20% have the greatest proportion of beneficiaries who were short
      term unemployed, had literacy problems and were returning to the labour market. The
      remaining four quintiles are all projects where the vast majority of individuals
      supported have been unemployed long term or disabled people.

      Colleges and voluntary organisations tended to deliver those projects that appeared
      in the best performing quintiles whilst Local Authorities were responsible for delivering
      the majority of projects in lower quintiles. It was found that projects in the top quintile
      had a smaller number of average beneficiaries than those in 2, 3 and 4th quintile but
      not in comparison to the bottom quintile. There was no relationship found between
      geographical location and performance, as there is a mix of both urban and rural
      projects in the top and bottom quintiles.



      TABLE 2.5: HIGHLANDS AND ISLANDS: QUINTILES OF % OF COMPLETERS/LEAVERS MOVING
      INTO EMPLOYMENT, EDUCATION OR TRAINING

      Ranking of Projects              Employment, education or             Employment
                                              Training

      Top 20% of projects                         51%                           18%

      21-40% of projects                          21%                            5%

      41-60% of projects                          9%                             3%

      61-80% of projects                          4%                             0%

      81-100% of projects                         0%                             0%




      A total of 18% of beneficiaries in the top 20% of projects are now in employment. This
      falls to 5% of beneficiaries the second quintile, and no individuals in the bottom fourth
      and fifth quintiles.

      As with analysis of the data from the outcomes of those beneficiaries placed into
      employment, education or training, the majority of projects support the long-term
      unemployed or disabled. However, similar to what was stated previously those
      projects, which support other groups such as short term unemployed, people with
      literacy/numeracy difficulties and labour market returners again are more successful
      at placing beneficiaries into employment. Colleges were mainly responsible for
      delivering those projects that were in the top performing quintiles and again councils
      delivered the largest proportion of projects in the bottom performing quintiles.

EKOS Limited                                                                                  21
                  Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      Contrary to the finding above, there appeared to be no relationship between project
      size and success. However, it was found that urban projects outperformed their rural
      counterparts in terms of employment, education and training outcomes, possibly due
      to more opportunities being available to leavers/completers.

      Table 2.6 below demonstrates the relationship between project performance in
      employment, education and training and average funding per head. There is a
      significant difference between the cost per head between the top and bottom quintile
      of projects, the cost per head of the lowest performing projects is considerably higher
      than both the top performing quintile and the middle quintiles. However, there does
      not seem to be a consistent pattern in the middle quintiles between performance and
      cost as the second lowest performing quintile has the lowest cost per head and the
      second best performing quintile has the second highest cost.

      TABLE 2.6: HIGHLANDS AND ISLANDS: QUINTILES OF AVERAGE FUND SPEND PER HEAD

      Ranking of Projects in employment,                            Average Fund spent per head (£)
      education and training

      Top 20% of projects                                                           1,776

      21-40% of projects                                                            1,919

      41-60% of projects                                                              952

      61-80% of projects                                                              814
                                                                                            1
      81-100% of projects                                                           9,515



      OBJECTIVE 3 PROGRAMME

      In the Objective 3 Programme, a high proportion of beneficiaries supported in the top
      20% of projects move into employment, education or training (90%); more than two-
      thirds of those supported in the second quintile; and 19% of those beneficiaries
      supported by projects in the bottom quintile (Table 2.7).

      Projects in the top quintile are not similar in types of target group, consisting of a
      range of projects that focus on supporting disabled people, lone parents, workless
      households or ethnic minorities. More than one-half of the projects in the second
      quintile are assisting lone parents and returners to the labour market. Examination of
      the data also establishes that almost all of those projects in the bottom performing
      quintiles are aiming to tackle literacy issues.

      Projects that appear in the top quintile in terms of performance tend to be those
      delivered by voluntary and training organisations, whilst Councils and colleges tend to
      appear in the lower quintiles. Those projects in the top quintiles have a considerably
      lower number of beneficiaries than those in the bottom three quintiles.

      Explanations for these outcomes may be that the college and council projects tend to
      be larger and target those with literacy problems where it is perhaps more difficult to
      place the individuals into employment or education. It may be that lone parents,
      workless households and ethnic minorities are perhaps more attractive to employers
      and closer to the labour market than these particular individuals.


      1
       This was skewed by one particular project that had a very high cost to beneficiary ratio (A.S.E.T. by Argyll & Bute
      Council)


EKOS Limited                                                                                                          22
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      The successful projects run by voluntary organisations tend to have smaller numbers
      of beneficiaries and are perhaps dealing with smaller niche target groups, this may
      make it easier to focus more specifically on each individuals needs. It is possible that
      because the college projects are larger it is more difficult for them to flexibly respond
      to the varied support needs of all beneficiary groups.



      TABLE 2.7: OBJECTIVE 3: QUINTILES OF % OF COMPLETERS/LEAVERS MOVING INTO
      EMPLOYMENT OR TRAINING

      Ranking of Projects               Employment, Education or            Employment
                                               Training

      Top 20% of projects                        90%                            52%

      21-40% of projects                         66%                            31%

      41-60% of projects                         39%                            18%

      61-80% of projects                         32%                            6%

      81-100% of projects                        19%                            2%

      Just over one-half of Objective 3 Programme beneficiaries in the top performing
      quintile of projects move into employment. This falls to 31% in the second quintile and
      18% in the third, 6% in the fourth and 2% in the bottom quintile. The pattern between
      employment outcomes and nature of projects in each quintile is largely similar to the
      relationship between the nature of projects and beneficiaries moving into
      employment, training and education. Similar to that found in the analysis of education
      and training outcomes, the top performing delivery organisations are voluntary
      organisations whilst colleges are less likely to appear in the top quintiles.

      Analysis of the data identified no relationship between project performance and
      geographical location. Projects that took place in 2004 did appear to be mainly
      urban. However, it is likely that there are rural areas within the local authority areas
      that the data does not identify. The results of the project manager survey in the
      following section provides a more detailed analysis on this issue as it is likely that the
      project managers are better able to identify which projects were delivered specifically
      in urban areas.

      OBJECTIVE 2 WEST PROGRAMME

      Table 2.8 below illustrates that those projects that fall within the top quintile place
      83% of beneficiaries into employment, education or training; this falls to 62% for the
      top 21-40% and to 15% for those projects in the bottom range of projects.

      A pattern emerges in relation to the top 20% performing projects in terms of moving
      beneficiaries into employment/ education or training. The greatest proportion of
      beneficiaries in the top quintile is short-term unemployed. Also, in one project almost
      half of those supported are lone parents. The second quintile encompasses projects
      that focus on short and long term unemployed as well as those with no qualifications.

      Those in the fourth quintile (61-80%) of projects support greater proportions of
      individuals that face barriers such as long-term unemployment, being a lone parent or
      holding no qualifications.

      The bottom quintile of projects consists of three projects that overall successfully
      place 15% of beneficiaries into employment, education or training. These projects

EKOS Limited                                                                                 23
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      include support to individuals that tend to face multiple barriers to employment such
      as ex-offenders and drug users. Top performing delivery organisations were
      identified, as voluntary organisations and colleges. There does not appear to be any
      relationship between project size or geography and performance.



      TABLE 2.8: OBJECTIVE 2 WEST: QUINTILES OF % OF COMPLETERS/LEAVERS MOVING INTO
      EMPLOYMENT, EDUCATION OR TRAINING

      Ranking of Projects              Employment, Education or           Employment
                                              Training

      Top 20% of projects                       83%                           63%

      21-40% of projects                        62%                           33%

      41-60% of projects                        32%                           25%

      61-80% of projects                        27%                           14%

      81-100% of projects                       15%                           4%




      Table 2.8 also shows that the top 20% of projects overall successfully place 63% of
      beneficiaries into employment. The pattern is similar to that observed above where
      beneficiaries are placed into employment, education or training. The top quintiles
      tend to consist of projects that support the short-term unemployed and lone parents.
      Those in the bottom quintile that only place 4% of beneficiaries into employment
      again tend to be projects that support those who face a multitude of barriers.
      However, a greater number of projects that support the long-term unemployed appear
      in the bottom tier, perhaps due to the fact that along with those that face a variety of
      barriers they are likely to be less attractive to employers than those that have only
      been unemployed for a short term or lone parents who may face lesser barriers such
      as availability of childcare.

      Voluntary organisations and colleges were mainly responsible for delivering projects
      that appeared in the top performing quintile of projects whilst local development
      companies tended to deliver those in the bottom quintiles. However, those colleges
      that did appear in the bottom quintile supported a larger number of beneficiaries than
      their better performing counterparts; this is similar to the finding of the Objective 3
      Programme, again suggesting that smaller ‘niche’ projects are more successful,
      perhaps as its easier to provide more flexible tailored support. It was found that as
      with the Highlands and Islands and Objective 3 Programmes, smaller projects had
      better employment outcomes than the larger ones. This was not the case with
      regards to employment, education and training outcomes. There was no direct
      relationship established between project location and performance, using this
      approach of data analysis.

2.3.3 Summary
      Analysis of the data establishes that in the Highlands and Islands Programme the
      largest proportion of beneficiaries on all projects are disabled people and the long-
      term unemployed. However, the small number projects that tackle other barriers are
      more successful in terms of employment and training outcomes. In terms of delivery
      organisations, colleges perform best whilst Councils tend to appear in the lower
      quintiles. Smaller projects perform best on employment, education and training
      outcomes whilst urban projects perform best on employment outcomes only. The

EKOS Limited                                                                               24
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      data also identifies that the lowest performing projects have an average cost per head
      far above that of the top performing projects.

      There appears be no specific type of project that performs exceptionally well in the
      Objective 3 Programme. However, projects that have a high percentage of
      beneficiaries who are returners to the labour market or lone parents appear in the
      middle quintiles and those that support clients with literacy problems are least
      successful. Voluntary organisations tackling small numbers of beneficiaries have
      performed comparatively better than larger projects that have been delivered by
      colleges; these divergences may be explained in a number of ways, such as
      differences in the attractiveness of the beneficiary groups to employers, difficulties
      caused by project size or the fact that voluntary organisations may be offering more
      ‘niche’ services.

      The most successful projects in the Objective 2 West Programme have been those
      that support mainly short-term unemployed and lone parents. Projects supporting
      more of the long-term unemployed, those with literacy problems and those with no
      qualifications have achieved less positive outcomes. Those that achieve the lowest
      rate of employment/training and employment outcomes support those that face
      multiple barriers such as being an ex- offender or drug user. In addition, smaller
      projects tend to perform more successfully as well as those delivered by voluntary
      organisations and colleges.

      Over all of the Programme areas added together it has been found that:

              generally, those that tend to support the short term unemployed are most
               successful, whilst those that support the long term unemployed, those who
               face multiple barriers and labour market returners are less so;

              projects that perform well tend to do so in terms of employment and in
               education and training, and this is also the case for projects that perform
               poorly;

              the Highlands and Islands Programme underperforms both the Objective 3
               and Objective 2 West Programmes in employment, education and training
               outcomes in all quintiles. This may suggest that rural projects encounter
               greater external barriers to positive outcomes;

              projects in the Objective 3 Programme focus more specifically on particular
               target groups, whilst those projects in Objective 2 West have a mixed group of
               clientele. Objective 3 projects perform better on education and training
               outcomes but underperform on employment outcomes in all quintiles against
               Objective 2 West;

              it seems that smaller projects have stronger performance than their larger
               counterparts; often these are delivered by voluntary organisations; and

              there is no obvious relationship between geographical location and
               performance. However, a large number of projects are delivered in both
               urban and rural areas and so it is very difficult to establish a geographical
               relationship. The delivery organisations are therefore best placed to make
               this judgement and a more detailed analysis of this issue is provided in the
               Chapters 3 and 4.




EKOS Limited                                                                              25
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

2.4 NET IMPACT ANALYSIS

2.4.1 Introduction
      This section estimates the gross and net employment impacts of the ESF Social
      Inclusion Measures within the Highlands and Islands, West of Scotland Objective 2
      and Objective 3 Programmes at the Local Authority Level. Net economic impact at
      the national level will also be assessed. Before doing so gross outcomes will be
      arrived at by allowing for:

      Attribution: the extent to which gross job outcomes are attributable to the EU funded
      intervention;

      In order to generate robust net impacts from gross outcomes, the following factors will
      be taken account of:

      Additionality: accounting for the extent to which the projects would have taken part
      without EU funding and considering the differences in project timescale and size;

      Displacement: taking account of the extent to which jobs the EU funded beneficiaries
      may have been taken from other individuals who did not benefit from support; and

      Multipliers: applying multipliers to reflect the additional knock on effects elsewhere in
      the local and national economy.



2.4.2 Assumptions
      Providing a net impact for the study is somewhat problematic in that no beneficiary
      interviews have been undertaken and no leavers survey data is available (a new
      MORI Beneficiaries Survey will be published later in 2005). Therefore in order to
      estimate net impacts it has been necessary to apply the following assumptions:

      ATTRIBUTION

      It has been assumed that all the employment impacts are attributable to the EU
      funded intervention. This assumption is based on the fact that the client groups for
      these interventions exhibit personal and behavioural characteristics that make them
      unattractive to employees, which would largely explain their non-employment status
      prior to involvement with the project, i.e. they had been unable to secure employment
      prior to joining the Programme;

      ADDITIONALITY

      As part of the work programme we assessed, via telephone consultations with a
      sample of project managers, the extent to which individual projects would have
      proceeded without EU support.

      A stratified random sample of projects was selected for the purpose of the report to
      ensure that a relevant range of projects was represented. These projects were
      assessed on the following to establish to what extent they would have gone ahead
      without EU funding:

              absolute additionality: where all gross jobs are additional and none of the
               jobs outcomes would have occurred in the absence of funding;


EKOS Limited                                                                                26
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
              time additionality: we allocated 10% additionality for every year for which
               the reported changes were brought forward; and

              scale additionality: establishing and accounting for the effect EU funding
               had on the scale and quality of projects.

      The following additionality factors were applied:

              projects that would not have gone ahead without support were assigned an
               additionality factor of 1;

              projects that would have been delayed by more than 24 months and been of
               significantly lower scale and quality in the absence of funding were assigned
               a factor of 0.5;

              projects that would have proceeded but would have been delayed by up 24
               months, were given an additionality factor of 0.2; and

              if a project would have gone ahead regardless of funding then zero
               additionality factor was assigned.

      DISPLACEMENT

      Displacement was considered as it relates to the labour market in each Local
      Authority area and the extent to which the jobs obtained by project beneficiaries have
      been at the expense of other unemployed persons. Displacement was assessed as
      being low (20%) in areas where unemployment was below the Scottish average, 40%
      in areas where unemployment was on par with the average and 60% when
      unemployment was above the average.

      MULTIPLIERS

      The non-displaced employment impacts in each Local Authority area were carried
      forward and a combined supplier and income multiplier was applied, taken from the
      HM Treasury’s A Framework for the Evaluation of Regeneration Programmes and
      Projects. These were assumed to be;

              1.15 for the combined supplier and income multiplier at the Local Authority
               level; and

              1.48 for the combined supplier and income multiplier at the National level.

2.4.3 Gross Impacts
      GROSS EMPLOYMENT

      Table 2.9 outlines the gross level of job creation generated by ESF projects in each
      of the Local Authority Areas. The highest proportion of employment generated is
      concentrated in Glasgow City (42%). The remaining 28 Local Authority areas outwith
      Glasgow individually account for less than 10% of all gross employment outcomes.


      TABLE 2.9: ATTRIBUTABLE GROSS EMPLOYMENT BY LOCAL AUTHORITY AREA
                 Local Authority              Gross Employment               %
      Glasgow City                                 2,829                   42%
      Fife                                          506                     8%
      Highland                                      434                     7%

EKOS Limited                                                                                 27
               Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

      West Lothian                                313                    5%
      Dundee City                                 300                    5%
      North Ayrshire                              300                    5%
      City of Edinburgh                           233                    4%
      South Lanarkshire                           205                    3%
      North Lanarkshire                           203                    3%
      Renfrewshire                                174                    3%
      East Ayrshire                               155                    2%
      West Dunbartonshire                         149                    2%
      Aberdeen City                               143                    2%
      Midlothian                                  131                    2%
      Inverclyde                                  114                    2%
      Shetland Islands                            103                    2%
      South Ayrshire                               95                    1%
      Western Isles                                54                    1%
      Falkirk                                      43                    1%
      Argyll & Bute                                41                    1%
      Perth & Kinross                              37                    1%
      Angus                                        29                    0%
      Stirling                                     29                    0%
      Orkney Islands                               11                    0%
      Moray                                         9                    0%
      Aberdeenshire                                 7                    0%
      Dumfries & Galloway                           7                    0%
      East Dunbartonshire                           2                    0%
      East Lothian                                  1                    0%
      Total                                      6,657                  100%

      The projects have generated a total of 6,657 gross jobs in Scotland overall. Table
      2.10 summarises the number of gross jobs that were generated under each social
      inclusion Measure.



      TABLE 2.10: ATTRIBUTABLE GROSS EMPLOYMENT BY PROGRAMME AREA
               Programme area               Gross Employment              %
      Highlands and Islands                       643                   10%
      West of Scotland Objective 2               3,424                  51%
      Objective 3                                2,590                  39%
      Total Scotland                             6,657                  100%

      More than one-half of all employment outcomes were generated as a result of the
      West of Scotland Objective 2 Programme (51%). A total of 2,590 (39%) were created
      through the Objective 3 Programme and 643 (10%) through the Highlands and
      Islands Programme.



2.4.4 Gross to Net
      ADDITIONALITY

      An average additionality factor over all projects was calculated.       Based on the
      responses received, we found that:




EKOS Limited                                                                            28
                   Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
                24 project managers reported absolute additionality, so were allocated an
                 additionality factor of 1;

                9 projects reported significant time delay, quality and scale and were
                 assigned a factor of 0.5;

                1 project manager reported only time delay of 24 months and so was
                 allocated an additionality factor of 0.2; and

                No project managers reported zero additionality.

      The results generated an average additionality factor of 0.84, therefore 84% of
      gross job outcomes were carried forward from each project for further analysis.
      Employment outcomes were then assigned and summed to each Local Authority
      area.

      DISPLACEMENT

      The displacement consideration took into account levels of unemployment in the local
      authority labour markets that would reduce the impact of gross employment
      outcomes.The assessed levels of displacement for each local Authority area are
      shown in Table 2.11.



      TABLE 2.11: LEVELS OF DISPLACEMENT BY LOCAL AUTHORITY AREA
      Unemployment above Scottish Average - Displacement Factor: 0.6
      Dundee City                                              East Ayrshire
      Fife                                                     Glasgow City
      Inverclyde                                              North Ayrshire
      West Dunbartonshire                                      Western Isles


      Unemployment at Scottish Average - Displacement Factor: 0.4
      Angus                                                   Argyll and Bute
      Dumfries and Galloway                                       Falkirk
      Highland                                               North Lanarkshire
      Renfrewshire                                            North Ayrshire


      Unemployment below Scottish Average - Displacement Factor: 0.2
      Aberdeen City                                           Aberdeenshire
      East Dunbartonshire                                      East Lothian
      Edinburgh City                                            Midlothian
      Moray                                                   Orkney Islands
      Perth and Kinross                                      Shetland Islands
      South Lanarkshire                                           Stirling
      West Lothian



      MULTIPLIER EFFECTS

      The knock on employment generated from the various ESF projects were taken
      account of using a multiplier of 1.15 at the local level and 1.48 at the national level.


EKOS Limited                                                                                29
                   Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      NET ADDITIONAL EMPLOYMENT

      Applying additionality, displacement and multiplier effects to gross employment
      outcomes of the Highland and Islands, West of Scotland Objective 2 and Objective 3
      Programme generates net additional employment. This method and the outcomes at
      each stage are outlined in Table 2.12 below:



      TABLE 2.12: NET ADDITIONAL EMPLOYMENT BY LOCAL AUTHORITY AREA
                                                                    Net                          Net
                                           Gross                   direct                     Additional
                        Gross   Less non    Direct       Less     Addition        Plus        Employme
                        Direct additional Additional Displacement    al          Multiplier      nt
      Glasgow               2829      452        2377        1426        950       143          1093
      W. Lothian             313       50        263          53         210        32           242
      Fife                   506       81        425         255         170        26           196
      Edinburgh              233       37        196          39         157        23           180
      Highland               434       71        363         218         145        22           167
      S. Lanarkshire         205       32        173          34         138        20           159
      N Lanarkshire          203       32        171         68          103        16           117
      Dundee City            300       48        252         151         101        15           116
      North Ayrshire         300       48        252         151         101        15           116
      Aberdeen City          143       23        120          24         96         14           111
      W.Dunbartonshire       149       97        236         141         95         15           108
      Midlothian             131       21        110          22         88         13           101
      Renfrewshire           174       28        146          59         87         14           101
      East Ayrshire          155       25        130          78         52          8           60
      South Ayrshire          95       15        80           32         48          7           56
      Inverclyde             114       19         95          58         39          6           44
      Western Isles          54        9          45          18         27          4           31
      Perth & Kinross        37        6          31          6          25          4           29
      Falkirk                43        7          36          14         22          3           25
      Stirling               29        5          24          5          19          3           22
      Shetland Islands       103       15         88          70         18          3           20
      Angus Council          29        5          24          10         15          2           17
      Argyll & Bute          41        7          34          20         14          2           16
      Moray                  9         1          8           2          6           1           7
      Aberdeenshire           7        1          6           1           5          1              5
      Dumfries &
      Galloway                7        1          6           2           4          1              4
      Orkney Islands         11        2          9           7           2          0              2
      E.Dunbartonshire        2        0          1           0           1          0              1
      East Lothian            1        0          1           0           1          0              1
      Total                 6,657    1,138      5,702        2,964      2,739      413          3,147




      A total of 3,147 jobs have been generated at the local level in Scotland. A large
      proportion of these jobs have been created in Glasgow City (1,093).

      Table 2.13 below demonstrates the calculation of net additional employment at the
      national level.



EKOS Limited                                                                                   30
                 Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

      TABLE 2.13: NET ADDITIONAL EMPLOYMENT BY PROGRAMME AREA
                                                                                                      Net
                                  Less non     Gross Direct    Less     Net direct    Plus         Additional
                   Gross Direct   additional    Additional Displacement Additional   Multiplier   Employment
      Scotland        6,657         1,138         5,702       2,964       2,739       1,315          4,054




      Applying a combined income and supplier national multiplier calculates that a total of
      4,054 net additional jobs were created in Scotland as a result of ESF funding.

2.4.5 Summary
      Net additional employment generated as a result of the ESF social inclusion
      Measures is estimated at a total of 3,147 at the local level and 4,054 at the Scotland
      wide level.

      Within each Local Authority area, net additional employment generated was varied in
      value. Glasgow City accounted for 35% of all jobs, whereas the remaining 28 Local
      Authorities individually accounted for less than 10% of the total net employment.



2.5 BENCHMARKING PERFORMANCE AND EFFECTIVENESS
2.5.1 Introduction
      A central objective of this evaluation is to assess progress and effectiveness,
      including cost-effectiveness. This requires the use of appropriate benchmarks. This
      Section therefore looks at performance in three ways:

                at Measure level, against Programme targets;

                at UK level, across comparable social inclusion Measures; and

                against other funding mechanisms with social inclusion objectives.

2.5.2 Benchmark 1: Fund Objectives/Targets
      Initial Programme objectives/targets provide a key point of reference against which to
      assess progress. Taking findings from the PME’s own MTEU reports (in the case of
      Objective 3 this information was further updated by PME information), we now
      compare actual against forecast expenditure, outputs and results. This is
      accompanied by an assessment of the extent to which initial objectives/targets were
      appropriate and realistic. In addition, some of the key performance indicators in each
      of the Programme Measures are put in the context of overall EU expenditure incurred.

      Unit costs are used to compare actual with planned/anticipated outturns, allowing for
      an assessment of the effectiveness and efficiency of the Programme. Similar
      comparisons can also be undertaken between Programmes. The strength of unit
      costs is that they are independent of any actual over or under-spend of allocated
      financial resources.

      We begin by examining the progress of each relevant Programme Measure.




EKOS Limited                                                                                        31
                 Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

      Table 2.14, below, illustrates progress in the Western Scotland Objective 2
      Programme Measure 3.2 at 31 March 2005.



      TABLE 2.14: WEST O2 MTEU FINANCIAL AND PHYSICAL PROGRESS AT 31 MARCH 2005
                                     Funds        Funds       Claimed to   % Funds     % Funds
                                     Available    Committed   Date         Committed   Claimed
                                          £26.47m     £18.29m      £10.62m         69%         40%

                                     PC Target        Forecast         Actual            % PC Target % PC Target
                                                                                         met by      met by
                                                                                         Forecast    Actual
      Outputs
      Total no. beneficiaries              19,852           19,157              10,916            96          55
      Total no. of employed                 2,589            1,388                 967            54          37
      beneficiaries
      No. of beneficiaries getting          5,179                176               51              3           0
      self-employment help
      Number of companies                        25              231               69            924         276
      helped
      Results
      No. of positive outcomes             10,841           10,673               3,603            98          33
      for leavers/completers
      No. of leavers/completers             6,905            7,022               2,510           102          36
      gaining full/part
      qualification
      No. of beneficiaries                 15,881           11,767               6,135            74          39
      completing their course
      Source: West O2 MTEU


      This Measure is forecast to perform well across most indicators with the exception of
      beneficiaries receiving self-employment support (an indicator that is also not
      achieving to target in other Programmes). Actual performance is broadly in line with
      the % of funds claimed to date. One area of over-performance is assistance to
      companies, although the number of employed beneficiaries assisted is not as high as
      expected.

      It should be noted that the MTEU did not record progress towards other results such
      as beneficiaries entering education/training and employment. The actual unit cost
      calculations set out in Table 2.15 (below) are therefore drawn from Scottish Executive
      data. This indicates that unit costs per output indicators are slightly lower than the
      SPD predicted. The unit costs for qualifications and employment results meets
      expectations, while education/training outcomes are significantly lower.

      Turning to the Objective 3 Programme, Table 2.15 shows progress in the Objective 3
      Measure 2.1 at December 2004 as per information provided by the PME in
      September 2005.

      The original funds available for this Measure amounted to £48.8m, however, following
      virements, further funds have been allocated to meet demands, bringing the total
      amount available to £78.4m.




EKOS Limited                                                                                                  32
                 Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

      TABLE 2.15: OBJECTIVE 3 MTEU FINANCIAL AND PHYSICAL PROGRESS AT DEC 2004
                                   Funds        Funds               Claimed to   % Funds     % Funds
                                   Available    Committed           Date         Committed Claimed
                                      Following    £64.9m                 £41.2m          83                   53
                                      virements
                                         £78.4m
                                   OP Target    Forecast            Actual            % OP Target % OP Target
                                                                                      met by      met by
                                                                                      Forecast    Actual
      Outputs
      Total no. beneficiaries              50,544          43,109            51,808            85             103
      No of people given                   38,690          40,592            40,625           105             105
      guidance
      No of people trained                 46,430          38,613            48,752            83             105
      Results
      No. of positive outcomes             29,251          25,477            32,639            87             112
      for leavers/completers
      No. of people trained                15,322          24,228            16,088           158             105
      gaining a (part)
      qualification
      No. of beneficiaries                 27,858          27,628            29,251            99             105
      completing their course
      No of people going onto              19,036          14,431            21,241            76             112
      further education/training
      No of people going into                7,893         10,637             8,289           135             105
      employment
      Source: Objective 3 MTEU and updated actual data to 31 December 2004 provided by the PME in September 2005

      This Measure is performing well against Operational Programme targets. It is
      exceeding expectations for this stage of the Programme in all of the selected
      performance indicators, and indeed projects have exceed their own expectations for
      all output indicators and many results indicators (the exceptions being numbers of
      people trained gaining a qualifications and numbers into employment). The actual unit
      costs have been significantly lower than anticipated (75-80% of forecast) for all of the
      output and results indicators as illustrated further in Table 2.16 below.

      Finally, Table 2.16 describes the progress for the HISTP Measure 3.2 as captured in
      the MTEU.




EKOS Limited                                                                                                   33
                 Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

      TABLE 2.16: HISTP MEASURE 3.2 MTEU FINANCIAL AND PHYSICAL PROGRESS TO MAY 2005
                                    Funds        Funds        Claimed to   % Funds      % Funds
                                    Available    Committed    Date         Committed    Claimed
                                          £8.47m       £6.49m       £4.75m           77                               56

                                    PC Target         Forecast          Actual           % PC Target % PC Target
                                                                                         met by      met by
                                                                                         Forecast    Actual
      Outputs
      No of individuals given an                  -          1,230                924
      one-to-one interview and
      individual action plan
      No of people receiving                3,000            5,170               3,323           172                 111
      training to nationally
      recognised standards
      Results
      No. of leavers/completers             1200                    -            1059                  -              88
      gaining full/part
      qualification
      No of leavers/completers                  700              1294             568            185                  81
      entering employment
      No of leavers completers                  800              3138            1939            392                 242
      entering ed/training
      Source: HISTP MTEU


      The Measure is forecast to greatly exceed Programme expectations across a range
      of key performance indicators, and indeed, actual achievements have already
      exceeded targets for the numbers of people trained (111%), and those entering
      further education or training (242%).

      The Programme assumed an average cost per person trained as £2,823; per
      leaver/completer gaining a full/part qualification £7,058; per leaver/completer entering
      employment £12,100; and per leaver/completer entering education or training
      £10,585. Actual unit costs have been considerably lower across all of these indicators
      (see Table 2.17) indicating that the Measure has been more cost-effective than
      anticipated. Indeed, the Programme makers may have been unambitious in their
      targets.

      Table 2.17 draws these findings together to show the unit costs anticipated and
      actually achieved by the three Programmes.


      TABLE 2.17: UNIT COST OVERVIEW

                               OBJECTIVE 2 WEST                    OBJECTIVE 3                         HISTP

                                   Target       Actual           Target          Actual       Target           Actual

      No of beneficiaries          1,333         973              965             796            -               -

      No of people trained         1,365        3,215            1,051            846          2,823           1,429

      No of people gaining
                                   3,833        4,231            3,185           2,564         7,058           4,485
      qualifications

      No of people entering
                                   4,764        4,883            6,183           4,976        12,100           8,363
      employment

      No of people entering
                                   7,941        3,623            2,564           1,942        10,585           2,450
      education/training




EKOS Limited                                                                                                         34
                    Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

2.5.3 Benchmark 2: Performance Across Areas
      The national coverage of ESF activity also allows us to benchmark activity between
      areas and sectors. Such comparisons could highlight variations in performance
      across areas and thematic interests, and provide a basis for identifying and
      understanding the factors that contribute to Programme success.

      First, Table 2.18 examines the implementation of social inclusion Measures within the
      Objective 3 Programmes across the UK. We find that ESF contributions in Scotland
      and England are roughly similar at over £40m to the end of December 2004, whereas
      Wales has a smaller level of resources available.

      TABLE 2.18: COMPARISON OF OBJECTIVE 3 MEASURE 2.1 ACROSS THE UK

                                                          Scotland                 Wales                England

                                                        (At Dec 2004)          (At Dec 2003)         (At Dec 2004)

      No of Beneficiaries                                   51,808                  4,052                55,404

      % of Beneficiaries in Work on Leaving                  16%                    10%                   21%

                                                                                                                  2
      ESF Costs (£m)                                        41,245                  6,654                44,183

      ESF Cost per Beneficiary                               796                    1,642                  797

      ESF Cost per Beneficiary into Work                    4,976                  16,931                 3,797

      Total Eligible Expenditure                            91,655                 14,787                94,972

      Total Cost per Beneficiary                            1,769                   3,649                 1,714

      Total Costs per Beneficiary into Work                 11,057                 37,626                 8,163
      Notes: Calculations based on information supplied in the Wales Objective 3 AIR 2003; England Objective 3 AIR 2004;
      Scotland Objective 3 based on information received directly from the PME. The Wales Objective 3 AIR 2003 was used
      because the 2004 report will not be published until October 2005; however the unit costs calculations should be
      comparable as progress is calculated based on spend and performance as recorded at any one point in time.

      At this rather simplistic level of analysis it would appear that Scotland’s performance
      lies between England and Wales in terms of beneficiaries gaining work on leaving.
      Looking at cost-effectiveness, Scotland’s unit costs are lower than Wales and much
      the same as England in terms of spend per beneficiary, but slightly higher than
      England as regards costs per beneficiary into work. Factors that may contribute to
      variation in performance include the:

                 structure of projects: variances in the size of projects or types of support
                  being offered;

                 types of individuals supported: there may be variations in the types of
                  individual being supported in the Programmes, their distance from the labour
                  market and barriers to be addressed;

                 employment and unemployment rate variations between the three countries,
                  which will affect the availability of work for these beneficiaries to move into;




      2
          Conversion rate = £1:0.62€


EKOS Limited                                                                                                          35
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
              delivery mechanisms: it is not possible at this level to analyse the contribution
               of the characteristics of organisations delivering projects and their
               organisational capacity; and

              geographic barriers: in Wales, there are clearly barriers to project
               implementation associated with rurality, whereas it is likely that a large
               percentage of projects in England will be delivered in urban areas; however,
               again we do not know enough about the projects to assess how this may
               contribute to cost-effectiveness.

2.5.4 Benchmark 3: Comparator Funds
      A valuable sense of perspective can also be gained from identifying how ESF social
      inclusion Measures compare to similar or related funding mechanisms.

      The following analysis of five funding programmes targeted at disadvantaged groups
      provides a useful point of reference against which to assess current performance, and
      a potential source of learning.

      Table 2.19, over, provides an overview of the following programmes:

              New Futures Fund – a programme managed by SE/HIE aiming to reduce
               social exclusion and disadvantage amongst younger people (16-34) by
               providing skills and mentoring to help them find a job;

              Get Ready for Work – a SE/HIE programme for 16-18 year olds who require
               support to access training or employment after school;

              New Deal for Disabled people – a UK-wide programme offering advice and
               assistance to enable disabled people to enter or re-enter the labour force;

              New Deal for Lone Parents – a UK-wide programme offering advice and
               assistance to enable lone parents to enter or re-enter the labour force; and

              Beattie Inclusiveness Projects – a Careers Scotland-managed programme
               for young people aged 16-24 leaving education/in care who are disengaged
               from the labour market.

      Briefly, some interesting points from the comparison include:

              Programmes/projects need time to bed in; motivated and experienced staff
               are critical;

              Individualised services, and one-to-one support building relationships
               between a mentor and client are found to be effective;

              Use of partnership organisations and specialist support services to take a
               more holistic approach to client progression adds value to the beneficiaries’
               experience;

              Aftercare can be difficult to manage and resource – job retention support is
               necessary to ensure sustainability of outcomes; and

              Forming relationships with employers is key, and good quality work
               placements invaluable to client progression.



EKOS Limited                                                                                 36
                                                                                                                 Scottish Executive: Review of the Effectiveness of Social Inclusion Measures


      TABLE 2.19: OVERVIEW OF COMPARATOR FUNDING MECHANISMS

                                          Characteristics                   Finance/Outcomes                    Strengths/Good Practice         Weaknesses/Issues                Lessons and Future
                             3
      New Futures Fund                    Focuses on a range of             Phase 2: £14.2m                     Takes holistic approach         Aftercare not consistent         Takes time to develop
                                          disadvantaged       groups        15% into employment/self            Additionality    –    extra     Barriers to mainstreaming –      effective projects – funders
                                          (mainly      16-34)      to                                           services, new ways of           including budget holders         need to invest in the
                                          engage/re-engage       with       36% into education/training                                                                          capacity building stage
                                                                                                                working, wider access           perception      of       cost-
                                          labour market                     Av cost per start: £2100                                            effectiveness/priorities         Challenge to pass lessons
                                                                                                                Good results in job and job-
                                          Managed      by   SE/HIE,         (further cost analysis below)       related outcomes – but                                           into the mainstream
                                          delivered by the voluntary                                            other progression routes                                         Larger projects most cost-
                                          and public sector                                                     identified and monitored                                         effective
                                 4
      Get Ready for Work                  For 16-18 years who need          Y1: £14.02m in SEN area;            Concentration            on     Early – needs time to bed in     More       resources    for
                                          support to access training        £0.68m in HIE area                  addressing individual needs     Varied quality of training       development of training
                                          or employment after school        Av. Cost per training week:         One-to-one work                                                  materials,    for  Personal
                                                                                                                                                Opportunities for formal         Advisory Service, quality
                                          Package: assessment and           £135 SEN; £149 HIE                  High     quality        work    accreditation limited
                                          action plan process by CS;                                                                                                             assurance
                                                                            Av cost per beneficiary:            placements                      Limited aftercare
                                          4    strands   of   training                                                                                                           Design aftercare system
                                          delivery; ongoing review by       SEN: £2,817; HIE: £1,639            Stimulates new networks
                                                                                                                and partnerships                                                 Improve      performance
                                          CS      Personal   Advisory                                                                                                            measurement and tracking
                                          Service.
                                                                                                                                                                                 Offer accreditation options
                                      5
      New Deal for Lone Parents           Help for lone parents to          317,000 clients 1998-2002           Job         quality/sustained   Locational factors influence     Factors in effectiveness:
                                          take up/increase paid work,       51% left Income Support             outcomes better than for        work outcomes, including         motivated    PAs;       PA
                                          improve job readiness, and        and into work of at least           non-participants                deprived wards and rural         autonomy and flexibility to
                                          employment opportunities          16hrs per week                      Personal Advisors range of      areas                            tailor to client needs;
                                          Personal Advisers provide                                             services expanding and          Less effective with those        caseload     management
                                                                            Increased   exits    from                                                                            skills
                                          help with finding work,           benefits to work by 24              highly valued                   further from the labour
                                          accessing training etc            percentage         points,                                          market and parents looking       Future: Improve local links;
                                                                            measured over 9mths                                                 for higher level jobs            extend toolkit; meet needs
                                                                                                                                                                                 of    repeaters;   improve
                                                                                                                                                                                 employability/employment
                                                                                                                                                                                 retention



      3
          TERU et al (2005) Evaluation of the New Futures Fund Initiative
      4
          Smart Consultancy et al (2003) Get Ready For Work Programme: First Year Evaluation and Future Development Options
      5
          Evans, M et al (2003) New Deal for Lone Parents: Second Synthesis Report of the National Evaluation




EKOS Limited                                                                                                                                                                                                    37
                                                                                                                     Scottish Executive: Review of the Effectiveness of Social Inclusion Measures



                                            Characteristics                   Finance/Outcomes                   Strengths/Good Practice          Weaknesses/Issues               Lessons and Future

      New Deal          for   Disabled      Personal Adviser Service to       10% of clients left benefit        Intervention            Fund     Increasingly       outcome      Improve info to employers
            6
      people                                help people find/return to        22% into work                      (discretionary,   used     to    focused and less holistic       Flexibility      in     time,
                                            work – set goals, provide                                            assist clients move towards      Clients mixed view on how       resources, expertise to deal
                                            advice, job search, contact       21%                     started    employment) opened up
                                                                              training/education                                                  well advisers understood        with diverse client group
                                            employers,       refer  for                                          opportunities and plugged        illness/ability to work
                                            specialist help.                                                     gaps in existing services                                        Improve monitoring/quality
                                                                                                                                                  Hard to educate employers       insurance
                                            Take-up tended to be by                                              Partner         organisations
                                            those closer to the labour                                           contributed    to improved       Employers satisfied when:       Need for      continuity    of
                                            market:      younger,    with                                        service                          contact    with    adviser;     service
                                            qualifications,       driving                                                                         knowledgeable;        good      Ensure advice on benefits,
                                                                                                                 Occupational psychologist        understanding of employer
                                            licence         and      car,                                        support valued                                                   tax credits and financial
                                            economically active partner.                                                                          Variation in success rates in   support for move into work
                                                                                                                                                  getting clients into work       Better alignment of benefit
                                                                                                                                                  directly related to labour      and employment provision
                                                                                                                                                  market demand
                                                                                                                                                                                  Job retention support
                                        7
      Beattie           Inclusiveness       16-24 leaving education/in        £15m 2001-2004                     Distinctive localised delivery   Better   management        of   Regional       inclusiveness
      Projects                              care and needing support          circa. 8,400 clients               Clients value customised         ending of support               strategies?
                                            for transition to labour                                             support by Key Workers           Need more support for           Review links with          Get
                                            market – clients disengaged       40% into training
                                                                                                                 and felt involved in planning    clients leaving education       Ready for Work
                                            and distanced from LM             circa. 20% into unsupported        their options                    and training for work -
                                            Key Worker support before,        employment                                                          aftercare
                                                                                                                 New/improved      partnership
                                            during and after transition                                          working   for      vulnerable    Less effective joint working
                                            stage – guidance and                                                 young people                     at strategic level
                                            support    and   individual
                                            action plans                                                                                          Need improved monitoring




      6
          Lournidis, J. et al (2001) Evaluation of the New Deal for Disabled People Personal Advisor Service Pilot
      7
          SQW Ltd (2005) National Evaluation of the Careers Scotland Inclusiveness Projects




EKOS Limited                                                                                                                                                                                                       38
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

      Various approaches have been taken to assessing cost-effectiveness and good
      value:

              New Deal for Lone Parents: evaluators concluded that the Programme is
               cost-effective. Economic gain to society was calculated at £4,400 per
               additional job and the net exchequer saving was estimated at just under
               £1,600 per additional job with substantial social benefit reported.

              The New Deal for Disabled People evaluation did not make an assessment
               of value for money because the Programme was a pilot. Significantly,
               however, although over a fifth of clients had found work, there was no
               evidence that the Personal Adviser Service had significantly increased the
               movement of disabled people into work – possibly due to a short monitoring
               period or because movements to intermediate outcomes temporarily delayed
               movement to employment;

              New Futures Fund: Various approaches were taken to assessing cost-
               effectiveness – comparisons by target group; type of delivery organisation;
               size of project; and maturity of project. Factors associated with higher cost-
               effectiveness were identified as:
                     Smaller scale NFF funding
                     Lower NFF funding dependency
                     Local authority over voluntary sector provision
                     Projects operating in both Phase 1 and Phase 2 as opposed to Phase
                      2 only (therefore more mature)
                     Larger scale projects in terms of number of client starts.
               It was further concluded that comparing NFF to other employability enhancing
               programmes with similar client groups yielded few reliable benchmark figures;

              Beattie Inclusiveness Projects: no explicit value for money assessment,
               possibly due to data deficiencies. Concluded that the initiative had been
               effective in enhancing client employability and assisting clients into positive
               outcomes – however, many of these outcomes are short-lived and aftercare
               systems require greater attention.

              Get Ready for Work: noted variations in the cost of delivery between the SE
               and HIE areas. No other relevant comment.




EKOS Limited                                                                               39
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

2.5.5 Benchmarking Cost-effectiveness by Target Group
      The New Futures Fund Evaluation provides a useful analysis of cost-effectiveness at
      target group level. This examined cost per start and closure, as well as positive
      outcomes at three levels:

              Level 1: move into New Deal/New Deal Gateway, pre-vocational training,
               voluntary work and other employability projects;

              Level 2: moving into supported employment, FE/HE, Training for Work,
               Skillseekers, Get Ready for Work, ILM project;

              Level 3: moving into employment and self-employment.

      As many of the client groups targeted within this programme are mirrored by ESF it is
      possible to conduct a comparative analysis of cost-effectiveness.

      Due to the structure of projects, it was necessary to restrict the analysis to projects
      that could be easily identified as targeting a single client group, thus the ESF
      comparator is the Objective 3 Measure.

      Taking project information supplied by the Objective 3 PME, projects were allocated
      to target categories according to their main client group. Certain adjustments to the
      data were made, e.g.

              some projects which had been classified as “Other” target group could be
               filtered into standard headings (e.g. Long Term Unemployed);

              a small number of projects had to be removed from the analysis because of
               multiple or difficult to categorise target group identification; and

              where non-reporting of key indicators was found, e.g. where possible a value
               was identified from examining other indicators, or the project was removed.

      New Futures Fund and ESF output and result indicators were aligned as follow:



      New Futures Fund                         Objective 3

      No. of starts                            No. of beneficiaries

      No. of closures                          No. of beneficiaries completing their course

      No of positive outcomes – Level 3        No. of leavers/completers into employment

      No. of positive outcomes – Level 1+2     No. of leavers/completers into education /
                                               training



      Table 2.20, over, presents the findings of this cost comparison.




EKOS Limited                                                                               40
                                                                                   Scottish Executive: Review of the Effectiveness of Social Inclusion Measures


TABLE 2.20: COST AND PERFORMANCE COMPARISON OF OBJECTIVE 3 WITH NEW FUTURES FUND
                                                                               Homeless           Lone Parent          Drug User            Disabled          Ex-Offender          Ethnic Minorities
                                                                             ESF       NFF       ESF       NFF       ESF       NFF       ESF       NFF       ESF       NFF          ESF      NFF
% of Beneficiaries Completing Course                                             67        66        78         52       52         68        57        70        37        75          66        81
% of Beneficiaries Entering Employment                                           11        11        17         15       18          7        15         7         8        16          12        20
% of Beneficiaries Entering Ed/Training/Employment                               47        30        56         30       29         31        47        42        26        36          50        52


Total Cost per Beneficiary                                                    2,328     1,700     2,730     3,100     2,267     2,400     3,077     2,955     1,712     2,000        2,022    1,600
Total Cost per Completer                                                      3,472     2,500     3,510     6,000     4,397     3,600     5,407     4,233     4,685     2,700        3,075    2,000
Total Cost per No. into Employment                                          20,921 15,500 16,140 21,200 12,750 36,000 19,994 41,423 21,136 12,900 16,349                                      8,000
Total Cost per No into Ed/Training/Employment                                 4,938     5,600     4,901 10,400        7,846     7,800     6,569     7,204     6,698     5,500        4,079    3,100
Notes:
Extreme caution is advised in the interpretation of cost comparisons. The ESF Cost information is based on an estimate of total project costs (actual ESF contributions = 45% of
total costs). The NFF data is Total NFF Contract Actual Spend. This is likely to be an underestimate of total project costs as contractors were free to supplement projects with
non-NFF funding. We do not have information on the extent to which this took place.
The New Futures Fund data for Disabled and Mental Health groups has been merged to compare more closely with the ESF Disabled definition. Note, however, that almost
three-fifths of NFF clients experienced mental health issues, and that average costs for this client group were significantly lower than for the (physically) Disabled.




EKOS Limited                                                                                                                                                                 41
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

Analysis of the above data reveals that:

              higher course completion rates generally lead to a higher percentage of
               beneficiaries entering education, training and employment;

              for both Programmes, Lone Parents, Disabled People and Ethnic Minorities
               achieved the highest levels of positive outcomes. Interestingly, the highest
               spend per head was for Disabled People and Lone Parents, whereas Ethnic
               Minorities were among the groups with the least spend per head;

              in terms of employment outcomes, NFF was significantly more successful
               with Ethnic Minorities and Ex-Offenders, whereas ESF achieved better with
               Drug-Users and Disabled People. There is no direct correlation between the
               spend per beneficiary and the achievement of these results;

              ESF recorded higher or equivalent overall positive outcomes across most
               categories, with the exception of Ex-Offenders; and

              in terms of costs per beneficiaries into employment, education or training,
               both Programmes were broadly similar with the exception of Lone Parents,
               where ESF was significantly more cost-effective.




EKOS Limited                                                                             42
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

3. FINDINGS FROM STRATEGIC INTERVIEWS

3.1 INTRODUCTION
      The study team carried out a total of 30 interviews with organisations with an active
      involvement or interest in the social inclusion Measures of the three Programmes
      under review. A full list of consultees is provided in Appendix 1.

      These interviews gathered stakeholder views on the effectiveness of the Programmes
      in reaching disadvantaged groups, their ability in adding value to domestic funding,
      and their success in achieving and demonstrating targeted outcomes.

      The findings are presented below, where appropriate, disaggregated by Programme.

3.2 IMPACT AT STRATEGIC LEVEL

3.2.1 Compatibility with domestic funding
      National and local agendas with regard to the implementation of ESF social inclusion
      Measures seem to be well aligned with the strategic priorities of other partner
      organisations.

      Indeed, a number of consultees pointed to the positive and continuing alignment of
      funding streams over time. That said, a number of interviewees in lowland Scotland
      commented on an increasingly difficult climate for securing domestic funding,
      particularly among voluntary organisations.

      The key issues in terms of funding streams were highlighted as:

              a lack of grant funding for projects targeting young people;

              some difficulties in aligning ESF with Social Inclusion Partnership (SIP)
               funding and strategies. Albeit, it was noted that a more concerted effort is
               being made to align ESF with the Community Regeneration Fund (which has
               now replaced SIP funding);

              an increasing administrative burden in managing multiple funding sources,
               including incompatible reporting requirements and difficulties in the attribution
               of outcomes; and

              the need for more cross-departmental integration within the Scottish
               Executive itself, with more attention paid among policy makers to European
               funding.

      Although other funding has been increasingly tied to outcomes, many interviewees
      commented that ESF, in not having an outcome based approach, has enabled
      organisations to take risks and innovate, and develop flexible and tailored
      approaches.

3.2.2 Dependency on ESF?
      Given that the ESF resource is becoming increasing limited year-on-year, there is a
      need to manage individual and community expectations and to continue to forge new
      funding partnerships in order to sustain project activities.



EKOS Limited                                                                                 43
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      There is, a broad consensus that the majority of ESF project activity could not survive
      in the absence of public funding. Question marks are raised over the ability of bodies
      such as Jobcentre Plus, Local Authorities and the Scottish Executive to find
      alternative funding to help to sustain projects.

      From a general overview of the ESF Programme there is concern that not enough
      organisations are developing their own exit strategies from ESF funding after
      2006/2007. Organisations have built up with much assistance from ESF and are now
      highly dependent on the Funds. Without obvious alternatives many projects have
      struggled to focus on long-term funding strategies.

      That said, there are notable exceptions. For example:

              the North Highland College is currently developing online hospitality courses
               through private funding from MacDonald Hotels and thus moving away from
               ESF support.; and

              Merkinch Enterprise and TAG Highland have been successful in generating
               commercial income. In Merkinch, the Childcare Centre and the web design
               service operate commercially with any income realised being reinvested in the
               local community.

      The Objective 2 West Programme in particular has been encouraging project
      sponsors to gear up for the post-ESF situation. However, it appears that applicants
      are not yet addressing the issue of financial sustainability well.

      Across our consultations senior level discussion is advocated, to consider the
      implications and options in the case of ESF withdrawal.

3.2.3 Influence on social inclusion approaches
      It is clear from feedback from the consultees that ESF funding, both in cash and in
      kind, has provided a buffer to enable risks to be taken in order to trial and test various
      training projects that would not have happened otherwise.

      One example of where European Funding has led to different approaches to reaching
      target groups include: coordination through Fife European Group, comprising
      colleges, the LEC, local authority and voluntary organisations, which takes a more
      strategic approach on bidding and ensures synergy in the activities of the different
      organisations. This ensures that clients are directed to the best source of training to
      meet their needs, that there is no competition for clients or funding.

      Other organisations have been able to test different approaches to reaching target
      groups, e.g. Fairbridge, which has developed a softer and step-by-step approach and
      does not emphasise job outcomes at the outset. The Wise Group on the other hand,
      has attracted a large client base and built up a reputation for quality training and high
      job outcomes.

      Another interesting approach has been the Glasgow Re:focus Programme, which has
      developed collaboration with social services in employability activities; for many
      individuals, social services are the main or only contact with public services and this
      presents an opportunity to engage with individuals who are far from the labour
      market. This work could be developed further under a future Programme.

      It had been hoped that some new approaches trialled under the Equal Community
      Initiative would have been developed further under Objective 3, however, the timing
      of these pilots and the quick absorption of funding under Priority 2 has prevented this.

EKOS Limited                                                                                 44
               Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      Positive movements, from Jobcentre Plus towards extending support for those on
      inactive benefits and NHS Scotland towards pre-employability assistance, as well as
      the engagement of social services in employability support, suggests a shift in
      domestic policy which will assist ESF social inclusion objectives in the future.

      The PME in the Highlands and Islands has a close working relationship with each
      project and has adopted a more proactive approach with communities and individuals
      from the initial project identification and design stage through to fostering a more
      joined up network between projects to enable the dissemination and sharing of good
      practice and learning with regard to the overall project development and
      management. Indeed the PME in partnership with the UHI (which includes the
      network of colleges throughout the Highlands and Islands) is proactively aiming to
      attain more buy-in from communities through a thematic area approach to issues and
      not just an ESF Programme focus. Furthermore, since Objective 1 the role of the
      voluntary and private sector has changed, particularly with regard to supported
      employment, resulting in more valuable jobs being created.

      The ESF funding has also given projects the initial financial foundation on which to
      attract match funding through, for example, the LECs, the Local Authority Social Work
      Departments, Careers Scotland, Job Centreplus, the National Lottery et al. However,
      the availability of match funding has been an issue for some organisations or for
      some areas not targeted with area-specific programmes.

      For some projects the use of ESF funding has delivered a much more diverse
      approach to projects with the development of commercial components than originally
      anticipated.

      For the Supported Employment Programme, the ability under the ESF Programme to
      couple job training with a wage plus a general increase in the education for the
      individual with special needs has exceeded expectations of what could be realised.

      One consultee noted that the role of ESF in meeting social inclusion needs has
      changed since the Government began to introduce policies specifically addressing
      economic inactivity; it was argued that now a more targeted approach is required to
      fill gaps not met by mainstream programmes.

3.2.4 Management and Delivery Processes
      The Programmes have also clearly contributed to the development of organisational
      capacity within project sponsors in terms of:

              financial management and accountability;

              project monitoring arrangements; and

              internal project management processes and skills.

      The development of organisational capacity, however, is attributed more to smaller
      organisations, sub-contractors and project partners than to larger organisations
      (which report pre-existing management capacity).

      The other area cited by a large number of consultees was the role of the Programmes
      in stimulating the development of stronger and more varied partnerships for project
      delivery. It was argued that voluntary organisations have achieved an enhanced and
      more balanced role in partnership arrangements over the years.




EKOS Limited                                                                            45
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

3.3 PROGRAMME LINKAGES

3.3.1 Objective 2 West and Objective 3
      Despite steps having been taken following the MTE to improve coordination between
      Objective 3 and Objective 2 West, there remains a perception that the social inclusion
      Measures have not been well linked and that mechanisms are inadequate to ensure
      synergy.

      This lack of integration is attributed to:

              the practicalities of the administrative arrangements;

              the lack of consultation between the Objective 3 and Objective 2 West
               Programmes at the initial planning stage; and

              the ambiguity in relation to the availability of ESF funding for Objective 2 West
               Transitional areas.

      Where the integration of Objective 2 West and Objective 3 is evident, this is reported
      to be as a result of a considered approach by project sponsors and a helpful PME,
      rather than systematic Programme integration.

      That said, the awareness and approach of sponsors varies considerably. It was
      commented that organisations tend to approach either Objective 2 West or Objective
      3. Some, however, have been highly effective in utilising both funding streams for
      discrete elements of project activity.

      It was suggested that improved strategic management and more joint monitoring
      visits to projects would be of value to ensure a rounded ESF overview of project
      activities.

3.3.2 ERDF and ESF
      Opinions were fairly evenly divided as to whether ERDF and ESF had been
      successfully integrated.

      On the one hand, multifunctional local development organisations commented on the
      ability to make linkages between physical infrastructure developments and socio-
      economic activities, thereby achieving complementarity. Representatives of the FE
      sector were able to identify linkages through the various infrastructure projects that
      have been supported.

      On the other hand, these linkages were noted at project or organisational level,
      perhaps achieved at local discretion rather than by Programme design. There was a
      call for a more strategic and coordinated approach to ERDF and ESF at Programme
      level. The use of Community Planning structures was highlighted as an appropriate
      mechanism through which to ensure strategic co-ordination on an area-wide basis.

      Another issue reported is that there is not the same requirement placed on ERDF
      projects to consider ESF linkages as are expected of ESF applicants to consider
      ERDF. Objective 3 ESF application forms, however, require applicants to discuss
      ERDF linkages even if the project is not located in an Objective 2 area.




EKOS Limited                                                                                 46
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

3.4 EFFECTIVENESS IN REACHING TARGET GROUPS
      In general, consultees indicate that all targeted groups have benefited from the
      delivery of ESF funded projects.

      A range of methods of reaching target groups were raised during interviews,
      including:

              funding of outreach workers, e.g. in the Full Employment Area project –
               effective in reaching economically inactive people in their communities;

              set up of referral networks, e.g. TAG Highland trainees are referred to the
               project via mental health agencies, hospitals and GP surgeries, and therefore
               there is often a waiting list for courses;

              cross-over from other internal activities, e.g. in North Ayrshire, users of
               flexible learning centres are encouraged to move on to the Access to Training
               and Employment initiative; and

              taster sessions, commonly used in Colleges to ease beneficiaries into the
               learning environment.

      It is not possible from the project monitoring system to analyse outcomes per target
      group, although it has been pointed out that manual examination of claim reports
      would achieve this.

      It should also be noted that in Objective 3, the effectiveness in reaching the target
      groups envisaged by Programme Measure 2.1 will be hidden by the project reporting.
      As projects only select one main target group, there was a tendency to use the
      broader categories such as literacy/numeracy and no qualifications as a catch-all to
      reach the 80% quota required for beneficiary groups.

      Consultees point to a drop in the participation of smaller organisations in this
      programming period, potentially affecting the Programme’s reach to certain niche
      groups; examples being people with alcohol and drug problems and ex-offenders.
      The reasons for this may be lack of capacity to secure and administer funding, as well
      as the nature of the issues, which require more flexibility and possibly less job output
      orientation than can be permitted in ESF.

      However, other organisations noted that the Measure structures of Objective 3 meant
      that it would be easier for smaller organisations who target niche groups, whereas
      organisations whose activities crossed Measure boundaries or target categorisations
      had to make false distinctions, dividing up their activities for project applications on a
      thematic and/or geographic basis to fit with Measure eligibility criteria.

      Although consultees noted that target groups are being reached throughout the
      Highlands and Islands on a project-by-project basis, the dispersed geography of the
      area is always challenging in terms of accessibility for such groups in remote rural
      areas to access training support.

      However, ESF funding has allowed the participant training providers to reach more
      clients by more innovative delivery, e.g. through the use of open learning, online
      learning and taking learning in the candidate’s own home. It has also been known for
      a course to run in another location rather than at the training provider’s base in order
      to work around the specific requirements of the client group.



EKOS Limited                                                                                 47
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      In order to strengthen Programme targeting, there has been some debate within the
      PMC as to whether to introduce a specific round to target groups which appeared to
      be under-represented in the Programme, however, as no targets had been set for the
      Programme to achieve in this regard, it was considered unnecessary in terms of
      fulfilling Programme objectives to introduce a further barrier to project applicants.

      There has also been discussion regarding the possibly ranking reserve list projects to
      prioritise projects that target underrepresented groups, however, it was rejected as
      being unfair to retrospectively add a condition post-application stage. Furthermore,
      there were only a few projects that would have met this objective, e.g. specifically
      targeting ex-offenders, and these were not originally ranked highly in the reserve list.

      A further discussion was held over whether to introduce thresholds for
      literacy/numeracy difficulties in order to ensure that individuals in real need of
      assistance were being targeted. The introduction of a requirement for evidence of
      literacy/numeracy needs of a certain level was considered inappropriate at this late
      stage in the Programme; however, this type of testing would seem worthwhile to
      ensure that organisations do not assume certain characteristics but provide evidence
      which can be comparable with other projects, in order to ensure that those most in
      need are being prioritised. This is a point for consideration in any future Programme.

3.5 IMPACT ON TARGET GROUPS

3.5.1 Change in perceptions to training
      On the question of whether the Measures have achieved a change in perception
      among target groups towards training, consultees noted a number of elements of
      added value in the training experience offered via ESF social inclusion Measures.
      The Measures have:

              enabled organisations to work flexibly and target support to individual’s
               needs;

              reduced the emphasis on accredited qualifications, which has attracted a
               client group not normally attracted to traditional training interventions;

              brought a more holistic focus on health and employability support that have
               provided positive steps towards rehabilitation;

              provided a vital workplace experience element, combined with interview skills
               training which assists candidates improve their recruitment prospects;

              enabled “hooks into learning” such as taster sessions and confidence building
               activities that have attracted those beneficiaries most disaffected with
               mainstream provision.

      Overall, there is widespread agreement that the flexibility offered by the ESF
      Programme is vital in responding to the needs of the target groups, who would
      otherwise have drifted away from mainstream programmes and become increasingly
      excluded. For example, it gives training providers more scope to:

              waive fees compared with normal mainstream training and development
               programmes, thus removing financial barriers to participation; and

              allow candidates who have dropped out to repeat courses and gain the
               qualifications to progress in further education or mainstream training such as
               Get Ready for Work and New Deal programmes.

EKOS Limited                                                                               48
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

3.5.2 Bringing beneficiaries closer to the labour market
      In the main, consultees agreed that the training offered to individuals as a result of
      ESF funding had indeed brought these beneficiaries closer to the labour market.

      The thematic and geographic targeting within ESF means that applicants are
      encouraged to consider potential barriers to the labour market and build in
      supplementary support to alleviate difficulties (e.g. travel expenses, additional one-to-
      one mentoring, and aftercare support). In the course of time, as these features have
      been validated as contributing to successful integration into the labour market, it has
      been noted that they have been increasingly incorporated into mainstream
      programmes, such as New Deal.

      For Objective 3, the Measures appear to have achieved much better results than
      anticipated at the outset. In part this has resulted from improved economic conditions
      and tighter labour markets, bringing increased opportunities for employment among
      more disadvantaged groups.

      For Objective 2 West area, it was reported that success in the labour market could be
      partially attributed to the emphasis the PME placed on economic cohesion which
      directed projects towards placing beneficiaries in employment situations, and linking
      with mainstream programmes such as Training for Work, particularly within Scottish
      Enterprise. Indeed, this type of activity also occurs under different Measures in the
      other Programmes.

      Suggested activities contributing to successful integration into the labour market were
      identified as:

              engagement of employers in the development and implementation of the
               project;

              work placement activity as an integral component of the training; and

              ensuring a support network within the workplace for those moving into
               employment.



      Case Study Examples: Successful Integration into the Labour Market

      The Talking Tayside project at Dundee College was devised in consultation with
      local call centre employers, careers advisors and public and private sector
      employment agencies. The private sector’s high profile support for the project, with
      workplace open days, employment information packs, and most importantly
      guaranteed interviews, has provided a real incentive for beneficiaries to engage with
      and complete the training. It should be noted, however, that this level of support
      required a sustained period of planning and networking prior to submission of the
      application. Furthermore, while many employers and supporting organisations are
      willing to provide informal assistance it can be difficult to achieve formal agreements
      that can be written into applications.

      SAMH encourages both work experience within the project itself, and work placement
      and other forms of engagement with the “outside world” as much as possible. During
      the two-weekly interviews, beneficiaries are reminded of the ultimate goal to move on
      from the project and re-engage fully with society. Within the project itself, more able
      beneficiaries are given the opportunity to work as volunteers in providing, e.g. IT
      training to the wider group, thus building confidence.

EKOS Limited                                                                                49
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

      For the TAG Highland project, mentoring and aftercare are viewed as future priorities
      to bring people closer to the labour market. The use of named mentors in the
      workplace is seen as a natural support network to enable people with mental health
      problems to integrate back into a real-life working environment. This support network
      should also include the employer to eradicate the stigma and misunderstanding of
      mental health issues.



      Barriers remain, however, in reaching labour market outcomes, particularly:

              geographic – where there is not the capacity in rural areas to develop
               sufficient projects and to overcome the lack of employment opportunities for
               those labour-market ready (Objective 3 Measure 2.3 also addresses this);

              thematic – where those with severe and sometimes multiple disadvantage
               require long-term, intensive and ongoing support to secure job outcomes and
               maintain these positions;

              childcare – where the availability of childcare places (e.g. within Highlands
               and Islands) and the operation of the Childcare Tax Credit system pose
               particular barriers;

              benefits – where many adults have money and debt issues which make it
               difficult to make the transition from benefits to earning. A money advice
               element as integral to support was suggested;

              job placements – where organisations can struggle to find suitable work
               placements in certain sectors; and

              health – where for example mental health issues can pose a barrier to entry
               to the labour market, given the stigma attached to such problems.

      It should be noted that addressing all of these barriers is not necessarily within the
      scope or remit of the ESF Programmes to address.

3.6 IMPACT AT PROJECT LEVEL
      The effect of ESF Measures on projects has been described as:

              increasing scope for innovation;

              upscaling of providers;

              enabling a more client centred approach;

              resourcing more robust management;

              providing flexibility to alter and extend provision; and

              fostering partnership working to provide a more holistic service to clients.

      ESF project performance is reported in terms of beneficiaries that have completed
      their course and who progress onto further education or training or employment.
      These ‘hard’ performance measures are crucial to demonstrate how projects are
      making a difference, but this is only part of the picture.


EKOS Limited                                                                                  50
               Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      Although some candidates leave without completing their course, there are still many
      intangible achievements such as building self-confidence that are realised but not
      recorded. Many of the softer indicators, such the building self confidence, self esteem
      and work stamina are difficult to record other than in terms of a case study, but are
      often just as, if not more, important as an outcome achieved by project participants.

      A number of consultees raised issues with Programme reporting requirements, in that
      the outcomes are reported directly on completion of the projects, whereas job
      outcomes are likely to be significantly higher once a period for job search has been
      allowed for, e.g. at six months. The revised MORI Beneficiary Survey is due later in
      2005, which will provide further information on outcomes and impacts.

      One interviewee felt that the Programme should be more focused on reviewing
      project performance before continuing funding and ensuring that projects consult with
      stakeholders to keep improving project delivery.

3.7 GEOGRAPHICAL ISSUES
      HIGHLANDS AND ISLANDS SPECIAL TRANSITIONAL PROGRAMME

      ESF funding has given the PME the freedom to target vulnerable target groups more
      proactively. This was not possible for the PME at the beginning as it is the smallest in
      Scotland but covers the widest geographical area.

      With the nine TAG centres located across the Highlands and Islands, impact is being
      demonstrated in more remote areas by providing individuals with the same access to
      services that are offered in an urban context.

      However, the ongoing issue is that job opportunities for trained individuals are still
      more limited in a rural environment than the variety of openings available in urban
      locations such as Inverness. The cultural environment was also raised as hindering
      female participation in certain areas of the labour market.

      In particular, the strategic partners have highlighted that it can often be more difficult
      to achieve an impact in a rural area. This is an issue to be taken into consideration,
      with the level of performance set lower than would be expected for an urban setting.

      OBJECTIVE 3

      Clearly, there have been issues in the take up of funds in rural areas. This has been
      attributed to the lack of capacity to take projects forward, as well as difficulties in
      assembling viable groups in more sparsely populated areas.

      There has been some progress in the South of Scotland in developing links between
      Borders and Dumfries & Galloway institutions to cooperate on project applications,
      which can bring economies of scale. Some good work has also been highlighted in
      terms of cooperation between local authority and colleges in Dumfries and Galloway
      to test new approaches.

      A number of suggestions were made to improve take up in rural areas: bring together
      players in local areas to jointly develop projects; employ a rural facilitator to help
      organisations develop solutions to issues of delivery in rural areas; and establish a
      Rural Sub-Group in Programme Management with experience of rural development
      issues.




EKOS Limited                                                                                 51
               Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      OBJECTIVE 2 WEST

      Most consultees argued that ESF has been well targeted and responsive to the
      particular issues and scale of need faced by urban areas in the West of Scotland.

      The effectiveness of the Programme in meeting providers’ roles has been slightly
      hampered by the difficulties for area-based providers in reconciling eligible and non-
      eligible areas. In many cases boundaries are not co-terminous with regeneration
      priority areas.

      There was a call for Measures to be better integrated with local strategies to ensure
      synergies of effort. In particular, future support must be aligned with Community
      Planning.

3.8 HORIZONTAL THEMES
      It was generally accepted that horizontal themes have been a worthwhile element of
      the Programmes. Most consultees felt that horizontal themes have been effectively
      addressed and some of them are becoming self-evident. ICT, for example, is
      increasingly well addressed, and integrated into most learning environments.
      However, some consultees were more sceptical about the impact this had actually
      had on organisations, and felt there was still some room for improvement.

      Given the very nature of the projects, one of the main thrusts is that of equal
      opportunities with respect to age, geographical location, ability (mental and physical),
      gender and ethnicity. In the Highlands and Islands it was noted that many projects
      work closely with the Equality Forum which actively promotes social inclusion for all at
      all levels. The presence of a National Advisor on Equal Opportunities is also
      considered worthwhile in disseminating good practice and directly advising projects.

      In the Highlands and Islands, there are now more opportunities for women in the
      science and engineering field as a consequence of the decommissioning work at
      Dounreay. The ‘Jamie Oliver/Gordon Ramsay’ factor has also encourage more men
      to consider the hospitality industry. In addition, Highlands & Islands Enterprise has a
      role to assist the development of equalities in the region. However, the Western Isles
      based interviewee considered that there remains a strong cultural barrier that inhibits
      female participation in traditional male dominated industries, and indeed mothers from
      entering the labour market.

      Across Scotland, a number of projects are addressing equal opportunities objectives
      directly with employers – this may involve changing attitudes towards the employment
      of target groups such as disabled people, mental health suffers, ex-offenders, etc.,
      providing practical advice to employers on what support is available for workplace
      adjustments, or advice on how to help individuals integrate into the working
      environment. It was suggested that there is scope for further work in the field of
      disabilities.

      Some other concepts have been more difficult to engage with – the sustainable
      development objective was the theme most raised as being hard to define and to
      incorporate within projects. Further definition, and identification and dissemination of
      good practice would be desired.

      Good practice is evident however. For example, the Shirlie Project first considered
      itself to have limited linkage with an environmental definition, however, found more
      relevance with a broader concept of sustainable communities. This has meant taking
      an approach that, while focused on achieving employment outcomes, supports


EKOS Limited                                                                               52
               Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      individuals in a holistic way that enables them to make a positive contribution to life in
      the community.

      Views were mixed on the Lifelong Learning theme. Some consultees argued that
      further cooperation is required between institutions. Others noted much improved
      integration of Further Education into social inclusion projects, which has increased
      their reach into communities and helped them to build up a client base, moving from
      employability onto further education and training.

      The Objective 3 PME noted that a change in the application form has improved the
      integration of horizontal themes to make them appear more relevant to project
      development and assist with mainstreaming. Survey work is underway to find out how
      projects have been applying horizontal themes considerations in practice; this type of
      evidence will help to inform future programming. Generally, consultees involved in
      Objective 3 agreed that this integration into the body of applications was a positive
      move.

      One consultee suggested that there had been a missed opportunity in not requiring a
      mandatory element of training to include equal opportunities/sustainable
      development. However, projects can be constrained by the requirements of other
      institutions, e.g. the SVQ model provides limited capacity to adjust training content.

3.9 GOOD VALUE
      It is widely appreciated that the unit cost of assisting the social inclusion target group
      is likely to be much higher than for mainstream training programmes. It is also
      generally agreed that cost per job/beneficiary is not an ideal measurement of value
      for money in the context of social inclusion Measures. The wide range of needs, types
      of support, varying geographical circumstances affecting running costs and “distance
      to travel” means that project-to-project comparisons are meaningless.

      A number of organisations assess the distance travelled by beneficiaries in terms of
      their progression up the learning ladder, with regard to the attainment of specific skills
      sets as well as life skills such confidence building. A range of different techniques
      have been trialled and used by project sponsors, however, it has been argued that
      there is no “one size fits all” technique that could be applied across ESF activities to
      provide a comparable measure.

      Jobcentre Plus and Careers Scotland utilise the methodology of comparing the cost
      to society (i.e. in benefits, allowances, etc) of individuals not participating in project
      activities, against that of providing support and moving individuals off benefit and out
      of other public support services such as day care centres, and into work (and paying
      tax) or training. This analysis has been undertaken by the Shirlie Project for example
      to evidence value for money in relation to its supported employment scheme.



      Case Study: Shirlie Project Value For Money Assessment
      Client: 19 year old male, not attended school since 14, chaotic home life; interacts
      little with others; on Jobseekers Allowance £43.25 per week.
      Support: Job coach needed much time to get person engaged - 181 hours of support
      provided. Cost: £3,620
      Now: In full-time employment, earning £309 pw.
      Gains to tax payer per year: JSA: £2,249; Tax: £3,299 Total: £5,548



EKOS Limited                                                                                 53
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

      It was reported that no formal guidance is provided to Advisory Groups in relation to
      the assessment of value for money. A few interviewees called for more information on
      benchmarks and cost comparisons to be developed. However, it was recognised that
      the diversity within project types and client groups would necessitate the development
      of a range of indicators.

      Other interviewees argued that, given that ESF is valued for the added quality,
      innovation and flexibility it brings to training delivery, it may not be helpful to set
      stringent criteria or benchmarks for performance that may act as barriers against the
      very differentiators that ESF has over other funding sources.

      The HISTP ESF Advisory Group are now requesting that projects build in proper
      evaluations in order to demonstrate where the added value of ESF funds has been
      realised and to share learning with similar project across the Highlands and Islands.
      As a consequence of the first year of evaluation activity, projects themselves are
      becoming more focused on what needs to be done to add more value. Subsequently
      it is hoped that this method of sharing good practice will avoid unnecessary project
      duplication, such as course preparation.

3.10 FUTURE PROSPECTS
      A continued focus on social inclusion support within ESF is widely supported. Views
      on the future programming can be split into two key strands:

3.10.1   Strategic Direction
      A number of consultees called for more strategic targeting of ESF, greater integration
      with local strategies, and alignment with local contexts, for example using Community
      Planning structures. There is an opportunity also to align the next Programme with
      current domestic priorities towards moving individuals off incapacity benefit.

      Competitive bidding rounds have attracted a high volume of projects, however, a
      more strategic approach through collaborative working between delivery
      organisations may prove more effective in ensuring that target groups are reached,
      that the quality of delivery is maintained and that potential synergies between delivery
      organisations, and funding streams, are realised.

      A number of interviewees cited the EQUAL experience, whereby Development
      Partnerships have been formed as “umbrella organisations” composed of a range of
      partners from across key sectors to work on thematic priorities, as a potentially
      replicable model.

      It is hoped that the Scottish Executive Employability Framework will be valuable in
      providing direction on the priorities for support and action in the future.

      There were calls from consultees for ESF to focus on a number of key areas,
      including:

              clients further from the labour market, where 6 months in a mainstream
               programme is not adequate to access and sustain employment;

              providing a higher level of support, into work, sustaining employment and
               progressing;

              in rural areas, reaching out to remotely displaced target groups;

              continued focus on fostering innovation and creativity; and
EKOS Limited                                                                               54
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
              continued focus on aligning ERDF and ESF support towards agreed strategic
               priorities.

      The most widespread call among interviewees was for Programmes to be as flexible
      as possible. In this regard, a broader Programme structure with fewer Measures was
      favoured.

3.10.2    Operational Issues
      At an operational level there were calls for:

              improved organisational synergies – increased collaboration among
               employment support agencies, and avoidance of inter-agency rivalry,
               particularly with regard to credit for project outcomes;

              building capacity – proactive support for organisations to effectively target
               remote, fragile areas under the HISTP and other rural areas, and to generally
               promote and support widespread engagement in ESF;

              simplified application process – including and improved project continuation
               process;

              streamlined administrative processes – where possible bringing alignment
               with other funding sources;

              better accessibility – for example ring-fenced funding for small voluntary
               groups to enable the Programme to reach the most marginalised target
               groups;

              increased promotion – a general increase in Programme resources towards
               awareness-raising and publicity aimed at beneficiary recruitment; and

              contingency funding – the issue of delays between programming periods is of
               significant concern, particularly in terms of loss of project momentum and in
               relation to the ongoing viability of smaller organisations.




EKOS Limited                                                                             55
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

4. PROJECT SURVEY FINDINGS

4.1 INTRODUCTION
      A key component in assessing the effectiveness of ESF social inclusion activities was
      the project fieldwork carried out by the study team. In total, 34 telephone surveys
      were completed with project managers. A list of the projects included in the sample is
      detailed in Appendix 2.

      A range of issues were covered with project managers during the discussion,
      including:

              project fundamentals: including delivery partnerships, aims and objectives;

              level of project additionality;

              target groups: identification and recruitment;

              key issues and challenges;

              project progress and achievements to date;

              outcomes and impacts;

              added value as a result of ESF support;

              complementarity with wider strategies and ERDF; and

              future aspirations for project development and social inclusion oriented
               training support.

      The following presents a summary of the key findings from the project fieldwork stage
      of this study. In addition, six case studies covering each of the three Programmes
      and highlighting areas of best practice are included in Appendix 3 and referred to
      where appropriate throughout this Chapter.

4.2 SCOPE OF SOCIAL INCLUSION FOCUSED PROJECTS
      The ESF supported projects sample covered a range of activities and were delivered
      by a wide variety of organisations as illustrated below:

         -     38% delivered by local authorities;

         -     29% delivered by voluntary organisations (e.g. One Plus);

         -     12% delivered by Local Economic Development Companies (e.g. Drumchapel
               Opportunities and Govan Initiative)

         -     9% delivered by “other organisations” such as Enable Scotland;

         -     6% delivered by Further Education Establishments; and

         -     6% delivered by Scottish Enterprise National (e.g. Careers Scotland).




EKOS Limited                                                                                 56
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      Evidently, local authorities play a significant role in terms of the delivery of ESF social
      inclusion projects, however smaller organisations such as LEDCs and voluntary
      organisations were also well represented in the project sample.

      Analysis of each of the 34 projects in terms of their respective aims and objectives
      identified a number of common project priorities:

              skills development: increasing the skills levels of beneficiaries, targeting
               both specific employment sectors and general skills development e.g.
               effective CV writing, team working, vocational skills and interview skills;

         -     personal development - improving confidence, assertiveness and self
               esteem;

         -     progression with respect to employability distance travelled – assisting
               those furthest removed from the labour market and helping them make
               positive steps towards employment, training or further education;

         -     promoting life long learning – encouraging both individuals and wider
               communities to recognise the importance of life long learning;

         -     raising the profile of economic activity – encouraging the ethos of
               employment within communities and highlighting economic “activity” as
               opposed to economic “inactivity” as the social norm;

         -     engaging with socially excluded groups – conducting outreach work to
               engage with those individuals and households least likely to engage in training
               or employability development programmes;

         -     work experience or placements – providing vital “world of work” experience
               and potential financial remuneration to improve longer term employment
               success; and

         -     guidance and counselling – offering guidance and support on a range of
               issues including job search methods, national employability programme
               criteria, state benefit advice and available further education and training
               opportunities.

      Clearly the remit of the project sample was varied and tailored to the requirements of
      the ESF Measure under which it was supported as well as the beneficiary group. The
      wide range of project priorities illustrates that a number of approaches were adopted
      by projects to tackle socially excluded groups.

4.3 REACHING TARGET GROUPS
      Social exclusion is a complex concept, resulting from individuals being detached in
      some way from either the social or economic mainstream. It is an issue that can
      affect a wide number of groups in society and can occur at multiple levels. For this
      reason it is inherently difficult to develop a project tailored to meet just one socially
      excluded group.

      Each of the ESF projects examined as part of this study focused on a particular
      socially excluded group as shown in Table 4.1. There was however a high incidence
      of projects supporting numerous socially excluded groups due to the multifaceted
      nature of social inclusion.




EKOS Limited                                                                                  57
                 Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      An example of this over-lap is illustrated in Case Study 2, Appendix 3. The Full
      Employment Areas Initiative (FEA) targeted workless households within three SIP
      areas in Glasgow. Due to the nature of this target group, many beneficiaries fell into
      additional categories such as alcohol/drug abusers, lone parents, ex-offenders and
      ethnic minorities. Consequently, although the remit of the FEA Initiative was to
      address workless households and improve the employment rate within the targeted
      communities, the actual impact of the Initiative addressed wider social exclusion
      issues.

      TABLE 4.1: PRINCIPLE TARGET GROUPS (ESF PROJECT SAMPLE)
      Socially Excluded Group                                   No. Projects           %
      Disabled people                                                7                 21
      Literacy/Numeracy                                              5                 15
      SIP Residents                                                  4                 12
      Alcohol/Drugs                                                  2                  6
      Lone Parents                                                   2                  6
      No qualifications/Educationally Disadvantaged                  2                  6
      Workless Households                                            2                  6
      LTU                                                            2                  6
      Other - Young Unemployed                                       2                  6
      Homelessness                                                   1                  3
      Pre-school Leavers                                             1                  3
      Older Unemployed                                               1                  3
      Young People Leaving Care                                      1                  3
      Ex-offenders                                                   1                  3
      Ethnic Minorities                                              1                  3
      TOTAL                                                         34                 100

      Table 4.1 summarises the principal target groups of the ESF project sample,
      identifying that the most common groups were disabled people (21%), those with
      literacy/numeracy difficulties (15%) and SIP residents (12%). It is important to note
      that target groups such as “SIP Residents” encompass a wider range of socially
      excluded groups i.e. LTU, lone parents and ex-offenders. This table is therefore only
      an indication of the principal target groups addressed.

      Recruitment of beneficiaries within the project sample was pursued using a number of
      approaches:

            -   referrals - a number of projects were delivered in partnership with their local
                SIP, social work department or other local economic development agency and
                referrals from these groups accounted for much of project recruitment;

            -   good reputation - some projects had established a high quality reputation as
                a result of client achievements or overall project management, benefiting the
                reputation of the overall project;

            -   positive word of mouth - many projects benefited from being highly reputed
                in local communities;

            -   outreach work - specific projects recruited through speaking to individuals in
                their homes or communities or by issuing information leaflets in areas where
                target groups were likely to visit. i.e. community centres, village shops; and

            -   Jobcentre Plus - frequently stated by the project sample as being a key
                referral organisation.


EKOS Limited                                                                                 58
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      The recruitment challenges experienced by the project sample tended to focus on
      enticing those on state benefits to become involved and also sourcing suitable
      candidates that met project criteria. Many individuals identified as potential project
      beneficiaries were reluctant to participate in projects because they were concerned
      about losing benefits and consequently reducing their disposable income. This
      particular challenge was addressed by providing advice and helping individuals to
      calculate the gains and losses of training and employment. One project offered a
      completion bonus in response to this specific problem.

      The stigma felt by individuals in admitting to literacy and numeracy problems was
      identified by some projects as a particular recruitment challenge. One project tried to
      address this problem through actively raising awareness among employers and
      referral organisations of issues experienced by those with literacy and numeracy
      difficulties.

      A number of projects reported difficulties in attracting sufficient numbers of
      beneficiaries to their project due to funding requirements e.g. the need to specifically
      target unemployed 18 to 24 years olds from a SIP area. This difficulty was further
      cited as the reason for target numbers of project beneficiaries being underachieved.

      The Youthbuild Paisley project summarised in Case Study 1, Appendix 3 is a good
      illustration of how a project evolved to meet funding criteria difficulties. Youthbuild
      attracted an additional match funder to allow the project to target a wider group than
      was previously eligible (i.e. the 16-18 years group as well as the 18-24 years group
      was supported, due to Get Ready for Work funds being incorporated by the project).

4.4 ISSUES AND CHALLENGES
      The main challenges and issues experienced by the project sample with regards to
      implementing their projects was a further issue examined by the study team.
      Typically, challenges identified were related to overall project management:

         -     level of documentation - a common difficulty reported by a number of
               projects was the large amounts of paperwork associated with delivering
               projects, with respect to ESF monitoring and annual application submissions.
               One organisation attempted to overcome this by setting up an online recording
               system, which is now starting to prove effective. However many projects
               lacked sufficient resources to implement such formal systems;

         -     project staff recruitment - projects commented on the difficulty of recruiting
               suitable employees to deliver their project. Often staff were only required for a
               temporary period due to time bound project funding which discouraged
               suitable candidates;

         -     meeting employment outcomes – difficulties meeting ESF employment
               outcomes were reported by some project managers. Whilst the requirement of
               achieving employment outcomes for ESF funding was understood, actual
               project experience showed some client groups moved significantly forward yet
               did not achieve full employment status within the ESF monitoring period. This
               issue made it difficult to demonstrate the success of some projects and project
               managers believed it meant the real progress achieved with clients in terms of
               distance travelled and longer-term employment prospects was not accounted
               for; and




EKOS Limited                                                                                 59
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
         -     partnership working - the partnership structure of many projects was
               perceived as challenging by project managers as each partner tended to have
               varying degrees of ‘openness’ and different objectives and ways of operating.
               This caused difficulties in terms of moving projects forward and developing
               consensus. It also meant that a range of funding criteria had to be met which
               could dilute the focus of the project.

4.5 GEOGRAPHICAL IMPACT ON PROJECT PERFORMANCE
      The ESF project sample encompassed three Structural Fund Programmes, covering
      different regions of Scotland. This provided the study team with an opportunity to
      examine the specific issues faced by social inclusion orientated projects as a
      consequence of their geographical reference area.

      Analysis of the project sample identified that:

         -     56% of the sample targeted an urban project reference area;

         -     23% of the sample targeted a rural project reference area; and

         -     21% of the sample targeted a mixed rural/urban reference area.

      The geographical remit of a project was regarded as an issue that could both
      positively or negatively affect project performance. Project managers provided the
      following comments in relation to the extent the geographical context of their project
      affected project performance:

         -     very much (21%);

         -     a little (26%);

         -     neither/nor (29%);

         -     not Much (9%);

         -     not at all (15%);

      Evidently the geographical context of projects was perceived by many project
      managers (47%) to have some affect on project performance. This was certainly the
      case with each of the six rural projects supported under the Highlands and Islands
      Special Transitional Programme, five of which stated their project reference area
      affected project performance “very much”.

      The following outlines some examples of how the geographical context of projects
      impacted project performance:

      Positive Impact

              the urban setting of the Youthbuild Paisley (Case Study 1, Appendix 3) and
               FOCIS projects meant they could benefit from concentrated urban
               construction sector activity;

      Negative Impact

              the rural remit of some projects was considered to make barriers such as
               transport difficulties and childcare more of an issue contributing to social
               exclusion; i.e. the “Argyll Supported Employment Team” project was unable to
               roll out its activities to local islands due to transport difficulties for project staff;

EKOS Limited                                                                                         60
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
              the cost of outreach activities in rural localities is invariably higher and
               consequently limited, however usage of video conferencing had been used by
               some projects to tackle this problem; and

              recruiting suitable project staff was perceived to be more challenging in rural
               localities, i.e. the LEAD North Project had to take into account the residence
               of project staff applicants and that they held a driving licence to ensure the
               could travel to meet clients.

4.6 ADDRESSING HORIZONTAL THEMES
      The ESF project fieldwork further examined how Horizontal Themes were being
      addressed by social inclusion orientated projects. A summary of the Horizontal
      Themes for each specific Programme is given in Table 4.2.



      TABLE 4.2: HORIZONTAL THEMES
      Programme             Horizontal Themes
                            Sustainable Development, Equal Opportunities, Life Long
                            Learning, Support for local initiatives,
      Objective 3           Promoting the information society
                                   Sustainable Development, Equal Opportunities, Innovation,
      West of Scotland Objective 2 Information Society
      Highlands & Islands Special Equal Opportunities, Sustainable Development,
      Transitional Programme      Sources of Information



      The ESF project sample was highly successful in incorporating horizontal themes. A
      number of the key horizontal themes such as sustainable development and equal
      opportunities were intrinsically linked with the aims and objectives of the project
      sample. The wish by each of the projects to address social exclusion in some form
      and improve social cohesion and economic growth was felt to suitably address
      sustainable development priorities. Where applicable, green office policies were
      adopted e.g. with the recycling of paper and ink cartridges by project management
      and the use of video conferencing and the internet to reduce the need for private
      travel.

      As illustrated in Table 4.1, the project sample targeted a wide range of excluded
      groups e.g. disabled people, ethnic minorities and individuals with literacy/numeracy
      difficulties. By targeting some of the hardest to reach groups, the ESF sample
      ensured equal access to all groups in society.

4.7 DEMONSTRATING PROJECT ACHIEVEMENTS
      An important objective of the project survey stage was to establish any difficulties
      social inclusion orientated projects may have had in demonstrating their
      achievements. First, the ease at which projects could demonstrate that their activities
      linked directly to Programme Measure performance indicators or targets were
      examined. Project managers made the following comments in relation to how well
      activities linked with performance indicators:

              very well (17%);

              quite well (65%);


EKOS Limited                                                                                   61
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
              not very well (15%); and

              not at all (3%).

      Project activities were largely considered to fit Measure performance indicators
      across the project sample. Those projects that had experienced difficulties tended to
      be a result of inexperience of the project management organisation in handling ESF
      monitoring and/or due to the project target group – e.g. one project that focused
      specially on helping disabled beneficiaries into further employment or training within a
      rural location felt that the ability of its target group to demonstrate quantifiable
      progress in early years of support to be minimal.

      The need for greater emphasis on softer outcomes within Programme monitoring was
      cited by a number of project managers. They felt softer outcomes such as increased
      client confidence and improvement in beneficiary timekeeping should also be a
      priority of ESF monitoring.

      Views on the effectiveness of guidance and information provided on how to measure
      and calculate performance differed. Some consultees found the ESF documentation
      easy to understand and relatively simple to follow, mainly due to the fact that they
      already had considerable experience of the process and/or were supported by
      European Officers within their local authority. Others found that support was lacking
      when problems were encountered and the documentation was lengthy when making
      application submissions and documenting progress.

      Typically, project management staff that had some level of experience in delivering
      similar employability or social inclusion related programmes were involved in
      quantifying initial project targets. Many looked at similar project models and worked
      with local referral agencies to ensure their targets were both realistic and achievable.

4.8 OUTCOMES AND IMPACTS
      Quantitative analysis of how social inclusion orientated projects under each
      Programme performed against targets set is covered in Chapter 2. However, to
      capture a more qualitative view, project managers were asked how their individual
      project had performed against ESF targets set and any underlying issues affecting
      project outcomes and impacts. This provided us with a useful insight into the
      problems faced by social inclusion orientated projects specifically in meeting targets
      and common factors that affect project outcomes.

      In terms of how well the project sample performed against its targets, project
      managers reported the following:

              very well (32%);

              quite well (47%);

              not very well (18%); and

              not at all (3%).

      The above results demonstrate that the vast majority of projects believed they had
      performed very or quite well against ESF targets set. Reasoning for any short fall in
      reaching targets included:

              client health issues – those projects working with client groups that were not
               only far distanced from the labour market but also affected by personal health

EKOS Limited                                                                               62
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
               issues, e.g. disabled people and those with mental health problems. Often
               this group experienced periods of ill-health which affected their performance
               and in turn impacted on the outcomes of the project;

         -     personal problems – for some beneficiaries additional difficulties such as
               homelessness, alcohol/drug abuse, family childcare issues etc, negatively
               effected their performance within projects;

         -     barriers surrounding identification and communication with the target
               group – once again the stigma felt by beneficiaries of admitting to difficulties
               associated with numeracy and literacy often prevented projects from
               supporting those most in need;

              pilot projects – some of the projects were piloting new approaches in
               addressing social inclusion and were consequently less experienced in
               reaching outcomes;

              external influences – some external influences impacted project
               performance e.g. delays within the Criminal Record Office in vetting ex-
               offenders further delayed them entering employment;

              capacity building - those projects which relied heavily on engaging with the
               client group in their homes and communities required considerable time to
               build up trust with beneficiaries, lengthening the process in which a positive
               project target could be met; and

              referrals - lack of or inappropriate referrals affected the ability of some
               projects to target sufficient numbers of beneficiaries to meet targets, resulting
               in lower project outcomes.

      In addition to Programme monitoring requirements, a number of projects recorded
      softer outcomes, such as increased confidence, improved sense of achievement,
      increased morale, heightened self-esteem and improved decision making and
      problem solving skills. As previously stated, a number of project managers believed
      these softer outcomes helped to demonstrate less tangible but still highly significant
      project achievements.

      The difficulty in effectively measuring softer outcomes was also identified. Whilst
      project staff could witness real progress in terms of clients “distance travelled”
      through their day-to-day working with them, they found it difficult to effectively
      demonstrate these outcomes. Some projects used Rickter Scale type systems to
      assist in recording soft outcomes, however the time and cost implications of these
      formal systems was identified as a barrier. The need to demonstrate ESF and other
      funding partner targets was given priority by the project sample and therefore whilst
      many projects would have liked to record softer outcomes formally, limited resources
      meant it was not feasible.

4.9 KEY AREAS OF SUCCESS AND GOOD PRACTICE
      Almost half (16) of the ESF projects within the study sample had been independently
      evaluated at the time of our consultation stage. From both these evaluations and
      discussion with project managers, areas of good or best practice in terms of
      delivering social inclusion orientated support and achieving desirable outcomes have
      been identified and are discussed below:

         -     wider role of partnerships - the development of partnership approaches in
               the delivery of projects has resulted in improved synergies – i.e. local colleges

EKOS Limited                                                                                 63
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
               working in partnership with SIPs and other similar local economic
               development organisations, helps to ensure that further education
               establishments and the option to become involved in training and life long
               learning generally were seen as a more viable option for SIP residents;

         -     pro-active linkages with employers – the importance of engaging not only
               with the client group but also potential employers was recognised by some
               projects e.g. Youthbuild Paisley engaged directly with private sector housing
               contractors and housing associations to ensure sufficient levels of quality work
               placements;

         -     exchanging information – projects were aware of the value of exchanging
               information and building on the experience of other projects . One of the
               organisations leading a study sample project, established an ESF
               coordinators forum where numerous projects could exchange experiences
               and learn from each other, enhancing the quality of project delivery;

         -     provision of flexible, client centred support - beneficiaries within some
               projects were allocated a specific project officer responsible for their
               development. Specific training plans suited to the clients’ needs were drawn
               up, to offer an individual support package, and beneficiaries could discuss
               wider issues that might affect their performance such as state benefit queries,
               housing and general home life issues. Offering this level of tailored
               assistance was seen as crucial to assisting those groups furthest from the
               labour market and was a major factor that distinguished ESF projects from
               existing employability support projects. In one case, clients could chose from
               a range of support services offered in various occupational areas. This high
               level of support and flexibility ensured that clients needs could be addressed
               on an ongoing basis, leading to improved beneficiary retention and project
               outcomes; and

         -     celebrating success – this contributed to a heightened sense of achievement
               by beneficiaries. In one particular project, clients were invited to the local
               radio station to discuss their success and experiences; this event both
               promoted the project and assisted in raising the confidence of the participants.

4.10 ADDED VALUE
      A key objective of the project survey stage was to gauge the added value of ESF
      support. This section of Chapter 4 will specially address this issue and illustrate the
      overall impact ESF funding had on the project sample.

      Looking firstly at the level of project additionality as a consequence of ESF, the
      sample of 34 ESF projects were developed as follows:



              27% of projects were developed specifically as a response to the one of the
               Structural Fund Programmes;

              35% of projects were an adaptation of an existing project idea; and

              38% of projects were a continuation/expansion of an existing project.



      With respect to the level of additionality of the project sample:

EKOS Limited                                                                                64
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
              71% of projects would not have gone ahead without ESF;

              26% of projects would have been significantly different in scale and quality
               than any alternative planned or would have been delayed by up to 24 months
               in the absence of ESF;

              one project (3%) of similar scale and quality would have proceeded but would
               have been delayed by up to 24 months in the absence of ESF; and

              none of the projects of a similar scale or quality would have been undertaken
               in the same timescale without ESF.

      The above figures demonstrate the high level of project additonality as a
      consequence of ESF support. Each of the projects was additional at some level with
      the majority (71%) wholly additional to ESF. Given that approximately a third (27%)
      of the project sample were developed specially as a response to one of the three
      Programmes, it is evident that ESF support has led to the development of social
      inclusion orientated support.

      ESF funding also had a number of additional impacts on the project sample:

              allowed projects to provide additional support to clients i.e. assistance with
               transport costs, childcare etc. This support was vital in assisting client groups
               facing multiple barriers to employment;

              regular ESF monitoring ensured that project progress was monitored and
               operationally the management of projects was more formalised. This
               monitoring requirement also encouraged regular beneficiary surveys and
               feedback sessions and consequently any issues identified by clients could be
               addressed effectively;

              some projects used ESF to provide a “waged period” to clients on longer-term
               work placements, this provided a further incentive to beneficiaries to
               participate and complete projects;

              a particularly valuable outcome of the ESF funds was that project managers
               were able to recruit skilled staff, they could allocate a dedicated worker to
               particular projects and this increased the quality of delivery;

              ESF funding further enabled the scale of many projects to increase,
               especially with regards to projects addressing a rural location – funding was
               used to provide additional local services;

         -     marketing of projects – in many cases funding to cover aspects such as
               advertising and marketing of projects to reach key target groups would not
               have been available, ESF funding assisted with this aspect;

         -     attracting additional match funding and project testing – securing ESF funding
               was perceived by some project managers as raising the profile of a project
               and helping to source match funding and delivery partners. In one specific
               case, ESF funding enabled a pilot project to be implemented and later rolled
               out as a mainstream Careers Scotland programme.

4.11 COMPLEMENTARITY
      The project sample indicated a high level of complementarity with wider national and
      regional strategies. A fundamental reason for this close fit was the fact that each of

EKOS Limited                                                                                 65
               Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      the projects was addressing one, if not multiple social inclusion issues. Tackling
      social exclusion is currently a key priority across Scotland, with numerous recent
      policy developments such as “Closing the Opportunity Gap” and the advent of
      Community Planning Partnerships.

      Given the high level of partnership delivery, complementarity with organisations and
      their wider polices such as Jobcentre Plus, local economic development companies,
      social work and health departments and further education establishments was
      particularly high. At present the importance of social inclusion is recognised by many
      economic, social and community development organisations, therefore there is a real
      opportunity for ESF projects to meet not only European but also local and national
      agendas.

      Examples of national programmes which ESF projects were able to complement
      include:

              New Deal – UK Government Programme;

              Get Ready for Work – Scottish Enterprise Programme; and

              Training for Work – Scottish Enterprise Programme.

      Each of the programmes above has been developed to address unemployment and
      have been incorporated by the ESF project sample. ‘New Deal’ tackles a wide
      category of unemployed groups over 18 years of age, whilst ‘Training for Work’
      assists adults over 25 years only and ‘Get Ready for Work’ targets 16 to 18 year olds.
      Each of the programmes offer guidance and assist in offering training or a work
      experience placement to clients, however a number of the ESF projects have
      supplemented these programmes and offered additional support. An example of this
      is the Youthbuild Paisley project, Case Study 1, Appendix 3 which incorporated both
      New Deal and Get Ready for work programmes whilst offering additional financial and
      personal support to young unemployed SIP residents.

      In relation to complementarity with ERDF projects, half the ESF sample stated that
      their project complemented an ERDF project, whilst 9% were unsure and the
      remaining 41% of projects were not considered to be complementary. The project
      sample reported only one common linkage to ERDF supported projects, in that ESF
      projects benefited from usage of buildings or IT suite based ERDF projects often
      based in local further education establishments or community centres. Consequently
      ERDF projects assisted with the provision of IT training elements of a number of ESF
      projects or indeed facilitated local provision.

4.12 FUTURE PROSPECTS
      The future outlook for the ESF project sample was varied. A number of projects
      anticipated carrying on in future years, addressing the same client group and tackling
      the same issues, whilst others were complete and would use their project experience
      to date to incorporate it into the effective development and delivery of new projects.

      Ever tighter funding budgets and increasing competition for available project funding
      were reported as having a severe impact on the longer-term future projects.
      Uncertainty over ESF eligibility post 2006 was a further cause for concern by a
      number of project managers, an issue that is unsurprising given the high number of
      projects wholly additional as a result of ESF support.

      A number of projects stated important lessons had been learned and these would be
      fed into new projects. For example, some projects achieved a better understanding

EKOS Limited                                                                             66
               Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      of where the real support needs existed as a result of engagement with target groups,
      and future project activity would consequently be focused on complementing existing
      provision and providing added value for those in most need.

4.13 CONCLUSION
      The project survey stage of the study has provided a useful insight into the remit,
      management, delivery, and outcomes of ESF social inclusion orientated projects.
      The information attained by project mangers helps to contextualise the issues raised
      in other chapters of this report and greatly assists in demonstrating the added value
      ESF support as brought to the project sample.

      Generally, the 34 projects examined were successful in achieving their objectives in
      terms of ESF targets, but importantly they were making progress in addressing the
      wider issue of social inclusion. Common issues in terms of recruitment difficulties and
      measurement of softer outcomes were identified, as well as examples of good
      practice in addressing such challenges. The six case studies within Appendix 3
      further help to demonstrate a range of the activities incorporated by the project
      sample and provide an insight into current practice for addressing social exclusion.




EKOS Limited                                                                              67
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

5. CONCLUSIONS AND RECOMMENDATIONS
      This Chapter presents the overall             conclusions    of   the   study,   including
      recommendations and lessons learnt.

5.1 MAKE-UP OF CLIENT GROUP
      The beneficiary data shows that the social inclusion Measures have predominately
      supported the long-term unemployed (3 years+) in Scotland (33%). The over 25 (7 to
      12 months) group has received the lowest proportion of support in Scotland (5%).

      In terms of target groups, it is notable that HISTP is strongly associated with disabled
      projects with 70% of this group among the beneficiaries. Disabled people also formed
      the largest share of beneficiaries in the Objective 3 Programme (30%) with people
      with literacy/numeracy issues also receiving a large share of support. The Objective 2
      West of Scotland Programme on the other hand was concentrated on people with no
      qualifications and workless households.

      It is recommended that clearer definitions be provided on target groups such as
      literacy/numeracy. This will provide an evidence base, which can be used to ensure
      comparability between projects, and ensure that those most in need are being
      prioritised during selection processes.

      The study attempted to look at the reach of the Measures in terms of populations of
      individuals within the target group categories as a proportion of total populations in
      Scotland registered in these groups. This exercise was difficult to accomplish, given
      that a) target group populations are likely to be underestimated by Programme
      monitoring systems, as projects only select one defining target group characteristic
      per individual; b) it was not clear precisely how projects had defined groups, e.g.
      unemployed over 3 years may have included a high proportion of economically
      inactive as well as claimant unemployed beneficiaries; and linked to this, c) it was not
      easy to find statistical sources against which to reliably compare beneficiary data.

      Recommendation: target group categorisations be linked to standard statistical
      sources in order to enable quantification of the number of individuals within the
      reference population that are being reached by the Programme.

5.2 ADDED VALUE OF ESF
      Our project survey found that almost three-quarters of projects would not have gone
      ahead without ESF funding, with the remainder reporting increased scale or quality.
      Added value is achieved in a range of ways. Our survey identified that ESF:

              allowed projects to provide additional support to clients e.g. assistance with
               transport costs, childcare etc. This support was vital in assisting client groups
               facing multiple barriers to employment;

              contributed to the development of good project management practice.
               Regular ESF monitoring ensured that project progress was regularly
               monitored and operationally the management of projects was more
               formalised. This monitoring requirement also encouraged regular beneficiary
               surveys and feedback sessions and consequently any issues identified by
               clients could be addressed effectively;

              gave the projects scope to incentivise beneficiaries. Some projects used ESF
               to provide a “waged period” to clients on longer-term work placements, thus

EKOS Limited                                                                                 68
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
               providing a further incentive to beneficiaries to participate and complete
               projects;

              enabled organisations to augment their resources. A particularly valuable
               outcome of the ESF funds raised in the survey was that project managers
               were able to recruit skilled staff, and allocate a dedicated worker to the
               projects, thus increasing the quality of delivery;

              enabled the scale of many projects to increase, especially with regards to
               projects delivered in a rural location – funding was used to provide additional
               local services;

         -     provided resources for marketing of projects – in many cases funding to cover
               aspects such as advertising and marketing of projects to reach key target
               groups would not have been available; and

              assisted with attracting additional match funding– securing ESF funding was
               perceived by some project managers as raising the profile of a project and
               helping to source match funding and delivery partners.

      Projects typically combine funding from a number of sources, and this has
      implications for the administrative resource burden. A more standardised approach to
      project management requirements would be desirable, although it is recognised that
      this will be difficult to achieve given the range of funding sources utilised.

      Recommendation: it is recommended that project administration requirements
      are looked at again in relation to key match-funding programmes, seeking to
      streamline and complement where possible.

5.3 COMPLEMENTARITY
      During strategic interviews, linkages between ERDF and ESF were noted at project or
      organisational level primarily by the FE sector and local development companies,
      perhaps achieved at local discretion or due to the multifunctional nature of
      organisational remits rather than by Programme design.

      There was a call for a more strategic and coordinated approach to ERDF and ESF at
      Programme level. The use of Community Planning structures was highlighted as an
      appropriate mechanism through which to ensure strategic co-ordination on an area-
      wide basis.

      The project sample reported only one common linkage to ERDF supported projects,
      in that ESF projects benefited from usage of ERDF supported accommodation or IT
      suites, often in further education establishments or community centres.
      Consequently, ERDF projects assisted with the provision of IT training elements of a
      number of ESF projects or indeed facilitated local provision.

      Recommendation: there continues to be a requirement for improved strategic
      coordination of ERDF and ESF funding streams. It may be appropriate to use
      structures such as Community Planning to facilitate this discussion at a local
      level.




EKOS Limited                                                                               69
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures

5.4 ACHIEVEMENT OF POSITIVE OUTCOMES
      The project survey found that most managers were comfortable with the range of
      indicators provided, and were able to monitor progress towards targets without
      difficulty.

      Around one-half of projects had performed “quite well” against targets and one-third
      “very well”, according to project managers. The main difficulty was experienced in
      achieving employment targets. Whilst the requirement of achieving employment
      outcomes for ESF funding was understood, actual project experience showed some
      client groups moved significantly forward yet did not achieve full employment status
      within the ESF monitoring period. This issue made it difficult to demonstrate the
      success of some projects and project managers believed it meant the real progress
      achieved with clients in terms of distance travelled and longer-term employment
      probabilities was not accounted for.

      The Leavers Survey was intended to capture this information, however, the revised
      MORI Beneficiaries Survey will be published later in 2005. It will provide further useful
      information on outcomes and impacts, which unfortunately could not be drawn on for
      this report.

      Recommendation: Longer-term tracking of individuals would be desirable in
      order to provide a more realistic time frame for job seekers to achieve results,
      as well as to demonstrate sustainability of outcomes.

      A range of good practice was identified that contributed to positive outcomes,
      including:

              development of partnerships to ensure holistic multi-faceted support to
               beneficiaries, and to facilitate beneficiary transition into “the next stage”,
               whether further training, education or employment;

              networking with other projects to exchange experience and help to improve
               quality of provision;

              engagement of employers in the development and implementation of the
               project;

              provision of flexible, client-centred support including one-to-one mentoring,
               phased progression plans with milestones, etc.;

              work placement activity as an integral component of the training; and

              ensuring a support network within the workplace for those moving into
               employment.

      Benchmarking analysis of other social inclusion oriented programmes typically
      backed up these findings.

      The monitoring of soft indicators was advocated by a number of interviewees, as
      being valuable in reaching a comprehensive overview of individual achievements.
      Some organisations have successfully tested and applied models, however, others
      have struggled or have not worked up measurement systems.

      The Leavers Study did try to capture softer outcome information, however, again, its
      discontinuation leaves a gap in the data set available for Programme evaluation.


EKOS Limited                                                                                70
                Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      Our fieldwork also identified continuing barriers experienced by beneficiaries in
      reaching labour market outcomes, particularly:

              geographic – where there is not the capacity in rural areas to develop
               sufficient projects and to overcome the lack of employment opportunities for
               that labour market ready. In our project survey, the geographical context of
               projects was perceived by the majority of project managers (46%) to have
               some affect on project performance. This was certainly the case with each of
               the six rural projects supported under the Highlands and Islands Special
               Transitional Programme, five of which stated their project reference area
               affected project performance “very much”;

              multiple disadvantage – those with severe and sometimes multiple
               disadvantage require long-term, intensive and ongoing support to secure job
               outcomes and maintain these positions. We found that projects which
               engaged with the client group in their homes and communities required
               considerable time to build up trust with beneficiaries, lengthening the process
               in which a positive project target could be met;

              childcare – where the availability of childcare places (e.g. within Highlands
               and Islands) and the operation of the Childcare Tax Credit system pose
               particular barriers;

              benefits – where many adults have money and debt issues which make it
               difficult to make the transition from benefits to earning. A money advice
               element as integral to support was suggested;

              job placements – where organisations can struggle to find suitable work
               placements in certain sectors; and

              health – where for example mental health issues can pose a barrier to entry
               to the labour market, given the stigma attached to such problems.

      It should be noted that addressing all of these barriers is not necessarily within the
      scope or remit of the ESF Programmes to address.

5.5 IMPACT ANALYSIS
      Providing a net impact for the study was somewhat problematic in that no beneficiary
      interviews have been undertaken and no leavers survey data is available. Therefore
      in order to estimate net impacts it has been necessary to apply a range of
      assumptions as agreed with the study steering group. Furthermore, as noted,
      employment outcomes are likely to be underestimated given the short monitoring
      period.

      Net additional employment generated as a result of the ESF social inclusion
      Measures is estimated at a total of 3,147 at the local level and 4,054 at the Scotland
      wide level.

      Within each Local Authority area, net additional employment generated was varied in
      scale. Unsurprisingly, Glasgow City accounted for the largest share with 35% of all
      jobs, whereas the remaining 28 Local Authorities individually accounted for less than
      10% of the total net employment.

      With a lack of detailed information on beneficiaries it was not possible to take this
      analysis further to determine the precise effects on unemployment and benefit
      claimant rates. However, individual projects, which have analysed the cost of

EKOS Limited                                                                               71
               Scottish Executive: Review of the Effectiveness of Social Inclusion Measures
      supporting individuals versus their claimant status prior to support, have evidenced a
      substantial net gain to the exchequer.

5.6 GOOD VALUE
      It is widely appreciated that the unit cost of assisting the social inclusion target group
      is likely to be much higher than for mainstream training programmes. It is also
      generally agreed that cost per job/beneficiary is not an ideal measurement of value
      for money in the context of social inclusion Measures. The wide range of needs, types
      of support, varying geographical circumstances affecting running costs and “distance
      to travel” means that project-to-project comparisons are meaningless.

      That said, attempts were made to benchmark performance; three methods were
      used: against the Programmes’ own targets; against other ESF Programmes; and
      against other social inclusion interventions. The first benchmark showed that
      Highlands and Islands had performed much better than the Programme anticipated
      across a range of key output and result indicators. The Objective 3 Programme is
      performing very well against its Operational Programme targets. Output unit costs
      were in line with expectations, however, the achievement of results was higher than
      anticipated. The Objective 2 West Measure is forecast to perform well against the
      majority of its targets, and actual performance is in line with funds committed. The
      unit costs for employment outcomes are only slightly higher than anticipated, while
      the cost per education/training outcome is less than anticipated .

      Our comparison of the three Objective 3 Programmes in the UK found that Scotland
      general performs better than Wales but that England achieves greater impacts than
      both Wales and Scotland by some considerable margin. We discussed various
      reasons why this might be the case.

      We also undertook a benchmarking study of five other social inclusion-oriented
      programmes in Scotland and the UK, using evaluation materials. While some useful
      insight could be gained from the analysis in terms of good practice and lessons, cost
      comparisons were made difficult by lack of information or differences in the types of
      activities funded. A comparative analysis was undertaken between Objective 3 and
      the New Futures Fund, which looked at performance against certain common target
      groups; each programme was found to have a set of strengths in outcomes achieved.
      In terms of costs per beneficiaries into employment, education or training, both
      Programmes were broadly similar with the exception of Lone Parents, where ESF
      was significantly more cost-effective.

      Recommendation: Further examination of the English experience with their
      social inclusion Measure may be valuable, in understanding why the data
      would suggest a significantly better cost-effectiveness achievement than
      Scotland.




EKOS Limited                                                                                 72

								
To top