Document Sample
740 Powered By Docstoc
					ELWa – Evaluation of Learning
              Challenge Fund
         First Round Evaluation Report

                         SHARED INTELLIGENCE

                              1 Fitzroy Square
                                      W1T 5HE

               SIGNED OFF 22 September 2005

1    Introduction ................................................................................ 1
2    Canolfan Sgiliaith ........................................................................ 5
3    Carmarthenshire Basic Skills Strategy ......................................... 9
4    Cyfrwng (Welsh For Adults) CYMAD ......................................... 17
5    The Learning Vale ...................................................................... 19
6    Methyr Tydfil Lifelong Learning Campus .................................... 22
7    National Strategy for the Tourism Training Forum for Wales ..... 26
8    The Oaks Project ....................................................................... 30
9    Powys Basic Skills ..................................................................... 33
10 Swansea College City Centre Feasibility Study .......................... 36
11 Training for the Social Economy ................................................ 38
12 Vale Learning Network .............................................................. 43
13 Video Conferencing Project - CYDAG ......................................... 50
14 Non-contracted projects ............................................................ 52
15 Completed projects ................................................................... 54
16 Scheme progress ....................................................................... 56
17 Conclusions ............................................................................... 65
                                                    Learning Challenge Fund Evaluation
                                                          First Round Evaluation Report


      The Learning Challenge Fund evaluation

1.1   This report outlines findings from the first round of evaluation work
      that Shared Intelligence has carried out of Learning Challenge Fund
      projects, on behalf of ELWa.

1.2   Shared Intelligence (Si) was employed by ELWa in 2003 to conduct an
      ongoing evaluation of the Learning Challenge Fund (LCF). The first
      phase of the evaluation involved a review of the process of developing
      and setting up the LCF, and the production of a monitoring and
      evaluation guide for external project managers to use. Si is now
      carrying out mini-evaluations of each of the funded projects, and using
      this data both to write reports on the individual projects and to
      produce an assessment of the progress of the scheme in meeting its

1.3   The key objectives and success criteria for the LCF were to:

         engender innovation and new approaches in learning provision;
         promote a strategic approach at national, regional and local
         encourage collaboration between learning providers leading to
          improved learning networks;
         improve access and widen participation;
         deliver more and high-quality learning outcomes from existing
          resources; and
         effectively mainstream successful projects.

1.4   In total, 19 projects were approved for LCF funding, although one did
      not eventually proceed. Most of these projects have now contracted
      with ELWa and some are already complete. The LCF projects and their
      current status at end-July 2005 are as follows:

 1                            Shared Intelligence                      September 2005
                                                       Learning Challenge Fund Evaluation
                                                             First Round Evaluation Report

           Name                                 Proposer                   Status
      1    Cyfrwng – Welsh for Adults           Cymad                      Continuing
      2    Wales Institute for Sustainable      CAT                        Being
           Education                                                       contracted
      3    Canolfan Sgiliaith                   Coleg Meirion Dwyfor       Completed
      4    Learning Vale                        Deudraeth Cyf              Continuing
      5    Denbighshire Post-16 Learning        Denbighshire CC            Being
           Centres                                                         contracted
      6    Powys Basic Skills                   Powys CCET                 Continuing
      7    Competency Cube for                  WEF                        Not proceeding
           Electronics Sector
      8    Trainlink Skills and Survey          Trac                       Completed &
      9    Enable South West Wales              Neath Port Talbot          Completed &
                                                College                    Evaluated
      10   Carmarthenshire Basic Skills         Carmarthenshire CC         Continuing
      11   Oaks Project                         Blaenau Gwent BC           Continuing
      12   Merthyr Tydfil Lifelong Learning     Merthyr Tydfil BC          Continuing
      13   National Strategy for Tourism        TTFW                       Continuing
      14   Achieving Closer Integration         UWCN/Coleg Gwent           Completed &
      15   Vale Learning Network                Vale of Glamorgan          Continuing
      16   Training for the Social Economy      Wales Co-op Centre         Continuing
      17   Accreditation Pilot                  Prince‟s Trust             Completed &
      18   City Centre Location Feasibility     Swansea College            Completed
      19   School Videoconferencing             CYDAG                      Completed

      What does this report cover?

1.5   Si drafted a baseline evaluation report in January 2005, outlining
      findings from initial visits to projects and consultation with contract
      managers at ELWa. Since then, mini-evaluation reports have been
      written on four completed LCF projects, and evaluation work has
      continued on the remaining projects. Each of the projects is being
      evaluated separately and Shared Intelligence have developed
      „Integrated Evaluation Plans‟ that have been agreed with the project
      managers. These plans cover the evaluation activity that projects will
      be carrying out as part of their contract, and the evaluation activities
      that Shared Intelligence will undertake to complement these.

 2                               Shared Intelligence                      September 2005
                                                    Learning Challenge Fund Evaluation
                                                          First Round Evaluation Report

1.6   This „first round evaluation‟ report sets out findings from evaluation
      work carried out between January and July 2005. The evaluation is by
      Si but also draws and builds on the projects‟ own evaluation findings,
      where these have been available. The report is largely concerned with
      projects‟ progress to date. Because each evaluation includes slightly
      different activities, and because projects are at different stages of
      development, the information in the progress assessments below
      varies from project to project. In all cases we have included a
      description of activities carried out to date, and an appraisal of
      achievement of targets and milestones. In addition, we have drawn in
      outcome and impact data from beneficiaries, stakeholders, project
      managers and contract managers where available.

1.7   The report also looks at scheme-wide issues emerging from the
      evaluation so far. This section is centred around aspects such as:
      scheme management; monitoring and evaluation; achievement of LCF
      objectives; and mainstreaming and sustainability.

1.8   The 12 live or recently completed projects are covered first in the
      progress sections in alphabetical order:

         Canolfan Sgiliaith – Coleg Meirion Dwyfor
         Carmarthenshire Basic Skills Strategy - Carmarthenshire
          County Council
         Cyfrwng Iaith Gwaith (Welsh for Adults) - Cymad
         The Learning Vale - Deudraeth Cyf
         Merthyr Tydfil Lifelong Learning Centre – Merthyr Tydfil CBC
         National Strategy – Tourism Training Forum for Wales
         The Oaks Project - Blaenau Gwent Education Department
         Powys Basic Skills – Powys CCET
         Swansea College City Centre Feasibility Study – Swansea
         Training for the Social Economy – Wales Co-operative Centre
         Vale Learning Network – Vale of Glamorgan Council
         Video Conferencing - Cydag

1.9   In addition, a brief outline is given of findings from the four projects
      for which mini evaluations are already complete. These are:
      Accreditation Pilot – Prince‟s Trust Cymru; Achieving Closer
      Integration – University of Wales Newport and Coleg Gwent; Enable
      South West Wales – Neath Port Talbot College; and TrainLink –

 3                            Shared Intelligence                      September 2005
                                                     Learning Challenge Fund Evaluation
                                                           First Round Evaluation Report

1.10   Finally, there are two projects that have not yet contracted with ELWa,
       so have not started their LCF activities. These are Denbighshire
       Post-16 Learning Centres and WISE. A brief description of these
       projects‟ aims is given in the section on „non-contracted projects‟.

 4                             Shared Intelligence                      September 2005
                                                     Learning Challenge Fund Evaluation
                                                           First Round Evaluation Report



2.1   The Canolfan Sgiliaith (Skills Centre) LCF project is now complete. The
      Centre was established in 2001 with the basic aim of supporting
      further education colleges in Wales to develop bilingual provision. Until
      2003, the Welsh Language Board funded Canolfan Sgiliaith directly,
      but later took the view that funding should come from elsewhere.
      From March 2003 until February 2005 it was funded by ELWa through
      LCF (£149,000 over two years) with an additional contribution of over
      £30,000 a year from Fforwm. The Centre answers to the Fforwm Board
      and is located in Coleg Meirion Dwyfor.

2.2   Historically, bilingual provision was offered by only two FE colleges in
      North Wales, while another four colleges offered limited provision. The
      aims of the project were therefore to encourage cooperation by means
      of training activities and innovative curriculum panels; and to
      strengthen the learning network by means of increasing the capacity
      of FE colleges to deliver bilingually.

2.3   Canolfan Sgiliaith employs one FTE co-ordinator and three part-time
      staff (one 60% fte administration and two 20% fte academic). The
      Steering Group includes representatives from ELWa, Fforwm, Coleg
      Meirion Dwyfor and CYDAG.

      Progress to date

2.4   The project is considered by the ELWa contract manager to have been
      well managed although it appears that it would have benefited from
      additional staff resources. The table below shows target and actual
      outputs for the two years in which Canolfan Sgiliaith received LCF

2.5   In terms of particular aspects of the scheme, the following progress
      was made:

         Resources Panels – these were established to identify key
          vocational subjects that would benefit from new Welsh-medium
          provision and to develop the materials and resources to make this
          possible. Over the two years Sgiliaith contributed to ELWa‟s
          commissioning and publishing programme by consulting with
          providers and other agencies to recommend bilingual or Welsh-
          language teaching and learning resources. This was carried out in
          dedicated Panels, including a Strategic Panel (high level) and
          various Subject Panels;

 5                             Shared Intelligence                      September 2005
                                                          Learning Challenge Fund Evaluation
                                                                First Round Evaluation Report

            Training and development with colleges, schools and
             providers – promotion of bilingualism in the curriculum included
             a range of activities such as visits to institutions, advice and
             support with implementation of Welsh Language Schemes and
             staff training events. Sgiliaith worked with CYDAG on related
             projects on Welsh medium computer training and on
             videoconferencing. Feedback sheets from 104 trainees found that
             94% considered the Sgiliaith training to be “good” or “very good”.
             The detailed analysis of these forms will be included in the mini
             evaluation of this project.

2.6   The LCF funding also allowed the Centre to get out and work with staff
      in colleges and private providers that were not previously responsive
      to offers of support in relation to bilingualism. This was true
      particularly in non-Welsh speaking areas. Sgiliaith has held events in
      all colleges in Wales.

2.7   The draft internal evaluation includes output related data. The project
      achieved and surpassed all its operational targets set by ELWa and
      incorporated in their contract. The table below shows target and actual
      outputs for the two years in which Canolfan Sgiliaith received LCF
      funding. Some items (marked NA) did not have targets set by ELWa.

                     Canolfan Sgiliaith: Target and actual outputs, Years 1 and 2
                                              YEAR 1                   YEAR 2
                                              Target       Actual      Target       Actual
          Resources panels –
          Participating advisers              50           60          50           107
          Development work with colleges
          Days’ work                          50 days      80 days     50 days      55 days

          No of Colleges                      12           18          12           23

          Staff involved                      NA           95          NA           115
          Development work with NVQs
          Days’ research work                 6 days       6 days      9 days       23 days

          Organisations involved              6            9           8            20

          Staff involved                      NA           10          NA           50
          Individuals trained                 NA           0           75           318

 6                                  Shared Intelligence                      September 2005
                                                     Learning Challenge Fund Evaluation
                                                           First Round Evaluation Report

       Project outcomes and fit with LCF scheme objectives

2.8    Sgiliaith has prepared a draft internal evaluation report and Shared
       Intelligence has held discussions about this with the ELWa Contract
       Manager. It is understood that ELWa officers have supported the
       preparation of a bid for continuation funding and Shared Intelligence
       contributed to this process with a brief submission.

2.9    Our interim evaluation findings, based on discussions with the ELWa
       contract manager and the project manager, regarding the success of
       Sgiliaith are as follows:

          the Resources Panels were innovative and the idea is seen as
           successful, but in future need to be more sophisticated and
           regular to maintain momentum and influence. Remit, membership
           and the process for selecting curriculum areas should be specified
           in more detail;
          more regional Subject Panel meetings would help as would linking
           panels to other events to maximise attendance; and
          the resulting production of commissioned materials has been
           impressive but ELWa could help to speed the process up.
2.10   The project has been innovative in the ways in which it has taken
       Welsh medium provision to the FE sector and in increasing vocational
       provision in schools. Also the progress being made with private
       training providers is seen as ground-breaking. The Resources Panels
       have been innovative as practitioners have never been involved in
       identifying needs and developing resources and materials before.

2.11   The project is considered to have made real strategic impacts. The
       profile of Welsh-medium and bilingual vocational learning is being
       given a higher priority in schools and colleges, but increased resources
       - and a greater practical involvement by Fforwm - would enable this to
       be extended further and to get to more private providers. The project
       has informed providers of ELWa and WAG strategies and initiatives and
       established stronger collaboration between Sgiliaith and CYDAG
       throughout Wales.

2.12   The ELWa Contract Manager stated that “Sgiliaith was a necessary
       vehicle to drive forward the bilingual agenda”. ACCAC has
       specialist curriculum officers for the school sector but ELWa does not.
       Therefore it was essential for ELWa to develop a partnership approach
       and identify a delivery arm for FE and vocational provision.

 7                             Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

       Mainstreaming and sustainability

2.13   Canolfan Sgiliaith is bidding for funding from ELWa for a further
       programme of influencing and development of learning resources. If
       successful, the project will have built on its pre-LCF activity and
       achieved sustainability after LCF. The long-term objective of
       mainstream provision of Welsh language opportunities vocational
       courses in schools and FE throughout Wales will be one step closer.

       Conclusions and learning

2.14   Si will carry out a mini-evaluation of Canolfan Sgiliaith in September
       2005. Our interim conclusions are that the project has been effective
       in achieving its targets and has made some important progress in
       preparing learning resources for vocational Welsh-medium in colleges
       and schools and in getting staff and institutions prepared for delivering
       such resources. It has also met LCF objectives in terms of innovation,
       strategic focus and, if further funding is made available, sustainability.

2.15   Should the project be awarded further funding, the ELWa contract
       manager believes that it must focus more strongly on making national
       impacts with FE colleges and private providers by developing its
       influencing capabilities.

2.16   One of the wider learning points for ELWa LCF scheme managers is
       how the setting and agreement of output targets can make the
       projects very numbers-driven and can lead to anxiety in project
       managers about not achieving their targets – sometimes at the
       expense of focusing on the more qualitative or outcome-based
       objectives. Good communication between ELWa contract managers
       and project managers can help establish the balance between output
       targets and more qualitative objectives.

 8                              Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report



3.1   The Carmarthenshire Basic Skills Strategy (CBSS) seeks to support,
      develop and co-ordinate the delivery of basic skills within
      Carmarthenshire. The CBSS vision is for people in Carmarthenshire
      with low basic skills “to gain, or regain, the will to learn, improve their
      basic skills and increase their confidence and self esteem to enable
      them to reach their potential”. CBSS was intended to contribute to
      Carmarthenshire CCET‟s objectives around basic skills, which include a
      target for 90% of adults to have functional literacy and 80% to have
      functional numeracy by 2005.

3.2   Carmarthenshire CCET put together its proposal for the CBSS
      specifically in response to the Learning Challenge Fund‟s call for
      funding bids. The project was not running prior to LCF. In its original
      bid for LCF funding, CBSS proposed three main strands to the project:

          funding basic skills projects within Carmarthenshire;
          training basic skills tutors; and
          following up learners who have dropped out of mainstream

3.3   The ELWa project began in March 2004 and runs until March 2006. The
      Carmarthenshire Basic Skills Strategy received £262,000 from the
      Learning Challenge Fund. The project received in-kind match funding
      (in the form of use of office space, etc) from Carmarthenshire
      Association of Voluntary Services (CAVS) but has no additional sources
      of funding except LCF.

3.4   Since the project started its staffing structure has changed. The
      project manager has moved from being based at Carmarthenshire
      Association for Voluntary Services (CAVS) and has taken on a new role
      at Carmarthenshire County Council (CCC), as Senior Basic Skills
      Officer. Her post is now funded by CCC, but she continues to manage
      the project as it fits closely with her remit at the Council. ELWa
      funding is now being used to employ two part-time development
      workers, one of whom works with the voluntary sector and the other
      of whom focuses on workforce development. The development workers
      are based at CAVS.

 9                              Shared Intelligence                      September 2005
                                                         Learning Challenge Fund Evaluation
                                                               First Round Evaluation Report

      Progress to date

3.5   The following table summarises key target and actual outputs of the
      project to date. More detail on the individual strands is given below.

          Carmarthenshire Basic Skills Strategy: Target and Actual Outputs to
                                    end June 2005
                                               Target                  Actual
          Individuals supported                319                     429
          Contact learners who have            150                     0
          dropped out
          CBSS event                           1                       1
          Provide BS support for adults with   8                       8
          learning difficulties

      Basic Skills Projects

3.6   Funding basic skills projects is a core strand of CBSS‟ work. To date,
      CBSS has exceeded its targets in supporting individuals through the
      basic skills projects, even though there has been a slight underspend
      in budget to fund these projects (£23,500 was planned and £20,500
      had been spent by end June 2005). To give a feel for the types of
      basic skills projects that CBSS has funded, past and current funded
      projects include:

             Ty Ni family Centre, Carmarthen - peripatetic basic skills
              support provided by an outreach worker in an area of high
             Snooker Project, Whitland – project involving 8 participants
              from Furnace Bank Homeless Hostel who have been initially
              assessed at below entry level 3;
             Kidwelly Home-Start‟s „Money-Go-Round‟ – financial literacy
              for Home-Start beneficiaries;
             Cwtsh Gloyn - a series of sessions for local Welsh language
              „confidence building‟, aimed at improving the reading skills of
              local young mums in particular; and
             Cam Cyntaf – basic skills classes for people with learning
              disabilities, leading to accreditation at entry level 1 and 2 in
              numeracy and literacy.

 10                                Shared Intelligence                       September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

                           Park Hall Family Centre – Fit Club

          The Fit Club is a gym and basic skills course that runs for six weeks. It
          is run by the Park Hall Family Centre manager and a basic skills tutor,
          who have linked up with a local gym to deliver the project.

          The project is aimed at the people who use Park Hall Family Centre,
          which is a new centre on the deprived Park Hall Estate in Carmarthen.
          Most of the participants are young mothers.

          Learners come in on Monday lunchtimes for a weigh-in, and they learn
          some basic skills around calorie counting, writing down what they have
          eaten for a week, working out their ideal weight for their height,
          working out targets for weight loss, and so on. After this, they go to
          the gym and use fit ball, machines and weights. The course is based on
          a similar one that was originally developed in Pembrokeshire.

          The members are mostly attracted to the Fit Club because they want to
          lose weight, so it‟s important that basic skills is „embedded‟ so that
          they are not put off by the learning element. As well as doing basic
          maths with the tutor, the members also learn key facts about nutrition
          and the tutor encourages them to eat breakfast, drink a lot of water
          and make sure they eat proper meals. So as well as having a strong
          basic skills element, the project is aiming to improve health and help
          the learners establish healthy eating routines.

          The project has a good retention rate: it started off with 20 members,
          and 14 were still coming in the fourth week. For the learners it has
          helped increase their confidence, get them off the estate, and help
          them lose weight. Several now express interest in taking hairdressing

3.7   Stakeholders and project management agree that it took longer than
      expected for CBSS to start funding basic skills projects, but that this
      strand of work was now beginning to pick up pace. Reasons for the
      slow start include:

         changes in CBSS staffing structure which meant that there was
          no-one in place to develop projects for several months – this was
          exacerbated by difficulties in recruiting new development staff,
          though both posts are now filled;
         an initial overestimation of the capacity of the voluntary
          sector to bid for funding for basic skills projects. CBSS had
          envisaged that its role would be mainly to appraise project
          proposals and distribute funding, but as the project progressed, it

 11                             Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

            became clear that voluntary and community organisations in fact
            needed a high level of support in developing proposals. This is a
            primary reason for the underspend in project budget;
           a lack of ready-qualified basic skills tutors in the area who
            could deliver training; and
           difficulties in securing action from employers, even though they
            have signed up to the Employer Pledge.

       Training basic skills tutors

3.8    Training basic skills tutors was part of CBSS‟s original bid, but became
       increasingly important as the LCF project progressed. This is because
       it became clear that the original mapping exercise underpinning
       CBSS‟s bid for funding had not recognised the full extent of the
       shortage of basic skills tutors in the county. CBSS has engaged in
       basic skills tutor training and in updating the qualifications of existing
       basic skills tutors in response to changes in the national framework.
       Tutor training is ongoing and the first group who enrolled are due to
       finish shortly. More information on numbers completing the training,
       and what they go on to do afterwards, will be provided by the project
       manager in due course.

       Following up learners who have dropped out

3.9    The intention to follow up learners who have left provision and find out
       the reasons that they have dropped out has not yet been achieved.
       Work on this element started in January 2005, but response was low,
       partly because large numbers were not dropping out at that time. The
       project also had some difficulties in getting information on dropouts
       from Coleg Sir Gar. This has now been resolved and renewed attempts
       will be made in September 2005. This element of the project focuses
       on 16-19 year old learners in Level 1 and 2 provision.

       Project outcomes and fit with LCF scheme objectives

       Widening participation and improving access to learning

3.10   A key outcome of the project is that it has widened participation, as
       it is been able to reach beneficiaries that mainstream funding does not
       reach. These groups include beneficiaries who are at entry levels 1 and
       2, travelling communities, people with ESOL needs, and a range of
       people who would not access provision in college or learning centre

 12                             Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

       locations, owing to lack of confidence, bad prior experiences of
       education, and so on.

3.11   CBSS is able to reach these groups for several reasons. First,
       provision does not have to be accredited, and having to undertake
       formal accreditation can put many learners off. Secondly, CBSS
       funding allows providers to work with beneficiaries who are not
       covered by other funding streams. The provider quotations below
       helps to illustrate this:

                  “The problem I’ve got is that we get some funding from
                  ELWa for work based learning, and most of the rest of our
                  funding comes through Jobcentre Plus for New Deal. There
                  are a lot of people who I’d really like to teach but who don’t
                  fit into these funding streams…CBSS funding allows us to
                  accommodate people that aren’t covered through other
                  funding streams.” – Learning Provider, Carmarthenshire

                  “It’s just got such a varied scope – it is looking at the
                  hardest to reach, and target groups below level one – some
                  are way below level 1. And is looking to signpost people
                  into other provision. Some people are moving on who
                  would never have come into basic skills by the normal
                  routes.” – CBSS Stakeholder, all-Wales remit

3.12   CBSS also enables providers to deliver to small groups, which
       otherwise would not be financially viable. The project manager and
       stakeholders have pointed out that this is often a real hurdle for
       learning providers when setting up basic skills courses. It is possible to
       make these sustainable in the longer term, because courses often
       attract greater numbers of learners once they have been running for a
       while and awareness spreads. However, getting sufficient numbers of
       learners to engage in a new basic skills course can be difficult.

3.13   Finally, by engaging with the voluntary sector, CBSS reaches groups
       of beneficiaries that colleges and private providers would find difficult
       to reach.

3.14   Nevertheless, one stakeholder questioned whether some of learners
       that are being reached – particularly travellers and ESOL clients –
       should be included because they may not be permanent
       Carmarthenshire residents, so delivering to them would not necessarily
       achieve the aim of raising basic skills levels within the county. The
       CBSS project bid did not make clear whether these groups should or
       should not be targeted. However, it would be difficult, legally and
       morally, to exclude these groups from provision.

 13                             Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

       Encouraging collaboration and a strategic approach

3.15   Stakeholders contributing to the evaluation to date have agreed that
       CBSS has successfully set up collaborative ways of working with
       other networks, providers and agencies in the county and beyond. One
       strong example is CBSS‟s close working with the Carmarthenshire New
       Learning Network (which is ESF funded). CNLN‟s core objective is to
       get people into learning – of any type. Once people are engaged in
       some form of learning, they are more likely to consider basic skills
       courses. CNLN often identifies basic skills needs among its clients,
       then works with CBSS to set up projects. Two examples have been the
       „money-go-round‟ project with Home-Start Kidwelly (financial literacy)
       and an ESOL course for a group of Polish immigrants working in a
       meat factory in the county. In addition, CBSS works closely with the
       Basic Skills Agency (BSA) in its national support project for voluntary
       and community organisations and the Employer Pledge. CBSS has also
       supported the Probation Service in delivering a basic skills project that
       the BSA views as highly successful. The BSA has disseminated
       information on this project to the Probation Service across Wales.

3.16   Meanwhile, CBSS has engendered a more strategic approach to the
       way basic skills provision is planned and delivered in the county
       through the dual role of the project manager. Because she is now
       based at the Council, this reduces the possibility of duplication
       between CBSS and other provision. The project manager also provides
       a single point of contact for basic skills provision within the County.
       The added value of this role will be explored further with stakeholders
       within the Council as the evaluation research continues.

       Other outcomes relating to LCF objectives

3.17   Other project outcomes to date that fit clearly with the objectives of
       the LCF include:

          innovation and new approaches to forms of learning and
           engagement: for example, the Park Hall Family Centre is
           developing a gardening club that will incorporate basic skills; and
          raising quality of basic skills provision, through training tutors
           and encouraging networking and sharing of good practice:
                  There are more resources available, in terms of the
                  workplace courses that have been designed and are
                  therefore available for other employers. And I feel that the
                  standard of delivery has also improved because the tutors
                  have had the benefit of adapting their work to the
                  workplace rather than the classroom. - CBSS Stakeholder,
                  national remit.

 14                             Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

       Other outcomes

3.18   There is also evidence that CBSS projects are leading to other
       outcomes for individuals that are not specifically related to LCF
       objectives. These include:

          encouraging progression among learners: again, this has not
           been quantified but case studies of individuals suggest distance
           travelled – for example three of the participants in Kidwelly
           Home-Start‟s Money-Go-Round project have enrolled at the
           Avenue Learning Centre in Llanelli for further accredited Basic
           Skills courses in Literacy and Numeracy and one at Carmarthen
           for ESOL; and
          raising learners‟ aspirations: for example, several of the learners
           at Park Hall said they would now like to do further learning and
           were interested in taking hairdressing courses at Trinity College;
          raising awareness among employers, learning providers and
           voluntary and community organisations about the importance of
           basic skills: this is seen by stakeholders as a key function of the
           project and as a strength. The quotation below explains why:
                  Basic skills are the bottom line in regeneration, though not
                  everyone sees it like that… That’s why raising awareness
                  with Jobcentre Plus, employers and people in regeneration
                  is so important. – CBSS stakeholder, Carmarthenshire


3.19   The project manager is currently preparing an exit strategy and it is
       hoped that several of the projects will be able to become mainstream
       funded. For example, the Cam Cyntaf project for people with learning
       disabilities may be able to attract funding through Carmarthenshire
       Transition Service. Similarly, where the project has trained basic skills
       tutors for a particular organisation, these projects can become self-
       sustaining. An example of this is Kidwelly Home-Start‟s Money-Go-
       Round project.

       Conclusions and learning

3.20   After a slow start, the CBSS project is now progressing well and
       demonstrates a good fit with LCF objectives. Stakeholders interviewed
       have all agreed that the project brings coherence to the planning of
       basic skills provision in the county, and that the basic skills projects
       funded through CBSS are additional to provision funded through other

 15                             Shared Intelligence                      September 2005
                                                     Learning Challenge Fund Evaluation
                                                           First Round Evaluation Report

       streams. Evidence gathered to date suggests that individuals involved
       in the basic skills projects are progressing well and meeting personal
       goals. The projects are also reaching non-traditional learners in
       innovative ways.

3.21   The scale of the project, however, means that it is unlikely that
       provision provided through CBSS will make a noticeable difference to
       overall levels of basic skills within Carmarthenshire before the project
       finishes in March 2006. Another question has been raised over whether
       the project is more focused towards helping individuals achieve
       personal development rather than achieving economic goals in line
       with ELWa‟s own targets. The full mini-evaluation of this project
       (which will also be completed by the end of March 2006) will look at
       levels of literacy and numeracy within Carmarthenshire, and make an
       assessment of how far CBSS has contributed to any improvements in
       these, drawing on our evaluation work with stakeholders. However,
       the view of the project manager and providers involved in the
       evaluation is strongly that this type of small scale provision, that
       meets learners „where they are‟ and draws them into basic skills
       through other areas of interest, is an essential first step in raising
       basic skills levels over the longer term.

3.22   The project has already brought out several lessons that could be built
       into future initiatives. These include:

          the importance of recognising multiple barriers to learning and
           developing provision that can break some of these down (e.g. by
           employing peripatetic tutors, or by building basic skills into more
           recreational courses);
          the importance of having flexible funding streams to address the
           way in which accreditation can act as a barrier to learners and a
           disincentive for providers to deliver to the lowest-qualified and
           support distance travelled as a crucial first step; and
          the fact that voluntary and community organisations need support
           with developing proposals for basic skills projects.

3.23   The project has uncovered an extensive need for ESOL provision which
       was not foreseen at the beginning of the project as well as showing
       that there was a need to train more basic skills tutors than had been
       anticipated. The ELWa contract manager feels that the lessons from
       CBSS will be able to shape future mapping exercises that the CCET
       carries out.

 16                            Shared Intelligence                      September 2005
                                                     Learning Challenge Fund Evaluation
                                                           First Round Evaluation Report



4.1   The aim of the Cyfrwng project is to increase the number of people
      learning Welsh, defined as the number of people in the process of
      learning Welsh. The objectives are to:

         provide opportunities for 3 specific audiences to access Welsh
          courses: parents, non-Welsh speakers in Welsh speaking areas,
          and people who need Welsh in the workplace;
         increase capacity and establish provision, delivered in non-
          traditional ways; and
         increase the demand for Welsh language provision.

4.2   The project aims to address both demand and supply. The demand-
      side methodology is particularly innovative, with a focus on three
      specific market segments: tutor organisers and area-animateurs; the
      use of a call centre; promotional materials and a range of innovative
      delivery models.

4.3   The project suffered delays in contracting and the funded period was
      from December 2003 till March 2006 (extended from December 2005).
      The project was allocated £989,000 of ELWa LCF funding. Cymad are
      delivering the project alongside their other activities. This has allowed
      the project to gain access to Cymad support. For instance, when the
      project manager was on maternity leave, the Cymad director was able
      to oversee the project. An external evaluator has been appointed (the
      only such case among LCF projects).

4.4   There is a Steering Group with members from ELWa, Welsh Language
      Board, the Universities of Bangor and Glamorgan, and the Welsh
      Medium Nurseries.

      Progress to date

4.5   Shared Intelligence will be discussing progress with Cymad during
      September and will then be better placed to provide an update. We
      have met with the ELWa contract manager and ELWa‟s Head of the
      Bilingual Unit to discuss the project.

4.6   The project has met or come close to many of its output targets but
      problems in recruitment of staff in the field have reduced the overall
      impact. The required skills are persuasiveness and knowledge of

 17                            Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

       provision, and it is believed that these have not always been available.
       The numbers of available providers remain low and it is likely HE will
       be the centre of provision going forward. It has proved difficult to
       develop local provision to serve the pockets of demand that have been
       created by the local activism approach. Marketing materials have been
       considered to be patchy but are now becoming more focused. The call
       centre has been successful and is likely to be integrated into
       Learndirect eventually.

4.7    It has been suggested by ELWa staff that the management of the
       project has lacked drive and direction in some respects, partly because
       of the arrangements put in place to cover maternity leave. The
       Steering Group has lost some momentum and has not driven the
       project strategically. The project required energy and buzz – ELWa
       believe Welsh for Adults has been a static area in recent years, and
       more infrastructure and capacity was required to take forward the
       momentum being created by Welsh language schemes.

4.8    The work in the autumn will provide a more up-to-date picture of the
       progress and outputs being achieved, and whether some of the issues
       raised have been effectively addressed.

       Project outcomes and fit with LCF scheme objectives

4.9    It is not clear yet the extent to which Cyfrwng has been able to make
       enough difference to the demand for – and provision of – Welsh for
       Adults learning in their key target areas.

4.10   The ELWa officers believe Cyfrwng is an innovative project –
       particularly in the area of raising awareness with a focus on 3 specific
       market segments; tutor organisers and area-animateurs; the use of a
       call centre; and promotional materials.

4.11   It also has a core aim of improving access and widening
       participation in a particular subject area. The other LCF scheme
       objectives will be assessed during the next round of evaluation.

       Conclusions and learning

4.12   The interim conclusions that will be explored further with the project in
       October include the fact that staff continuity was not adequately
       secured, that the innovative project plans proved difficult to deliver
       because of recruitment challenges, and that the demand created by
       the LCF project proved difficult to serve because of local provision

 18                             Shared Intelligence                      September 2005
                                                     Learning Challenge Fund Evaluation
                                                           First Round Evaluation Report



5.1   The purpose of the Learning Vale (LV) project is to enable rural Welsh-
      speaking communities to access a wide range of IT learning
      opportunities between Blaenau Ffestiniog and Porthmadog. The
      scheme aims to run courses for 1,200 learners, supported by a
      programme of marketing and promotion. The scheme uses existing
      training resources such as the Deudraeth Training Room and the
      Digital Vale Broadband Network.

5.2   In 2000, a community group formed to address economic, social and
      environmental welfare of the local rural area. A skills study found low
      levels of ICT skills and, as a result, 100 laptops were distributed to
      create a “networked village”. Colleges were seen as geographically and
      institutionally distant and local people were under-confident to enter
      college-based learning. In response to this, the group bought the old
      police station and created a range of community learning facilities
      there. LCF funding allowed this foundation to be built upon with
      focused provision.

5.3   The project was originally expected to start in 2002, but ELWa funding
      was delayed until February 2004. The project kicked off in December
      2002 without ELWa funding in a relatively low-key way. ELWa back-
      filled funding to meet costs of the first year and a Training Co-
      ordinator was employed. It was originally planned to operate until
      October 2005 but this has now been extended to June 2006.

5.4   There have been personnel changes both within ELWa and the project,
      and this has had an impact on momentum. While such changes are
      inevitable in a medium-term programme it is essential that continuity
      is achieved by effective handover plans in ELWa and the projects.

      Progress to date

5.5   The new ELWa contract manager believes that the project looks “safe”
      and the activity fits well with the original objectives of the project. The
      project is meeting wider objectives of re-engaging people with the
      labour market, meeting employers‟ needs, creating awareness of
      lifelong learning. It is also gathering good information on case studies
      and progression – showing benefits for individuals and businesses.
      There is a call centre in Porthmadog that has most new staff trained by
      LV, while local government, health board and local surgeries are all
      using LV services. The project is making effective partnership
      connections with Coleg Meirion-Dwyfor, Coleg Menai and a local
      enterprise agency.

 19                            Shared Intelligence                      September 2005
                                                         Learning Challenge Fund Evaluation
                                                               First Round Evaluation Report

5.6    The project is getting good local press coverage, which is raising its
       profile and helping to demonstrate benefits for the labour market and
       for progression of individuals.

5.7    One challenge has been the lower than anticipated levels of
       accreditation. Only 70% of beginners and CLAIT learners take the
       exam; in response, ELWa have agreed a new payment profile that
       reflects a modular approach to accreditation.

5.8    The project‟s targets were for 1,200 learners over 3 years – 390
       beginners and CLAIT; 300 ECDL; 510 specialist (Sage, Photoshop etc).
       After 18 months the project had achieved 450 learners altogether, so
       had some way to go to meet the challenge. Targets were re-profiled
       and it is now understood that the learner numbers are ahead of target.
       Shared Intelligence will be seeking updated output information in

       Project outcomes and fit with LCF scheme objectives

5.9    The project is considered to be innovative in the way it has gone
       about creating demand for ICT courses as a means of addressing
       social and economic exclusion in a local rural area. The community-
       wide approach, linked to other initiatives such as physical
       regeneration, broadband and the distribution of laptops, has led to
       improved access and widening participation and to the delivery
       of more learning outcomes. Further work with the project will
       explore how far it is possible to identify outcomes that are directly
       attributable to the LCF project.

       Mainstreaming and sustainability

5.10   The ELWa contract           manager believes that the project has
       opportunities for a         sustainable future and could provide an
       effective service rolled    out throughout Meirioneth. The opportunities
       for this will be explored   during the next round of evaluation.

       Conclusions and learning

5.11   The Learning Vale is certain to provide important information about
       how a project like this can make a difference throughout a small area
       and in a focused area of learning – adult ICT training. An intensive
       approach in a relatively confined rural area and an integrated approach
       using a range of project types (not all LCF-funded) will prove

 20                                Shared Intelligence                      September 2005
                                                    Learning Challenge Fund Evaluation
                                                          First Round Evaluation Report

5.12   One learning point is the need to ensure, where there are changes in
       personnel in the project or in the ELWa contract managers, that
       momentum is maintained and continuity achieved.

 21                           Shared Intelligence                      September 2005
                                                    Learning Challenge Fund Evaluation
                                                          First Round Evaluation Report



6.1   The Merthyr Tydfil Lifelong Learning Campus is a research and
      feasibility study aimed at investigating the potential for raising
      demand and interest in learning and training through an integrated
      provision of learning within the Borough. The concept of the Campus is
      to provide a seamless portfolio of learning opportunities to support
      individual achievement and business competitiveness in Merthyr, that
      will also be a potential flagship for similar developments throughout

6.2   Through a series of research papers, the project aimed to develop a
      detailed analysis of learning within the Borough, and an assessment of
      the future needs of the community so that a Business Case could be
      drawn up for the development of the preferred model. It was agreed
      that the project will culminate in a report to ELWa on the integrated
      elements of the Lifelong Learning Campus that covers:

         consultation and stakeholder involvement;
         links with ELWa Corporate and Operational Plans;
         operation and management of the new campus;
         funding and finance;
         project design and construction;
         synopsis of all research reports; and
         clear recommendations and Implementation Plan.

6.3   Discussions about the concept of an integrated lifelong learning
      provision began in 2001 and a feasibility study identified components
      such as enhanced post-16 provision, a university presence, business
      incubation and a high-tech information and technology facility. The
      partnership that took the LCF funding bid forward included ELWa, the
      Council, University of Glamorgan, Merthyr Tydfil College and Tydfil
      Training Consortium. The project is staffed by 3 people – Project
      Manager (seconded from an adjacent Council), Project Officer (from
      local College) and Project Support. The Project Manager was appointed
      in September 2004.

6.4   Getting the project started necessitated extensive negotiation with
      ELWa and WAG. The contract was only signed in June 2004 and a 2-
      phase approach was agreed with total funding of £752,000. The
      timescale for Phase 1 was up to April 2006.

 22                           Shared Intelligence                      September 2005
                                                        Learning Challenge Fund Evaluation
                                                              First Round Evaluation Report

          Phase 1 – Commissioning consultant reports (by end 2004),
           carrying out research (by March 2005), submitting these to ELWa
           and WAG for review and feedback, drawing up an overall Business
           Case, and reaching a decision on whether to proceed to Phase 2.
           This would be completed by April 2006 and would have a budget
           of £446,000.
          Phase 2 – Subject to approval this would involve planning and
           implementation of the strategy approved at Phase 1. A total of
           £306,000 would be available. The timescale was not specified.

6.5    There was believed to be some concern about the ultimate objectives
       of the different stakeholders – it has been suggested that local
       partners favour a physical build concept while ELWa favour more of a
       virtual concept based around collaboration and networking. This may
       prove to be a lingering issue throughout the project.

       Progress to date

6.6    A Project Advisory Group (PAG) has been established with members
       from a wide range of organisations, chaired by Merthyr Tydfil Council
       and meeting monthly.

6.7    Six key research papers have been commissioned and the following
       five reported by March 2005:

          Skills for Business;
          Learning through the Outdoors;
          Adult Community Learning;
          Arts Media & Culture Learning; and
          Work-Based Learning.

6.8    These reports sat with ELWa for some time awaiting feedback but
       feedback has now been provided on all but one report. The PAG were
       concerned that, without this feedback, it was not possible to begin to
       agree a plan for proceeding to the Curriculum Plan and subsequent
       Business Case.

6.9    The sixth report (Further Education) is due to be completed in
       September 2005. A study on Learning Disabilities is being developed,
       not funded by LCF, and two further papers are proposed if ELWa will
       approve a carry-over from 2004-05 underspend.

6.10   There is a considerable degree of uncertainty about the final strand of
       research into 16-19 Provision. The project has agreed not to

 23                               Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

       commission this research, as this has been overtaken by the
       Geographical Pathfinder study in this subject area. but to take on
       board the findings of that study. Since the project has not had the
       opportunity to establish the research brief there is some concern that
       the Pathfinder will reach conclusions that are at odds with other
       aspects of the research and that raise serious issues for
       implementation. At the very least the project is concerned that the
       Pathfinder will stall the process (not reporting until end October) and
       make it impossible to complete substantive work by the end of 2005
       and reach agreement by March 2006 on whether to proceed to Phase

       Project outcomes and fit with LCF scheme objectives

6.11   The outcomes of the project will be dependent on progress to Phase 2.
       The Project Manager is concerned that the Pathfinder study on 16-19
       Provision will create a major conflict at the heart of the Lifelong
       Learning Campus project, and that this will exacerbate the delays in
       feedback to the first five reports. It is believed that there are problems
       of “expectation management” and differing agendas being promoted
       by different, politically influential partners.

6.12   ELWa have said that the reason for the delayed feedback was the need
       to identify and secure appropriate expertise in each of the key
       research areas (e.g. arts, media and culture and outdoors learning) to
       review and provide value-added feedback. This required a mix of in-
       house and external capacity.

6.13   Shared Intelligence will carry out consultation with key strategic
       stakeholders in September to assess the issues about expectations
       and the more contentious issues surrounding the project.

6.14   The Lifelong Learning Campus is an innovative approach to
       developing a forward strategy for learning throughout a Borough by
       building an evidence base around commissioned papers and then using
       a partnership to take it forward. It is strategic at a local level, with
       real possibilities of having relevance throughout the whole of SE
       Wales. There is a strong focus on collaboration between learning
       providers in principle (college, sixth forms and private providers).
       Finally, it would be expected that a successful strategy that met the
       objectives of the project would also improve access, widen
       participation and deliver more high-quality learning outcomes.

       Mainstreaming and sustainability

6.15   The project is a two-stage process with Phase 2 depending on the
       successful completion of Phase 1 and the approval of funding.

 24                             Shared Intelligence                      September 2005
                                                     Learning Challenge Fund Evaluation
                                                           First Round Evaluation Report

       Mainstreaming will depend on a number of factors and it is not
       possible to make a realistic assessment at this stage. However the
       track record of LCF in achieving approval for follow-on funding for
       feasibility-style projects has not been strong.

       Conclusions and learning

6.16   Merthyr Tydfil Lifelong Learning Campus is perhaps the most politically
       sensitive of the LCF projects because it has an influential group of
       stakeholders and a strong sense of preferred outcomes. It is also seen
       by some stakeholders as having ramifications well beyond Merthyr.
       Such strong but differing expectations can have implications for the
       management of the project. Perhaps more could have been done from
       the outset to ensure an agreed set of expectations between the local
       partners, ELWa and even WAG.

 25                            Shared Intelligence                      September 2005
                                                    Learning Challenge Fund Evaluation
                                                          First Round Evaluation Report

7     NATIONAL   STRATEGY  FOR                            THE         TOURISM


7.1   The purpose of the Tourism Training Forum for Wales (TTFW) is to
      provide leadership, guidance and co-ordination for tourism training
      and education in Wales. The Forum – established in 1998 - aims to
      highlight the role education and training has in securing the industry‟s
      future. Tourism contributes almost £2.5 billion to the Welsh economy
      every year and employs almost 10% of the entire national workforce.
      By highlighting the contribution that education and training can make,
      TTFW is endeavouring to raise standards, improve quality and
      guarantee the industry‟s future, in a vibrant and competitive Wales.

7.2   The LCF project began in March 2003 and runs until the end of
      February 2006. TTFW will receive £369,000 of LCF funding over three
      years and this sits alongside WTB and ESF funding.

7.3   The overall aim of TTFW is to change the culture of small tourism
      businesses by increasing their awareness of how good HR practice can
      improve their business performance, changing their attitude to training
      and staff development and improving access to high quality provision.

7.4   The LCF project aims are as follows:

         to encourage education and training providers to ensure they
          offer relevant, accessible and high quality services to individuals
          and the whole breadth of the tourism industry in Wales;
         to encourage tourism businesses to take advantage of relevant
          training and business development and facilitate their access to
          provision and services; and
         to promote effective partnerships to encourage the development
          of human resources for the tourism industry in Wales.

7.5   The project has several innovative strands that are aimed at achieving
      these objectives by means of conferences, websites, newsletters,
      identification of exemplar businesses and training of their key
      managers, partnership development and strategic engagement with
      key regional and national stakeholders. It is a complex project in
      terms of strands, activities and targets. These are the delivery
      indicators against which the LCF contract will be measured – as agreed
      between ELWa and the project - but they do not in themselves
      adequately show evidence of integrated progress nor reflect the

 26                           Shared Intelligence                      September 2005
                                                           Learning Challenge Fund Evaluation
                                                                 First Round Evaluation Report

       strategic aims of the project as they relate to changing attitudes and
       the effectiveness of partnerships.

       Progress to date

7.6    TTFW has been delivering on these LCF targets for over two years and
       will complete in February 2006.

7.7    The recent internal Year 2 evaluation by TTFW suggests “good overall
       progress against the majority of targets”. They particularly draw
       attention to the level of response to two key strands of the project – website and the Tourism Training Guide, although it is
       not possible yet to assess the actual impact of accessing such

7.8    In terms of key progress measures, the following statistics show how
       the project has exceeded its targets in a number of key areas. The
       table below summarises the output data so far:

              Tourism Training Forum for Wales: Target and Actual Outputs
                                         Target                  Actual
           Whodoiask website users       4,000 a year            6,000 sessions per
           Tourism Training Guide        800 a year              500 separate visits per
           database users                                        month
           Newsletter recipients         4,000 twice a year      5,300 twice a year

7.9    Some further points to note are that:

              whodoiask website has a target of 4,000 users a year, but the
               actual interest is much greater than this. This level of usage is
               increasing following site development and ongoing promotion;
              the Tourism Training Guide online database has a target of 800
               beneficiaries a year, but it is receiving over 2,000 hits per
               month and over 500 separate visits; and
              evidence shows that most other milestones have been met or are
               planned, including regional events, connections with CCETs,
               partnership agreement with People 1st, HR co-ordinators attached
               to Regional Tourism Partnerships

7.10   The area where progress against targets has been most challenging is
       in the identification and recruitment of best practice exemplar
       business managers and in enrolling these people on to the
       relevant training modules. So far 15 exemplars have been identified
       against a target of 20. Further work will take place this year to try to

 27                                  Shared Intelligence                      September 2005
                                                     Learning Challenge Fund Evaluation
                                                           First Round Evaluation Report

       reach the target, and a further 15 possible exemplars have been
       identified and are being approached.

7.11   Additionally, six managers from exemplar businesses have undertaken
       Trinity College Module 1 training to be able to share their experiences
       with other businesses and six are due to start Module 2 in the autumn.

7.12   TTFW believe it has been difficult to attract exemplar staff to
       undertake this training as, by their nature, exemplar businesses are
       busy and therefore find it difficult to allocate time to complete their
       training. Training was an essential component of the project because
       of ESF match funding, but TTFW believe that a more informal and
       networking approach to mentoring might have been more attractive to
       potential exemplars.

       Project outcomes and fit with LCF scheme objectives

7.13   As stated above, it is difficult at this stage to assess the wider
       outcomes of the TTFW project. In terms of partnership development
       and strategic engagement there is evidence of real progress – for
       example, the memorandum of understanding with People 1st and
       the development of a joint strategic plan for HR in tourism in
       Wales; and the establishment of HR co-ordinator posts in regions.

7.14   The evaluation of TTFW in September and the following months will
       need to focus on interviews with partners and stakeholders (Chair of
       Welsh Tourism Alliance, Strategy Director of SWW Tourism
       Partnership, Strategy Director of Regional Tourism Partnership for
       North Wales, UWIC, Trinity College, People 1st and Fforwm) about the
       strategic impact of TTFW; and on building the evidence base regarding
       the benefits of the successfully disseminated products such as the
       website, Training Guide, toolkits and events.

7.15   TTFW has had innovation at its core in a number of respects – the
       planned use of exemplar businesses and mentors to take the message
       out to small SMEs throughout the tourism sector; the development of
       innovative learning tools such as a website, online toolkit and TV
       programme. It has been focused on a strategic approach at national
       and regional level. It has been successful in bringing several
       universities and colleges into collaboration. It certainly is aiming at
       widening participation in training and staff development amongst
       small businesses.

 28                            Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

       Mainstreaming and sustainability

7.16   The whole area of tourism development in Wales is uncertain with the
       planned merger of the Wales Tourist Board, WDA and ELWa. The
       project manager believes the evidence points to a definite need to
       maintain TTFW as an independent force not tied to government at this
       stage. The links with People 1st and the relationships with the regional
       tourism partnerships are considered to augur well for the sustainability
       of the organisation, which already receives funding from other sources
       for non-LCF activities.

       Conclusions and learning

7.17   The project has achieved a lot during its life – in terms of awareness,
       low intensity information and advice through websites and toolkits.
       The effectiveness of these activities will be assessed by the project in
       the coming months. One learning point would be to evaluate new
       media as they are launched – as a mainstream media company would
       do. This is perhaps a resource issue and suggests that the TTFW
       project was under managed in the first instance – a problem that has
       been rectified. In this way the monitoring and evaluation plan will now
       be capable of implementation.

7.18   The project perhaps lacked a strong focus initially, reflected in the raft
       of quantitative targets, most of which are linked to specific strands of
       activity but which do little to add to the understanding of the
       difference the project is making.

7.19   The recruitment of exemplar businesses – champions who could
       spread the word amongst the small tourism businesses – has proved
       more difficult than originally expected. This was a key thrust of the
       TTFW project. It will be important to consider in the next stages
       whether this was because the whole approach was too ambitious or
       whether there have been any delivery issues that have reduced the
       success of the approach in practice – for instance the need to link
       recruitment to a training course as a requirement for ESF match

 29                             Shared Intelligence                      September 2005
                                                    Learning Challenge Fund Evaluation
                                                          First Round Evaluation Report



8.1   The Oaks Project is an outreach project to encourage adults into
      learning. When it was established there were some 23,000 adults in
      Blaenau Gwent not qualified to NVQ3 or equivalent (out of 33,000
      working age adults), and 10,000 of these were estimated to have basic
      skills deficiencies. The project set out to establish four Community
      Computer pilot centres in collaboration with Community First
      Partnership Boards, and to deliver learning, primarily through the
      medium of high-quality ICT. A target was set to engage 600 individual
      beneficiaries, drawn from both economically inactive and active
      groups, including employed and older-age individuals. The project
      would address basic skills, essential skills and ICT skills.

8.2   The aims of the Oaks Project are:

         through a partnership approach, to increase flexible learning
         to offer guidance, advice and support to enable beneficiaries to
          embrace lifelong learning;
         to complement other learning initiatives in Blaenau Gwent; and
         to work with Communities First Partnerships to establish 4
          community computer pilot centres in Communities First wards.

8.3   The Oaks project was established in December 2003 and was due to
      end in December 2005 but has recently been extended to March 2006.
      The project is funded with £392,000 of LCF resources plus £48,000 of
      match funding.

      Progress to date

8.4   Within the first 15 months of the 2-year programme all key
      operational targets have been met or almost met, with the
      exception of the progression targets. The main performance figures
      are as follows:

 30                           Shared Intelligence                      September 2005
                                                           Learning Challenge Fund Evaluation
                                                                 First Round Evaluation Report

               The Oaks Project: Target and Actual Outputs to Date
                                                       Target               Actual
         Beneficiaries engaged                         600                  603
         Students enrolled                             400                  464
         Beneficiaries progressing                     250                  232
         Community Computer Centres                    4                    4
         Learning programme for each CF area           10                   9
         Learning opportunities in schools             24 (80%)             20 (67%)

8.5   The balance between male and female enrolments is 29/71 in favour
      of females compared with an objective of 40/60. Within the male
      target population the young males under the age of 44 are particularly
      under represented, with only 16% of enrolments from this group
      compared with an objective of 30%. Younger women make up 42% of
      all enrolments, slightly ahead of target. The reason why the male
      targets were not reached will be addressed in subsequent rounds of
      the evaluation.

8.6   Entry interviews show that 82% of Oaks learners have not engaged in
      learning in the last six months and a similar number are unemployed.
      Real progress is taking place in local schools. Only 3 schools have
      declined to get involved. One course had to be doubled up to meet
      demand. Work is continuing to get outstanding schools involved. Most
      popular school-based courses are in IT, Welsh for Parents and Family
      Literacy. Basic Skills has been a particular success.

8.7   The Oaks are planning to achieve the progression target by the end of
      2005, requiring just an additional 18 cases of progression. The ELWa
      Contract Manager believes this was always going to be the most
      ambitious target and believes the appointment of two Development
      Officers (Community and Schools) will support this. The officers attend
      enrolment and 50% of classes to build rapport and encourage
      progression and take completers to see RISE centres and colleges.
      Service agreements are in place with Careers Wales, Jobmatch and
      RISE to ensure beneficiaries get advice and guidance.

8.8   The project is achieving better accreditation than originally
      envisaged. Collaboration with WEA is leading to the development of
      OCN accredited courses in Beauty, Welsh and Design a Newsletter. The
      ELWa Contract Manager supports the project positively and believes
      the management and development of the project is effective and

 31                              Shared Intelligence                            September 2005
                                                     Learning Challenge Fund Evaluation
                                                           First Round Evaluation Report

       Project outcomes and fit with LCF scheme objectives

8.9    The Oaks Project has achieved its operational targets very effectively
       but it may also be making a difference at a more strategic level.
       From October Si will be assessing if there is evidence that enrolling
       over 600 local people, the majority of them unemployed and not in
       touch with learning, has made a significant impact on the local labour
       market in Blaenau Gwent and on broader attitudes to learning.
       Without a doubt it is improving access and widening
       participation. The innovative location of courses in a whole range of
       community and informal venues is believed to be a strong attraction.
       The project reports that some progress is also being made with local
       employers, including the Council itself. This was not an initial target
       for the project but impacting on business competitiveness and
       workplace skills is an added benefit of the Oaks. Si will consult with
       the employers who have been engaged with the project.

8.10   The project staff believe that the project provides value for money and
       effective achievement of targets and wider objectives. They believe
       they provide an attractive model for development throughout South
       East Wales. Si will explore the potential for wider roll-out with
       stakeholders from October.

       Mainstreaming and sustainability

8.11   The project has proved to be a best practice LCF example of
       sustainability. It has received ESF funding that will extend its
       operation until August 2007 and is aiming to increase targets to 600
       enrolments (from 400) and 350 progressions (from 250).

       Conclusions and learning

8.12   The Oaks has performed well and has been a very well managed
       project with professional standards and effective procedures. It has
       created real demand and interest amongst non-learners and has even
       begun to raise its game in terms of progression. The approval of ESF
       funding till 2007 shows the confidence in the management team and
       their approach.

 32                            Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report



9.1   The purpose of the Powys Basic Skills project, delivered by Powys
      CCET, is to provide a sustainable and innovative infrastructure for
      engaging with basic skills learners in communities throughout Powys.
      The project funding was delayed at the start – it was originally planned
      to run from August 2003 until July 2005, but LCF funding was delayed
      until September 2004. The project lifespan has recently been extended
      to March 2006. Powys Basic Skills will receive £201,000 of LCF funding
      and this sits alongside Basic Skills Agency, ESF and in-kind funding.
      The project actually kicked off before LCF funding was contracted and
      outputs were funded retrospectively.

9.2   The LCF project is the second phase of a three-phase CCET plan to
      improve basic skills provision in Powys. The first phase brought
      together 49 organisations who committed to basic skills and provided
      training for 45 learners. The project was seen to fit with ELWa‟s
      “Essential Skills Goal for Mid Wales”. It set out to target learners in
      those geographic communities with above average rates of low literacy
      and numeracy through working with „communities of interest‟ to help
      them access learners.

9.3   The objectives of the LCF project are as follows:

         to develop and deliver basic skills awareness training for a wide
          range of organisations;
         to identify and train basic skills tutors;
         to identify delivery       organisations           and   signposting/referral
          processes; and
         to develop further       the    basic      skill    partnership   with    key

9.4   Following the LCF second phase, it is planned to deliver the learning to
      identified small groups and individuals out in the community using ESF

      Management structures

9.5   The project is sponsored by CCET but is managed by Coleg Powys. The
      project is overseen by a Management Group which is a sub-group of
      the CCET with representatives from WEA, Powys LEA, ELWa, Coleg
      Powys and PAVO (voluntary organisations). An Operational Group with
      a wider membership considers the operational aspects of the project.

 33                            Shared Intelligence                       September 2005
                                                       Learning Challenge Fund Evaluation
                                                             First Round Evaluation Report

      Progress to date

9.6   Powys Basic Skills has been delivering on a number of March 2006 LCF
      targets and a selection are shown below.

               Powys Basic Skills: Target and Actual Outputs to Date
                                               Target for Mar        Actual - April
                                               2006                  2005
        Trainers engaged in training           40                    47
        Trainers completed training            40                    18 (+14 Mod 1)
        Trainers supporting learners           40                    16
        Learners accessing support             400/200/100           287/141/107
        (initially/further/fully engaged)
        Voluntary organisations identified     100                   42

9.7   The C&G Level 2 Certificate for Adult Learner Support courses held in
      Brecon and Newtown are complete and new courses are being
      advertised for October 2005. They have evidence of sixteen trainers
      already working with learners, representing seven organisations. No
      outputs are claimed unless all relevant paperwork and certificates are
      issued. The outstanding numbers of trainers yet to complete training is
      partly due to the delay in availability of the Level 3 materials from
      BSA. The Level 3 course began in February 2005 and since trainees
      are required to work with learners as part of course requirements this
      is expected to increase the numbers of trainers supporting learners
      and the number of learners further/fully engaged.

9.8   Meetings have been held with PAVO to increase the engagement with
      voluntary organisations. A targeted action plan is being developed.
      Project promotion has been stepped up with stories in newsletters of
      partner organisations. BSA have been involved in awareness and
      information provision for partners.

      Project outcomes and fit with LCF scheme objectives

9.9   It is difficult at this stage to assess the wider outcomes of the Powys
      Basic Skills project since it is very much a developmental stage of a
      three-phase programme. Not until the individual and small groups of
      learners are trained in basic skills will the wider impacts on labour
      markets, business performance and progression of learners be evident.
      However, the outcomes that have been achieved relate to the
      development of the basic skills partnership and the increase in
      awareness of the basic skills agenda amongst local partners.

 34                              Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

9.10   One of the key risks relates to the potential sustainability of the basic
       skills partnership after the end of LCF funding in March 2006. There is
       some uncertainty regarding the robustness of the partnership if there
       is no specific funding to maintain the developmental work, particularly
       once any ESF funding for provision of learning is completed. Some
       problems over chairmanship of the Management Group suggest that
       the involvement may not survive ending of funding. This will be
       assessed through partner interviews in September 2005. The project
       ends in March 2006 and the mini evaluation will be completed by May

9.11   This project has been effective in bringing basic skills to an area where
       it has never been prioritised before. It has brought partners to the
       table. It has created a strategic approach at county level, has
       encouraged collaboration between learning providers, has set the
       foundations for improving access and widening participation and
       for delivering more learning outcomes.

       Mainstreaming and sustainability

9.12   The success in sustainability and mainstreaming is not clear at this
       stage and will depend on funding decisions and in the robustness of
       the basic skills partnership once LCF ends.

       Conclusions and learning

9.13   The Powys Basic Skills project has been an important learning vehicle
       for the local CCET and has brought partners together, some of whom
       had never run such a project before. The project management team
       was enthusiastic and very keen to learn, building strong links with the
       Basic Skills Agency and getting them to address partnership meetings
       to raise knowledge and awareness for all members. The work carried
       out has been important in an area where no basic skills infrastructure
       existed before and it would be a real loss if the momentum was

 35                             Shared Intelligence                      September 2005
                                                     Learning Challenge Fund Evaluation
                                                           First Round Evaluation Report


10.1   The aims of Swansea College‟s LCF project were to:

          undertake a second stage of a feasibility study for a city centre
           site for Swansea; and
          to identify the implications of the site to the development of
           learning centres across Swansea, and the implications for other
           Swansea College locations.

10.2   The LCF-funded feasibility study followed on from the findings of a
       previous study by Swansea College, and from research undertaken by
       Swansea CCET into the development of a Swansea Learning Network.
       The College identified that a second stage of research was needed in
       order to take the CCET‟s findings into account, and also to ensure that
       Jobcentre Plus‟ strategic vision for Swansea was built into the study.

10.3   The key objectives stated for the study were as follows:

          to ensure that Swansea College‟s curriculum and Estates Strategy
           help to deliver ELWa‟s 2010 targets - specifically, to enable those
           currently economically inactive to become economically active;
          to help those currently in employment to expand their skills and
          to consider the implications of the main findings of the CCET
           study on Developing a Swansea Learning Network for the
           Strategic Partners regarding capital investment in the City Centre;
          to identify the Strategy being adopted by JobCentre Plus
           regarding support for their priority groups in the centre of
           Swansea and in Community Centre;
          to identify the opportunities for the College in response to the
           above initiatives;
          to identify the client groups and programmes to be offered in a
           city centre site;
          to identify the implications and opportunities regarding the
           College‟s existing sites; and
          to identify a site in Swansea to meet these objectives, including
           costs and potential sources of funding for this.

 36                            Shared Intelligence                      September 2005
                                                     Learning Challenge Fund Evaluation
                                                           First Round Evaluation Report

10.4   The feasibility study cost a total of £50,000 and was delivered by GVA
       Grimley on behalf of the College. The final report was produced early
       in 2005.

10.5   Si is currently awaiting further consultation with the project manager,
       which will clarify further evaluation activity that needs to take place.
       As the project is complete in terms of the part that was LCF funded,
       evaluation work will be undertaken as quickly as possible.

 37                            Shared Intelligence                      September 2005
                                                     Learning Challenge Fund Evaluation
                                                           First Round Evaluation Report



11.1   Training for the Social Economy is run by Wales Co-operative Centre
       (WCC). The project aims to strengthen WCC‟s training offer by
       bringing together the different strands that the WCC provides into an
       integrated programme. The project also aims to create a learning
       community within the co-operative sector in Wales and develop
       opportunities for good practice sharing.

11.2   The project received £464,000 of LCF funding. It runs for 3 years from
       October 2003 to September 2006, with the aim of enabling 2100
       individuals to participate in 6300 days of training.

11.3   The project includes the following key strands:

          development of new bespoke course materials and delivery of
           these courses to individual social enterprises;
          delivery of a wide range of training courses, through WCC‟s own
           tutors and through the WEA, to groups of learners within the
           social economy or those interested in getting into or starting a
           social enterprise;
          development and piloting of e-learning provision; and
          creation of learning networks for groups within the social

11.4   Wales Co-operative Centre is carrying out its own evaluation research,
       which includes gathering course evaluation forms from all learners,
       collecting feedback on conferences and one-off events, and carrying
       out a longitudinal survey of learners. Si‟s evaluation tasks aim to
       complement these activities by carrying out qualitative research with
       learners. Si‟s mini-evaluation will be complete by the end of March
       2006, although the project will still have six months to run at that
       point. It is intended that the data included in Si‟s mini-evaluation
       report will be able to feed into WCC‟s own end-of-project evaluation

 38                            Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

       Progress to date

       Key activities

11.5   Training delivered through the project falls broadly into 3 categories:

           specialist training – training for co-operative businesses and
            credit unions on regulation and disciplines specific to running
            these types of business;
           general training – training for co-operatives, credit unions or
            other members of the social economy on subject areas of
            common need; and
           foundation courses – training for members of the community who
            are either members of a co-operative or are individuals taking
            part in the social economy (delivered by South Wales Workers
            Educational Association, Coleg Harlech WEA and external tutors).

11.6   Most training has been general or foundation training, but as the table
       in the sub-section below shows, WCC has also delivered nearly 60
       bespoke training courses, developed specifically for individual social

11.7   The e-learning project is progressing – at the last meeting with the
       project manager in June, the tendering process had just been
       completed and the successful sub-contractor appointed to run the
       pilot. The e-learning project will make six of WCC‟s core units available
       to study online. There will be tutor support and it is a key aim of the
       project that learners will be able to interact with one another through
       message boards or similar technology. WCC will provide the course
       material, adapted from current modules – there will be three courses
       for credit unions, two for co-operative businesses and one generic one
       on customer care, which all social enterprises can make use of. The
       contractor, Llanelli College, will develop the material for online learning
       and run the pilot. It is intended that this project go live in January.

11.8   Three Learning Networks are currently running. These are
       developed in response to demand from particular types of social
       enterprise. The three current networks include regional North and
       South Wales employees‟ networks for Credit Unions, and a business
       succession network that covers the whole of Wales (there are about 20
       businesses going through transfer to social enterprises that are taking
       part). For each of the networks, WCC acts as facilitator, while
       members themselves set agendas and terms of reference.

 39                             Shared Intelligence                      September 2005
                                                        Learning Challenge Fund Evaluation
                                                              First Round Evaluation Report

11.9    WCC also held a conference recently on „Fitting the Pieces Together
        with Training‟. The day long conference was aimed at all organisations
        within, or interested in, the social economy. It was interactive and
        aimed to create a greater understanding of the diversity of the social
        economy, identify training needs and help participants learn from, and
        network with, one another. WCC asked delegates to fill in evaluation
        forms, and the response was extremely positive.

        Outputs to date

11.10   The project has exceeded its target for new beneficiaries. WCC has
        been successfully able to recruit learners through advertising to its
        existing database and through word of mouth. However, they have not
        met targets so far for numbers of beneficiary hours and days. The
        table below gives more details on key actual and target outputs to the
        end of April 2005.

           Training for the Social Economy – Core outputs to end April 2005

                                                           Target              Actual

           New beneficiaries                                1037               1040

           Beneficiary days                                 3138               1958

           Beneficiary hours                               18828              11748

           General training sessions delivered          No target set           190
           Bespoke training sessions developed
                                                        No target set            59
           and delivered

11.11   Part of the reason that targets for beneficiary hours have not been met
        is due to the way numbers were profiled. The original profile assumed
        that each new beneficiary would undertake three days‟ worth of
        training in the quarter that they joined up, when in fact it is more
        likely that they will take a series of courses over a longer time period.
        The project manager believes that targets will be met over the course
        of the project as earlier beneficiaries return to undertake more

11.12   The project manager also reports that there is a data input backlog,
        and that, in fact, more hours have been delivered but not yet
        recorded. WCC collects registration forms and evaluation data from all
        beneficiaries and data entry is a problem that was not foreseen at the
        start of the project. WCC has suggested employing a data entry clerk
        to help clear this backlog.

11.13   There have been no major barriers identified within the project so far.
        Minor setbacks include the fact that some courses have had to be

 40                               Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

        cancelled owing to low attendance – courses must have at least 8
        learners signed up in order to go ahead. In response to this, WCC has
        introduced a cancellation fee for learners, although courses themselves
        are free. One other issue was raised around WEA‟s delivery of courses
        – these are not clearly linked to the social economy in the same way
        that WCC‟s training is, and it was not clear what would and would not
        be acceptable for WEA to deliver under this contract. When reviewed
        by ELWa, it was found that WEA were delivering courses including
        „Healing through Colour‟ and „Introduction to Reike‟ that were not
        considered appropriate for the contract. These have now been

11.14   The most significant issue that WCC has faced in delivering the project
        has been managing clients‟ expectations. The WCC offer through the
        Training for the Social Economy project is slightly different to its
        previous training provision. One change is that now most courses are
        open and delivered to a range of people from different organisations,
        whereas before trainers went to individual organisations to deliver
        training. Learner evaluation will try and draw out whether this is seen
        by learners as an improvement; WCC hopes it will be seen this way,
        because the this change was introduced in order to help organisations
        learn from one another as well as from the tutor.

11.15   The biggest changes, though, are outwith the scope of the project.
        Previously, WCC had other funding streams that enabled it to offer
        marketing support and other small grants to social enterprises, which
        it can no longer do. Some clients have difficulty understanding that
        WCC no longer has funding for this.

        Project outcomes and fit with LCF objectives

        Outcomes relating to LCF objectives

11.16   The LCF objectives that the project has made the most clear
        contribution to so far are:

           innovation and new approaches: this is being achieved
            through the new ways in which courses are delivered to mixed
            groups from different organisations, and is likely to be achieved
            through the e-learning provision that is being developed;
           more/higher quality learning outcomes: WCC aims both to
            reach more learners (through expanding its offer to the whole of
            the social economy rather than focusing only on credit unions and
            co-operatives) and to deliver higher quality learning outcomes
            through its new approach to course delivery. Achievement of

 41                             Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

            these aspects will be researched further in the remainder of the

        Other outcomes

11.17   For the WCC itself, key outcomes from the project so far include
        broadening its client base. WCC previously reached clients who
        were mostly involved in co-operative businesses or credit unions, but
        now the client base is broader and includes other enterprises with a
        „social‟ element such as non-profit making companies, food co-
        operatives, and business successions. The project has also enabled
        WCC to pilot new provision and to improve its previous course

11.18   In terms of learner feedback, evaluation carried out by the WCC shows
        that almost all thought the course they attended was good or
        excellent. WCC analysed over 1000 course evaluation forms from
        learners on their courses (including new beneficiaries and those who
        had previously used WCC training). The evaluation forms also
        indicated that learners felt their level of knowledge on the subject
        studied had improved, and all who answered the question (88% of
        respondents) felt their learning objectives had been met.

11.19   WCC is also carrying out a longitudinal study through a questionnaire
        distributed 6-12 months after learners complete courses, while Si is
        carrying out visits to courses and qualitative research with learners in
        conjunction with the ELWa contract manager. Further outcomes will be
        appraised in the full mini-evaluation report on the project, which will
        be complete by March 2006.

        Mainstreaming and sustainability

11.20   The project bid did not include any detail about how the work would be
        sustained in the longer term. However, the project manager and
        contract manager are already thinking about how the provision could
        be sustained. One option would be for WCC to act as a sub-contractor
        to an organisation that could receive mainstream ELWa funding (such
        as WEA).

        Conclusions and learning

11.21   To date, the Training for the Social Economy project appears to be
        progressing well, with the project meeting its targets for beneficiary
        numbers and the concept of Learning Networks being taken up by
        organisations within the social economy. There may be an issue
        around delivery of beneficiary days and hours as targets do not appear

 42                             Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

       to have been met so far, although the project feels that at least part of
       this is down to a backlog in data entry rather than a shortfall in hours
       and days. This does have implications for ELWa contracts, however,
       suggesting a need for targets to be profiled differently in future as
       they seem to have been over-ambitious in terms of the numbers of
       hours each beneficiary is expected to undertake. For example, most of
       WCC‟s courses last one day, so to expect each beneficiary to
       undertake three days of training is perhaps unrealistic, at least in the
       short term.




12.1   The aim of the Vale Learning Network (VLN) project is to develop a co-
       ordinated and integrated network of community-based and other
       learning providers. It is delivering basic skills, ICT and other provision
       across community-based learning centres and with employers. It
       plans, develops and implements co-ordinated learning provision where
       information, guidance and learning opportunities are easily available to
       all and, in particular, to those groups that have traditionally been
       unwilling or unable to access learning opportunities.

       VLN projects and funding

12.2   The VLN grew out of the existing Vale Consortium for Education and
       Training (VCET). With Vale of Glamorgan Council (VOGC) as the lead
       body, the VCET bid for funding from the Learning Challenge Fund and
       from ESF (Objective 3), to develop the network and to run four
       projects that addressed particular gaps in provision that the VCET had

12.3   The Vale Learning Network received £500,000 from the Learning
       Challenge Fund to develop the network, and a further £400,000 from
       ESF Objective 3 for the four learning initiatives. The project is being
       run as one integrated project. This evaluation focuses on the ELWa-
       funded strand, but as the five projects are integrated there is some
       reference to the ESF-funded strands are well.

12.4   The VLN received LCF funding in July 2004, and funding runs out in
       December 2006. There was a delay in receiving the LCF funding, which
       meant that the project‟s timescale was reduced from 3 years to 2.5

 43                             Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

       years. The VLN could have continued using its LCF funding for a
       further 6 months, but would have lost out on some of its ESF matched
       funding if it did this. Instead it was decided to reduce the timescale for
       the whole project.

       Project objectives

12.5   The project has a set of amalgamated objectives for the LCF and four
       ESF projects. The objectives that relate most clearly to the LCF funded
       strand are as follows:

           to improve the quality and range of post-16 provision through the
            establishment of formal networking arrangements for all network
            providers where provision is planned, developed and implemented
            in partnership; and
           to engage the full range of providers in the development of the
            learning network.
12.6   In the long term, VLN members describe their goals as including joint
       development of corporate strategic plans, and being able to identify
       learning needs and gaps in provision, and commission provision to
       address them.

       Project management structures

12.7   The project is managed by a Project Team based at Barry College. A
       sub-set of the VLN, known as the Project Management Group (PMG),
       has been formed to oversee the project, and meets monthly. The PMG
       reports to the VLN Steering Group, which reviews and ratifies decisions
       that the PMG makes.

       Progress to date

12.8   The VLN does not have quantitative output targets related to the LCF
       funding, but has agreed a detailed project plan, setting out milestones
       and deliverables, which forms part of its contract with ELWa.

12.9   Activities undertaken so far in relation to the ELWa contract include:

           development of a VLN website for use by providers and learners;
           an annual conference for all VLN members to set priorities for
            the VLN and inform ELWa‟s Regional Statement of Needs and
           an audit of community learning provision across the Vale,
            including staff training needs analysis of providers, that will result

 44                             Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

            in a database of learning opportunities that providers can access,
            as well as highlighting gaps in provision and duplication;
           employment of IT staff who can deliver support to smaller
            providers in the Vale to ensure all can access the VLN‟s on-line
            resources; and
           production of e-newsletter and bulletins for members.

12.10   There has also been a range of other activity around addressing VLN
        objectives, including development of service level agreements between
        providers; discussions around sharing strategic plans; and even
        meetings of the Policy and Strategy sub-group to discuss rates of pay
        within the sector:

                   “We have discussed aligning pay and concessions, and
                   fees. We decided to take an audit of provision first, to find
                   out how rates of pay etc vary, and we will then look at
                   actions to be taken from this.” – PMG member

12.11   However, the project has not progressed as quickly as was set out in
        the project plan. The barriers to progression that PMG members have
        noted include:

           delays in contracting and receiving funding from ELWa;
           recruitment difficulties; and
           problems around one key staff member who has been on long
            term sick leave but could not be replaced because there was no
            additional funding to bring in another member of staff.

                   “Late arrival of employees has affected the critical path –
                   but this has been handled very well by the project team
                   and manager. Late arrival of money has been the problem,
                   people have had to be moved around to cover tasks.
                   Recruitment takes a long time and perhaps some money
                   should be released up front for recruitment.” – PMG

12.12   Further research with the project team and others involved in delivery
        of the project will be undertaken to see if there are other reasons for
        delays that the project could learn from – such as incorrect
        assumptions about how long aspects of the project would take or
        whether certain activities would be feasible.

12.13   One area of slight confusion is over the accountability of the project
        team to ELWa for delivery of the ESF-funded projects. ELWa funding is
        tied directly to network development, not to delivery of provision to

 45                             Shared Intelligence                      September 2005
                                                       Learning Challenge Fund Evaluation
                                                             First Round Evaluation Report

        learners. However, ELWa would not have provided the LCF funding if
        the ESF projects had not been part of the „package‟. The VLN also has
        one integrated delivery plan and it is hard to separate out activities
        that are funded under the different streams. As such, the ELWa
        contract manager is keen to see that the ESF projects are progressing
        according to plan, but feels that it is difficult to get a sense of whether
        targets that WEFO have set for the ESF projects are in fact being met.

12.14   Nevertheless, interviews with other members of the Project
        Management Group have shown that most members are happy with
        the quality and content of the information that they receive from the
        project manager before each meeting.

        Project outcomes and fit with LCF scheme objectives

12.15   As mentioned above, the key objectives of the ELWa-funded strand of
        the VLN project were to develop the provider network and ensure
        closer joint working, planning and delivery of provision. This fits
        closely with the LCF‟s objectives to promote a strategic approach at
        national, regional and local levels and to encourage collaboration
        between learning providers leading to improved learning networks.
        To test whether this was happening, members of the Project
        Management Group were interviewed for the evaluation (six interviews
        were complete when this report was written up).

12.16   PMG members were very enthusiastic about the value of the network.
        Consultees felt that the VLN was successful in reaching all sectors.
        The Steering Group, for example, consists of two representatives from
        each of 9 relevant sectors in 16+ learning provision, and
        representatives are encouraged to consult within their sector. The
        network also reaches a very broad span of providers. The network has
        been proactive in trying to draw in those who were not already
        members. As one consultee explained,

                   “I was concerned at a recent meeting to find out that there
                   are some learning providers who still aren’t involved, but
                   the VLN is making attempts to bring them in now.”

12.17   The VLN is perceived to be open and transparent, and a key strength
        is that larger providers working well with smaller providers.

                   “Everybody’s represented, though I think the bigger
                   providers are obviously going to be more involved, I think
                   rightly so… [but] there doesn’t seem to be any sort of

 46                              Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

                   “Sometimes we can get pushed to the edges – working
                   with our target group - but within the network, we feel we
                   have a voice.”

12.18   The project team was also seen to be actively helping VLN members to
        network with one another. This was seen as a key strength,
        especially by providers in the voluntary sector who otherwise felt they
        would have been isolated from the wider 16+ learning community.

                   “It’s very accessible. If you have a problem they will get
                   back in touch with you within a day, and if they can’t help
                   they’ll put you in touch with other providers who can.”

                   “We felt very isolated here before we became involved –
                   we needed to find a way to network with other resources.”

12.19   Leverage of further funding was also seen as a key benefit. The
        VLN cannot bid for funding itself, which is why VOGC became the lead
        body responsible for delivering the contract. However, individual
        partners have been able to bid for funding for various projects and feel
        their membership of the VLN has helped to attract this:

                   “When we do grant applications as a charity, the fact that
                   we’re part of a prestigious network really helps. Also, when
                   a partnership application is needed, that partnership is
                   already there.”

12.20   An important issue for the evaluation is to find out how much
        additional benefit the LCF funding has had. Compared with project
        funding, LCF funding for developing the network is large. However, the
        network existed previously (in the form of the VCET) so the evaluation
        is trying to find out what qualitative difference the additional funding
        has made. PMG members felt that the network had changed, even
        though its structure was mainly the same:

                    “It’s become more confident and people can see we do
                   have weight now, because we have funding.”

12.21   There is clearly added value for partners from being involved.
        However, more quantification of the benefits is needed if possible, and
        the evaluation will aim to draw this out and encourage the VLN project
        team to do the same in their evaluation work. In addition, the
        evaluation will look at the benefits of VLN membership for partners
        who are less closely involved, i.e. those who are not on the Steering
        Group or any of the sub-groups. The VLN project team is looking at
        carrying out evaluation work with learners themselves, as part of the
        work of the ESF-funded projects.

 47                             Shared Intelligence                      September 2005
                                                       Learning Challenge Fund Evaluation
                                                             First Round Evaluation Report

12.22   The learner projects that the VLN is delivering also fit well will the LCF
        objectives around widening participation and delivering more and
        high quality learning outcomes – though it should be noted that, as
        previously mentioned, these projects are funded through ESF
        Objective 3 rather than LCF.

        Mainstreaming and sustainability

12.23   Looking to the future, the VLN members consulted felt there should be
        greater opportunities for the VLN to play a part in planning and
        managing provision in the Vale. They felt there was greater scope for
        providers to specialise and that more needed to be done to avoid
        duplication and make sure provision was filling gaps and meeting
        needs. They also felt that the VLN would be able to assist ELWa in
        planning provision strategically in the Vale and would welcome greater
        powers for the VLN (and other CCETs) in doing this.

12.24   Individual partners in the VLN are also thinking about how additional
        funding can be drawn in to fund projects delivered through the VLN. As
        previously mentioned, the VLN itself cannot bid for funding so it is up
        to individual partners to act as lead organisations to administer

        Conclusions and learning

12.25   Consultees for the evaluation to date, who have represented a range
        of sectors (e.g. further education, voluntary and community sector),
        have all been very positive about the operation and outcomes of the
        Vale Learning Network‟s work so far. It has been difficult to separate
        the significance of network development (funded through LCF) and
        learning projects (funded through ESF), because the process of
        developing and delivering learning projects is key to development of
        the network itself. However, it is the strong view of consultees that the
        VLN has been strengthened by LCF funding and is now functioning
        effectively as an organisation bringing together learning providers to
        better meet needs of learners within the Vale of Glamorgan.
        Nevertheless, it is recognised that those who have contributed to the
        evaluation so far are closely involved with the VLN (all were on the
        Project Management Group) and thus may have a more positive view
        of it than learning providers who have not been so closely involved.
        Therefore, subsequent stages of the evaluation will explore the
        difference that the VLN has made to learning providers that are not
        part of the management structure of the VLN.

12.26   In terms of learning, one operational lesson that has emerged from
        the project so far is around the responsibility on the lead partner.
        As mentioned, the VLN itself cannot bid for funding, so the Vale of

 48                              Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

        Glamorgan Council‟s Lifelong Learning Department, as the largest
        partner, took on the responsibility for contracting with ELWa. The
        Lifelong Learning Department has since found that the contracting
        process has taken up a lot of resource within the department, as well
        as in the legal, finance and HR departments. For example, ELWa were
        not happy for the Council to have service level agreements with the
        project deliverers, but stipulated that they had to sign full contracts.
        These included ELWa and WEFO guidance so were extremely long. This
        process was very resource intensive for VOGC and had not been
        factored into the original project costs. The Lifelong Learning
        Department feels that this would have to be costed for in future.

12.27   Another problem that the original project bid did not account for was
        the possibility of a key staff member being absent for a long time
        through sickness. It was suggested that some budget could in future
        be set aside to cover additional staff time in the event of such an
        absence. If this budget was not used, it could be clawed back.

 49                             Shared Intelligence                      September 2005
                                                       Learning Challenge Fund Evaluation
                                                             First Round Evaluation Report



13.1   The aims of the Cydag project were to install video-conferencing
       equipment in bilingual and Welsh-medium schools, enabling them to
       offer greater choice to sixth form pupils for A and AS level students;
       and to instigate a partnership that would allow bilingual and Welsh-
       medium schools to share expertise and resources and develop
       methodology skills in flexible teaching.

13.2   The objectives were to:

          get agreement between 6 schools to establish the project;
          adapt rooms in 6 schools and install equipment;
          train staff in use of video-conferencing equipment;
          provide teacher training and sharing good practice;
          prepare teaching materials in 9 subject areas;
          provide 200 hours of teaching in each school in 9 subject areas;
          provide an evaluation report and exit strategy.

13.3   The equipment enabled the schools (in Bangor, Mold, Wrexham,
       Cardigan, Cardiff and Powys) to offer greater choice of subjects to
       sixth form pupils who would be able to receive lessons delivered by
       teachers located at any of the other schools in the group. The schools
       were added to the Welsh Video Network which was established to
       allow universities and colleges across Wales to share tutorials and

13.4   The Project Co-ordinator on behalf of CYDAG – the Association of
       Welsh Medium Schools, the head teacher of Caereinon High School in

13.5   The project ran from December 2003 until July 2005, and was
       allocated £448,000 of LCF funding.

       Progress to date

13.6   Shared Intelligence are planning to visit in September 2005 to advise
       on the end-evaluation of the video-conferencing project. The brief
       analysis of progress is based on some monitoring papers and

 50                              Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

       discussions with the ELWa contract manager and Head of Bilingual

13.7   When the project was initiated, certain problems were foreseen and
       were agreed as important challenges to address. These include
       differing timings of the school day, different timetable structures, staff
       changes and lack of commitment from students.

13.8   The ELWa officers believe the project has made considerable progress
       but issues remain in a number of respects:

           problems in achieving common timetabling
           inadequate level of management and co-ordination resources
           inadequate staff training and networking
           too much emphasis on capital aspects
           lack of clear leadership and sign-up

13.9   This project will be evaluated in September 2005 and will be
       considered in terms of sustainability and roll-out beyond the LCF
       funding period.

 51                             Shared Intelligence                      September 2005
                                                     Learning Challenge Fund Evaluation
                                                           First Round Evaluation Report


14.1   There are two additional LCF projects that have not been covered in
       the preceding sections. These are:

          Denbighshire Post-16 Learning Centres; and
          Wales Institute for Sustainable Education (WISE).

14.2   Denbighshire and WISE have not yet agreed contracts with ELWa, so
       LCF-funded activities have not proceeded to date. A brief description of
       these projects‟ intended activities is given below.

       Denbighshire Post-16 Learning Centres

14.3   Contract negotiations for the Denbighshire Post-16 Learning Centres
       project are still ongoing. Conditional agreement between ELWa and
       Denbighshire County Council was reached in February 2004, when
       both parties agreed to work together to find a solution to ensure that a
       project to create learning centres across Denbighshire would go ahead.
       The ELWa Finance and Performance Review Committee asked for
       assurances that:

          the project will not result in duplication of services already offered
           by others;
          it will deliver an expansion of learning in communities; and
          arrangements will be put in place to ensure the ongoing financial
           viability and sustainability of the learning centres.
14.4   It is expected that a contract will be signed during September 2005,
       once WAG approval is granted. Once the contract is signed, work will
       be carried out by the project (assisted by Si) to set baselines against
       which to measure progress and success.

       Wales Institute for Sustainable Education

14.5   ELWa‟s participation in this project was originally planned as a capital
       input of approximately £500,000 towards a £5 million project to create
       a major new educational facility at the Centre for Alternative
       Technology. However, it was decided that ELWa‟s role would be more
       appropriate if they were directly related to learning outcomes, so a
       revenue approach is now being taken forward. ELWa also decided that
       the project needed a stronger FE focus to sit alongside the original HE
       linkages and a condition has been set to develop FE links with Coleg
       Meirion-Dwyfor and Coleg Menai.

 52                            Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

14.6   Funding is now being considered for a 3-year period from 2005/06, but
       if contracting takes place in October 2005 it is unlikely that more than
       £70,000 will be spent in the current year. The contract manager in
       ELWa believes that ELWa funding will be less than the original
       £500,000. Year 1 funding is likely to be for marketing, equipment and
       some course development.

14.7   Project management might be an issue because CAT are heavily
       committed on other activity through the summer season. It is
       anticipated that the new facility will be “part useable” by late 2006 and
       might start delivering learning outputs by the end of year 2006/07.

 53                             Shared Intelligence                      September 2005
                                                     Learning Challenge Fund Evaluation
                                                           First Round Evaluation Report


15.1   The following projects are now finished, and the full mini-evaluation
       reports have been completed by Shared Intelligence:

          Achieving Closer Integration – University of Wales College
           Newport and Coleg Gwent: a feasibility study looking at possible
           models of integration for the two institutions;
          Accreditation Pilot – Prince‟s Trust Cymru: a pilot looking at
           ways in which the Prince‟s Trust‟s team programme could be
           accredited in order to be able to be eligible for mainstream
          Enable South West Wales – Neath Port Talbot College: a study
           into the needs of learners with disabilities in South West Wales,
           and into provision and gaps in provision for these learners; and
          TrainLink – Trac (Antur Teifi Cyf): a continuation of a pilot that
           aimed to set up a referral system between work-based learning
           providers in Carmarthenshire, in order to deliver a better service
           to employers and meet learners‟ needs more effectively. Through
           the pilot, employers also completed survey questionnaires that
           helped the TrainLink partners to find out more about employers‟
           needs in the area.

15.2   Each of these was a short project lasting between five months and one
       year. Enable and Achieving Closer Integration were conceived as first
       phases of larger projects, while TrainLink and the Accreditation Pilot
       were both conceived as pilot phases of future projects.

       Themes from completed projects

15.3   The four projects were all seen as largely successful in their own
       right. The project managers and stakeholders interviewed felt that the
       research reports drafted by the Enable and Achieving Closer
       Integration projects were of high quality. The Prince‟s Trust was
       successful in finding an accreditation route that was beneficial for
       learners as well as suitable to its training programme. Meanwhile, the
       TrainLink pilot met most of its target outputs, and helped to develop
       joint   working    between     work-based      learning providers    in

15.4   Nevertheless, none of these projects continued to the further stages
       that were envisaged when the projects started. Project teams at Neath
       Port Talbot College and UWN/Coleg Gwent felt very disillusioned after
       finding out their projects would not be taken forward, and consultees

 54                            Shared Intelligence                      September 2005
                                                     Learning Challenge Fund Evaluation
                                                           First Round Evaluation Report

       believed that their hard work had been in vain. There was also a sense
       that opportunities to learn from the projects were missed, because
       findings were not disseminated.

15.5   The main reasons why projects were not continued to further phases
       varied between projects:

          with the UWN/Coleg Gwent integration study, ELWa (and HEFCW,
           who also funded the project) felt that they could not support the
           two institutions‟ plans to undertake a formal merger;
          the main reason why TrainLink did not continue was because
           most of the partners felt that the time and costs required
           outweighed the benefits of the project, and also because training
           providers were facing budgetary restrictions and were unwilling to
           put funding into non-essential work;
          Enable was unable to secure funding for further work, partly
           because no further rounds of LCF were available (as had originally
           been proposed), and because no other funding streams would fit
           with the project‟s aims, for example in terms of geographical
           location; and
          The Prince‟s Trust had aimed to accredit its provision so that it
           could access mainstream funding, but at the time the project
           finished, ELWa had not finalised its new Planning and Funding
           System and could not guarantee that the Prince‟s Trust would in
           fact be able to access funding as a result of its pilot.

       Lessons from completed projects

15.6   The main lessons arising from the completed project evaluations were:

          that expectations needed to be more carefully managed by ELWa
           from the start of the project, so that projects did not assume that
           they would get ongoing funding;
          that projects needed to think through exit strategies from the
           start. When the outcome of the project could not be predicted
           (i.e. it was a pilot or research project), projects could have come
           up with alternative scenarios for future funding, and/or for
           learning from the experience; and
          that in future, evaluation findings and lessons could be usefully
           shared amongst ELWa staff and externally amongst LCF project
           promoters and other organisations.
15.7   The full mini evaluation reports will be made available on ELWa‟s

 55                            Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report


16.1   This section looks at the more general lessons on progress of the LCF
       as a scheme that have started to emerge from the mini-evaluations to
       date. This section covers:

           fit with LCF scheme objectives;
           monitoring and evaluation; and
           scheme management.

       Fit with LCF objectives

16.2   As mentioned in the introduction to this report, the key objectives and
       success criteria for the LCF were to:

           engender innovation and new approaches in learning provision;
           promote a strategic approach at national, regional and local
           encourage collaboration between learning providers leading to
            improved learning networks;
           improve access and widen participation;
           deliver more and high-quality learning outcomes from existing
            resources; and
           effectively mainstream successful projects.

16.3   This sub-section looks at ways in which projects have met these
       objectives. As noted in previous reports, not all of these objectives
       were specified as criteria to projects when they were putting their LCF
       funding bids together. For example, it was not necessary for projects
       to say how they would be innovative, or how they would mainstream
       their activities. Therefore, projects are not being „judged‟ against these
       objectives, but Si is nevertheless assessing the extent to which the
       scheme as a whole is achieving these.

       Innovation and new approaches in learning provision

16.4   Although „innovation‟ was not specified in the project proposal form, or
       used as a scoring criterion, there are nevertheless good examples of
       innovation in some of the LCF projects. For example, CBSS has already
       had success in embedding basic skills into activities that would
       normally be seen as leisure activities, and is continuing to develop

 56                             Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

       more courses along these lines. This is also the case with Cymad
       where Welsh for Adults courses are being offered in non-mainstream
       ways, covering practical activities like shopping.

16.5   Other examples include the Oaks project and the Learning Vale
       project, which are offering adult courses in innovative locations such
       as community venues and local primary schools. The CYDAG video
       conferencing project is innovative in providing sixth form subjects in
       Welsh medium. CYMAD has used innovative methods to raise
       awareness and demand for Welsh for Adults courses. TTFW has piloted
       a wide range of new learning methods for a very hard-to-reach market
       of small tourism managers and owners – using video, websites and
       mentoring. The evaluation of these methods will provide valuable
       learning for the tourism learning sector in Wales and beyond.

       Strategic approach at national, regional and local levels

16.6   There are a number of examples of how the LCF projects are taking a
       strategic approach to delivery, or enabling learning providers within a
       particular area or sector to be more strategic in the way they plan and
       develop provision. For example, TTFW works at national and regional
       level – the latter through the network of four Regional Tourism
       Partnerships, and the former through connections with People 1st, WTB
       and WDA. Canolfan Sgiliaith has been working with Fforwm and is
       aiming to increase its connections at national level with private
       providers and other stakeholders to have a greater strategic
       influencing impact as it moves forward.

16.7   Local strategic work is evident in CBSS and Powys, both working at
       County level to establish basic skills partnerships and strategies. These
       projects have worked from a knowledge of local needs to set up a
       consistent approach to delivery of basic skills provision and to ensure
       that providers know what each other are doing and are working to the
       same ends. Vale Learning Network, similarly, has used LCF funding to
       deliver projects that are seen as strategic priorities for the Vale by VLN
       members. The VLN also sees a key role for itself in encouraging larger
       providers to develop their own corporate strategies in line with each

16.8   Meanwhile, the projects that are focused around research outputs also
       aim improve capacity of providers to act strategically. For example,
       Merthyr LLC is aimed at generating a strategic approach to lifelong
       learning throughout the borough, with potential roll-out implications
       for the South East Wales area.

 57                             Shared Intelligence                      September 2005
                                                       Learning Challenge Fund Evaluation
                                                             First Round Evaluation Report

        Collaboration between learning providers

16.9    There is clear evidence of collaboration in a substantial number of LCF
        projects. CBSS, VLN, Canolfan Sgiliaith, Powys Basic Skills and the
        CYDAG school videoconferencing project are good examples. The
        TTFW has been approached by University of Glamorgan with a view to
        turning an online business course into a tourism-specific course that
        the University would write and make available through the whodoiask

16.10   However, setting up collaborative working has not always easy. CYDAG
        has found problems in co-ordinating timetables between the six
        schools to make the sharing of subjects possible. Canolfan Sgiliaith has
        found links with colleges and schools easier to establish than links with
        private training providers. Making more strategic contact through
        national representative bodies is an objective for Phase 2 and the
        pressure being brought by Estyn through their Common Assessment
        Framework is expected to help in this respect.

        Improving access and widening participation

16.11   Many of the LCF projects aim to improve access and widen
        participation, either through delivering to non-traditional learners,
        widening previous learner bases, or carrying out research that will
        enable bespoke provision to be developed where gaps currently exist.

16.12   A significant way in which LCF has enabled providers to widen
        participation is that learning does not have to be accredited. This is
        really important in bringing in „non-traditional‟ learners and those at
        the lowest end of the basic skills scale. Good examples of where this
        has happened are the CBSS, Learning Vale and Oaks projects. This is a
        wider issue than LCF, but one that several consultees have
        commented on. In some cases the bridge between non-accredited
        courses and progression to accredited courses is being successfully
        made. The Oaks project is nearing the achievement of its project
        targets for accreditation – in some cases by agreeing for WEA to take
        over successful groups of learners and run accredited courses for it.
        The idea of recognising partial accreditation – on a modular basis – is
        a good idea for recognising that bridge.

16.13   One of the key objectives of the Merthyr LLC project is the creation of
        a culture of lifelong learning throughout the Borough by developing a
        comprehensive curriculum plan that links opportunities and access at
        all levels.

16.14   Other projects such as Learning Vale and Oaks, while working largely
        at the community level, are achieving real positive learning outcomes

 58                              Shared Intelligence                      September 2005
                                                       Learning Challenge Fund Evaluation
                                                             First Round Evaluation Report

        with individuals and groups who have not engaged with learning in
        recent years.

        Delivering more and higher quality learning outcomes

16.15   In the original LCF guidance, there were two criteria relating to this
        objective. One was to „deliver more learning outcomes from existing
        resource levels‟ – i.e., projects should deliver better value for money
        than previous arrangements. The other was a „priority area‟ that
        specified that projects should „improve the quality and content of
        learning‟. These two areas have been amalgamated into one objective
        for the purposes of evaluating the LCF because there is some overlap
        between the two.

16.16   LCF projects are, on the whole, delivering provision that is one-off, or
        different from anything that had happened previously, so it is not
        possible to show whether they are delivering better value for money
        than previous provision. Nevertheless, there is clear evidence that
        some LCF projects are delivering more, and/or higher quality learning
        than was previously available. Wales Co-operative Centre is a good
        example, as it has not only broadened its client base, but revised its
        curriculum to be more effective. WCC is now delivering courses to
        mixed groups rather than groups from single organisations, so that
        participants can learn from each other as well as from the tutor.

16.17   Canolfan Sgiliaith is another example, where the Resources Panels has
        worked up new materials covering six subject areas and these are
        starting to be delivered in FE colleges and schools offering
        opportunities for accredited vocational Welsh-medium learning.
        Projects such as „Oaks‟ are achieving progression – sometimes passing
        their learners on to providers such as WEA who take them forward to
        higher quality and accredited provision.

        Sustainability and mainstreaming

16.18   The four completed mini evaluation reports did not make optimistic
        reading in terms of sustainability and continuity of those LCF projects.
        In each case the project had left some stakeholders disappointed,
        following decisions not to proceed to follow-up funding.

16.19   However, the research into continuing and recently completed LCF
        projects has led to more optimism. For example, the mini evaluation of
        Canolfan Sgiliaith is currently taking place, and it is evident that there
        is clear support at ELWa officer level for continuation funding to allow
        the project to move to a next stage. The Oaks project, which is still
        receiving LCF funding, has just been awarded ESF funding that will
        maintain its activities until 2007, as a confirmation of the success it

 59                              Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report

        has achieved under LCF. CBSS is thinking about sustainability and so
        is VLN – both have plans for continuation.

16.20   There are still some issues with sustainability and partner expectations
        that should be resolved by better communication between ELWa and
        the projects. For example, the Merthyr LLC project is suffering from
        lack of transparency and political uncertainty and the project
        managers believe that ELWa might be playing for time with no
        intention of funding the key strands of the emerging strategy.

16.21   There is still a sense from projects that the likelihood of continuation
        or the processes for agreeing follow-up funding were not made clear
        enough in the process of setting projects up at the start. As one
        stakeholder in the VLN project commented,

                   “LCF should be pump priming, but there wasn’t enough
                   emphasis on sustainability and what would happen after
                   the original funding finished. Also the process of getting the
                   money took too long. By the time the funding was received,
                   the needs could have changed completely – we were lucky
                   that in our case, they didn’t.”

16.22   Projects could also be looking more widely at how their approaches
        could be mainstreamed in other ways than securing mainstream
        funding for continuation of project activity. In other words, there are
        ways in which projects can influence the mainstream, even if project
        activities are not continued. This might include adoption by
        mainstream providers of a particular way of engaging one client group,
        or continuation of partnership working arrangements after project
        funding runs out. A key outcome for mainstreaming would be if LCF
        projects were able to influence ELWa‟s funding mechanisms in any
        way. The mini evaluations will continue to explore these aspects with
        projects and probe as to which working practices could be
        mainstreamed in absence of continued project funding.

        Monitoring and evaluation


16.23   As reported in the Baseline Report of January 2005, all but one of the
        projects are using the bespoke LCF monitoring system. Some projects
        feel that monthly reporting is a heavy burden, particularly for projects
        that are not involved in big volume training and work with
        beneficiaries. In most of these cases, ELWa contract managers have
        agreed that reporting should be on a quarterly, rather than a monthly

 60                             Shared Intelligence                      September 2005
                                                       Learning Challenge Fund Evaluation
                                                             First Round Evaluation Report

16.24   Generally, the LCF monitoring system seems to be working fairly well
        for project managers and contract managers, although if it were to be
        used for future schemes, it would be beneficial to consult project and
        contract managers further about how it could be improved (for
        example, by adding formulae to the MR1 Excel form so that it totals
        columns automatically). One contract manager has introduced a „traffic
        lights‟ system so that outputs are coloured green, amber or red
        according to whether they are on track – the project manager feels
        this is helpful and it is this type of modification that could be adopted
        across the piece if ELWa were interested in taking the monitoring
        system forward.


16.25   The projects‟ own internal evaluation activities vary quite considerably
        in terms of content, depth and scope. For example, at one end of the
        scale, Wales Co-operative Centre is doing extensive evaluation and
        Cymad has appointed its own external evaluation consultant to carry
        out an agreed programme. On the other hand some projects are doing
        little of their own evaluation work on an ongoing basis and would
        benefit from gathering more evidence of impact as they go along.

16.26   The level of evaluation work that is ongoing relates to projects‟
        previous experience of evaluation but also to the variable level of
        resources available to each project for evaluation. For example, the
        previous ELWa contract manager for the Learning Vale project
        commented that the project had no money in its budget for evaluation,
        so would need to do it “smartly” within the resources available for

16.27   Where evaluation work is not happening on an ongoing basis, Si has
        raised this with projects. However, there is a need to work closely with
        ELWa contract managers on this as Si does not have the power or
        remit to „require‟ projects to carry out evaluation tasks. This is an
        ongoing issue that Si will continue to address with contract managers
        and projects.

 61                              Shared Intelligence                      September 2005
                                                       Learning Challenge Fund Evaluation
                                                             First Round Evaluation Report

        Scheme management

        Role of ELWa contract managers

16.28   The interviews carried out so far suggest that LCF contract managers
        generally have close working relationships with the projects they
        manage. In several cases, LCF contract managers are involved in
        project steering groups or other formal mechanisms to help the
        projects to progress.

16.29   However, projects that involve ongoing delivery of tangible outputs
        (e.g. direct learning provision) seem to work more closely with their
        ELWa contract managers than those that are focused on carrying out
        research or studies. This has become a problem with some of the
        projects that are now completed, where it seems that ELWa did not
        take a strong enough role in steering project activities.

16.30   One further issue that has emerged is that, in many cases, contract
        managers have changed during the course of projects. This causes two
        main problems. First, it has had an impact on the management and
        monitoring of the projects, causing some of the projects to lose
        momentum. Secondly, it sometimes means that current contract
        managers do not „buy in‟ to the aims and objectives of the project, as
        they were not involved at the development or approval stage.

16.31   Some of the projects have mentioned that their contract manager‟s
        main concern is around delivery of contracts, achievement of outputs,
        milestones and spend targets – all things that the contract manager
        should be concerned about – but that there is little interest in the real
        achievements of the project, its significance, or its wider outcomes.
        This issue will be explored further with contract managers but it is
        accepted that personnel changes have been inevitable given the period
        of change that ELWa is going through currently.

        Contracting issues

16.32   Several projects faced delays in contracting, which were discussed in
        the Baseline Report drafted by Shared Intelligence in January 2005. As
        noted above, two projects have still not been able to agree contracts
        with ELWa.

16.33   In addition to the issue of delays, the relative inflexibility of contracts
        has been brought to our attention in some cases. Contracts are very
        detailed and projects are tied down to a comprehensive and detailed
        set of outputs and objectives. While this is undoubtedly to protect

 62                              Shared Intelligence                      September 2005
                                                       Learning Challenge Fund Evaluation
                                                             First Round Evaluation Report

        ELWa‟s investment and make sure that projects do not default on their
        commitments, it also means that there is little scope to change any of
        the scheduled activities, even if things turn out not to be working. This
        can be an issue in longer lasting projects where the project can feel
        they are likely to be held to a target output that has become less
        relevant as the project has developed. It also means that project
        managers and steering groups do not have much influence over how a
        project develops and responds to circumstances as it progresses. One
        further point about contracting and targets is that some projects feel
        that the wide range of targets is a problem. This is because it leads
        them to deliver across a plethora of activities rather than focusing on
        the key activities.

16.34   Several projects have found, as they progressed, that for various
        reasons they have not been able to deliver the activities that were
        initially planned. As projects were intended to be innovative, pilot
        projects, this is unsurprising. However, as a result of this, project and
        contract managers have had to spend a lot of time and effort
        renegotiating contracts, reprofiling outputs and spend, and getting
        approval for this at different levels within ELWa. Where projects have
        been able to revise their contracts, though, this has been very helpful
        to them. For example, in one case (Wales Co-operative Centre) the
        budget was revised to include marketing and reasonable catering costs
        for courses, where this had been overlooked in the original bid.

        Funding issues

16.35   Linked to the issues around contracting, some projects have faced
        difficulties over receipt of LCF funding. Delays in contracting first
        meant that projects received funding later than expected. Most
        projects were able to delay project starts or use other pots of funding
        to support LCF project activities in the short term. However, as
        mentioned in the section on Vale Learning Network above, this caused
        real problems for some providers who came close to „going under‟
        without this funding.

16.36   Some projects then experienced initial delays in receiving payment,
        even after contracts had been signed. One of the short-term projects,
        now completed, did not receive any payment from ELWa until the
        project had almost been delivered in full. Clearly, if this were to
        continue to happen in future, it would present a real risk for some
        contractors, especially those that do not have multiple streams of
        funding that can be shifted to support project activity in the short

16.37   Some of the projects have also faced uncertainty over whether funding
        could be carried over if project activities were delayed until the next
        financial year. Wales Co-operative Centre, for example, had to run a

 63                              Shared Intelligence                      September 2005
                                                       Learning Challenge Fund Evaluation
                                                             First Round Evaluation Report

        conference not knowing if they would receive funding for this, because
        the decision over whether spend could be carried over had not been
        made. Merthyr LLC also have a similar issue relating to underspend in
        2004/5, which was caused by delayed contracting. This amounts to
        £35,000 and the project would like to allocate this to carry out two
        further pieces of research but is having difficulties getting approval or

        Feedback from ELWa

16.38   The funding and contracting problems both highlight issues some
        projects have faced in getting feedback from ELWa, over and above
        that which the ELWa contract manager can provide. Several project
        managers have complained that contracting and funding decisions
        have taken far longer than they would have expected. This has also
        put contract managers in a difficult position, having to mediate
        between projects and the larger ELWa organisation.

16.39   Additionally, when projects have submitted research pieces to ELWa,
        feedback (for example on whether further funding could be granted)
        has taken a long time to come through. Coleg Gwent and University of
        Wales Newport, for example, waited nearly eight months before
        receiving formal feedback from ELWa on their „Achieving Closer
        Integration‟ report. Merthyr LLC waited for up to five months for
        feedback after submitting their key research reports in March 2005.
        Lack of a formal response hindered their ability to move forward
        towards their curriculum plan and they were concerned it could mean
        they were unable to meet their scheduled deadlines.

 64                              Shared Intelligence                      September 2005
                                                      Learning Challenge Fund Evaluation
                                                            First Round Evaluation Report


17.1   This First Evaluation Report brings the awareness and understanding
       of the ongoing LCF programme up-to-date as at the end of July 2005.
       It builds on the Baseline Report drafted at the beginning of 2005. The
       findings are based on consultation with the projects and with ELWa
       contract managers, assessment of monitoring reports and evaluation
       activities that have been undertaken by the projects and by Si.

17.2   The broad conclusions are that the LCF programme is being rolled out
       effectively on most fronts and that the projects are delivering activities
       and outputs in line with their contracts and targets. Management of
       the process is fair – but there is little evidence of a cohesive approach
       to LCF project management within ELWa. Much appears to depend on
       the commitment and interest of the contract manager and their line
       manager. The number of job switches has been a problem and some
       contract managers do not seem strongly engaged in the outcomes of
       the project, although all are carrying out their monitoring role
       effectively. Nevertheless, many contract managers have developed a
       real interest in their project and provide an effective mentoring service
       for their projects.

17.3   Project managers are strongly committed to delivering against their
       project contracts and are diligent in achieving relevant outputs. There
       are some good examples of ELWa achieving flexibility with reference to
       targets, reprofiling of funding and extension of end dates.

17.4   In the period between September and December 2005, we will be
       carrying out mini evaluations of completed projects such as Canolfan
       Sgiliaith and CYDAG, as well as assisting ongoing projects with their
       evaluation activity. This will involve Si carrying out interviews with
       strategic stakeholders and beneficiary research in some cases. The
       projects will be assessing the effectiveness of their activities and
       delivery strands. The Integrated Evaluation Plans will be implemented
       and updated through this period.

 65                             Shared Intelligence                      September 2005

Shared By: