Docstoc

Evaluation of Sector Strategies - Center for Energy Workforce

Document Sample
Evaluation of Sector Strategies - Center for Energy Workforce Powered By Docstoc
					State Sector Strategies: Get the
Most from Your Workforce
Development Investment
CEWD Summit
October 9, 2009
Indianapolis, IN
Erin Andrew, National Governors Association
Scott Cheney, PAROS Group
The most efficient way to produce
anything is to bring together under
one management as many as
possible of the activities needed to
turn out the product. – Peter Drucker
Agenda

What are Sector Strategies?

Why Sector Strategies?

Project Overview

Evaluation Examples
         Characteristics of Sector
               Strategies
• Address the needs of employers -- by focusing
  intensively on the workforce needs of a specific industry sector
  within a region over a sustained period of time, often concentrating
  on a specific occupation or set of occupations within that industry
• Address the needs of workers -- by creating formal
  career paths to good jobs, reducing barriers to employment, and
  sustaining or increasing middle class jobs
• Bolster regional economic competitiveness                 -- by
  engaging economic development experts in workforce issues and
  aligning education, economic, and workforce development planning
• Engage a broader array of key stakeholders --
  through partnerships organized by workforce intermediaries
           Sector Initiative Model

                                            Recruitment
                                             Services


                               Workforce
                             Intermediary


                  Business                   Support
Employers in an   Services                   Service      Job Seekers
                               Training
                                                s         and Workers
Industry Sector                Services
 State Sector Strategies: Common
      Elements Across States
1. Engaging leadership
2. Co-creating core principles
3. Promoting regionalism
4. Strategic use of funding
5. Data-driven decision making
6. Alignment of public resources and strategies
7. Capacity building
8. Promotion and positioning
9. Evaluation
10. Legislation
Agenda

What are Sector Strategies?

Why Sector Strategies?

Project Overview

Evaluation Examples
                    Results for Workers
Random assignment
evaluation of 1,014
individuals showed that
sector initiative
participants earned
18.3 percent—about
$4,500—more than
members of the control
group over the 24-month
study period.
Public/Private Ventures: Job
Training that Works: Findings from
the Sectoral Impact Study, 2009
           Results for Employers
A third-party evaluation of sector initiatives in
Massachusetts showed:
•   Turnover: 41% reduction
•   Re-work: 19% reduction
•   Customer complaints: 23% reduction
•   Companies who said partnerships with other
    companies were valuable: 100%
      Systems Change Impact
Sector initiatives address key problems for:
  – An industry’s businesses
  – The industry’s workforce and/or potential
    workforce
By doing so, they gain leverage to make systems
change in the areas of:
  – Workforce and economic development
    (including education/training, support
    services, and business services)
  – Public policy
  – Industry practice
          Benefits to Regions
–Decreased skill and labor gaps
–Retention, expansion, and attraction of higher
 wage employers
–Reduction of poverty and unemployment
–Increased regional cooperation and collaboration
–More efficient and effective use of public
 resources
Agenda

What are Sector Strategies?

Why Sector Strategies?

Project Overview

Evaluation Examples
   Accelerating State Adoption of
     Sector Strategies Project
• Purpose: Advance regionally targeted industry
  strategies as a state strategic framework around human
  talent development
   – Includes increasing state policy support for sector initiatives that
     focus on low-income and at-risk workers
• Four-year project in two phases
• Partners:
   – National Governors Association (NGA) Center for Best Practices
   – National Network of Sector Partners (NNSP)
   – Corporation for a Skilled Workforce (CSW)
• Funders:
   – Charles Stewart Mott Foundation
   – Ford Foundation
   – Joyce Foundation
             Project Tracks
• Policy Academy
  – MD, MS, NH, NV, NY, WI
• Learning Network
  – AR, CO, GA, IL, MA, MI, MN, NC, OK, OR,
    PA, WA
• Knowledge Exchange
  – Website: www.sectorstrategies.org
  – Virtual Policy Academy
  – Toolkit
              Policy Academy
• States in the Policy Academy have generally:
   – Secured participation of leadership in the
     Governor’s office and in key state agencies
   – Formed multi-agency teams, often including
     regional sector initiatives
   – Developed principles or a policy framework for a
     state sector strategy
   – Created and started on an action plan
• Example: Wisconsin
  Learning Network: Quick Facts
• Over $150 million dollars have been invested in sector
  strategies across the 12 states, ranging from $500,000 in
  North Carolina to $65 million in Arkansas (primarily
  Career Pathways) for an average investment of just over
  $13 million per state.
• The majority of states in the Learning Network have
  been funding sector strategies for the past three to
  four years, with Massachusetts particularly active since
  2000
• All 12 states have had or plan to have competitive
  RFP’s to promote sector strategies in their state.
  Learning Network: Quick Facts
There are over 400 “local sector partnership projects”
spanning 15 industries funded and supported by the 12
states in the Learning Network, an average of just over 36
per state.
Most common industries are:
• Manufacturing
• Health Care
• Biotechnology / Bioscience
• Logistics
• Agriculture and Food Production
• Energy – Traditional and Renewable
Agenda

What are Sector Strategies?

Why Sector Strategies?

Project Overview

Evaluation Examples
        Evaluation of Sector Strategies:
        What We Know Matters to States
Impact on Jobseekers and Workers                          Impact on Employers and Industry
    • Increased availability of good jobs                 • Shared costs and risks
    • Improved working conditions                         • Increased availability of skills training
    • Expanded work supports                              • Guidance on ways to improve human resource
    • Increased opportunities for education and             practices
      training for high-demand occupations                • Reduced turnover
    • Increased employment and wage gains


Effectiveness of the Partnership                          Systems Change
•       The right partners “at the table”                 •   Changes in how education, workforce, economic
•       Industry challenges identified                        development and employers work together
•       Appropriate solutions designed/implemented        •   New social and business supports
•       Plan of action, road map, goals and outcomes      •   Changes in public policy
•       Partners agree the Partnership is valuable        •   Changes in employer practices
•       New and leveraged funding

Two Notes: 1) NGA White Paper and Evaluation Framework offers full detailed discussion on these categories;
2) a 5th category of growing interest is “impact on community” (such as reduced poverty/unemployment)


                                                                                                              19
         Is There One Formula for
                Evaluation?
First, can we fairly compare        Second, do we assume training
outcomes? Consider . . .            outcomes? Consider . . .
• Over 1,000 healthcare service     • A small manufacturing sector
  workers trained every year by       partnership that focuses on
  a metropolitan sector               career awareness
  partnership                       • Or an energy partnership that
• About 130 students per year         develops skills standards (that
  trained and placed in               can be used to develop
  employment by a solar               training) but that primarily is
  installation sector partnership     valuable to align titles and HR
• 3 Journeyman apprentices            selection criteria across
  supplied to Grand Coulee Dam        employers
  as a result of a Power          No single formula, but there are
  Generation partnership          templates to capture some consistent
                                  data across partnerships.
                                                                   20
From Washington State
        From Massachusetts
                       Annual Report August 2009*
Purpose: To provide an opportunity for each project to reflect on its progress toward meeting its goals
and share what is learned with Commonwealth Corporation. These reports offer a means to learn more
about how projects are unfolding - both at the level of individual projects and across the WCTF initiative

Category of                           Type of Information Requested
Performance
Part I: Quarterly Update              Describe activities during last Qtr, successes, innovations, challenges, next steps
Part II: Program Goals (of            # participants; # un-/underemployed; # incumbent; #                On Track? Y/N
jobseeker or worker participants)     completers; # new credentials; # wage increase; other              Will reach goal when?
Part III: Outreach, Recruitment       Describe any successes/challenges, changes you plan to make within each area. On
and Selection (of participants)       scale of 1-5, rate your success with each.
Part IV: Training                     How are providers selected? How do they participate? Please provide your assessment
                                      of training to date. What changes will you make?
Part V: Participant Supports          What types of supports do your participants need (paid release time, coaching, case
                                      management, tutoring, childcare, etc)? Describe challenges and changes.
Part VI: Partnership                  List partners, their roles and indicate in which types of activities they participate.
                                      Describe benefits for partners, challenges, future partner contributions.
Part VII & VIII: Employer             How are employers involved? Is engagement low, medium or high? Using the “Getting
Engagement and Biz Impact             Started” plan, what baseline and impact data do you have?
Part IX: Products of Project          Describe progress on contractual deliverables. Describe other products such as new
                                      curriculum, new certificate program, new partnerships, others.

                               *Note: Adapted and condensed from 10-page WCTF – Annual Report, Round Two, Year 1,
                               August 2009
         Some Evaluation Resources*
Evaluations of Sector Initiatives
Targeting Industries, Training Workers, and Improving Opportunities: Final Report of the Sectoral Employment
Initiative, Public-Private Ventures, 2008: http://www.ppv.org/ppv/publications/assets/263_publication.pdf

BEST (Building Essential Skills through Training) Benefits: Employer Perspectives, Volume 2, Issue 4,
Research and Evaluation Brief, Commonwealth Corporation, 2004:
http://www.commcorp.org/researchandevaluation/pdf/ResearchBrief2-04.pdf

Benefits of a Sector-Based Approach, Volume 2, Issue 3, Research and Evaluation Brief, Commonwealth
Corporation, 2004: http://www.commcorp.org/researchandevaluation/pdf/ResearchBrief2-03.pdf

Performance and Evaluation Models for Sector Initiatives
An Evaluation Framework for State Sector Strategies, product of 11-state project of the National Governors
Association, The Corporation for a Skilled Workforce and the National Network of Sector Partnerships, 2008:
http://www.sectorstrategies.org/system/files/EvaluationFrameworkWhitePaper.pdf
Evaluating Industry Skill Panels: A Model Framework, Commissioned by the Washington State Workforce
Education and Training Coordinating Board, Corporation for a Skilled Workforce with the Paros Group, June
2008: http://www.wtb.wa.gov/Documents/2008SkillPanelEvaluationReport.pdf

Business Value Assessment for Workforce Development Organizations: Handbook. The Aspen Institute –
Workforce Strategies Initiative, 2005: http://www.aspenwsi.org/BVA/BVA_HAND11-22FINAL.pdf


                          *All available at www.sectorstrategies.org
      Field of Evaluation is Ever- Evolving
•   Since March ‘09, six states in the project (MI, MA, PA, WA, IL, WI) convened four times via
    conference call to discuss key issues in sector strategy evaluation
•   Short (but rich) summaries of each call are on the blog at www.sectorstrategies.org
•   Topics for discussion have included:
     – “Systems Change” can be a nebulous, confusing term; there’s a need to de-mystify; concrete
        indicators of systems change already exist within our repertoire of sector evaluation, including
        career ladder development, changed HR protocols, new certificate/degree programs, others.
     – A “lessons learned” discussion about measuring impact on employers from MA’s experience in
        the healthcare/long-term care industry; Common data points: turnover, vacancy rates,
        reductions in agency fees – i.e. for firm to contract out for skilled work –, and overtime costs.
        Lesson learned: must collect qualitative testimony in addition to quantitative because latter not
        always available, and certainly not quickly.
     – Measurement Consistency across sector partnerships: WA takes the “choose 3” approach; not
        perfect. Also need common definitions for indicators (such as reduced cost of turnover). PA
        drew heavily from the NGA projects framework to develop the 2008-2009 annual report. Two
        other issues: how do we get honest results? How do we account for differences across
        industries?
     – PA put a healthcare impact report together summarizing efforts and outcomes of 13 healthcare
        partnerships – this is potentially a powerful way to capture impact more specifically.

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:1
posted:4/17/2013
language:English
pages:24