Roi on Training Feedback Template - DOC

							                                       Initiative Overview
 1. Name              GWDC ROI Initiative – Phase I
 2. End Date          June 25, 2010
                              Name                Phone #               E-mail Address
                      Bryan Lindsley           651-259-7572           bryan.lindsley@state.mn.us
 3. Sponsor
 4. Project                                       651.259.7581        cristine.leavitt@state.mn.us
                      Cristine Leavitt
    Manager
 5. Process           Cheryl Kartes                                              kartes@aol.com
    Facilitator
 6. Committee       Paul Anton                                                  Paa2@wilder.org
    Members         Art Berman                                    ahberman@twincitiesrise.org
                    Mark Brinda            612-673-6231 mark.brinda@ci.minneapolis.mn.us
                    Susan Carter                                  Susan.carter@so.mnscu.edu
                    Carol Dombek                                     carol.dombek@state.mn.us
                    Jeremy Estenson                                jestenson@mnchamber.com
                    Martha Hegewisch                           martha.hegewisch@state.mn.us
                    Richard Joerger                             Richard.joerger@so.mnscu.edu
                    Randy Johnson                                              rjohnson@wfdi.ws
                    Susan Lindoo                                      susan.lindoo@state.mn.us
                    Nick Maryns                                  Nicholas.maryns@state.mn.us
                    Devon Meade                               Meaded@unitedwaytwincities.org
                    Anne Olson                                          aolson@mncounties.org
                    Brian Paulson                            paulsonb@unitedwaytwincities.org
                    Raymond Robertson                                robertson@macalester.edu
                    Mary Rothchild                               mary.rothchild@so.mnscu.edu
                    Deb Serum                                           deb.serum@state.mn.us
                    Joann Simser                                  joann.simser@so.mnscu.edu
                    Todd Wagner                                       todd.wagner@state.mn.us
                    Luke Weisberg                                      Weisberg@lukeworks.net
 7. Scope, Purpose, Objectives, and Deliverables (to be refined by committee members)

     Scope: Minnesota workforce development training and employment programs.

     Purpose: To pool our shared knowledge and expertise to create through consensus a
     standard return on investment methodology for workforce training and employment programs
     that meets the objectives below by June 2010.

     Objectives:
        1. ROI Committee members agree on what success looks like for the ROI Initiative and
            are committed to the success of the initiative and the objectives.
        2. Clarity on how and who will make decisions on project issues.
        3. Collect and share information on ROI methods.
        4. Develop a standard ROI methodology for workforce development and training
            programs that best satisfies the following criteria:
                a. Manageable (easy to apply/administer; minimizes work and training for service
                   providers to implement)
                b. Useful (relevant and timely; helps providers improve programs, helps
                   legislators and others understand program impacts)
                c. Understandable (transparent; easily understood by experts, users, and the
                   general public – assumptions and limitations defined, rules of use defined)
                d. Credible (trusted; measure is seen as accurate/credible by experts, users, and
                   the general public – relies on verified data as much as practical, process
Adopted: August 13, 2009, Last Updated: 9/14/09                                             1
                                           Initiative Overview
                   integrity - ensures data and model integrity of applications),
               e. Adaptable (measure can be used by different programs/providers)
               f. Sensitive to change (collect and report data at a frequency that detects
                   significant variations)
        5. Develop an action plan for each phase of the project:
                Phase I: Evaluate existing ROI models and develop a standard model
                Phases II - pilot test, evaluate, and refine the ROI measure
                Phase III: Make policy and implementation recommendations as part of the
                  January 2011 GWDC Report to the Legislature. Include in the report:
                         i. Recommendations for communicating ROI results, including language
                            on the appropriate uses and limitations of ROI measures
                        ii. Recommendations for implementing a ROI workforce development and
                            training methodology statewide.
                       iii. Identify training and other implementation costs.
        6. Team members feel that their perspectives were understood, that their involvement in
           the project was a valuable use of their time, and that the process to achieve the
           objectives was conducted efficiently and effectively.

      Deliverable(s): 1) ROI methodology/template (assumptions, standard process, unique
 application components/discussion), 2) Introductory page that provides background/context on
 methodology (e.g., what the ROI methodology is, who the intended audience is for the measure,
 value of the measure, how to apply the methodology)
 8. Timeline (milestones and dates)

 Phase I
  Timeframe          Activity
  Jun-Aug 2009       Launch ROI Initiative, form committee, prepare project plan and action plan
  Aug-Mar 2009       Review and evaluate ROI models
  Jan–Apr 2010       Create standard ROI methodology
  May-Jun 2010        Share recommended ROI methodology/approach with others
                      Gather feedback on methodology
                      Adjust methodology
                      Begin to plan Phases II & III
  June 25, 2010      Adopt standard ROI methodology

 9. Vision (what success will look like – what good results are we striving for?)
 Our vision of success is pooling our shared knowledge and expertise to create through
 consensus a return on investment methodology that is:
  Useful (helps providers improve programs, helps legislators and others understand program
    impacts),
  Transparent (easily understood by experts, users, and the general public – assumptions and
    limitations defined, rules of use defined),
  Trusted (measure is seen as credible by experts, users, and the general public – relies on
    ―real‖ data as much as practical, process integrity - ensures data and model integrity of
    applications),
  Adaptable to different programs/providers, and
  Easy to apply/administer (minimizes work and training for service providers to implement).
 10. Performance Measures
 Achieving ROI Initiative Objectives as defined above.

 11. Committee Member Engagement Principles (Ground Rules) How we will conduct
Adopted: August 13, 2009, Last Updated: 9/14/09                                               2
                                         Initiative Overview
 ourselves and work with each other

     1.   Contribute your thinking and experience
     2.   Listen to understand
     3.   Connect and build on ideas
     4.   Set cell phones on silent, check messages during breaks
     5.   Communicate your comments, questions, or concerns
     6.   Stay on topic and on time
     7.   Fulfill Participant Roles and Responsibilities

 13. Context Boundaries of the project
             Time Participating in a 2 hour meeting every 4-6 weeks and up to three hours of
     Commitment: work between meetings. Annual individual time commitment: (30-60 hours)
                         Participant              Role And Responsibilities
                         Sponsors:                Serve as the official leader, chair, or chief
                         Bryan Lindsley,          operating officer for the project. Some specific
                         GWDC Executive           responsibilities include:
                         Director                   1. Determine the purpose, goal, scope of the
                                                       work, and participant roles and
                                                       responsibilities;
                                                    2. Designate project manager;
                                                    3. Fund meeting basics;
                                                    4. Invest the necessary time;
                                                    5. Provide sufficient resources (e.g., support
                                                       services);
                                                    6. Invite participants;
                                                    7. Provide needed background information and
                                                       direction
                                                    8. Define how decisions will be made
                         Project Manager:         Manage the ROI Initiative so that it achieves its
                         Cristine Leavitt         purpose and objectives. Some specific tasks
   Member Roles                                   include:
              and                                   1. Conduct research and analysis
 Responsibilities:                                  2. Work with sponsor and committee members
                                                       to develop and implement a project plan and
                                                       action plan
                                                    3. Prepare meeting materials (may include
                                                       agendas), and meeting summaries
                                                    4. Provide progress updates
                         Process Facilitator      Coordinate meetings to assure fair, efficient,
                         Cheryl Kartes            timely, and productive meetings. Some specific
                         (Cristine Leavitt for    tasks include:
                         Kick-off mtg only)         1. Review project participants’ roles and
                                                       responsibilities
                                                    2. Design a fair and empowering process
                                                    3. Develop and enforce ground rules
                                                    4. Prepare meeting agendas and facilitation
                                                       materials.
                                                    5. Facilitate ―productive‖ meetings – adapt the
                                                       process to meet evolving needs and
                                                       changing circumstances (see list on p. 180 of
                                                       ―Clues to Achieving Consensus‖ by Mira P.
Adopted: August 13, 2009, Last Updated: 9/14/09                                                3
                                   Initiative Overview
                                                 Hanson)
                                              6. Mediate conflicts
                                              7. Remain neutral and objective, but at the
                                                 same time challenge the team to achieve
                                                 more
                   Committee Support: Sustain the lean transformation by ensuring
                   Shannon Scheurich        implementation of the new standard of work
                                            process and sustainability practices. Some
                                            specific responsibilities include:
                                              1. Set meeting dates and secure meeting
                                                  space
                                              2. Coordinate food and beverage
                                              3. Ensure that meeting materials are available
                   Committee Members: Contribute your expertise and talent to achieve
                   Paul Anton               the ROI Initiative purpose and objectives. Some
                   Art Berman               specific responsibilities include:
                   Mark Brinda                1. Share pertinent information, expertise, &
                   Carol Dombek                  opinions at meetings.
                   Martha Hegewisch           2. Serve as ambassador of ROI information.
                   Randy Johnson              3. Represent unique ―stakes‖ or perspectives of
                   Susan Lindoo                  partners, customers, stakeholders
                   Nick Maryns                4. Listen to understand
                   Anne Olson                 5. Build upon the ideas of others
                   Brian Paulson              6. Develop Committee relationships
                   Raymond Robertson          7. Participate in consensus deliberations
                   Mary Rothchild             8. Promote collaborative solutions
                   Deb Serum                  9. Respect collective and individual efforts
                   Todd Wagner                10. Complete Committee assignments (e.g.,
                   Luke Weisberg                 share and review ROI methodologies)
                                              11. Participate in Committee events (e.g.,
                                                 meetings, conference calls, etc)
                   Stakeholders/            Review and comment on proposed ROI
                   Interested Parties       methodology.
                   WDD program
                   directors, Mark C.
                   Larson (DOT)…
    Assumptions: see ROI Project Overview

       Strengths:       Experience with ROI measures
                        Diversity of Representation
                        Private sector driving measure
                        Openness/willingness to share/ willing to move forward
                        Shared purpose
                        Some private sector experience
                        Have other experts to invite
     Weaknesses:        Private sector engagement on team
                        We represent ―insiders‖
                        Politics—can be ―written off‖
                        Team needs other perspectives for broader/more diverse view
                        Data assumptions (no perfect model, need more information)
                        Lack of data on self-served customers (i.e., Wagner Peyser customers

Adopted: August 13, 2009, Last Updated: 9/14/09                                            4
                                        Initiative Overview
        Benefits:  Value to legislators; taxpayers
                      Enrich the policy conversation—clarify meaning/content
                      Information for philanthropic uses/help raise funds
                      Leverage additional resources
                      ROI as a way to compare value of programs
                      Explore different models and values and how to frame key points
                      Help us in the other work we do
 Dangers/Threats:  May motivate some to report false data - manipulate data
                      Go out and get feedback
                      Promoting conversations that still miss each other
                      Some programs won’t want to reveal their assumptions/won’t buy in
                      Political alienation to plan ahead
                      Getting real data on public assistance costs to lower assumptions
                      Invite legislative analyst to participate in ROI project
 14. Communication Requirements
    - Initiative launch message/invitation – Bryan Lindsley
    - Project Plan and Action Plan – Committee/Cristine Leavitt
    - Meeting agendas and summaries – Cristine and Cheryl Kartes




Adopted: August 13, 2009, Last Updated: 9/14/09                                            5