Program Design & Evaluation Matrix by u9GScg

VIEWS: 0 PAGES: 7

									                                                                   Blue Cross and Blue Shield of Florida Foundation
                                                                               EMPOWERMENT EVALUATION MATRIX

Introduction
The Blue Cross and Blue Shield of Florida Foundation (BCBSF Foundation), the philanthropic affiliate of Florida Blue, supports community-based solutions that address the many different health care challenges
facing Florida both today and in the future. The mission of the BCBSF Foundation is to improve the health and well-being of Floridians and their communities.

To ensure that its philanthropic grantmaking has the greatest positive impact, the BCBSF Foundation places a tremendous value and importance on its grantees’ abilities to track, measure and evaluate the
progress of their programs and services. Therefore, the BCBSF Foundation has incorporated the following evaluation criteria (among others) into its grant application review process:

       A demonstrated commitment to a detailed, high-quality planning process that engages the necessary resources and leadership and includes a credible evaluation strategy; and
       Potential favorable impact with:

                o   An expected significant outcome;
                o   High-quality intervention(s);
                o   Number to be served relative to goals and resources;
                o   Likelihood to influence other organizations;
                o   Possible chance for positive systemic change; and
                o   A practical evaluation plan.

The BCBSF Foundation uses the “Empowerment Evaluation” approach to planning and evaluation for its grantees. In addition to describing projected outcomes and results, as well as measurement and
evaluation processes in the “Program / Project Narrative” section of the IMPACT grant application, the BCBSF Foundation also requires applicants to complete and attach an Empowerment Evaluation Matrix to
their applications.

Please carefully read the “Tips” section before completing the Empowerment Evaluation Matrix. Remember to save this document to your computer often so as not to lose changes.

Overview of Empowerment Evaluation
Pioneered in the 1990s by Dr. David Fetterman, Empowerment Evaluation is the use of evaluation concepts, techniques and findings to foster improvement and self-determination. Although it can be applied to
individuals, organizations, communities and societies or cultures, the focus is usually on programs. As Fetterman explains in Foundations of Empowerment Evaluation (2000), “Empowerment Evaluation was
designed to help people help themselves and improve their programs using a form of self-evaluation and reflection. Program participants – including clients – conduct their own evaluations; an outside evaluator
often serves as a coach or additional facilitator depending on internal program capabilities.” Empowerment Evaluation was also designed to mainstream evaluation as part of the planning and management of the
program/organization (Fetterman 2000).

Empowerment Evaluation is based on the premise that people who design, implement, staff and consume the program’s services are in the best position to evaluate outcomes. According to Dr. Barbara Morrison-
Rodriguez, president and CEO of BMR Consulting and accomplished research methodologist, “The more that people engage in the act of conducting their own evaluations the more likely it is that they will find the
results credible and act on the recommendations.” Dr. Morrison-Rodriguez further states, “Empowerment Evaluation can help you clarify your values. It can help you to know if your activities are reaching your
intended groups. Empowerment Evaluation uses evaluation concepts to develop skills for on-going self-assessment (build capacity) that can be used to obtain and maintain funding and support (evidence of
effectiveness). It documents your efforts and allows you to inform yourself and others about what did and did not work. Additionally, Empowerment Evaluation utilizes a strengths- or assets-based approach that
can guide policy and inform practice.”



                                                                                                        1
Tips for Completing the Empowerment Evaluation Matrix
Beginning on page 4 of this document, you will find four (4) Empowerment Evaluation Matrix worksheets, one (1) per page. While you do not need to complete all four worksheets, please try and limit one goal per
worksheet and one objective per row. Please feel free to add additional rows as necessary.

Below, please find the bolded headings and corresponding definitions of the various sections of the Empowerment Evaluation Matrix worksheets you need to complete. Underneath each heading are bulleted tips
for completing each section of the worksheet.

The main idea or concept behind the Empowerment Evaluation process is for organizations to collectively define – with the input of program participants / clients – what everyone involved in the program / project
(i.e., members of the evaluation team) feel the intended outcomes of the program should be. Therefore, to best complete the Empowerment Evaluation Matrix, you (or your group) may want to work backwards by
first stating the outcomes / results you hope the program / project activities will achieve.

NAME OF ORGANIZATION

       Please state the name of your organization at the top of each Empowerment Evaluation Matrix worksheet.

PROGRAM / PROJECT TITLE

       Please list the program / project title at the top of each Empowerment Evaluation Matrix worksheet.
       Be sure that the title on each Empowerment Evaluation Matrix worksheet matches the program / project title on your grant application.

GOALS: Specific aims of a particular program or project consistent with the mission of the organization.

       Goals should be clearly stated at the top of each Empowerment Evaluation Matrix worksheet.
       Your Empowerment Evaluation Matrix should contain a minimum of one (1), but no more than four (4), program / project goals.
       If you have more than four (4) goals, please state only the four (4) most important goals.
       Goals relating to different aspects of the program / project should be stated on separate Empowerment Evaluation Matrix worksheets [i.e., one (1) goal per worksheet].
       Goals should address the outcomes your program / project hopes to achieve and NOT the activities related to the program / project.
       Goals should address how your program / project will benefit either your clients (i.e., target population) or your organization.
       Goals should be realistic given the scope of the problem / need, characteristics of the target population, duration and funding level of the program / project.
       The goal(s) of your program / project should be consistent with the overall mission of your organization.

    Example: To increase access to low- or no-cost, quality dental services for homeless adults in XYZ counties.

PROCESS OBJECTIVES: These are the very large “buckets” of work that need to be performed to achieve the stated goal.

       Process objectives state, in relatively broad terms, the various actions you will take to achieve the desired outcome(s) associated with a particular goal.
       Please list one (1) process objective per row on each worksheet.
       Please state one (1) or more measurable and appropriate (logical) process objectives for each or your program’s / project’s stated goal or goals.
       Feel free to insert additional rows on any / all Empowerment Evaluation Matrix worksheets.

    Example: Recruit certified dental professionals to provide dental exams and cleanings.



                                                                                                           2
ACTIVITIES: In essence, these are the individual steps that need to be taken in order to “fill” / accomplish the larger “buckets” of work / your process objectives.

       In the ACTIVITIES column, please separately list each task your staff will perform to implement and achieve the corresponding process objective.
       Stated activities should be logically linked to the accomplishment of one (1) or more process objectives.

    Examples: Obtain list of dental professionals from local dental society chapter. Contact by phone local dental offices and professionals. Develop and mail recruitment letters to dental professionals. Secure
    signed contracts and/or memoranda of understanding from local dental professionals.

TIME FRAME: The date(s) when specific activities and / or services are expected to occur and / or be completed.

       Each stated activity (or group of like activities) should have its own start and stop / due date.
       Dates may be expressed in terms of days, months or even years.
       For those activities that will continue throughout the life of the program / project, please simply list these activities as “on going.”

REQUIRED RESOURCES: The skills, staff and other resources needed to carry out each stated activity.

       The required resources needed to perform activities also should be reflected in the program / project budget.

RESPONSIBLE PERSONS: The person(s) who will oversee or execute the activities relative to the stated process objective(s).

       Specific individuals or position titles are indicated for each activity.
       There should be a match between the Responsible Persons and the budget.

OUTCOMES: The desired changes in the individual client(s), community health status, organization status or health system status that the program / project hopes to achieve as a result of accomplishing the
process objectives.

       Outcomes should be stated as numerical performance targets that express a desired degree of change (expressed as a percentage).
       Stated outcomes should describe how and the degree to which the status quo will change by using such action word as increase, decrease, improve, gain, etc.
       Outcomes should include clear performance baselines whenever possible.
       There may be multiple desired outcomes for a process objective and that is OK. Similarly, two or more process objectives may share the same outcome(s), and that’s OK, too.
       The target population of the outcome should match the target population of the process objective.
       Outcomes should be reasonable given the characteristics and needs of the target population and the level of the intervention.

    Example: To increase the number of dental professionals providing low- or no-cost exams and cleanings to homeless adults in XYZ county by 20% (baseline: 100 dental professionals; desired outcome: 20
    new dental volunteers for a total of 120).

INDICATORS AND TOOLS: Specific measures and / or benchmarks used as evidence that outcomes have actually been achieved.

       Indicators are valid and reliable measures of outcome variables.
       Tools refer to the processes and / or devices used to track and measure the indicators.
       Indicators should be appropriate and specific to each stated outcome.
       When available and feasible, indicators and tools should follow evidence-based practices / models.
       It is not only permissible, but also encouraged, to have more than one (1) indicator and / or tool for a single outcome. Evidence of performance is often strengthened when there is more than one (1)
        indicator to measure the outcome from different sources (triangulation).

                                                                                                               3
NAME OF ORGANIZATION
PROGRAM / PROJECT TITLE
GOAL #


     PROCESS OBJECTIVES                           ACTIVITIES                   TIME FRAME       REQUIRED   RESPONSIBLE               OUTCOMES                      INDICATORS AND
                                                                                               RESOURCES     PERSONS                                                    TOOLS

 Major strategic approaches used to   The individual tasks/steps performed     Due dates for                             Projected benefits / degree of change   Measures and processes
accomplish desired outcomes and the   by the staff to achieve each objective     activities                                to clients, program or community       used to demonstrate
          anticipated results                                                                                              often expressed as percentages          outcome attainment

              (What)                                 (How)                       (When)                       (Who)                     (Why)                          (Evidence)




                                                                                                4
NAME OF ORGANIZATION
PROGRAM / PROJECT TITLE
GOAL #


     PROCESS OBJECTIVES                           ACTIVITIES                   TIME FRAME       REQUIRED   RESPONSIBLE               OUTCOMES                      INDICATORS AND
                                                                                               RESOURCES     PERSONS                                                    TOOLS

 Major strategic approaches used to   The individual tasks/steps performed     Due dates for                             Projected benefits / degree of change   Measures and processes
accomplish desired outcomes and the   by the staff to achieve each objective     activities                                to clients, program or community       used to demonstrate
          anticipated results                                                                                              often expressed as percentages          outcome attainment

              (What)                                 (How)                       (When)                       (Who)                     (Why)                          (Evidence)




                                                                                                5
NAME OF ORGANIZATION
PROGRAM / PROJECT TITLE
GOAL #


     PROCESS OBJECTIVES                           ACTIVITIES                   TIME FRAME       REQUIRED   RESPONSIBLE               OUTCOMES                      INDICATORS AND
                                                                                               RESOURCES     PERSONS                                                    TOOLS

 Major strategic approaches used to   The individual tasks/steps performed     Due dates for                             Projected benefits / degree of change   Measures and processes
accomplish desired outcomes and the   by the staff to achieve each objective     activities                                to clients, program or community       used to demonstrate
          anticipated results                                                                                              often expressed as percentages          outcome attainment

              (What)                                 (How)                       (When)                       (Who)                     (Why)                          (Evidence)




                                                                                                6
NAME OF ORGANIZATION
PROGRAM / PROJECT TITLE
GOAL #


     PROCESS OBJECTIVES                           ACTIVITIES                   TIME FRAME       REQUIRED   RESPONSIBLE               OUTCOMES                      INDICATORS AND
                                                                                               RESOURCES     PERSONS                                                    TOOLS

 Major strategic approaches used to   The individual tasks/steps performed     Due dates for                             Projected benefits / degree of change   Measures and processes
accomplish desired outcomes and the   by the staff to achieve each objective     activities                                to clients, program or community       used to demonstrate
          anticipated results                                                                                              often expressed as percentages          outcome attainment

              (What)                                 (How)                       (When)                       (Who)                     (Why)                          (Evidence)




                                                                                                7

								
To top