NYSERDA PROGRAM LOGIC MODEL

Document Sample
NYSERDA PROGRAM LOGIC MODEL Powered By Docstoc
					                         PROGRAM LOGIC MODEL REPORT
               BUYING STRATEGIES AND ENERGY AWARENESS PROGRAM
                                      Final Report




                                      Prepared for
                                  The New York State
                       Energy Research and Development Authority




                                      Prepared by
                                  GDS Associates, Inc.




NYSERDA
April 2007  
NOTICE

This report was prepared by GDS Associates in the course of performing work contracted for and
sponsored by the New York State Energy Research and Development Authority (NYSERDA) (hereafter
the “Sponsor”). The opinions expressed in this report do not necessarily reflect those of the Sponsor or the
State of New York, and reference to any specific product, service, process, or method does not constitute
an implied or expressed recommendation or endorsement of it. Further, the Sponsor, the State of New
York, and the contractor make no warranties or representations, expressed or implied, as to the fitness for
particular purpose or merchantability of any product, apparatus, or service, or the usefulness,
completeness, or accuracy of any processes, methods, or other information contained, described,
disclosed, or referred to in this report. The Sponsor, the State of New York, and the contractor make no
representation that the use of any product, apparatus, process, method, or other information will not
infringe privately owned rights and will assume no liability for any loss, injury, or damage resulting from,
or occurring in connection with, the use of information contained, described, disclosed, or referred to in
this report.
        NEW YORK STATE ENERGY RESEARCH AND DEVELOPMENT AUTHORITY
                 BUYING STRATEGIES AND ENERGY AWARENESS PROGRAM
                                 PROGRAM LOGIC MODEL REPORT
                                              (Final – 4/6//07)


INTRODUCTION
This document provides:
   1.       A table showing a list of documents relating to the Buying Strategies and Energy Awareness
            Program that were used to provide insights during development of this program logic model
            report;
   2.       A high-level summary of the context of the markets within which this program operates, the
            other NYSERDA programs it works with to accomplish the New York Energy $martSM
            goals, and a brief program description;
   3.       Key program-specific elements including ultimate goals, market barriers, targeted market
            actors, program activities, inputs and potential external influences;
   4.       A program logic model diagram showing the linkages between activities and outputs and
            outcomes, and identifying potential external influences;
   5.       A table listing the key outputs and outcomes, including identification of relevant measurement
            indicators and potential data collection approaches to guide later prioritization, and
            development of a monitoring and evaluation plan; and
   6.       A list of potential researchable issues for consideration within evaluation planning.


1 – RELATED NYSERDA DOCUMENTS
The following table identifies NYSERDA documents and other potentially relevant documents that were
reviewed for this report:

                                 Table 1 – Relevant Documents Reviewed
                                             Document Descriptions

        System Benefits Charge Proposed Plan for New York New York Energy $martSM Programs (2006-
        2011). March 2006.
        New York Energy $martSM Program Evaluation and Status Final Report. May 2006.
        Low-Income Oil Buying Strategies Pilot Program RFP #765. July 2003.
        NYSERDA HEAPOIL website http://Heapoil.org
        NYSERDA Final Low-Income Sector Logic Model Report. March 2006.
        Low-Income Forum on Energy (LIFE) website www.nyserda.org/lifenynews/index.asp
        LIFE Conference Management and Newsletter Publication RFP #1007
        Technical Assistance for HEAP Oil Buying Component & Implementation of NYSERDA Heating
        System Service Project Draft RFP #1016
        2007-2008 HEAP Oil Buying Component PowerPoint Presentation for NYOHA & OHILI Meeting
        (December 18, 2006)


GDS Associates, Inc.                                                                                   Page 1
Final – Buying Strategies and Energy Awareness Program Logic



                                               Document Descriptions

         NYSERDA Heating System Clean & Tune Project: One Year Report (November 2005 thru October
         2006) PowerPoint Presentation (not dated)
         Residential Energy Affordability Programs (REAP) Presentation at
         www.nyserda.org/roadshowpresentations/REAP.ppt (not dated)
         Quality Assurance Services for NYSERDA Residential Programs RFP #1005
         Low Income Home Energy Assistance Program (LIHEAP) State Plan for Fiscal Year 2007. Final,
         September 1, 2006.


2 – CONTEXT AND PROGRAM DESCRIPTION
The Buying Strategies and Energy Awareness Program is part of NYSERDA’s portfolio of New York
Energy $martSM programs that serve low-income households in the state. The Buying Strategies and
Energy Awareness Program consists of four initiatives:

     •     HEAP Oil Buying Strategies – The Low Income Home Energy Assistance Program (LIHEAP)
           is a federally-funded program overseen by the New York State Office of Temporary & Disability
           Assistance (OTDA) and locally administered by local county Departments of Social Service
           (DSS). In 2003, NYSERDA partnered with OTDA to identify and demonstrate strategies to
           expand the buying power of LIHEAP funds used to purchase home heating fuels for low-income
           customers. 1 After two years of pilot projects, OTDA officially incorporated these strategies into
           New York’s Home Energy Assistance Program (HEAP) and began a three-year phased
           implementation, beginning with the 2005-2006 Program Year.
          Although NYSERDA does not currently provide direct funding to the HEAP program, it does
          contribute contractor/consultant and staff resources to help administer and implement the
          program (e.g., monitor energy markets, develop pricing schemes, recruit oil vendors).
          During the 2005-2006 heating season, the Oil Buying initiative served 20 counties, and about 200
          oil vendors participated. The initiative expanded its offerings to 39 counties during the 2006-
          2007 heating season, and will expand to all 62 counties in New York for the 2007-2008 heating
          season. Currently over 300 oil vendors are participating in the Oil Buying initiative, and there is
          good coverage in the 39 active counties. The goal of the initiative is to get all oil vendors to
          participate ultimately, so customers will not have to switch between non-HEAP and HEAP
          vendors and disrupt their existing service relationships.
          Prior to the implementation of this initiative, LIHEAP funds were charged full retail price for
          home heating oil. The Oil Buying initiative was developed to leverage the buying power of
          LIHEAP funds by developing pricing strategies that provide HEAP with more favorable
          purchasing terms. The pricing strategies developed for New York State are based on models
          operating in other states, industry research and communication, and experience gained through
          the pilot programs. In the past, the initiative has increased the buying power of LIHEAP funds
          for heating oil by 7 to 13 percent.
          The Oil Buying initiative currently includes two pricing options: Margin Over Rack (MOR) and
          Discount Off Retail (DOR). MOR is the lesser of the oil dealer’s retail price or the Oil Price
          Information Service (OPIS) average plus 35 cents per gallon for HEAP purchases, or 39 cents per
          gallon for all oil sold to HEAP customers through the season. DOR pricing is the dealer’s posted

1
 HEAP eligibility is set at the greater of 60 percent of the state median income or 150 percent of the federal poverty
guidelines by household size.

GDS Associates, Inc.                                                                                            Page 2
Final – Buying Strategies and Energy Awareness Program Logic


        price less 13 cents per gallon for HEAP purchases, or 9 cents per gallon for all oil sold to HEAP
        customers through the season.
        The Oil Buying initiative requires oil companies to sign a Uniform Vendor Agreement (UVA)
        with OTDA and commit to one of the following pricing options in order to be considered a
        ‘Participating HEAP Vendor’:
            • Option A – Margin Over Rack – HEAP Only
            • Option B – Margin Over Rack – All Season
            • Option C – Discount Off Retail – HEAP Only
            • Option D – Discount Off Retail – All Season
        Oil dealers are exempt from MOR/DOR pricing for customers with confirmed Price Protection
        Agreements or Service Contracts. Companies that do not sign a UVA or select a pricing option
        are considered ‘Non-Participating Dealers’ and are not eligible to receive direct HEAP payments.
        Non-Participating Dealers receive HEAP payments instead through two-party checks that require
        the customer’s signature. These two-party checks increase administrative workload and are
        intended to be a disincentive to non-participation.
        Going forward, OTDA will “centralize” vendor payments to address a history of slow DSS
        payments, and will aim to make payments within 30 days. Vendors will receive one remittance
        statement with total payments for all clients posted in a particular week.
        As the initiative expands to include more downstate counties, several known barriers will need to
        be addressed:
             •   The MOR option must allow the vendor to cover their operating costs and provide a
                 profit. The current 35/39 cent margins may not be sufficient to cover higher business
                 operating costs downstate, and higher margins may be warranted (or DOR may be the
                 only viable option).

             •   Each county has a group (or pool) of oil vendors that serve HEAP clients, and the number
                 of vendors varies in each county, ranging from less than 10 to about 50 (20 to 30 is
                 average). However, Nassau and Suffolk counties and New York City (NYC) have
                 hundreds of vendors each. The initiative has traditionally been introduced via face-to-
                 face vendor meetings held in April, followed by personal recruitment telephone calls in
                 the following months. The large number of vendors downstate will require changes to
                 this very labor-intensive recruitment approach.

             •   The Human Resources Administration runs HEAP in NYC and uses a completely
                 different computer system than the rest of the State, which will introduce
                 programming/coding complexity.

             •   The current initiative uses two-party checks as a disincentive for oil vendors; if vendors
                 do not participate they do not receive payment directly, but receive a two-party check that
                 requires the client’s signature instead. However, in Nassau and Suffolk counties, vendors
                 are always paid by two-party check, and so this disincentive for non-participation is less
                 effective.

        As noted previously, NYSERDA’s support for this initiative is provided mainly through
        contractor/consultant and staff resources to help monitor energy markets, develop pricing
        schemes, recruit oil vendors, etc. For the current (third) round of SBC funding, the Oil Buying




GDS Associates, Inc.                                                                                  Page 3
Final – Buying Strategies and Energy Awareness Program Logic


           initiative has a one-year goal of increasing the buying power of HEAP funds by $4 million, and
           leveraging an additional $20 million over five years. 2
       •    Heating Equipment Repair and Replacement (HERR) – Under the Emergency Component of
            HEAP, recipients can receive heating equipment repair and replacement assistance for inoperable
            furnaces through OTDA. 3 This component of HEAP considers both customer financial
            resources and income eligibility. If a customer has financial resources that are immediately
            available, then these resources must be spent prior to receiving a HEAP repair/replacement
            benefit. Heating equipment replacement benefits are capped at $6,000, and OTDA approval is
            required for benefits in excess of $2,500. Replacement costs that exceed $6,000 are the
            responsibility of the customer, and this money must be readily available prior to installation of
            the heating equipment. Customers can only receive one replacement per dwelling unit in a ten-
            year period. Heating equipment repair benefits are capped at $2,500 per customer.
           To be considered for HERR services, customers are responsible for getting bids from
           repair/replacement vendors, who must have vendor agreements on file with the local county DSS
           to be considered for HERR services. Customers then send the bids to DSS for review and
           selection (usually the lowest bidder). Alternatively, some counties contract out the vendor
           review/selection process to local Community Action Agencies.
           Approximately 3,000 heating equipment replacements and 1,200 repairs are provided each year
           through HERR. Going forward, NYSERDA will enhance the services delivered through HERR
           by developing standards for heating equipment installation, providing installation training for
           participating contractors and requiring that installed equipment meet specified standards,
           including ENERGY STAR® ratings.

       •    Clean & Tune (C&T) Project – Under HEAP, customers are offered emergency heating
            equipment repair and replacement assistance, but are not offered preventive maintenance
            services. The Clean & Tune Project was created by NYSERDA based on input from the industry
            noting that many low-income households do not regularly maintain their heating systems
            regularly, with the result that some of the most inefficient and unsafe systems operate in the
            homes of New York’s most vulnerable citizens. The goals of the C&T Project are to further
            stretch Oil Buying assistance dollars (by improving heating equipment efficiency), improve
            household health and home safety, and provide a meaningful incentive for companies to
            participate in the HEAP Oil Buying initiative.
           The C&T Project was originally piloted in 2005 and gives vendors participating in the HEAP Oil
           Buying initiative the opportunity to provide heating system Clean & Tune services and minor
           repair work, at no cost, to eligible HEAP households. Following are key elements of the C&T
           Project:
               •    Companies must be a Participating Vendor in the HEAP Oil Buying initiative to
                    participate in the C&T Project.
               •    Companies must sign a Participation Agreement with NYSERDA.
               •    Companies can service up to 20 boilers/furnaces in each county in which the oil vendor
                    participates in HEAP. NYSERDA reimbursement is capped at $150 per service ($120
                    for mobile homes) 4 .


2
    SBC Operating Plan, p. 8.14, Table 8.7.
3
    Funding is provided by the federal LIHEAP program, and not OTDA or NYSERDA.
4
 The customer covers costs in excess of the reimbursement limits. However, most service costs do not exceed the
caps, and the caps are considered prior to scheduling the service.

GDS Associates, Inc.                                                                                       Page 4
Final – Buying Strategies and Energy Awareness Program Logic



             •   NYSERDA provides companies up to $1,200 for each county served to pay for minor
                 parts and repairs that may be needed beyond the C&T services.
             •   Customers that rent their dwelling require landlord approval.
             •   Services must be performed to NYSERDA’s specifications, as outlined in the C&T
                 Checklist and Certification Form.
             •   NYSERDA offers an equipment purchase incentive to participating companies to
                 encourage the purchase of eligible Combustion Analyzers.
             •   Training is provided to participating companies. Training covers the NYSERDA
                 specifications and procedures, the use of combustion analyzers and the introduction of
                 building science principles 5 .
         Customer-owned heating equipment that is deemed to be irreparable (i.e., “red-tagged”) or
         requires major repairs is referred to the HERR component of HEAP. If the customer is ineligible
         for assistance under HERR, they may be referred to the EmPower New YorkSM program.
         Landlord-owned heating equipment is referred to NYSERDA’s Multifamily Building
         Performance Program for potential major repair/replacement assistance.
         Currently there are more than 40 participating dealers offering coverage in 39 counties. From
         November 2005 through October 2006, the project serviced 465 systems at a cost of over $70,000
         (for C&T and related repairs). Program staff estimate that 50 percent of the households served
         will require one less service visit in the two years following the C&T, and that average
         combustion efficiency will be improved by 2 percent. Going forward, the C&T Project is
         expected to serve between 1,000 and 2,500 systems per year as the Buying Strategies initiative
         expands to serve the whole state.
     •   Low-Income Forum on Energy (LIFE) – LIFE provides a forum where energy industry
         professionals, policy makers, low-income assistance agencies, and energy program implementers
         can identify, discuss and address energy issues that pertain to the low-income sector. LIFE is led
         by a Steering Committee that works together to develop a series of statewide conferences and
         regional meetings. To date, LIFE has conducted four sets of regional meetings and two
         statewide conferences. Future series of regional meetings are planned in 2007, 2009 and 2011,
         and statewide conferences are planned in 2008 and 2010.
         The Steering Committee consists of approximately 30 volunteers representing utility companies,
         State and local government, non-profit community-based organizations and low-income
         advocates. The LIFE database also includes approximately 4,000 individuals that participate in
         this on-going “dialogue” to address energy issues facing low-income consumers.
         Going forward, a contractor will assist the Steering Committee to plan and conduct the regional
         and statewide meetings and conferences, and will publish a new quarterly newsletter. The
         contents of the newsletter are expected to include: updates to NYS low-income energy programs,
         profiles of low-income energy professionals, case studies of low-income assistance best practices,
         and marketing of LIFE events.




5
 Combustion analyzers must measure oxygen, air temperature, stack temperature, and carbon monoxide (CO), and
must calculate combustion efficiency, excess air and carbon dioxide (CO2).

GDS Associates, Inc.                                                                                   Page 5
Final – Buying Strategies and Energy Awareness Program Logic




3 – KEY ELEMENTS SUMMARY
Based on a review of program documents, the following is a summary of the key elements of the Buying
Strategies and Energy Awareness program.

3.1     Ultimate Goals
The following program goals have been established for the Buying Strategies and Energy Awareness
program components:
1.      Increase the buying power of LIHEAP funds
2.      Provide reduced-price heating fuel to more low-income customers
3.      Maintain relationships between low-income customers and oil vendors
4.      Minimize administrative burden to local DSS offices as Oil Buying component grows
5.      Educate fuel customers about the benefits of moving from “cash on delivery” to automatic
        delivery and budget plans
6.      Improve heating system efficiency so as to reduce fuel consumption
7.      Health and safety improvements
        A. Reduce threat of carbon monoxide poisoning
        B. Reduce risk of fire through unsafe wiring, proximity of flammables, bypassed safety devices
        C. Greater comfort and confidence of homeowners through heating season
8.      Improve “whole-house” service approach and testing equipment availability through C&T
        training and equipment incentives
9.      Enhance awareness and communication about low-income energy issues

3.2     Market Barriers
Several barriers that inhibit the adoption of energy-efficient products and activities affect the low-income
residential market. These barriers can be separated into three general groups: barriers affecting supply-
side, mid-market/infrastructure, and demand-side (and associated end-use) market actors. Supply-side
and mid-market/infrastructure barriers include policies and business practices that deter the development
and/or delivery of energy efficient products and services, or indicate an insufficient availability of or
commitment to such energy-efficient products/services. Demand-side barriers in the low-income
residential sector include insufficient financial resources, lack of awareness or understanding about
energy efficiency options and benefits, and the low priority of energy efficiency given competing uses of
funds. Table 2 lists the specific barriers and market actors inhibiting low-income customers’ ability to
obtain reasonably priced heating fuel, obtain efficient heating equipment and maintain their heating
equipment (the order of these barriers does not reflect significance or priority; the numbers are for
reference purposes only).




GDS Associates, Inc.                                                                                   Page 6
Final – Buying Strategies and Energy Awareness Program Logic


Table 2 - Barriers and Actors Affecting the Ability of Low-Income Customers to Obtain Affordable
    Heating Fuel, Efficient Heating Equipment and Preventive Maintenance (“the Problem”)
Market Area                                              Barriers                                                  Market Actors
 Supply-Side      S1 – Oil dealers’ resistance to efforts to secure lower prices for low-income customers       Oil wholesalers
 (upstream        S2 – Higher oil vendor business costs in downstate market area make uniform pricing           Retail vendors
 actors)          schemes/participation less attractive
                  S3 – Increasing oil vendor business costs in all markets (e.g., labor, environmental and
                  security requirements) increase oil prices and complicate pricing schemes
                  S4 – Diverse population of oil vendors with different business models and varying
                  levels of interest (e.g., small rural family run businesses, large urban corporate vendors)
                  S5 – History of slow DSS and HEAP customer payments reduces oil vendor interest
 Market           M1 – High staff turnover at DSS agencies                                                      OTDA
 Infrastructure   M2– Fragmented HEAP implementation through 62 counties/HEAP programs                          DSS agencies
 / Policy (mid-
 stream actors)   M3 – History of poor DSS payment turnaround (90-120 days)                                     Oil heating system
                  M4 – Established DSS relationships with oil vendors hinders proactive recruitment of          service providers
                  new vendors to HEAP program; DSS needs to be convinced of public policy merits                Low-income energy
                  M5 – Lack of energy service contractors with interest or willingness to effectively           program administrators
                  deliver assistance to low-income populations                                                  and implementers, low-
                                                                                                                income service agencies,
                  M6 –Reluctance of repair and maintenance service providers to work with landlords             policy makers
                  M7– Lack of industry standards for cleaning, tune-up and maintenance services;
                  providers do not perform CO testing
                  M8 – Lack of equipment to do required tests among oil heating service technicians
                  M9 – Lack “whole-house” mindset towards energy efficiency among oil heating
                  service technicians
                  M10 – Incomplete understanding of low-income energy issues by policy makers and
                  staff in a variety of low-income assistance efforts
                  M11– Lack of information dissemination about low-income energy issues
 Demand-Side      D1 – Limited financial resources of income-eligible customers to buy home heating             Low-income households
 (downstream      fuel, repair/replace heating equipment, and address energy efficiency in their homes          Multifamily building
 actors)          D2 – Lack of market power by separate low-income customers (compared to pool of               owners and managers
                  HEAP participants)
                  D3 – Split incentives for rental units (building owners often do not pay the energy bills
                  and have little incentive to improve energy savings. Tenants do pay the energy bills but
                  have little incentive/ability to improve the rental property)
                  D4 – Language and geographic barriers (e.g., rural locations) that hinder energy
                  information dissemination
                  D5 – Lack of awareness among consumers of the value and benefits of energy efficient
                  equipment
                  D6 – Resistance to new and/or innovative energy technologies
                  D7 – Energy-efficient equipment performance uncertainties and lack of resources for
                  proper maintenance of more advanced equipment
                  D8 – Low-income customers thinking they earn too much to be eligible for assistance
                  programs
                  D9 – Low-income customers forgoing resources believing that others have greater need
                  D10 – Limited awareness by income-eligible customers of weatherization, fuel
                  assistance, and other services related to energy use in homes
                  D11 – Customers’ established relationships with oil vendors creating reluctance in
                  switching to different HEAP vendors
                  D12 – Some low-income customers wishing to avoid the perceived stigma of receiving
                  any form of financial assistance


GDS Associates, Inc.                                                                                                     Page 7
Final – Buying Strategies and Energy Awareness Program Logic


3.3     Targeted Market Actors
The Buying Strategies and Energy Awareness program provides outreach and services targeted to
multiple market actors in both the supply side (and associated market infrastructure) and demand side
areas. Table 3 shows a list of the specific market actor groups that are involved with the program.

      Table 3 - Market Actors Targeted by the Buying Strategies and Energy Awareness Program
           Supply-Side and Market Infrastructure                                   Demand-Side
      Heating oil wholesalers and retail vendors              Low-income residential customers
      Heating system service providers                        Multifamily building owners and managers
      DSS agencies
      Low-income energy program administrators and
      implementers, low-income service agencies, policy
      makers

3.4        Program Implementation Approach (“Activities”)
The Buying Strategies and Energy Awareness program is designed to provide reduced-cost home heating
fuel for HEAP recipients, preventive maintenance and repair/replacement services for home heating
systems, and information dissemination about low-income energy issues to a large and diverse audience.
A broad range of program activities is conducted to provide these services. Table 4 shows the major
categories of program activities and their related sub-activities.
                   Table 4 – Buying Strategies and Energy Awareness Program Activities
                                Information Sharing on Low-Income Energy Issues

       •    Identify low-income energy issues
       •    Identify low-income energy stakeholders
       •    Recruit LIFE steering committee members
       •    Assist LIFE steering committee in planning for statewide conferences and regional meetings
       •    Maintain and enhance LIFE website to improve information exchanges
       •    Develop and distribute LIFE newsletters
                                          Education and Technical Support
       •    Develop and distribute oil buying guidance information to LIHEAP customers (for all oil purchases)
       •    Develop and distribute Oil Buying informational materials to oil vendors
       •    Maintain and update Oil Buying website and phone Price Line (e.g., MOR pricing)
       •    Work with Oil Pricing Information Service (OPIS) to obtain current information (subscription and billing,
            addition/deletion of racks)
       •    Provide technical assistance to oil vendors through telephone Help Line, regular oil Vendor (Group)
            Meetings, and personal meetings
       •    Conduct bi-monthly oil Dealer Advisory Group conference calls
                                              Recruitment and Training




GDS Associates, Inc.                                                                                                    Page 8
Final – Buying Strategies and Energy Awareness Program Logic



      •    Train DSS staff on Oil Buying component
      •    Recruit oil vendors to Oil Buying component through letters, meetings and telephone calls
      •    Encourage oil vendors to sign Uniform Vendor Agreements (UVAs) and choose MOR/DOR pricing option
      •    Encourage participating oil vendors to provide heating equipment clean and tune services
      •    Train heating equipment clean and tune providers on testing procedures and testing equipment use
      •    Train heating equipment repair/replacement providers about required installation procedures and heating
           equipment standards/selection
                                   Heating Equipment Services and Incentives

      •    Develop clean and tune Services Checklist
      •    Provide incentives for clean and tune services and testing equipment purchases

      •    Develop heating equipment repair/replacement guidelines
                         Quality Assurance, Impact Reporting and Program Refinement
      •    Oil Buying vendor data tracked in program database (pricing option selected, counties served, etc.)
      •    Clean and tune providers and services performed recorded in (separate) program database
      •    Prior to payment, review clean and tune services provided for adherence to Checklist
      •    Inspect samples of clean and tune and repair/replacement services and perform Combustion Safety Tests.
           Report unsatisfactory work to NYSERDA and resolve with providers
      •    Conduct fuel price surveys, monitor wholesale energy markets, revise Oil Buying MOR/DOR pricing as
           needed
      •    Report Oil Buying impacts (e.g., increased buying power in $, gallons)
      •    Report clean and tune impacts (e.g., number of companies, services completed, minor repairs made,
           repair/replacement referrals, increases in system efficiency/longevity)
      •    Summarize clean and tune training evaluations

3.5       Program Inputs and Potential External Influences
The ability of the Buying Strategies and Energy Awareness Program efforts to accomplish a level of
outputs that will, in turn, cause the anticipated outcomes and the associated causal chain leading to the
program’s ultimate goals is dependent on the level, quality, and effectiveness of the inputs that go into
these efforts. There are also external influences that can help or hinder the development of anticipated
outcomes. Key program inputs and potential external influences are presented in Table 5.




GDS Associates, Inc.                                                                                                 Page 9
Final – Buying Strategies and Energy Awareness Program Logic


    Table 5 – Buying Strategies and Energy Awareness Program Inputs and External Influences
                                                         Program Inputs
 SBC and other funding sources
 NYSERDA staff resources and experience implementing SBC programs
 NYSERDA staff experience coordinating local, State and federal public agencies
 OTDA, DPS, and DSS staff experience implementing HEAP program
 Awareness of NYSERDA among market actors
 Attitudes of home fuel vendors toward assisting low-income and other groups
                                              External Influences and Other Factors
 Prices of home heating fuel
 Poor quality low-income housing stock and heating equipment
 Weather impacts on customer actions and energy use
 Costs and performance of newer, more energy efficient technologies
 Changes in state political priorities and regulations
 Changes to state building codes and standards
 Federal low-income housing programs structure and legislated processes
 Federal funding for LIHEAP and other government (non-NYSERDA) funding for low-income programs (e.g., weatherization)
 Level of interest rates, which affect home improvement activities
 Local, regional and national economic conditions and energy prices
 Other low-income household expenses (e.g., medical expenses, transportation, education, child care, etc.)
 Diversity and dispersion of the low-income population across state regions
 Diversity and large numbers of oil vendors across state regions



4 – PROGRAM LOGIC MODEL DIAGRAM
The following page contains the Buying Strategies and Energy Awareness Program logic model diagram
showing the linkages between activities, outputs and outcomes, and identifying inputs and potential
external influences. The diagram presents the key features of the program. The logic diagram presented
here is at a slightly higher level than the tables in this report, aggregating some of the outcomes in order
to provide an easier-to-read logic model. (Evaluation research should use the more detailed tables, in
addition to the diagram, to examine the anticipated linkages and performance through the various
outcomes.)




GDS Associates, Inc.                                                                                            Page 10
Final – Buying Strategies and Energy Awareness Program Logic




GDS Associates, Inc.                                           Page 11
Final – Buying Strategies and Energy Awareness Program Logic




5 – PROGRAM OUTPUTS, OUTCOMES AND ASSOCIATED MEASURMENT INDICATORS
It is important to distinguish between outputs and outcomes. For the purposes of this logic document,
outputs are defined as the immediate results from specific program activities. These results are typically
easily identified and can be counted, often by reviewing program records
Outcomes are distinguished from outputs by their less direct (and often harder to quantify) result from
specific program activities. Outcomes represent anticipated impacts associated with NYSERDA’s
program activities and will vary depending on the time period being assessed. On a continuum, program
activities will lead to immediate outputs that, if successful, will collectively work toward achievement of
anticipated short-, intermediate-, and long-term program outcomes.
The following tables list outputs (Table 6) and outcomes (Table 7) and associated measurement
indicators. For each indicator, a proposed data source or collection approach is presented. Where
appropriate, the need for baseline data is also noted. Items in this table should be prioritized and
subsequently considered as potential areas for investigation as part of a formal program evaluation plan.




GDS Associates, Inc.                                                                                  Page 12
Final – Buying Strategies and Energy Awareness Program Logic




             Table 6 – Program Outputs, Associated Indicators and Potential Data Sources
                                                                                       Data Sources and Potential
         Outputs (< 1 year)                             Indicators
                                                                                        Collection Approaches
  Low-income energy issues and          Number of LIFE topics considered for           LIFE program tracking data
  stakeholders identified               information sharing
                                        Number and types of LIFE members
 Oil buying guidance distributed to     Number and types of material distributed by    Program tracking data
 HEAP customers (for all heating oil    county
 purchases)                             Number and proportion of HEAP customers
                                        receiving materials
 Oil Buying component educational       Number and types of material distributed by    Program tracking data
 materials distributed to oil vendors   county
                                        Number and proportion of oil vendors
                                        receiving materials by county
 Technical assistance given to oil      Answers to common questions provided on        Program tracking data
 vendors                                program website                                Program staff interviews
                                        Number of vendor questions answered by
                                        telephone Help Line
                                        Number of personal meetings with oil
                                        vendors
 Participating oil vendor meetings      Dealer Advisory Group conference calls         Program tracking data
 conducted                              held                                           Program staff interviews
                                        Dealer attendance at oil Vendor Meetings by
                                        county
 DSS staff trained in Oil Buying        Number of trained staff by county              Program tracking data
 component                                                                             Program staff interviews
 Non-participating oil vendors          Number of recruitment letters sent to non-     Program tracking data
 contacted for recruitment              participating vendors by county and in areas   Program staff interviews
                                        with lowest participation rates
                                        Personal recruitment meetings with non-
                                        participating vendors
                                        Recruitment calls to non-participating
                                        vendors
 UVA agreements signed                  Number of new participating oil vendors by     Program tracking data
                                        county annually
                                        Total annual number of participating oil
                                        vendors by county
 Heating equipment service              Number of clean and tune contractors           Program tracking data
 contractors enrolled and trained on    enrolled by county                             Program staff interviews
 procedures, equipment use              Number of clean and tune contractors
                                        trained by county
 Heating system service procedures      Clean & Tune (C&T) Service Checklist and       Program tracking data
 established                            testing requirements developed
                                        Heating equipment repair and installation
                                        requirements developed
 Clean and tune service and testing     C&T service and testing equipment              Program tracking data
 equipment incentives developed         incentives available
 QA/QC procedures established           Descriptions of QA/QC protocols                Program tracking data
                                        Agreements with QA/QC contractors              Program staff interviews


GDS Associates, Inc.                                                                                                Page 13
Final – Buying Strategies and Energy Awareness Program Logic




Table 7 – Program Outcomes, Associated Indicators and Potential Data Sources
                                                                                     Data Sources and Potential
            Outcomes                                 Indicators
                                                                                      Collection Approaches
                                            Short-Term (1-4 years)
 LIFE meetings and conferences      Number of LIFE meetings and conferences          LIFE program data
 conducted                          conducted
                                    Number and types of attendees at LIFE
                                    meetings and conferences (e.g., policy makers,
                                    issue experts, low-income community leaders,
                                    low-income programs staff, non-experts)


 Increased awareness of low-        Level of awareness of low-income energy          Baseline assessment of low-
 income energy issues and           issues                                           income service staff and
 assistance best practices by       Number of topics presented at LIFE meetings      policy maker awareness, and
 experts and program participants   and conferences                                  surveys to measure change
 Development of new or more                                                          over time
                                    Frequency of LIFE newsletter distribution and
 effective ways to address energy   content volume                                   Interviews with program
 issues in the low-income                                                            staff and LIFE steering
 community                          Volume and types of information available at     committee
                                    LIFE website
                                                                                     Interviews with low-income
                                    Number and types of new initiatives developed    program and services staff
                                    (regardless of who will be undertaking the
                                    effort)                                          LIFE program data and
                                                                                     archives
 Increased Oil Buying component     Level of awareness of program                    Baseline assessment of DSS
 awareness by DSS agencies and                                                       agency and oil vendor
 oil vendors                                                                         awareness, and surveys to
                                                                                     measure change over time
 Heating oil available to low-      Number of participating oil vendors by county    Program tracking data
 income households at reduced
 price
 Low-income households buy          Prices low-income households actually pay for    Program tracking data
 heating fuel at reduced prices     heating fuel
                                    Number of HEAP households that purchase
                                    reduced price oil by county
                                    Amount of reduced price heating fuel
                                    purchased by county
 Heating equipment services         Number of C&T services provided by county        Program tracking data
 provided                           Amount of C&T service and testing equipment
                                    incentives spent by county
                                    Number of heating equipment repair and
                                    replacement services by county
 Reduced heating costs from new     Btu and therm savings                            Impact analysis
 or more efficient heating
 equipment
 Improved safety                    Number and type of safety violations found and   Program tracking data
                                    repaired                                         Participant and non-
                                    Reduced incidence of heating equipment-          participating customer
                                    related emergencies (CO poisoning, fires)        survey, survey with DSS
                                                                                     agencies or survey of fire
                                                                                     departments




GDS Associates, Inc.                                                                                               Page 14
Final – Buying Strategies and Energy Awareness Program Logic



                                                                                           Data Sources and Potential
            Outcomes                                     Indicators
                                                                                            Collection Approaches
 Quality of heating equipment          On-site visits conducted to assess heating          Program tracking data
 services measured for QA/QC           equipment services quality and contract
                                       compliance
 Feedback given to heating             Record of communications between QA                 Program tracking data
 equipment service providers           contractors and heating service providers           Q/A contractor interviews
                                       Number of service remediations performed
                                      Intermediate-Term (5-9 years) Outcomes
 Increased awareness of low-           Level of awareness of low-income energy             Baseline assessment of non-
 income energy issues among non-       issues                                              energy low-income service
 experts and the general public                                                            staff and general public
                                                                                           awareness, and surveys to
                                                                                           measure change over time
 Oil vendors find Oil Buying           Number of profitable HEAP accounts served           Survey of oil vendors
 participation is worthwhile and       by oil vendors                                      Program tracking data
 remain in program                     Oil vendor length of participation with program
 Heating equipment service             Number of profitable low-income accounts            Survey of heating equipment
 providers find low-income             served by heating equipment contractors             service providers
 services are profitable and remain    Contractor length of participation with program     Program tracking data
 in program
 More efficient low-income             Number/share of low-income units with energy        Low-income household
 residential building stock            efficient heating equipment                         survey (in and out of HEAP)
                                       Fuel consumed/unit                                  Impact analysis
 Oil Buying pricing strategies used    Number and types of low-income progams              Low-income program data
 to leverage other (non-HEAP)          utilizing Oil Buying pricing strategies to extend
 public funds for low-income           program dollars
 heating assistance                    Amount of additional public funds leveraged
                                        Longer-Term Outcomes (10+ years)
 Market infrastructure established     Number of DSS agencies working with                 Program tracking data
 in all regions of the state           HEAP/Oil Buying by county                           Interviews with program
                                       Number of oil dealers with UVA agreements           staff
                                       by county
                                       Number and types of participating heating
                                       equipment service providers by county
                                       Number and types of LIFE participants
 Reduced heating fuel costs for        Btu and therm savings                               Impact analysis
 low-income households                 Low-income share of monthly income used on          Surveys of HEAP and non-
                                       heating fuel has decreased                          HEAP low-income
                                                                                           households



6 – POTENTIAL RESEARCHABLE ISSUES FOR EVALUATION
Based on this assessment of the Buying Strategies and Energy Awareness Program, a number of potential
researchable issues have been identified and are noted below:
      •    Are the most expert low-income energy professionals and policy makers participating in LIFE
           meetings and conferences? Is their perspective and knowledge being effectively communicated
           to other experts and non-experts in the low-income energy field?




GDS Associates, Inc.                                                                                                     Page 15
Final – Buying Strategies and Energy Awareness Program Logic



      •   Do LIFE meetings, conferences, and other information distribution activities effectively address
          the full range of energy issues affecting the diverse low-income population throughout the
          state?
      •   Has awareness and understanding of the Oil Buying component changed among participating
          and non-participating oil vendors and DSS agency staff?
      •   Do participating oil vendors consistently remain in the program, or is there high turnover?
      •   Can low-income heating fuel customers throughout the state generally buy fuel from their
          preferred vendors through the Oil Buying component?
      •   Are low-income heating fuel customers moving from “cash on delivery” to other delivery and
          budget plans to reduce emergency fuel shortage situations?
      •   Is the training provided to heating equipment companies effective? Are they successfully
          adopting a “whole-house” service focus? Do they have the maintenance and testing equipment
          that is needed to improve heating equipment efficiency?
      •   Does the quality assurance effort ensure that heating equipment services are being provided
          according to program guidelines?
      •   Are the heating equipment maintenance, repair, and replacement components of the program
          resulting in notable health and safety benefits?
      •   Are heating equipment companies finding the provision of services to be profitable through the
          program? Is their participation in the program increasing over time? Are their services
          expanding to non-HEAP markets?
      •   Is the program resulting in notable energy cost reductions for low-income households? Is it
          increasing their ability to meet other priority spending needs?


Evaluation research addressing these questions will help to validate the reasonableness of the program
theory and will inform NYSERDA program staff of program progress and also potential areas for
program refinement.




GDS Associates, Inc.                                                                                    Page 16