Run ID Matches by gjjur4356


									Quality Control Review of E3
      Calculator Inputs
         Comparison to DEER Database

                   Brian Horii
    Energy and Environmental Economics, Inc.
              November 16, 2006

• Purpose was to review how well data entered into the E3
  Calculator matches the DEER database.
• A review of all IOU submissions reveals that very few
  measures actually state the DEER Run ID or Measure ID
• The review also shows many measures that were
  entered, but had no planned installations.
   – 9,236 total measures for all IOUs
       • 6,136 with some installations
   – 2,424 RunID matches
       • 1,304 with installations
   – 155 Measure ID matches
       • 151 with installations
• The lifecycle and peak impacts of these measures are
  shown in the next slides
               Lifecycle Gross kWh
share of
in E3

Values not
adjusted for
NTG ratio.
Values use
entered EUL
Gross Peak Reduction

Lifecycle Gross Therms

                 Run ID Matching
•   For the subset that had Run ID inputs, we compared
    how the entries match the DEER database.
•   Several criteria for “matches” were used. All matches
    were to only 2 or 3 significant digits to allow for
    1. Matches for either DEER common or code impacts AND either
       Incremental or Full costs.
    2. Matches that ignore a commodity (G or E) if no savings
    3. Matches on ratios of Impacts to Costs. Either Common or
       Code can match. Choice of Incremental or Full costs for
       denominator based on DEER Application and CostBasis.
       –   Full costs used with Common impacts 86% of the time.
       –   Incr costs used with Code impacts 69% of the time.
    4. Matches if entered ratios are lower than DEER ratios.
–   Note that these methods will NOT match cases where
    direct install costs are excluded from IMC and put into
    Admin.                                                         6
                   RunID Details
• 994 measures match DEER
• 1,070 match if entry ignored when no savings claimed
• 1,077 measures match for Impact/Cost ratios
• 1,893 match if all impact/cost ratios LOWER than DEER
  are deemed OK
• 244 measures do not match under any test.
    – 41 measures have negative peak impacts that were not entered
      (3.4 MW)
    – 244 measures have cost and impact units that do not match. Of
      those, 149 passed one of the matching tests, we did not perform
      any unit reconciliation tests.
• Impact of changing inputs for the 244 non-matches is
  shown on the next table. (i.e.: how different are those
                  Run ID Subset
• Impacts for the measures with a DEER Run ID that
  “match” the DEER database, versus the non-matches
  are shown in the top half of the table.
• The bottom half of the table shows the change in
  impacts if we modified the inputs for the non-match
  cases to match the DEER database.
• There is little effect on the kWh forecast, but significant
  impact on kW, and less so on Therms.

          kWh Detail for Measures w/

•   Match includes measures where entered ratios are less than DEER ratios.
•   Bottom table shows the change in kWh needed to match the DEER ratios.
•   Negative value indicates that entered values have larger impacts than
                    kW detail

• SCE overestimates are largely (if not entirely) due to
  “pasting error” mentioned at the prior QA/QC workshop.
            Lifecycle Therm Detail

•   Note that negative correction values for Sempra “matches” are due to cases
    where the measure had a negative therm reduction, but no therm impacts
    were claimed in the input section.                                       11
           Other Run ID Findings
• kW (Impact of matching to DEER shown in parentheses)
   – 104 cases where entered kW significantly above DEER. (228 MW)
   – 2 cases where DEER Watts entered as kW, plus 4 other conversion
     errors. (0.4MW)
   – 6 additional cases where negative DEER impact is entered as positive
     (0.7 MW)
   – 41 cases where negative kW impacts are not entered. Of those, 30
     have no installations. (3.2 MW)
• Therms
   – 5 cases where DEER values are Therms, not kBTU (24.4 MTh)
   – 4 cases where impact and cost units differ despite DEER indicating
     “same” (12.1 MTh).
• Other findings
   – 33 case where measure is RETROFIT, but base for impacts is CODE,
     not COMMON. This is conservative, and probably not an error, but
     highlights that users could use the DEER database to arrive at very
     different results by mixing the two sets of inputs and costs.

Review of Measures with No
          Run ID

• Develop avg & max ratios of impacts per $GIMC
  – Utility submissions that match DEER RunID, by
    measure end use categories (based on end use
    shapes in the E3 calculator)
  – DEER measures by 60 subcategories.
• Map Measures
  – Match measure end use shapes.
  – Manually map 1739 cases.
• Compare entered ratios to maximums from step 1

                   Ratio test overview
•   6916 measures did not have RunID matches in the input data
•   4681 have some installations
•   317 have no GIMC
•   Initial filtering using DEER ratios
     – 579 measures with kWh ratios in excess of the DEER sample’s maximum
          • But if average ratios are used, the utility submissions are very conservative in
            aggregate for kWh and kW.
     – 419 measures with Therm ratios in excess of the DEER sample’s maximum
          • The therm “overestimate” would be slightly higher in aggregate if the average ratios
            were used for all measures.
•   Problems with filtering analysis
     –   Impact and cost unit mismatches make comparisons difficult
     –   53,014 DEER runs have different units (out of about 120K)
     –   Results shown exclude all DEER runs where units are not the same
     –   Mapping of utility measures to categories is imprecise
     –   Aggregation into categories is imprecise

Impact Ratios by End Use
• Average and Maximum kWh ratios

• Watts
  Max &

• kBTU
  Max &

Results based on DEER Extract
• Based on reductions relative to CODE.

• Similar results if larger of CODE or COMMON is used

kWh ratios for DEER sub categories

Watt ratios by DEER subcategories

      Using DEER groups yields
         comparable results
• Based on ratios using 60 DEER subcategories,
  results are similar to end-use matching
• Matches for gas measures remains poor

              Unmatched Measures
• We used the maximum impact per GIMC as a very
  generous criteria
• Yet, even with that criteria, 562 cases where kWh ratio is
  exceeded, 406 cases for kW ratio, and 490 cases for
  Therm ratio. (1,045 unique cases).
     – Cases by utility
         •   PG&E:         393
         •   SCE:          121
         •   SDG&E:        167
         •   SoCal Gas:    364

•   Note: may indicate an input problem, or a problem with the assumed
    mapping or a problem introduced by the large number of DEER runs
    excluded due to unit mismatches.

With better data, the ratio test could
         be a useful screen
• Distribution of kWh ratios from the utility measures

• Largest ratios are for measures such as
   – Pre-rinse spray valve – electric water heating
   – Strip curtains
   – Lighting                                            23
   Identification of measures with
      largest ratio mismatches

• Based on sorting       # of     kWh   kW    Therms
  lifecycle amounts
• The measures will      100      88%   95%    93%
  differ from column
                          50      79%   88%    81%
  to column

    Some DEER Measure Gaps
•   Non-Res freezers and refrigeration
•   Non-Res ovens
•   Non-Res pool heaters
•   Computers
•   Non-Res gas measures

                 Summary of Findings
•   76% of cases with some installations have no easy links to RunIDs.
     –   Note that for PG&E, 27% of their 1668 cases w/o DEER RunIDs are for calculated
•   The Max Ratio test is a blunt instrument, but more precision would be very time
    consuming given the data in DEER and in the E3 calculators.
•   With that caveat…
     –   KWh estimates appear reasonable. (3% above max)
     –   kW appears high, compared to DEER data (11% above max)
     –   Therms are hard to judge via DEER (54% above max)
     –   The max ratios “pass” 80% of the measures (4,274 out of 5,319)

•   Recommendations
     –   Future tool should require users to explicitly indicate if savings are relative to common or
         code, and if costs are installation or incremental.
     –   Need a way to link to DEER sub categories, at a minimum, to allow for automated checks
     –   Need a process for creating new approved measures, and for updating DEER.
     –   EULs should be reconsidered for retrofits (remaining life gets common benefit, and EUL -
         remaining life gets code benefit)
     –   Secondary impact on other fuels should be considered.
     –   Direct install costs should be input on a measure basis (not moved into lump sum admin
         costs) to allow for QA/QC review.


To top