Performance Measurement and Reporting for Extended Producer Responsibility Programs

Performance Measurement and Reporting for Extended Producer Responsibility Programs Reporting Guidance Document Prepared for Environment Canada Environmental Stewardship Branch By Stratos Inc. 1404-1 Nicholas Street Ottawa, Ontario K1N 7B7 tel: 613 241 1001 fax: 613 241 4758 www.stratos-sts.com October 2007 Acknowledgements Environment Canada would especially like to acknowledge Patrick Kane and Brenna MacLennan of Alberta Environment for their participation and contributions to the development of the terms of reference for this project and for providing ongoing and valuable input during the drafting and final editing of the Guidance Document. In addition Environment Canada would like to thank the following individuals for providing constructive comments and suggestions. Their input enhanced the relevance and usefulness of this guidance document for Producer Responsibility Organization (PRO) program managers and administrators. Cam Davreux, CropLife Canada; Gordon Day, Stewardship Ontario; Marie Dussault, Government of Québec; Jim Ferguson, Green Manitoba; April Gucciardo, Refrigerant Management Canada; Dennis Hambleton, Alberta Used Oil; Warren Heeley, Refrigerant Management Canada; Russel Hurst, CropLife Canada; David Lawes, Government of British Columbia; and John Sinclair, University of Manitoba. October 2007 Purpose The purpose of this guidance document is to provide managers of extended producer responsibility (EPR) and product stewardship programs with specific guidance on what performance data and information they should consider when reporting on their programs. This guide identifies a set of recommended performance indicators, additional indicators, and a suggested outline for an annual performance report. It has been developed with the aim of encouraging Producer Responsibility Organizations (PROs) to improve the quality and consistency of performance reporting across EPR programs in Canada, and to challenge managers of these programs to critically review the accomplishments and areas for improvement of their respective programs. This guide is not intended to prescribe a fixed process and content for an annual performance report. Rather, it provides simple guidance on what to measure and disclose to create a credible report for use by interested parties. It is expected that individual PROs will build on this guidance and devise the structure and content of their annual reports according to the nature of their business and the interests of their stakeholders. Who should use this Guide? PRO managers This guide has been written for PRO managers. They can use this guide to improve their internal performance management and reporting practices, as well as their external annual performance reporting. This guide will also be useful to policy makers and regulatory authorities to provide a more consistent understanding of the parameters on which PROs can monitor and report, as well as the suggested content of an annual performance report for a PRO. Other interested parties can use this report to develop a better understanding of the types of information and indicators PROs might use to manage their business. Policy makers and regulatory authorities Other interested parties Building on Annual Financial Reports Most PROs are required to produce an annual report that includes specific data and information, including audited financial statements. PROs must be familiar with these reporting requirements and ensure that their organization meets them. The guidance provided in this document goes beyond minimum reporting requirements, and can be used by PRO program managers to expand the breadth of performance information they gather, use and disclose. This additional performance information can be incorporated within the organization’s annual report; alternatively, it could be presented in a separate report. Performance Measurement and Reporting for EPR Programs i Structure of this Guide This guide is presented in ten sections to facilitate its navigation and use by PRO managers. The “Reporting Roadmap” below identifies the key steps to preparing a performance report and shows how the guidance contained in this document supports each of these steps. This guidance document focuses on the types of performance information that PROs should track and report. More specific guidance on how to prepare a report (i.e. the reporting process) can be found in other publications, such as the Sustainability Reporting Toolkit available at www.sustainabilityreporting.ca 1 . Reporting Steps 1. Establish purpose of report Look to sections: Why report (Section 2) 2. Establish report boundaries Principles (Section 3) Boundaries (Section 4) Reporting Roadmap Principles (Section 3) 3. Decide what to report on and collect data and information Report outline (Section 6) Organizational Information (Section 7) Performance indicators (Section 8) Forward-looking information (Section 9) 4. Determine how best to present information Reporting format (Section 5) 5. Review completeness of report Appendix 1 – Report review checklist 6. Learn from Others Appendix 2 - References 1 This is an online tool developed by Environment Canada in collaboration with Stratos Inc. Performance Measurement and Reporting for EPR Program ii Table of Contents 1 INTRODUCTION........................................................................................... 1 2 WHY REPORT? INTERNAL AND EXTERNAL BENEFITS OF REPORTING .......... 2 3 PRINCIPLES OF QUALITY PERFORMANCE REPORTING ................................ 3 4 REPORT BOUNDARY: DETERMINING WHAT’S IN AND WHAT’S OUT ............. 4 5 REPORT FORMAT: PRINT OR ELECTRONIC? ................................................. 6 6 SUGGESTED OUTLINE FOR ANNUAL PERFORMANCE REPORT ....................... 8 7 CONTEXTUAL AND MANAGEMENT INFORMATION ........................................ 9 8 REPORTING ON PERFORMANCE INDICATORS............................................ 14 9 LOOKING AHEAD: ENSURING A FORWARD-LOOKING REPORT................... 40 10 CONCLUSION.......................................................................................... 41 APPENDIX A: APPENDIX B: ANNUAL PERFORMANCE REPORT REVIEW CHECKLIST REFERENCES AND SUPPORTING LINKS Performance Measurement and Reporting for EPR Programs iii 1. Introduction Extended Producer Responsibility (EPR) is “an environmental policy approach in which a producer’s responsibility for a product is extended to the postconsumer stage of a product’s lifecycle.” 2 While EPR programs can vary substantially from one another, reflecting different local circumstances, different products, and different program objectives, they share one fundamental characteristic: producers and/or distributors are involved in the postconsumer management of their specific products (e.g. automobiles), product categories (e.g. electric and electronic products) or waste streams (e.g. packaging). Producer Responsibility Organizations (PROs) manage EPR programs and can be defined as not for profit industry associations structured in response to an EPR regulation or to manage a voluntary initiative. PROs can support both voluntary and mandatory take-back schemes, deposit/refund systems, and advance disposal fee programs. PROs can also have responsibility for functions extending beyond the management of product take-back, such as education and training of producers and consumers, collection of fees and other responsibilities as delegated under the EPR program. PROs, like other private and public sector organizations, are increasingly being challenged to produce measurable results and be accountable for their performance. PRO managers, government policy makers, and the general public require sound performance data and information to determine if programs are meeting their intended objectives. Performance data and information not only help PROs run their programs in a more efficient and effective manner, but can also provide the basis for assessing EPRs as a vehicle for pollution prevention Different jurisdictions use different terminology for organizations that manage EPR programs. Commonly used terms include: PRO: Producer Responsibility Organization DAO: Delegated Administrative Organization IFO: Industry Funding Organization This report uses the term “PRO” throughout. Organisation for Economic Co-Operation and Development (OECD). 2001. Extended Producer Responsibility, A Guidance Manual for Governments. Paris: OECD. 2 Performance Measurement and Reporting for EPR Programs 1 2. Why Report? Internal and external benefits of reporting Regulatory requirements are often the first, and most prominent, driver for preparing an annual report. At a minimum, PROs must ensure their annual reports meet the minimum requirements stipulated in applicable regulations. There are a number of more subtle drivers and benefits of reporting. Performance reporting can add business value to your organization directly by helping you to measure and analyze your own performance, and indirectly by driving improved performance through enhanced transparency. Public reporting also provides information to external stakeholders that help them make informed decisions about your organization, whether they be governments, EPR program participants, other PROs, or facility neighbours. There can be many benefits to reporting on organizational performance. Some of these benefits can be purely internal to the organization, while others derive from providing information to external parties. The most commonly cited benefits of reporting include: Internal Benefits Communicates goals and progress towards those goals Builds internal alignment and capacities of employees Improves the management of risks and opportunities – what gets measured gets managed Improves performance through cost savings and more effective use of internal resources Helps identify successes and challenges for the organization External Benefits Meets regulatory requirements Strengthens external relationships by publicly sharing information on your performance Enhances reputation Adds credibility to an organization Allows for comparison and exchange of information amongst organizations The benefits your organization will receive from reporting depend on the content of your report, and how well you integrate the reporting process into your business activities to drive improved performance. It is not necessary to invent a new reporting process to address the guidance in this document. Use your current annual planning and reporting processes as building blocks to enhance the breadth of your disclosure. Identify your current annual business planning and reporting processes. How can you use these processes as building blocks to enhance the breadth of your disclosure? Performance Measurement and Reporting for EPR Programs 2 3. Principles of Quality Performance Reporting Quality performance information is essential to ensure that internal and external decision makers have the right information to assess an organization’s activities accurately. As you determine what to include in your report and how to present this information most effectively, you should consider the following principles. Principle Question to Consider Does your report address all of the key issues that are relevant to your organization? Description Cover the important issues and indicators relevant to your organization’s activities. Be sure to consider the reasonable expectations and interests of your stakeholders. Sufficiently reflect all of your organization’s operational activities to enable stakeholders to assess your performance in the defined reporting period. Discuss how your performance has actually improved conditions in society or the environment and your business. Reflect both positive and negative aspects of your organization’s performance, explaining the reason for performance shortfalls and actions taken to correct them. “Warts and all” reporting enhances credibility and provides a true picture of overall performance. Present performance data over multiple years to facilitate an analysis of your organization’s performance over time, and consider using benchmarks to compare your performance to other organizations. Gather, record, compile, analyze, and disclose data and information in a transparent way that communicates the quality and relevance of the information. External audits can help to enhance the credibility of your report. Ensure that readers can readily understand the data and information that is presented within your report. Provide explanatory text for graphs and figures, as required, and identify limitations to use or extrapolation of presented data. 1.1 Relevance 1.2 Completeness Are you covering all of your operations and impacts? 1.3 Balance Do you discuss both positive and negative performance? 1.4 Comparability Does your report present performance data for three or more years, where available? 1.5 Reliability Do you have confidence in the methods used to calculate your data? Does your report present relevant contextual information and identify limitations associated with the presented data and information? 1.6 Clarity Performance Measurement and Reporting for EPR Programs 3 4. Report Boundary: Determining what’s in and what’s out Early in the reporting process, you will need to determine the boundaries – or scope – of your performance report. You will need to make decisions about what is included within the report, and what is excluded. It is important to determine the boundaries at the outset, so that the data and information you collect all respect the same boundaries. More specifically, you will need to consider the following: Geographic boundaries Organizational boundaries What facilities or regions will the report include? Will all business lines and activities be included in this report? How will sub-contracted activities (e.g. fleet transportation) be addressed? What time period will the report cover? (e.g. annual? Will it include trend information over three years?) Temporal boundaries In establishing the boundary of your report, consider your reporting aims and the intended audience. The boundary of the report should include entities and operational activities over which you have control or significant influence (e.g. program participants). The boundary also needs to be selected based on the impacts of your PRO and their significance (e.g. environmental, economic or social). Determining the appropriate report boundary will depend on your unique organization. Some considerations in determining the boundary of your report are presented on the following page. In addition to defining the boundary of entities/operations that are covered in the report, you should also clearly identify the reporting time period covered by the report. Do the data represent your organization’s fiscal year, or the calendar year? Do you report on an annual or biennial report cycle? Reporting can be most effective as a driver for change when it is aligned with your business planning cycle. Section 7.2 discusses the importance of clearly communicating the report boundaries within the report itself. Performance Measurement and Reporting for EPR Programs 4 Considerations Regulatory Requirements Implications If your PRO is regulated, the boundaries will be affected by the applicable legislation. There might also be specific reporting requirements tied to the kind organization your PRO represents (for example, if your PRO is a not for profit organization). The intended audience for your report may influence the report boundary. For example, if the local community is your key target audience, there may be more interest in a facility-level report than a broader report that covers all of your operations. The legal ownership of your organization may present a logical, defined boundary for your report. The boundary also needs to be selected based on the economic, environmental and social impacts of your PRO’s activities and their significance. For example, it would be reasonable to expect PROs to report on how they use or dispose of collected materials, since post-collection use and disposal can have significant impacts. The report boundary should be influenced by the entities over which you exercise control or significant influence, both upstream (e.g. program participants) and downstream (e.g. partners for reuse, recycling, recovery). For example, as a PRO you have no direct control over program participants. However, the education and awareness that you provide, and the quality of your service, could significantly influence uptake and participation in your EPR program. Therefore, reporting on activities related to program participants should be within the boundary of your report. The business relationships you have may also influence the report boundary for your report (e.g. partnerships with manufacturers and distributors, non-profit organizations). The boundary of your report may be partially driven by the availability of data that you have for some entities/aspects of your operations. If enough reliable data is not available for some aspects of your operations (e.g. no quantitative data on satisfaction levels), it may make sense to identify this deficiency and plan to address the aspect in more detail in future reports, after performance management systems have been developed or have begun to mature. In some cases, the costs of developing performance management systems to collect data at certain entities/ operations may not warrant the benefit of reporting. As such, these entities/ operations will not be included within the report boundary. Different entities/operations may also be collecting performance data using different equipment, measurements, etc. It may take some time to get all entities/operations collecting and reporting data in a consistent manner. This situation may result in excluding some entities/operations from the report until these systems are better aligned. Intended audience Legal ownership Major impacts Degree of control or influence Business relationships Availability of data Cost of data collection Ease of data comparability among entities and operations Performance Measurement and Reporting for EPR Programs 5 5. Report Format: Print or electronic? Determining how to communicate performance information is an important decision. What medium should you use to produce your report – print or webbased? If electronic, should it be a Portable Document Format (PDF) document, or an interactive Hyper Text Mark-up Language (HTML) report? Deciding whether to produce a print or fully electronic report will depend on: What medium best fits your needs; What medium would be best for your report’s key target audience(s); and What you have the necessary internal/external resources for – print or electronic? Reporting through electronic documents has become the minimum standard, since these documents are widely accessible to anyone with a computer and an internet connection. Some organizations choose to supplement electronic reports with a small number of printed reports or report summaries for distribution to certain segments of their audience. Some advantages and disadvantages of electronic and print reporting are identified in the table. Advantages can be produced using standard word processors can be distributed via your website content can be amplified through hotlinks to your organization’s website or other information on the Internet ability to track the number of downloads to provide information on distribution; on web-based reports, can track the types of information stakeholders are accessing reduces distribution costs maintain better control over distribution paper copies can be handed out at events, which can provide a good communications opportunity some readers prefer printed reports Disadvantages some members of the target audience may not have access to electronic reports (and some prefer hard copies) cannot be readily handed out at events (less communications impact in some situations) Electronic reporting Print reporting more expensive to produce and distribute greater direct use of resources to produce and distribute report (paper, ink, fuel) Performance Measurement and Reporting for EPR Programs 6 Tips for Electronic Reporting Ensure the electronic report information is user-friendly; Ensure the report can easily be found from your home page; Make sure it includes easy navigation; Create an attractive design with visual impact; Link to other organizational information and relevant external information; Design the report to allow different stakeholders to quickly access information of specific interest; and Create archives of previous years’ electronic reports to facilitate comparability and the monitoring of progress over time. Some organizations have also adopted a hybrid form of reporting. Hybrid reporting involves producing a concise paper report, and using the web to provide supplemental or the majority of the organization’s data and information. As an example, your print report could contain your PRO’s vision and strategy, a high-level roll-up of key performance information and a discussion of key issues. Your website could then contain background information that does not change frequently, such as policies and management systems, as well as detailed performance information. Using this approach can reduce costs and environmental impacts (such as paper consumption, ink use, shipping impacts) by keeping reports to a manageable size. One of the interesting advantages of electronic reporting is that it can provide you with insight into the behaviour of your report’s readers. Electronic reporting can help you to track the types of information your stakeholders are accessing, as well as the number of readers. Performance Measurement and Reporting for EPR Programs 7 6. Suggested Outline for Annual Performance Report Suggested Elements of Annual Performance Reports Program highlights Organizational profile Report profile Vision and strategy Context Management statement Strategy Governance Performance management Policies Programs Management system elements Stakeholder engagement Performance information Awareness Participation and accessibility Product collection Post-collection management Operational efficiency Quality of service Management performance Looking forward Note: Most PROs are required to publish annual audited financial statements. The elements recommended in this outline are intended to provide supplemental performance information that will be of use to the organization and its stakeholders. This information can be presented in the same report as the financial statements for ease of use and cross-referencing. The adjacent text box presents a suggested outline, or table of contents, for an annual performance report. Sections 7 and 8 are structured to mirror this suggested outline, providing detailed guidance on the types of information that could be provided for each of the suggested sub-headings. Section 7 provides guidance on reporting information related to organizational profile, vision and strategy, governance, performance management and stakeholder engagement. Section 8 provides guidance on presenting performance information, including a core set of key indicators and a small number of additional indicators for the seven categories of performance information shown in the suggested outline. Performance Measurement and Reporting for EPR Programs 8 7. Contextual and Management Information When deciding what to report, it is important to include both performance data and contextual information about your PRO and how it operates. This contextual information is crucial for understanding specific performance information and assessing your organization’s overall performance. In some areas, you may only have contextual information when you begin reporting, as performance management systems and indicators may not yet have been developed. Reporting is an iterative process and, as a starting point, your PRO’s report should include a discussion of issues where full data and information are not yet available, rather than not discussing an important issue in your report. Once your PRO identifies an important data or information gap (for example, related to one of the recommended indicators presented in Section 8), you should develop a plan to collect the required quantitative performance data and include these data in subsequent reports. The following sections outline recommended contextual and performance information that you should consider including in your performance report. Keep in mind that this outline should be used as a guide, and that you should ensure the report’s structure and content reflect the culture of your organization. Program Highlights Many organizations present a short summary of program management and operational highlights at the very beginning of the report. Presenting a high level snapshot of the organization and its achievements can serve almost as an executive summary to the report, and can set the tone for the remainder of the report. Highlights should include key performance indicators, including performance against stated objectives and targets. Organizational Profile Your performance report should clearly describe what your PRO does and where it operates. Providing information at the beginning of the report about your operations will ensure the reader has a general understanding of the scope of your activities, and will better inform the rest of the report. Performance Measurement and Reporting for EPR Programs 9 The organizational profile should include: Name of organization; Location of operations; The means of collecting products (pick-up or drop-off depots); Services provided, including the types and amount of products collected. It is important to be specific, especially if your organization collects complex products, such as electronics (e.g. Are the batteries collected with laptop computers part of the products your organization is mandated to collect?); If the organization has been established by regulatory means or under a voluntary initiative; A description of specific technologies or equipment you use to provide services; Program participants; Changes in operations compared to the previous reporting period, such as program expansions, or facility openings or closings; Nature of markets for collected materials; Number of employees; Total revenues, including breakdown by revenue source 3 ; Total expenditures; and Program budget/funding vs. expenditures/ costs vs. revenues. Design options to assist in presenting this information can include the use of a table that succinctly describes this information, or the use of a map identifying the location of your operations, communities serviced, the types of operational activities at different locations, number of employees, and so on. In its 2004 Annual Report, the Manitoba Association for Resource Recovery Corp. includes this map showing the location of its EcoCentres within the province. 3 PROs are often funded by multiple organizations, including through product levies, taxes and/or other user fees. PROs are encouraged to report on the breakdown (by percentage) of the funds received from each of the principal revenue sources. Where PROs are responsible for managing multiple products, they should also report on revenue breakdown by product category. Performance Measurement and Reporting for EPR Programs 10 Report Profile As discussed in Section 4, your report should include a short section that clearly articulates the scope of operations that are covered in the report, and the reporting time period covered (e.g. fiscal or calendar year, annual or biennial report cycle). If the report meets specific regulatory reporting requirements for your PRO, this should be clearly stated within the text. Clearly outlining the report boundary enables a reader to better understand the extent of operations for which you are reporting performance, and anything that is excluded. Vision and Strategy Context The report should provide a “situation analysis” which clearly describes areas of concern related to the sector, the product(s), their treatment and final disposition following the product’s useful life. This context section should explain the rationale for the Extended Producer Responsibility program, and should be forthcoming in acknowledging concerns surrounding the product(s) or the program. This section should also include a highlevel discussion of the environmental benefits associated with the product stewardship program (e.g. reduced greenhouse gas emissions, reduced energy consumption, reduced resource consumption). Management Statement Including a management statement from the most senior representative of your organization in your performance report is valuable to establishing your PRO’s commitment to addressing its strategic priorities, as well as bringing credibility to the report. Information that can be included within a senior management statement includes: Overall vision; Strategic priorities – short and longterm; Mission statement; Commitment to EPR; Major challenges; Key achievements, setbacks or trends related to strategic approach; and Performance against goals or targets. Strategy The report should present information about how your organization sets its strategic priorities and addresses key risks, impacts, and opportunities to your program and stakeholders. The report should articulate how the program’s goals address the key issues and concerns identified in the context section (e.g. related to the sector, product(s), treatment or final disposition of the product following its useful life). Information should be provided on any specific goals or target rates and dates (i.e. reduction, recycling, recovery, collection) by material or by product that have been set for the PRO. The performance indicators that you select and report on should be linked to these strategic priorities, goals and targets, where possible, demonstrating that the reported indicators are used in strategic and operational management and decision-making. Performance Measurement and Reporting for EPR Programs 11 Governance Presenting information on the program’s governance structure and systems provides important contextual information on how the PRO is managed, and how it fits within broader EPR programs. Describe the organization’s mandate, the underpinning regulatory structure of your PRO (i.e. established through voluntary or regulatory means), and how it is, or is not, achieving its mandate. Discuss the date of program initiation, and any governing bodies that may exist to provide an oversight function to your organization (i.e. Board of Directors or a Steering Committee that sets the strategic direction of the organization). Note that regulatory reporting requirements may require disclosure of more detailed information on specific aspects of organizational governance. In addition to identifying the roles and responsibilities of any governing bodies, briefly describe the PRO's overall operational management structure and how it carries out its business plan to achieve its goals. Since PROs can differ substantially from one another, it is important to clearly describe the roles and responsibilities of partners or members within the EPR program. This will provide the reader with an understanding of the complexity of dependencies within the program, and will help them to understand what part of the system is within the PRO’s direct control. Performance Management It is helpful to provide readers with an understanding of how performance is managed within your organization. To do this, you should include a description of management processes and systems that support day-to-day activities and support your overall accountability framework. Information in this section of the report can cover any policies, procedures, risk, environment, health and safety and other management systems and decision-making tools that help you to meet your commitments, and identify and manage relevant performance issues effectively. Specifically, the report could include a description of: Policies in place to guide performance; How the PRO sets priorities and targets; Major programs to improve performance; Internal and external communications and training; Performance monitoring; Internal and external auditing processes; and, Senior management review of processes and performance. Performance Measurement and Reporting for EPR Programs 12 Stakeholder Engagement Stakeholder engagement can be important to the success of an EPR program. Engaging your stakeholders in a meaningful dialogue enables both your PRO and your stakeholders to better understand and address each other’s concerns. Describing how your organization has identified its key stakeholders, how it solicits their input, and how their input is considered in your PRO’s decision-making processes are all important elements to report. By candidly acknowledging both positive and negative feedback, your report can enhance your PRO’s public credibility, and form a basis for continued stakeholder dialogue. It is important to ensure that your performance report meets your operational needs, as well as the needs of your key stakeholders. The report could include: A list of key partners and a description of the relationship between your PRO and those partners; A list of other stakeholders; Your approach to engaging stakeholders (e.g. use of surveys, community liaison committees or advisory panels, public meetings, etc.); and Key topics and concerns that have been raised through stakeholder engagement and how those concerns are being or were addressed. Stakeholders can be seen as individuals and groups who have an interest in, or believe they may be affected by, decisions respecting the management of the EPR program. Examples of PRO’s Stakeholders Communities in which it operates Certain government authorities Producers of products covered by your mandate Consumers of products covered by your mandate Employees Target communities (those parties who are, or can be, direct program participants) Other PROs Non-governmental organizations (NGOs) Program partners Suppliers and contractors Further guidance on the means of engaging stakeholders can be obtained from the International Association for Public Participation’s website (www.iap2.org). Performance Measurement and Reporting for EPR Programs 13 8. Reporting on Performance Indicators The main focus of your report should be on your organization’s performance over the reporting period, both good and bad. In order to accurately reflect performance, your report should include indicators that clearly reflect your organization’s operations and are consistent with the principles of quality performance reporting discussed in Section 3, and the attributes of quality performance information presented in the adjacent text box. Be sure to report against performance targets established in your PRO’s business plan to enable the reader to assess your performance. The selected indicators should be directly relevant to your mandate and your organization’s performance goals (including specific targets established in your business plan). Data and information on these indicators should be collected in a systematic and consistent way to ensure that they accurately represent your operations. The following sections present a core set of key performance indicators that all PROs should report, along with a small number of additional performance indicators and data collection advice for the indicators in seven performance categories: Awareness; Participation and accessibility; Product collection; Post-collection management; Operational efficiency; Quality of service; and Management performance. The following table presents a list of the specific performance indicators suggested in this guidance document for each category. Each indicator is described in substantial detail (including the rationale, explanation, calculation guidance and limitations) in the remainder of Section 8 of this document. What Makes a Good Indicator? Useful for operations, stewardship and public reporting Helps improve performance Communicates performance credibly Data can be collected reliably Data have been collected consistently over time to enable year-over-year comparisons Quality Performance Information Links policies, targets and performance Identifies key performance indicators Presents performance data in: Trends over 3+ years Absolute (i.e. total) and/or normalized (i.e. expressed in terms of the amount per key production variable, such as amount per unit collected, or amount per 10,000 population) terms Sets and communicates performance targets Explains shortfalls and actions taken Uses benchmarks for comparisons EPR Reporting Guidance Document 14 Recommended Performance Indicators for PROs Performance Category Awareness Section Key Indicators Percentage of population aware of the program Additional Indicators 8.1 8.2 Participation and accessibility Participation rate Population within a certain proximity of the drop-off depot Average travel distance to drop-off depot 8.3 Product collection Absolute collection Collection rate Absolute collection per capita Post-collection fate of material Distribution of expenses Cost per unit of collected material Target community and program partner satisfaction Progress against business plan goals and/or targets Regulatory noncompliances Natural resource consumption and efficiency Number and nature of complaints Service disruptions 8.4 Post-collection management Operational efficiency 8.5 8.6 Quality of service 8.7 Management performance Awards and recognition Performance Measurement and Reporting for EPR Programs 15 8.1 Awareness An EPR program can only be effective if the program’s target community is aware of and actively participates in it. Under this category, discuss the efforts that your PRO undertakes to increase the awareness of your program within the target community. This could include information on public outreach, communication/education and advertising activities, as well as investments in these areas. Some PROs track activity indicators, such as the number of visitors to their program website, the number of awareness education campaigns published in the media, and the number of enquiries from the target community. You could also discuss any circumstances that provide a challenge to increasing awareness of your program. This guide recommends that PROs report on one awareness indicator: percentage of the population aware of the program. Because of the high importance of awareness to a program’s success, this is identified as a recommended key performance indicator. Performance Measurement and Reporting for EPR Programs 16 Awareness Indicator Percentage of population aware of the program (key Discussion Rationale: The percentage of population aware of the program provides an indication of the effectiveness of your program in terms of reaching the target community for your program. Explanation: Population refers to the community of program participants that the EPR program aims to serve. This indicator requires active engagement with your program’s target community or program participants in order to determine their levels of awareness. Collecting this information may require an investment of resources to design, execute and analyse survey results. Calculation guidance: Calculate this indicator by determining the percentage of the target community that is aware of your program, compared with the total population of your targeted community. The indicator can be presented as a percentage. In determining awareness, it is equally important to assess the population’s understanding of the program. For instance, data can be presented against a number of awareness categories (e.g. very aware, somewhat aware, indifferent, and not aware). When gathering awareness information, carefully craft the survey questions so that you can assess the population’s actual understanding of the program. Also, consider partnering with other PROs in your area to conduct surveys jointly to reduce overall cost. Indicator limitations: This indicator requires the use of a survey tool to collect adequate information, as well as an understanding of the total population of your targeted community. Statistical methods can be applied to reduce the number of completed surveys needed for representative results. performance indicator) Performance Measurement and Reporting for EPR Programs 17 8.2 Participation and Accessibility Performance reports should communicate how accessible the EPR program is to members of the target community, and how many of the members of the target community participate in the program. Providing a measure of program participation, taking into consideration program accessibility to the target community, provides important contextual information on the scale of the program. Ultimately, participation and accessibility have strong direct impacts on the collection and recovery rates. It is therefore important to discuss and measure precisely both the participation and the accessibility of a program. This discussion needs to consider the product collection methods (pick-up or drop-off at collection depot). This section of the report can provide additional contextual information, such as plans for adjusting the program’s accessibility through expansion or reduction plans, changes in the number of collection facilities or the number of products involved in the program. You may also want to discuss any circumstances that limit participation and accessibility to the program (e.g. geographically dispersed target community, impacts of climate, etc.). This guide recommends that PROs report on one indicator in this category: participation rate. This is identified as a recommended key performance indicator for consideration and use. Where participation rates are difficult to calculate, this guide also identifies one additional indicator for consideration: accessibility of drop-off depots. Performance Measurement and Reporting for EPR Programs 18 Participation and Accessibility Indicator Participation Rate (key performance indicator) Discussion Rationale: The participation rate indicator provides the reader with important contextual information on both the scale of the program and its use. Many other performance indicators (such as those on collection rates, recovery rates or quality of service), need to be considered in light of both participation and accessibility to understand program performance. Explanation: The participation rate is expressed as the number of program participants relative to the target community that the EPR program serves. In other words, for a program that collects household recyclables, the target community would be the population of the city or particular region, whereas for industry-specific programs (such as the collection of refrigerants), the target community would be the companies that sell, service and dispose of cooling equipment in that city or area. Prior to determining the participation rate, one must first define and quantify the program’s target community. For instance, target community could be expressed in terms of the number of households to whom the program is conveniently available, or the number of service stations located within the target community. It is important not to overstate the size of the target community, since participation rate should be expressed relative to the realistic potential “reach” of the program. Calculation guidance: You will need to know the overall size of the target community to whom the program is conveniently available (this must be defined by each PRO – see additional guidance provided in the additional indicators below), and the absolute number of program participants (e.g. number of households or specific industries in a defined area). Participation rate is calculated by dividing the number of program participants (e.g. number of households that actively participate in the program or number of specific companies that actively participate within in a defined area) by the overall size of the target community to whom the program is conveniently available (e.g. number of households in a municipality or number of specific companies located in the defined area). # of program participants Participation rate = Overall size of target community Indicator limitations: In some cases, information may not be readily available on the overall size of the target community. An interim step might be to present information on total number of program participants over time, while working to determine access rate. x100 Performance Measurement and Reporting for EPR Programs 19 Participation and Accessibility Indicator Discussion Programs that operate drop-off depots may have difficulty determining the number of program participants. In such cases, consider using additional accessibility indicators (see below). Accessibility of drop-off depots (additional indicator) Rationale: PROs relying on drop-off depots may have difficulty identifying the number of program participants. While not as informative as the participation rate, the following indicators can be used to provide an indication of program accessibility. Explanation: There are two ways in which the accessibility of drop-off depots can be measured: 1. by identifying the population within a certain proximity to the drop-off depot; or 2. by identifying the average travel distance to a drop-off depot. Calculation guidance: 1. Population within a certain proximity of the drop-off depot As a first step, you need to determine the acceptable travel distance to the drop-off location. A number of factors need to be considered, such as urban or rural setting, availability and quality of public transport, travel time by different transportation modes (walking, cycling, public transit, driving), and the context of the product you collect (e.g. in an urban setting, you may expect people to drive for 20 minutes to drop-off an old fridge, but not to drop-off everyday household waste (such as beverage containers)). Once you have determined the acceptable travel distance to your drop-off depot, you will need to determine the population within that distance. Your municipal government should be able to help you with this. 2. Average travel distance to a drop-off depot To calculate the average travel distance to a drop-off depot, you need to know where the closest and farthest (potential) participants in your program are located. Identify how many (potential) program participants you have (that is, the size of the target community to whom the program is conveniently available), and roughly where they are located (e.g. within 1, 2, 3 kilometers, etc.). Calculate the average travel distance by building the sum of program participants (PP) multiplied by their respective distance (D), and dividing the result through the total number of program participants. Average travel distance = (PPxD1) + (PPxD2) + (PPxD3)…(PPxDn) Total # of PP Indicator limitations: Both of these indicators only provide information on the environment that affects accessibility, not on the level of actual participation in the program. Performance Measurement and Reporting for EPR Programs 20 8.3 Product Collection Product collection (that is, removing targeted products from the general waste stream) is one of the principal objectives for all PROs and, as such, is a key performance indicator. When discussing product collection, your report should describe the types of products your organization is responsible for collecting, the collection methods used (i.e. pickup versus depot drop-off system), and any difficulties in measuring product collection or challenges faced in improving collection rates (e.g. zoning issues that might limit siting of additional depots required to service the target community). Some industries may face significant limitations to presenting a normalized collection rate due to the nature of their products, in that products either have a very long life, get (partially) consumed (i.e. used oil or paint), or are no longer sold (e.g. certain refrigerants). In these instances, the report should clearly explain such limitations and the efforts undertaken to overcome them. Recognizing the importance of good quality baseline information to tracking the PRO’s progress towards its principal objectives, it would not be unreasonable to expect PROs to conduct or commission studies to establish the amount of products in the marketplace. Such studies could be undertaken as a collaborative effort by several PROs, or in cooperation with government departments. Your discussion of product collection should also include information on the products that are collected through your program, but are outside of the scope or mandate of your program. For example, this is a common problem for PROs collecting electronics. The original scope of the program may be to collect computers, but it is not intended to include third-party batteries in laptop or handheld computers, which are often returned along with the computers. Similarly, used oil programs often receive used anti-freeze containers. Performance reports should clearly identify and describe any contamination issues encountered and provide a measure or indication of the magnitude of this contamination. Making comparisons with product collection rates of similar programs, where appropriate, can provide a useful benchmark for readers. However, any comparison must consider the unique circumstances under which each PRO operates. This guide recommends that PROs report on three product collection indicators: absolute collection; collection rate; and absolute collection per capita. All three of these indicators are identified as recommended key performance indicators for consideration and use. Performance Measurement and Reporting for EPR Programs 21 Product Collection Indicators Absolute collection (key performance indicator) Discussion Rationale: Data on the absolute amount of a product collected by the EPR program provide an understanding of the program’s size and impact. Explanation: Absolute collection refers to the total amount of a product collected through the program and should be presented for each individual product in the EPR program. Absolute collection should be presented in terms of the number of units, mass (tonnes or kilograms), or volume (litres), depending on the specific product. For example, used oil should be presented in litres, whereas used oil filters could be presented in terms of the mass recovered. Beverage containers could be presented using the total number of units collected. Calculation guidance: To calculate absolute collection, add the total number of units, mass, or volume collected for each product in your program. Do not add product categories together. Indicator limitations: There are no limitations identified at this time. All programs should have access to this information. Collection rate (key performance indicator) Rationale: The collection rate indicator is important for determining program efficiency and effectiveness. It also allows for comparison across product categories and jurisdictions, although caution must be exercised in making such comparisons, as a number of variables can affect the collection rate (including program design, accessibility, awareness, and so on). Explanation: For any given product, the collection rate is the percentage of the total number of units sold that have been collected. Calculation guidance: Divide the number of units collected by the number of units sold for the specific product in question in the region of your program. Multiply by 100 to present this rate as a percentage. Performance Measurement and Reporting for EPR Programs 22 Product Collection Indicators Discussion Collectio n rate (%) = Units collected Unit sales X 100 OR Weight collected Weight of unit sales X 100 OR Volume collected Volume of unit sales x 100 Indicator limitations: In some cases, it may be difficult to determine how much of a product category is eligible for collection, particularly for products with a long life expectancy (i.e. automobiles), consumables (i.e. oil or paint), or products that are no longer sold (e.g. CFCs). Some jurisdictions are undertaking research in this area, which might provide a starting point for your work. For example, in Canada three studies were completed to establish baseline levels of certain refrigerants. In Europe, work has been completed to develop baseline data in electronics. The accessibility of a PRO’s program can distort the collection rate in cases where the whole target community cannot be reached (e.g. your PRO collects products through municipalities in a province, but not all municipalities in that province have agreed to collect the product on your behalf). This limitation underscores the interrelationships among different indicators, and the importance of providing explanatory information alongside performance indicators. Performance Measurement and Reporting for EPR Programs 23 Product Collection Indicators Absolute collection per capita (key performance indicator) Discussion Rationale: Absolute collection per capita can also provide an indication of the program’s efficiency and effectiveness. Explanation: Absolute collection per capita is the percentage of the product that has been collected relative to the size of the target community. Calculation guidance: Divide the absolute collection of a product (expressed in number of units, weight or volume, as appropriate for the material) by the size of the target community. For further guidance on calculating the size of the target community, see the Participation Rate discussion in Section 8.2. Absolute collection per capita Units collected = # of potential participants in target community Indicator limitations: This indicator does not allow for comparison across products, as it does not take into account that some persons in the target community will not use certain products. However, manufacturers may have information on the target community of their products that could help you estimate the users. Some products are targeted at certain age groups or gender. Comparing this with the demographics of your target community could be a first step. Performance Measurement and Reporting for EPR Programs 24 8.4 Post-Collection Management Post-collection management describes how PROs manage products in the postconsumer phase of the product cycle and where the products eventually end up. Performance data in this area help to identify the success of your program, its effectiveness, and provides some information on the environmental impact of your program. Data and information should include how your program manages collected materials, and their fate in terms of reuse, recycling, energy recovery, destruction, and landfilling. Your report should provide details on these uses to allow the reader to fully understand how the materials have been managed following their collection (e.g. incineration with energy recovery, incineration for destruction, or how materials have been recycled or reused). Challenges or limitations in increasing diversion rates should also be discussed (e.g. lack of commodity markets for certain products). This guide recommends that PROs report on one key performance indicator related to post-collection use: post-collection fate of materials. The absolute quantities and the percentage of collected material that is reused, recycled, incinerated, destroyed and landfilled should all be reported (as applicable). This is identified as a recommended key performance indicator. Performance Measurement and Reporting for EPR Programs 25 Post-Collection Management Indicators Postcollection fate of materials (key performance indicator) Discussion Rationale: Data on the post-collection fate of collected material provides an indication of how your organization is managing products in the postconsumer phase of the product life-cycle. It is recommended that the absolute quantity and percentage of materials treated in each of five categories be reported: reuse, recycling, incineration, destruction, and landfill. Explanation: The post-collection fate reports on the ultimate end of collected materials (that is, the allocation to reuse, recycling, incineration, destruction and landfill). The report should present both the absolute levels (either by quantity, weight or volume, as appropriate for the material) and the percentage of collected materials directed to each of these categories. Along with this indicator, you should also address the fate of the material that was collected through your program, but is outside of the program’s scope or mandate. Calculation guidance: Absolute quantities should be presented in terms of the number of units, mass (tonnes or kilograms), or volume (litres), depending on the specific product. For example, used oil should be presented in litres, whereas used oil filters could be presented in terms of the mass recovered. Beverage containers could be presented using the total number of units collected. Percentages should be calculated as follows: Postcollection fate reuse (%) Units reused = Units collected x 100 OR Weight/volume of unit reused Weight/volume collected x 100 Note that while the above formula is specific to percentage of material reused, this post-collection indicator should be calculated for reuse, recycling, incineration, destruction and landfilling for each product category. Indicator limitations: No limitations identified at this time. Performance Measurement and Reporting for EPR Programs 26 8.5 Operational Efficiency Operational efficiency indicators often draw upon internal financial indicators that are used by managers and analysts to benchmark performance in businesses and business sections, and to identify areas for improvement. Operational efficiency indicators in the context of PRO performance reporting are at a higher level. They provide an overview of the relative allocation of funds to the different functions of the organization, or put overall expenses or resource use into context with the amount of collected materials. PROs are often funded through multiple organizations, taxes, or user fees. Transparency on the efficient use of that money is important. This guide recommends that PROs report on two key operational efficiency indicators: distribution of expenses; and cost per unit of collected material. Both of these indicators are identified as recommended key performance indicators for consideration and use. This guide also recommends that PROs report on additional indicators related to natural resource consumption and efficiency. Performance Measurement and Reporting for EPR Programs 27 Operational Efficiency Indicators Distribution of expenses (key performance indicator) Discussion Rationale: Many PROs collect fees from participating industries or public institutions, or directly or indirectly from the population they serve. They also fulfill a function that directly affects tangible and intangible public goods (e.g. the environment and quality of life). Reporting on the relative allocation of funds to the different functions of the organization provides a degree of transparency which helps to demonstrate the program’s accountability to stakeholders. Explanation: For this indicator, the performance report should provide an overview of your program’s expenses for: • Product collection; • Program administration; • Post-collection activities, including reuse, recycling, incineration, destruction, and disposal to landfill; and • Awareness initiatives. Calculation guidance: The indicator can be expressed in absolute dollar values, or a percentage of overall expenses. The percentage of overall expenses is calculated as follows: Expense on activity Overall expenses x 100 Indicator limitations: This indicator is most useful when applied to a single organization over time. It is of limited use when reported for a single year only, or when it is used to compare different organizations (accounting definitions between organizations may vary and can distort the comparison). It is important to understand that operational efficiency is to a great extent driven by the environment in which an organization operates. For example, an organization with a geographically dispersed target community may incur higher cost for product collection than one operating in a big city. Some financial information (e.g. prices) is considered commercial confidential information and, as such, may not be disclosed. Performance Measurement and Reporting for EPR Programs 28 Operational Efficiency Indicators Cost per unit of collected material (key performance indicator) Discussion Rationale: Similar to expense distribution, measuring and reporting on the cost per unit of collected material helps to demonstrate accountability to stakeholders. Explanation: This indicator provides an understanding of the costs an EPR program incurs for each unit of collected material. Units vary by program type, but are typically expressed in weight (e.g. tonnes, kg), volume (e.g. litres, m3), or, less frequently, in absolute numbers (e.g. number of pop cans collected). If your program collects more than one kind of material, it is useful to calculate and report on the cost per unit for each material. This can provide important contextual information and provide a better understanding of overall program efficiency. For those programs that collect multiple products and do not calculate the cost for each one, it is important to explain the scope of materials covered within the indicator and to describe sources that could result in differing levels of cost efficiency per product. Calculation guidance: To calculate this indicator, you need to know the overall costs the program has incurred in a fiscal period, and the total amount of material that was collected. The cost per unit is calculated by dividing the overall cost by the number of collected units for each product. Total cost # of collected units Indicator limitations: Similar restrictions to those described for distribution of expenses apply to this indicator. Cost per unit cannot be easily compared between programs as they may operate in very different environments. Reporting on cost per unit for mixed product collection programs may result in reporting of an average cost. Natural resource consumption and efficiency (water, Rationale: PROs divert material from landfills for reuse, recycling or destruction and thereby help to reduce the environmental footprint of some products. The consumption and efficiency of resources applied to program activities provides an indication of the stresses that are put on the environment in fulfilling this function. Performance Measurement and Reporting for EPR Programs 29 Operational Efficiency Indicators energy, GHG emissions) (additional indicators) Discussion Explanation: This indicator provides information on the amount and efficiency of resources that are applied in the program’s operations. Typically, these resources include water and various energy sources (e.g. gasoline, oil, natural gas, electricity, etc.) but, depending on the nature and scope of program activity, may include other significant inputs. In order to provide an order of magnitude, absolute and relative natural resource consumption and efficiency indicators should be provided. Calculation guidance: The first step is to establish the boundaries within which you will quantify natural resource consumption; that is, for what scope of activities will you collect this information? While it would be most useful to collect information for the complete cradle-to-cradle product cycle, we recognize this would be a large undertaking, requiring involvement of multiple parties. This guidance document recommends that PROs consider and quantify natural resource consumption and efficiency (including GHG emissions) associated with all activities from initial collection of the products, to the delivery of these materials to market. This boundary reflects the portion of activities over which the PRO has the greatest control or influence. To calculate this indicator, you need to know the overall amount of natural resources that were consumed by the program (broken down individually) for each activity, and the total amount of material that was collected and processed. Resource efficiency is calculated by dividing the amount of a certain resource consumed (for example, kWh, kg, litres, and so on) by the amount of collected material (that is, by number of units, weight or volume, as appropriate for the material). Amount of natural resource consumed # of units, weight or volume of material collected When calculating GHG emissions, you need to know the key sources and amounts of energy consumption. These are typically electricity and fuel use. Consider any process-related energy use associated with the product collection and processing methods. To calculate the GHG emissions for your organization, you need to multiply the amount of energy used by the appropriate emission Performance Measurement and Reporting for EPR Programs 30 Operational Efficiency Indicators Discussion factors. This calculation has to be completed for each energy source you use separately. Be aware that emissions factors vary by energy type and geography. For instance, electricity emissions factors vary by province because of the fuel mix used to generate it. You can obtain specific emission factors from your energy supplier, or consult National Resources Canada’s Office of Energy Efficiency for further guidance (http://oee.nrcan.gc.ca/). Indicator limitations: In line with the other operational efficiency indicators, this indicator is not necessarily suitable for comparing resource efficiency between programs. Within the report, PROs should clearly identify other variables that affect resource efficiency, since the amount of material collected is only one variable. Performance Measurement and Reporting for EPR Programs 31 8.6 Quality of Service Quality of service indicators provide a measure of how well programs are serving their intended target communities and program partners. The most direct and accurate way to obtain an indication of service quality is to determine target community and program partner satisfaction. Other approaches to determining service quality are based on the assumption that certain events affect the quality of service. This assumption can be considered a weakness in these approaches; however, these approaches have an advantage in that they can be determined based on information generated from within program. This guide recommends that PROs report on one quality of service indicator: target community and program partner satisfaction. This is identified as a recommended key performance indicator for consideration and use. In the event that a true measure of target community and program partner satisfaction is not possible, this guide provides two additional indicators that could be used as a proxy for target community satisfaction: number and nature of complaints; and service disruptions. Performance Measurement and Reporting for EPR Programs 32 Quality of Service Indicators Target community and program partner satisfaction (key performance indicator) Discussion Rationale: Target community and program partner satisfaction provides the reader with information on the level of success a PRO has had in serving its target community and in collaborating with its partners. Explanation: Most PROs act as intermediaries in the end-of-lifecycle management of products. In fulfilling this role, they have two kinds of clients: the target community they serve through material collection, and program partners for post-collection activities. Satisfaction on both ends of the spectrum provides a good indication of the quality of service provided by your program. This guidance document strongly recommends that you undertake measurements of the satisfaction levels of both types of clients, as they will provide very different types of information about the quality of service. Calculation guidance: When considering how to determine target community and program partner satisfaction, consider these two approaches. One is to get a one-time snapshot of target community and program partner satisfaction rates, and the other is to develop an on-going measurement program. The one-time snapshot requires a survey (note: different surveys will be required for target community and program partners). These surveys can be done by mail, electronically or via telephone. The advantage of surveys is that you can ask a variety of questions and potentially reach a broad audience. Through this method, it takes relatively little time to obtain statistically significant results. The obvious disadvantage of this method is the requirement for resource investment in survey design, execution, and analysis. On-going program measurement can be achieved by combining routine program activities (e.g. invoicing for customers, regular meetings with program partners), with level of satisfaction questions. The disadvantages of this approach are that a limited target community is reached at one time, not as many questions can be asked at one time, and that the return rate may be lower than for a survey. As such, it might take a long time to obtain statistically significant results. Performance Measurement and Reporting for EPR Programs 33 Quality of Service Indicators Discussion Indicator limitations: No limitations identified at this time. Number and nature of complaints (additional indicator as a proxy for target community satisfaction) Rationale: The number of complaints from your target community provides an indication of level of satisfaction. The nature of those complaints provides further information on the areas of your program that are not working sufficiently. Explanation: This indicator refers to the number and nature of calls or complaint letters your organization receives from its target communities. It is important to distinguish between complaints from target communities, and from the general public. This distinction might be difficult for programs that collect household recyclables (e.g. blue box in Ontario). Calculation guidance: Simply count and document the number and nature of complaints your program receives from its target communities. Have a designated person/position within your organization to receive complaints, and consider developing a procedure for documenting and responding to complaints. Reach out to your partners for comments on your collaboration, and ask for suggestions on how it could be improved. Indicator limitations: This indicator will only capture complaints from very dissatisfied individuals within the target communities, assuming a certain tolerance level on the target community side. Target community complaints might not reach your program if you use subcontractors/third parties for product collection and/or delivery. Performance Measurement and Reporting for EPR Programs 34 Quality of Service Indicators Service disruptions (additional indicator as a proxy for target community satisfaction) Discussion Rationale: Service disruptions may lead to target community dissatisfaction in that product pick-up or drop-off is delayed or not possible. Explanation: This indicator is based on the assumption that a service disruption leads to target community dissatisfaction. Calculation guidance: To calculate this indicator, you need to track the number and duration of internal service disruptions. Use an appropriate scale to express duration (e.g. hours, days or weeks). Indicator limitations: The assumption that a service disruption leads to target community dissatisfaction might not always be correct. This indicator does not provide any information on the severity of the disruption. Performance Measurement and Reporting for EPR Programs 35 8.7 Management Performance A broad range of management performance indicators exists, especially for financial performance management. Financial information and indicators are most commonly disclosed in financial sections of annual reports, and are not included within this guidance document. A number of additional indicators provide readers with useful information about specific and overall management performance of a PRO. This guide recommends that PROs report on up to three management performance indicators: Progress against business plan goals and/or targets; regulatory non-compliances; and awards and recognition. The first two are identified as recommended indicators. Awards and recognition is presented as an additional indicator for consideration and use. Performance Measurement and Reporting for EPR Programs 36 Management Performance Indicators Progress against business plan goals and/or targets (key performance indicator) Discussion Rationale: Reporting against goals or targets documented in your program’s business plan is an indication that your program applies good management practice. Explanation: The five elements of good management practice are policy, planning, implementation, controlling and monitoring, and management review. Reporting against the goals and/or targets of your business plan shows the level to which you are implementing your plan, and demonstrates that you are monitoring and reviewing performance. Calculation guidance: The information a PRO can provide under this indicator varies with the detail and quality of its business plan. Ideally, one should report against SMART targets (Specific, Manageable, Assessable, Relevant, Time-specific). Report on action taken to achieve the goal/target, issues that were encountered and how they were resolved, and action that is yet to be taken to achieve the goal/target. Indicator limitations: The quality and usefulness of information provided related to this indicator varies with the level of detail and quality of goals and/or targets in the PRO’s business plan. Performance Measurement and Reporting for EPR Programs 37 Management Performance Indicators Regulatory noncompliances (key performance indicator) Discussion Rationale: The number of regulatory non-compliances resulting in penalties to your program provides a useful indicator for the presence and effectiveness of management systems. Regulatory non-compliances could include reportable spills, health and safety incidents, exceedances of operating noise levels, and so on. Explanation: The occurrence of incidents leading to penalties is often an indication of a lack of program knowledge of applicable legislation, or the lack of appropriate management of related operations. This indicator does not include reporting on program non-compliances, but focuses purely on non-compliance with regulations. This indicator also reports only on situations of non-compliance detected within the PRO’s own operations and activities, as well as those of its first-tier suppliers. It does not include monitoring and reporting on non-compliances of more distant sub-contractors and partners. This boundary reflects the portion of activities over which the PRO has the greatest control or influence. Calculation guidance: Regulatory non-compliances can be expressed as the absolute number of legal non-compliances resulting in penalties against the organization and its first-tier suppliers, and in the dollar value of fines the organization (or its first-tier suppliers) has had to pay. Indicator limitations: In many cases, regulatory non-compliances represent a failure of a management system. This indicator does not provide information on aspects of management other than compliance with applicable legislation. Awards and recognition (additional indicator) Rationale: Awards and recognition can provide a good indication of the quality of management performance. Explanation: Awards and recognition are given to progressive programs from their peers and/or stakeholder groups. They represent external recognition of program management from groups that understand the issues and implications of an industry. Calculation guidance: Where applicable, simply list and provide a description of the awards and recognition that your program has received. Performance Measurement and Reporting for EPR Programs 38 Management Performance Indicators Discussion Indicator limitations: Many times, awards and recognition are given only for certain aspects of management and/or program performance (e.g. outstanding levels of service, exceptional environmental track record, technical innovation), and they do not reflect performance in all aspects of program management. Performance Measurement and Reporting for EPR Programs 39 9. Looking Ahead: Ensuring a forward-looking report Effective performance reports are forward looking. While it is important to include data and information on past performance, it is equally important to provide the reader with an understanding of the PRO’s priorities and plans for the coming year. This allows readers to understand how management plans to respond to identified challenges and opportunities for the organization. It can also help to ensure expectations for the organization are realistic and reflect an understanding of the organization’s priorities. Performance reports often include information on the organization’s plans to enhance or expand its data and information collection processes. This level of transparency around the quality and adequacy of current processes communicates that performance management is not static within the organization. Rather, it demonstrates that the organization is working to improve its performance management and reporting practices. Forward-looking information can be included throughout the report; it does not need to be presented in a separate, stand-alone section. Some organizations prefer to include a separate section of their report that clearly presents their future plans and priorities. This section could include a list of specific targets for the coming year. Performance Measurement and Reporting for EPR Programs 40 10. Conclusion Performance reporting can add business value to your organization directly by helping you to measure and assess your own performance, and indirectly by driving improved performance through enhanced transparency. In order to maximize the benefits of reporting to your organization, the reporting process should consider and reflect: The principles of good quality reporting; The geographic, organizational, and temporal boundaries of the report; Adequate contextual information to allow for accurate interpretation of performance information; and, A set of key performance indicators that clearly reflect your organization’s operations. Suggested Elements of Annual Performance Reports Program highlights Organizational profile Report profile Vision and strategy Context Management statement Strategy Governance Performance management Policies Programs Management system elements Stakeholder engagement Performance information Awareness Participation and accessibility Product-collection Post-collection management Operational efficiency Quality of service Management performance Looking forward This guidance document presents a suggested outline for a PRO’s annual performance report. Most PROs are required to publish annual audited financial statements. The elements recommended in this outline are intended to provide supplemental performance information that will be of use to the organization and its stakeholders. This information can be presented in the same report as the financial statements for ease of use and cross-referencing. This guidance document recommends a core set of key performance indicators and a small number of additional indicators for use by PROs. All PROs should report on the recommended key indicators and, as your data collection and reporting systems mature, you may consider expanding the scope to include relevant additional indicators. Performance Category Awareness Participation and accessibility Product collection Postcollection management Operational efficiency Quality of service Management performance Key Performance Indicators Percentage of population aware of the program Participation rate Absolute collection Collection rate Absolute collection per capita Post-collection fate of material Distribution of expenses Cost per unit of collected material Target community and program partner satisfaction Progress against business plan goals and/or targets Regulatory non-compliances Performance Measurement and Reporting for EPR Programs 41 Performance Measurement and Reporting for EPR Programs EPR Appendix A: Annual Performance Report Review Checklist You can use the checklist provided below to assess the extent to which your report provides information related to each of the sections in the suggested outline for an annual performance report. Over time, you can strive to address all of these elements within your report. REPORTING OUTLINE QUESTIONS (Please check box if your response is yes) Does your report include an organizational profile, describing your PRO, what it does and where it operates? Do you clearly identify the report profile, articulating the scope of operations and the time period covered in the report? Is there a management statement in the report discussing your PRO’s overall vision, strategic priorities and key achievements, setbacks and challenges going forward? Does the report describe key risks, impacts, and opportunities to your program and stakeholders that affect your PRO’s strategy? Are the governance and management structures, such as your PRO’s mandate, underpinning regulatory structure, governing bodies and roles and responsibilities described? Is there a description of performance management processes and systems that support day-to-day activities, such as decision-making tools, programs to improve performance, communications and training initiatives or audit processes? Does your report discuss stakeholder engagement and identify key partners, other key stakeholders, mechanisms of engagement (e.g. surveys) and key topics and concerns raised by your stakeholders? Are the reported performance indicators reflective of your organization’s operations and the principles of quality performance reporting (e.g. comparable, reliable, etc.)? Do they link to your mandate and your specific organizational performance goals? Do they cover, at a minimum, the core set of key performance indicators? Awareness Percentage of population aware of the program Participation and accessibility Participation rate Population within a certain proximity of the drop-off depot Average travel distance to drop-off depot Product collection Absolute collection Collection rate Absolute collection per capita Post-collection management Post-collection fate of material Operational efficiency Distribution of expenses Cost per unit of collected material Natural resource consumption and efficiency Quality of service Target community and program partner satisfaction Number and nature of Complaints Service disruptions Management performance Progress against business plan goals and/or targets Regulatory non-compliances Awards and recognition Do they provide trend data over time, along with an explanation of changes over time? Does the report communicate future performance targets for key performance indicators, where these have been established? Key Performance Indicators Additional Performance Indicators Checklist Appendix B: References and Supporting Links B.1 Sample of Annual Reports Produced by PROs Alberta Beverage Container Recycling Corporation (ABCRC). 2005 Annual Report. Calgary: 2005. http://www.abcrc.com/cfm/index.cfm?It=103&Id=1&Se=37 Albert Dairy Council Milk Container Recycling Program. Swish'n Squish, 2004-2005 Annual Report. Leduc: 2005. http://www.milkcontainerrecycling.com/AB/publications.htm Alberta Recycling Management Authority. 2005-2006 Annual Report. Edmonton, 2006. http://www.trma.com/admin/dynamicpage/default.cfm?PageId=138 Alberta Recycling Management Authority. From landfills to leadership, report to Albertans (March 2006), Edmonton, 2006. http://www.trma.com/web/sitemap.cfm Alberta Used Oil Management Association. 2005 Annual Report. Edmonton: 2005. http://usedoilrecycling.com/en/province.aspx?prov=1 Auto Recycling Nederland BV. Millieuverslag Environmental Report 2005. Amsterdam: 2006. http://www.arn.nl/ Beverage Container Program. 2005-2006 Annual Report. Yellowknife: 2006. http://www.enr.gov.nt.ca/eps/beverage.htm Boissons Gazeuses Environment. Rapport Statistiques de Vente et de Recyclage (2004). St-Laurent: 2005. http://www.bge-quebec.com/en/industrie4.asp British Columbia Used oil Management Association. 2005 Annual Report. http://usedoilrecycling.com/en/province.aspx?prov=2 Manitoba Association for Resource Recovery Corporation. MARRC 2004 Annual Report. Winnipeg: 2005. http://usedoilrecycling.com/en/province.aspx?prov=3 Manitoba Product Stewardship Corporation. 2004-2005 Annual Report. 2005. http://www.mpsc.com/main.asp?contentID=35 New Brunswick Tire Stewardship Board. 2004 Annual Report. Fredericton: 2005. http://www.nbtire.com/e/100/105e.htm Post-Consumer Residual Stewardship Program. Medications Return Program Report (2004). 2005. http://www.medicationsreturn.ca/british_columbia_en.php References - 1 Product Care Association. 2005 Annual Report. 2006. http://www.productcare.org/index.html Rechargeable Battery Recycling Corporation (RBRC). 2005, the Year in Review. Atlanta: 2006. http://www.rbrc.org/call2recycle/corporate/index.html Refrigerant Management Canada. 2005 Annual Report. Mississauga: 2006. http://www.hrai.ca/rmc/ Resource Recovery Fund Board, Nova Scotia. New Journeys 2005 Annual Report. Truro: 2005. http://www.rrfb.com/pages/about.html Saskatchewan Association for Resource Recovery Corp. 10th Annual (Report 2005). Saskatoon: 2006. http://usedoilrecycling.com/en/province.aspx?prov=12 Stewardship Ontario. Annual Report 2005. Toronto: 2006. http://www.stewardshipontario.ca/plans/plans_docs.htm The Beer Store. Responsible Stewardship 2004-05 Annual Report. http://www.wdo.ca/content/?path=page82+item35783 Waste Diversion Ontario. 2005 Annual Report. North York: 2006. http://www.wdo.ca/reports/Default.aspx North York: 2005. B.2 Reporting Guidance Alberta Environment. May 2006. Delegated Administrative Organizations, Monitoring and Evaluation Workbook. Beverage Container Management Board (Management Consultants Alberta Ltd.) Undated. Performance Measures for Beverage Container Recycling Within Alberta. Department of Environment, Food, and Rural Affairs, Government of the United Kingdom. 2006. Environmental Key performance Indicators, Reporting Guidelines for UK Business. Global Reporting Initiative (GRI). 2006. Sustainability Reporting Guidelines, Version 3.0. Amsterdam: GRI. http://www.globalreporting.org/ReportingFramework/G3Online/ References - 2 Global Reporting Initiative. March 2005. Sector Supplement for Public Agencies. Government of Canada and Stratos Inc. 2003. Sustainability Reporting Toolkit. Government of Canada: Ottawa. http://sustainabilityreporting.ca National Resources Canada: Office of Energy Efficiency. http://oee.nrcan.gc.ca/ National Standard of Canada CA/CSA-ISO 14031-00. March 2000. Environmental management – Environmental performance evaluation – Guidelines. Ontario Ministry of Environment. May 1998. Environmental Performance and Competitive Advantage: A Business Guide. http://www.ec.gc.ca/cppic/En/refView.cfm?refId=1841 Stratos Inc. 2005. Gaining Momentum- Corporate Sustainability Reporting in Canada. Stratos: Ottawa. (2007 benchmark survey expected release in December 2007) http://www.stratos-sts.com/pages/publica011.htm B.3 General Extended Producer Responsibility Documentation Agence De l’Environnement et de la Maitrise se l’Energie. May 2003. European overview of Producer Responsibility Organizations and Bodies Assuming Producer Responsibility in Managing End-of-life Products. B. Fishbien. Undated. EPR: What does it Mean? Where is it Headed? Journal Article, Inform. http://www.informinc.org Commission of the European Communities. 2003. Communication from the Commission: Towards a Thematic Strategy on the Prevention and Recycling of Waste. Environment Canada. March 2007. Extended Producer Responsibility and Stewardship. http://www.ec.gc.ca/epr/en/index.cfm Environment Canada (Marbek Resource Consultants Ltd.). January 2002. Assessing When to Implement Extended Producer Responsibility, a Workbook. Environment Canada (Marbek Resource Consultants Ltd.) March 2006. Inventory of Canadian EPR & Stewardship Programs. Environment Canada. August 2001. Guidance Manual for Establishing, Maintaining and Improving Producer Responsibility Organizations in Canada. Institute for Local Self-Reliance. 2006. Wastes to Wealth – Extended Producer Responsibility Tools. http://www.ilsr.org/recycling/epr/tools.html References - 3 Organization for Economic Co-operation and Development, Environment Directorate – Working Group on Waste Prevention and Recycling. 2006. EPR Policies and Product Design: Economic Theory and Selected Case Studies. Organization for Economic Co-operation and Development, Environment Directorate – Environment Policy Committee. 2003. Working Party on National Environmental Policy, Proceedings of OECD seminar on Extended Producer Responsibility: EPR programme implementation and Assessment, Part 1: Taking Stock of Operating EPR Programmes. Organization for Economic Co-operation and Development, Environment Directorate – Environment Policy Committee. 2003. Working Party on National Environmental Policy, Proceedings of OECD seminar on Extended Producer Responsibility: EPR programme implementation and Assessment, Part 2: Assessing EPR Policies and Programmes. Organization for Economic Co-operation and Development, Environment Directorate – Environment Policy Committee. 2001. Extended Producer Responsibility Guidance Manual for Governments. Pollution Probe. 1999. Extended Producer Responsibility. The International Institute for Industrial Environmental Economics. 2003. EPR Programmes: Individual versus Collective Responsibilities. References - 4