Results of Implementation and Testing Soils and Riparian – What

							Results of Implementation and
 Testing Soils and Riparian –
    What Did We Learn?
Purpose of Routine Evaluations
• Unclear
• It is our understanding that this was driven
  by the need to assess results under
  results-based legislation, ie FRPA
• The underlying purpose common to most
  evaluations of forest resources is to learn
  about forest practices.
           Uses of Indicators
•   Compliance and Effectiveness Audits
•   Inspections
•   Long-term Research
•   Effectiveness Monitoring
•   Etc.
           What Did We Do?
• Compliance and Effectiveness Audits
• Testing the indicators was not the objective of
  the audits
• We used the indicators as the basis for a set of
  commonly accepted indictors of effective soil
  conservation and stream management.
• Can’t talk about the audit results, as not yet
  reported
• Can discuss the process and use of the
  indicators in the audits
    Use of Indicators in Audits
• Audit relates to the forest practices (versus
  streams or soils)
• Indicators used to establish common
  understanding of effectiveness of forest
  practices impacting soils and streams
• This required some adjustments to the
  draft indicators to ensure that the use of
  the indicators was informative about the
  underlying forest practices. (handout)
       Use of Indicators Cont.
• We also developed audit programs that
  incorporated the indicators to allow audit
  analysis. This provides the basis of determining
  whether or not an indicator is achieved, and
  why.
• This is an important point. The technical design
  of an evaluation is critical to proper application
  of the indicators.
• Development teams should consider the type,
  purpose and objectives of “routine evaluations”
  in developing indicators.
          What did we Learn?
           (Key Messages)

• The development teams developed
  indicators and methodologies; the audit
  teams used the indicators, not the
  methodologies
• Would have been beneficial if the
  development teams had involvement of
  persons with varied expertise that will be
  using the indicators.
             What did we Learn?
                   (Cont.)
• Compliance audit assessment at same time as the
  effectiveness assessment works well – provides linkage
  to the forest practices
• Implementation teams need to incorporate the necessary
  expertise – we found substantial benefit in having one of
  the development team members on each audit team
• Site level field assessments were necessary to use the
  indicators (for most assessments)
• Using professional judgement inherent to using
  indicators
   – When to apply and take detailed measurements
   – Interpreting results
   What did we Learn? (Cont.)
• Indicators need to be general enough to
  facilitate the use of professional judgement
• Indicators, as developed, were “commonly
  accepted” indicators.
• Each indicator should be supported by a
  short rationale
   What did we Learn? (Cont.)
• Recommend that the indicators be tested using
  different routine evaluations e.g. compliance and
  enforcement inspections
• As part of training, having indicator developers
  scientists and implementers in the field together
  prior to starting audits was very beneficial
• Can not evaluate achievement of indicators
  without the collection of data (checklists
  developed).
• Different forms of data for different evaluations
    What did we Learn? Cont.
• Indicators we used were “accepted” as
  common indicators (no argument from
  auditees – recognize that audits are not
  complete)
• The combination of using scientists,
  foresters, experts, auditors, etc. works
  very well.
• Licensees, stewardship groups should be
  involved in indicator development as well.
          Lessons Learned
• The model of MOF developing indicators
  and board auditors developing the audit
  program worked adequately
  – Stronger relationship between the indicators
    and forest practices needed
  – Indicators were essentially premised on best
    management practices
END