Outline
• Definitions of Research/Evaluation
• Purposes of Evaluation / Research
• Science and Scientific Management
• Research/Evaluation as Process
• Types of Research / Evaluation
Definitions
• Evaluation = Process of judging the merit or
worth of something
• Research
– application of scientific methods to answer
questions
– controlled inquiry directed at increasing
knowledge/establishing truth
• Evaluation Research - combine the two
Research Evaluation
Evaluation Research
Why Evaluate - the Academic list
• To assess merits of alternative programs
• To discover whether & how well objectives
are being fulfilled
• To determine the reasons for successes &
failures.
• To uncover the principles underlying a
successful program.
• To refine, revise, update or track a program
Why Evaluate- "The Real List"
1. Because we are required to
2. To make better decisions.
3. To learn from experience
4. To justify programs
5. To kill programs
Purposes of Research
• Answer management questions - applied research
• For sake of knowing - pure or basic research
• Answer research questions - methodological
• Develop/test management alternatives -
developmental research
• Assess worth or merit of programs - evaluation
research
Research Purposes - II
• Exploratory
• Descriptive
• Explanatory
• Predictive
Ways of Knowing
or Establishing belief or truth
• Tradition
• Authority
• Repetition/Tenacity
• Science
Science
• Body of Knowledge • Method of Inquiry
• systematic • logical
• abstract – induction
• general – deduction
• parsimonious • self-corrective
• empirical
Scientific Management
• Application of scientific principles to
management and decision making
– systematic information gathering
– empirical, objective, self-corrective
Process -- Steps
Research Evaluation
• define problem • describe program
• objectives/hypotheses • evaluation criteria
• literature review • program scoping
• research methods • evaluation methods
• gather data/analysis • gather data/analysis
• conclusions • conclusions
Types of Evaluation
by Program Stage
• formative (conceptualization/design)]
• process (implementation)
• summative (outcomes, impacts, efficiency)
Types - By Approach
• Standards
– norm-based
– criterion-referenced
• Goals and objectives
• Impacts or effects
Types- by method
• Qualitative - quantitative
• Secondary data - Primary
• Survey - Experiment
• Internal vs external
• Who - Peers, participants, superiors,
subordinates, scientists
Evaluation Criteria
• Effort - qnty and qlty of inputs
• Performance - qnty and qlty of outputs
• Adequacy - meet needs?
• Efficiency - benefits/costs
• Equity - distributional issues, fairness
Process Evaluation
• Identifies how and why program works
– attributes
– recipients
– conditions
– effects
• single or multiple
• intended or side effects
• timing & duration, long/short term
• cognitive, affective or behavioral
Types by program area/subarea
• Emphasis: Parks, Tourism, Interp,
Program Mgmt, Commercial Rec, ...
• Program : lands, facilities, personnel,
budget, marketing, PR, maintenance,
design, policies, plans
Types of Research
• Basic - Applied
• Exploratory, Descriptive, Explanatory,
Predictive
• In-house / out- house
• by discipline - sociological, psych, economics
• by topic - boating, wilderness, legal, tourism, ...
• by method - survey, expmt, ...
Examples of Evaluation Studies
• Community needs assessment
• Feasibility study, SWOT analysis
• Program Audits, Market audit
• Cost/benefit anaysis
• Impact assessment
• PPBS, MBO, GPRA, CAPRA, LAC, ...
• Peer review
• Importance-performance analysis