policy evaluation

							          Policy Evaluation

Goal is to determine whether a social
  intervention has produced the
  intended result.

Results are not always well received.

Probably the most difficult type of
  research to do well
1. Specifying Goals
    Ambiguous goals – Vietnam/Iraq war
    Conflicting goals – UNLV - final 4/Acad.
    success
2. Identifying Unintended Consequences
3. Identifying Who Benefits/Who Suffers
4. Developing Methods and Measures of
    Evaluation
    Qualitative
    Quantitative
Evaluation Research Designs:

Experimental designs
  Control/treatment group

Quasi-experimental designs
  Time-series design
  Correlational design

Qualitative evaluations
  Focus groups, interviews
Ethical Issues:

  Social interventions being evaluated may
    raise ethical issues.

  Identification of drug users, sex offenders

  Deciding who gets treatment and who
    doesn’t
Research may be a mask for unethical
  behavior.
  Police survey of prostitutes
  Alabama treatment of black men with syphilis


Are Evaluations simply used to legitimize a
  social intervention?

How are the evaluations used?
THE POLITICS OF POLICY EVALUATION

1. Who Evaluates?
Elites: Government Agencies, Public
    Universities, Think Tanks
Interest Groups: AAA, ACLU, AMA,
    Business groups, etc.
Citizens: Complaints; Whistle Blower laws,
    Public Opinion; Voting; Recalls
2. Evaluation and Bureaucratic Oversight
(at least) Two reasons for evaluation:
Are the policy solutions working
Are the policy solutions really being
    implemented

3. How is Evaluation done?
Haphazardly: takes time and money
Fire alarm v. policy patrol model of oversight
4. Why do governments usually fail at
   their own efforts at policy
   evaluation?

5. Why are government programs
   rarely terminated, even when
   evaluations show that the programs
   are ineffective?

6. Interpretations of Evaluations (i.e., is
   the EPA doing a good job?)
WHY EVALUATIONS MAY BE INGORED

Too Complicated: Implications may not be
   presented in a way that nonresearchers can
   understand.

Results can sometimes be counter-intuitive =
  Results sometimes contradict deeply held
  beliefs.

Vested interest in a program.

Who evaluates government policies?

Answer: the government.
Welfare Reform: Is it a Success?
What were the Goals/Objectives?
How do we measures of success?

Police Consolidation?

Voucher system?
Welfare Reform: Is it a Success?
• What were the Goals/Objectives?
  – Break the cycle of poverty
  – Save money/lower spending

• How do we measure success?
  – Decrease in the number of people on welfare
  – Decrease in the number of people below the
    poverty line
 School Choice: Is it a Success?
• What does the data say?
• Interpreting competing results
  – Mortality rate
  – Selection bias
  – Missing data
  – Outliers
          Concluding remarks
•   Understand the process
•   Evaluate the process
•   Democratic or not democratic
•   More democracy/less democracy?
•   Less regulation, more regulation?
•   Recommend changes?