____THE AMERICAN UNIVERSITY

Report as inappropriate Your report has been received

Shared by: ekx2ojOV

Tags

Stats

views:: 13
posted:: 12/3/2011
language:: English
pages:: 12

Public Domain

Document Sample

THE AMERICAN UNIVERSITY
SCHOOL OF PUBLIC AFFAIRS

SYLLABUS

PUAD604.001 Program Evaluation Laura Langbein
Fall 2006 Ward 346
Weds., 5:30-8:00 Office hrs:
Weds. 2:30-5pm
(except 1st Wed.
of every month)
Th. 2:30-6pm; and
by appt.; late hrs.
by appt. only
Phone: 202/885-6233
langbei@american.edu

Course Objectives

The purpose of this course is to introduce a variety of research designs
and related methodological tools useful for evaluating the
implementation and impact of public programs. The course focuses
explicitly on the use of statistics together with appropriate
experimental, quasi-experimental and non-experimental research designs
as ways of collecting and interpreting data for the systematic analysis
of program implementation and impact. The course includes a
consideration of measurement, including the use of surveys and in-depth
case studies as ways of collecting data for systematic measurement. The
course also includes practical applications and centers on the
advantages and disadvantages of alternative research designs and methods
of data collection.

Main Texts

Langbein, Public Program Evaluation: A Statistical Guide. (This text
will be available by Sept. 22 in the AU bookstore; the first
chapters assigned before that date will be in BB.)
Bingham and Felbinger, eds., Evaluation in Practice (2nd ed.)
Thomas Mangione, Mail Surveys
Robert Yin, Case Study Research (3rd ed.)
Jean Converse and Stanley Presser, Survey Questions: Handcrafting
the Standardized Questionnaire
Berry, Understanding Regression Assumptions (in by Oct. 1)
Berry and Sanders, Understanding Multivariate Research (in by Oct. 1)

Recommended:
Wholey, Hatry and Newcomer, eds., Handbook of Practical Program
Evaluation
Rossi and Freeman, Evaluation: A Systematic Approach (old editions ok)
Fowler, Improving Survey Questions: Design and Evaluation

Other readings will be in Blackboard, on the web, or in the periodical stacks.
Required readings are singly asterisked. Most of the required readings you
must purchase. The rest are in Blackboard. Strongly recommended readings are
doubly asterisked; many are in BB, or in the AU library. Many of the readings
on quasi- and non-experimental designs are examples; you need not read every
one. Many of the readings listed on the syllabus below are available on the
web. Use one of the social science/PAIS search tools provided by the library
to see if you can download the article. Many of the articles are in JSTOR or
Econlit.

1
2

Many other examples of program evaluations (usually non-experimental designs
and occasionally some quasi-experimental designs) appear in the following
journals, where you may find more varied examples than those in the syllabus:
Evaluation Review; Evaluation Practice; American Journal of Evaluation;
Journal of Policy Analysis and Management; Social Science Quarterly; American
Economic Review; American Journal of Political Science; Policy Studies
Journal; Journal of Law and Economics.

Annual Meeting: The Annual meeting of the Association for Public Policy
Analysis and Management (APPAM) will be in Madison, Wisc. this year, on Nov.
2-4. I recommend that you download some of the papers to see what the
"business" of policy analysis and evaluation looks like, and to find about
employment opportunities in the field. All of the information is available
at www.appam.org. (Warning: I have to be in Madison by 8am Nov. 2; I will not
hold Thursday office hours on Nov. 2. The Nov. 1 meeting of this class will
occur, as scheduled.)

Assignments
1. Take-home midterm and final examination (20% each). The exams focus on
knowledge and understanding of material from required readings, classroom
lectures, and discussions in class. The midterm will be handed out Oct. 18
and due the following week, Oct. 25. The final will be handed out on Dec.
6, due on Dec. 13. The rest of the final is a short presentation of your
term paper on Dec. 18, when your term paper is also due. NOTE: On both
exams, I will ask for your AU ID only; NO NAMES, please.

2. Two short reports (10% each). Each student must prepare a written 3 page
double space written report and critical analysis of a randomized
experiment AND a quasi-experiment. The paper should cover the following
topics briefly:

Intervention
Outcome measure(s) (how measured)
Design
Method of (statistical?) analysis
Results
Analysis/critique: do you believe the results???

The papers are due Sept. 27 and Oct. 18. Be prepared to present your
paper orally. Select your example from the syllabus (but NOT from one of
the required readings; however, the recommended readings may be used for
this purpose); from the journals listed above; or from another class,
your workplace, etc. If you are unsure, please check with me about
whether your example is appropriate.

3. A term paper (40%). The term paper is to be an original program
evaluation, using empirical data that you yourself collect or that
someone else has collected and that you wish to reanalyze. For
example, you may wish to evaluate whether a specific community
policing program is effective in reducing crime, or whether anti-
smoking efforts deter smoking. Two or more persons may wish to study
the same program--one could evaluate the program in a particular
community, and someone else could evaluate the same program in
another community. A 1-2 page written research proposal is due on
Oct. 11. The research proposal should state the program to be
analyzed, briefly outline the proposed design, and indicate the
probable source(s) of data. Each student is responsible for talking
individually or in small groups with the instructor about his or her
topic before Oct. 11. The term paper is due on Dec. 18, when there
will be brief oral presentations of your findings.

2
3

4. Class participation and attendance. These are required but they are
not directly graded. However, you will find that attending class and
participating in class and small group discussions and exercises will
probably improve your grade. I do not offer extra help or assistance
to those who do not regularly attend class, unless there is a medical
emergency.

A brief note on my grading policy: A is reserved for truly exceptional work;
A- is close to outstanding work, but not quite there. B and B+ represent
good, sound work. Grades of B- and below mean that some portion of the basic,
core concepts are missing or poorly understood. The best work is accurate,
clear, and creative. Clarity includes a well-organized paper or essay,
paragraphs that correspond to separate topics and subtopics, and sentences
with subject, verb, and object, with appropriate use of adjectives and
adverbs. Correct your spelling. Remember, the spell-checker cannot
distinguish between their, there, and they’re, or between discrete and
discreet. Reread and edit your work (twice!) before you hand it in. Please
read “Owed to the Selling Chequer” at the end of this syllabus. Your work must
be your own and must conform to AU standards of academic integrity.

Late papers and exams will not be accepted, unless you tell me before-hand why you will be late and when you plan to
hand in the work or take the exam. Work that is more than one week late will not be accepted. Late work usually
hampers performance in the class, because much of the work is cumulative. That is, understanding the material on one
week depends on mastering the material in the previous weeks. Violations of the university’s Academic Integrity Code
will result in serious sanctions, a grade of F in the course, or suspension from the university. Please read the sections
set forth below on Reading and Writing Standards and the Academic Integrity Code carefully.

Reading and Writing Standards

I highly recommend that you take the American University Library’s Information Literacy Tutorial,
http://www.library.american.edu/tutorial/index.html, if you have not already done so in your
academic career. It is an excellent introduction to researching and writing. Throughout this and
every other course careful attention should be given to writing style and proper listing and citing of
sources, and the library’s tutorial is very helpful. For questions, please refer to the library website on
citations, http://www.library.american.edu/e_ref/citation.html or a college writing text such as
Hacker, A Writer’s Reference, www.bedfordstmartins.com/hacker/writersref. In addition, you
should evaluate websites carefully for academic quality and reputation. For guidance on website
evaluation, please contact a site such as
http://www.library.cornell.edu/okuref/research/webeval.html. In general, the best sources are
peer-reviewed journals, university publications, and publications by reputable commercial presses
and research organizations.

Academic Integrity Code

Standards of academic conduct are set forth in the University's Academic Integrity Code,
http://www.american.edu/academics/integrity/code01.htm. All examinations, tests, written papers,
and other assignments are required to be completed according to the standards set forth in this
code.

3
4

All work must be entirely your own. When you quote from others’ work, you must give full credit
by footnote or endnote. Failure to use quotation marks when quoting, failure to give full credit
when paraphrasing, use of others’ ideas or work products, submission of work prepared through
impermissible collaboration, and also submission of work prepared by you for another class are all
examples of violations of the academic integrity code and will result in sanctions. (Your work in this
class may be related to your work in another class, but the work you submit for this class should not
duplicate that submitted for another class. If you anticipate an exception, do not hesitate to ask.)

By registering in this class, you have acknowledged your awareness of the Academic Integrity Code,
and you are obliged to become familiar with your rights and responsibilities as defined by the Code.
Violations of the Academic Integrity Code will not be treated lightly, and disciplinary action will be
taken should such violations occur. Please see me if you have any questions about the academic
violations described in the Code in general or as they relate to particular requirements for this
course. Failure to comply with the requirements of the Academic Integrity Code can result in failure
in the course, as well as more serious academic sanctions.

Academic Support

If you experience difficulty in this course for any reason, please do not hesitate to consult me. In
addition to the resources of the department, a wide range of services is available at the university to
support you in your efforts to be successful in this course.

Academic Support Center (202-885-3360, MGC 243, asc@american.edu,
www.American.edu/ocl/asc) offers study skills materials and workshops, individual academic
assistance, tutor referrals, and services for students with learning disabilities and ADHD. Writing
support is available in the Academic Support Center (ASC) Writing Lab or in the Writing Center,
Battelle-Tompkins 228, 885-2991. Please use these services. We can all use help with our writing;
editors can teach you how to write better.

Counseling Center (202-885-3500, MGC 214, www.american.edu/ocl/counseling) offers
counseling and consultations regarding personal concerns, self-help information, and connections to
off-campus mental health resources.

Disability Support Services (202-885-4415, MGC 206, dss@american.edu,
www.american.edu/ocl/dss) offers technical and practical support and assistance with
accommodations for students with physical, medical, or psychological disabilities.

If you have a disability and might require accommodations in this course, please notify me via
Disability Support Services (DSS) or the Academic Support Center (ASC) early in the semester so
that I can make arrangements to address your needs.

Schedule of Classes and Reading Assignments

Aug. 30 Program evaluation and public management: definition,
history, current practice, types of evaluations. Examples
of possible evaluation projects for term papers.

4
5

Reading:
*Langbein, Public Program Evaluation: A Statistical Guide,
Chapt. 1, 2 (in blackboard)
*Bingham and Felbinger, ch. 1, 4
*M. Bamberger, "The Evaluation of International Develop-
ment Programs: A View From the Front," American
Jnl. of Evaluation 21 (1), 2000: 95-102. (BB)
*American Customer Satisfaction Index. (BB)
http://www.theacsi.org/model.htm
*Stoessel and Shaywitz, “What’s Wrong With Money in Science?”
Washington Post, July 2, 2006, B03. (BB)
**Berk and Rossi, Thinking About Program Evaluation,
ch. 1, 3, 4 (pp. 66-82)
**Wholey et al., Handbook, Chapter 1, 2, 3, 5 (recommended)
**Shadish, Cook and Campbell, ch. 1.
**Rossi and Freeman, Evaluation: A Systematic Approach,
ch. 1, 3, 5, 6
C. Wye and R. Sonnichsen, eds., Evaluation in the Federal
Government: Changes, Trends, and Opportunities (New
Directions for Program Evaluation, 55, Fall 1992).
Burt Perrin, "Effective Use and Misuse of Performance
Measurement," Am. Jnl. Eval., 19 (3), 1998: 367-379.

Sept. 6- Validity and the design of impact evaluations (subtopics
Nov. 1 follow)

Sept. 6,13 Types of validity and threats to internal validity (2 classes)

Reading:
*Langbein, PPE: Guide, chapter 3, 4 (blackboard)
*Bingham and Felbinger, ch. 2, 3, 4
**Berk and Rossi, Thinking About Program Evaluation, ch. 2
**Rossi and Freeman, Evaluation, ch. 7
**Shadish et al., ch. 2, 3
Campbell and Russo, Social Experimentation (Sage. 1999).
Chapt. 3, "An Inventory of Threats to Validity and
Alternative Designs to Control Them."
HHS/OIG, Office of Evaluations and Inspections, Practical
Evaluation for Public Managers.
http://oig.hhs.gov/organization/OEI/other/pepm.pdf

Sept. 20, Experiments (2 classes)
Sept. 27
Reading:

*Langbein, PPE: Guide, ch. 5 (blackboard)
*Bingham and Felbinger, ch. 5-7; 20-22
*Crane, Social Programs That Work, ch. 1 (BB)
**Rossi and Freeman, Evaluation, ch. 8
**Wholey et al., Handbook of Practical Program Evaluation,
ch. 8
**Shadish et al., ch 8-10
*J. Grossman and J. Tierney, "Does Mentoring Work? An Impact
Study of the Big Brothers Big Sisters Program," Eval.
Review 22 (3), June 1998. (Blackboard)
*P. Wortman, "An Exemplary Evaluation of a Program That
Worked: The High/Scope Perry Preschool Project,"
Eval. Practice 16 (3), 1995: 257-265. (Blackboard)
*Howell, Wolf, Campbell and Peterson,"School Vouchers and
Academic Performance: Results From Three Randomized

5
6

Field Trials." J. Policy Analysis and Management (2002),
21(2):191-217 http://www.aladin.wrlc.org/Z-
WEB/Aladin?req=db&key=ALADINPROXY&url=http://proquest.umi.
com/pqdweb?did=112283203&Fmt=3&clientId=31806&RQT=309&
VName=PQD (and in BB)
*Duflo and Kremer, “Use of Randomization in the Evaluation
of Development Effectiveness.” Prepared for World Bank
Operations Evaluation Dept., Conf. on Evaluation and
Development Effectiveness, July 2003. (BB)
Ballou, Goldring and Liu, “Magnet Schools and Student
Achievement.” March, 2006. Natl Center for the Study
Of Privatization in Education (BB)
Grosser and Schram, “Neighborhood Information Exchange and Voter
Participation: An Experimental Study.” American Political
Science Review 100 (2), May 2006: 235-248. (BB)
Glazerman, Mayer and Decker, “Alternative Routes to Teaching:
The Impacts of Teach for America on Student Achievement
and Other Outcomes.” J. Policy Anal and Mgmt. (2006), 25
(1), 75-96. (BB) (and RAND website)
M.J. Puma and N.R. Burstein, "The National Evaluation of the
Food Stamp Employment and Training Program,"
Journal of Policy Analysis and Management, 13(2),
Spring 1994: 311-330.
J. Gueron, "Learning About Welfare Reform: Lessons from
State-Based Evaluations," New Directions for Eval.
76, Winter 1997: 79-94.
Robt. G. St. Pierre et al., "The Effectiveness of Comprehen-
sive, Case Management Interventions....," Am. Jnl. Eval.
20(1), 1999: 15-34. (random assignment, but not
random selection)
David Greenberg and Mark Shroder, The Digest of Social
Experiments
Robert F. Boruch, Randomized Experiments for Planning and
Evaluation (Sage, 1997).

Sept. 27 Student reports: examples from the field
Oct. 11 Paper topic proposal due (1-2 page proposal, design, data source)

Oct. 4, 11,
Oct. 18 Quasi-experiments (3 classes)

Reading:
Rossi and Freeman, Evaluation, ch. 9, 10
*Langbein, PPE: Guide, ch. 6
*Bingham and Felbinger, ch, 8-12, 17-19
**Berk and Rossi, ch. 4 (pp. 83-end)
**Wholey et al., Handbook of Practical Program Evaluation,
ch. 6-7
**Shadish et al., ch. 4-7
**Imai, King and Stuart, “The Balance Test Fallacy in Matching
Methods for Causal Inference.” Unpublished paper, June
2006 http://GKing.Harvard.Edu (BB??)
*Crane, Social Programs That Work, ch. 1 (Blackboard)
(again)
*M. Harmon, "Reducing the Risk of Drug Involvement Among
Early Adolescents: An Evaluation of DARE," Evaluation
Review 17(2), Apr. 1993: 221-239. (Example of non-
equivalent control group design) (BB)
*Steven Rock, "Impact of the Illinois Seat Belt Use Law on
Accidents, Deaths and Injuries," Evaluation Review 16

6
7

(5), Oct. 1992: 491-507. (Example of single interrupted
time series with statistical analysis--ARIMA adjusted
regression) (BB)
*Blossom Patterson et al., "Evaluation of a Supermarket
Intervention: The Giant Food-NCI Eat for Health Study."
Evaluation Review 16 (5), Oct. 1992: 464-490.
(Example of pretest-posttest comparison group design,
or non-equivalent control group, with statistical
analysis--time series, cross-section regression)
(BB)
*Langbein, "Responsive Bureaus, Equity, and Regulatory
Negotiation: An empirical view. J. Policy Anal and Mgmt
(2002), 21 (3): 449-466. (Post-test only w/ statistical
controls and matching) (blackboard) OR
*Langbein, “Regulatory Negotiation versus Conventional Rulemaking: Claims
and Counter-claims” (with Cornelius Kerwin), Jnl. Public Administration
Research and Theory, 10 (3), 2000: 599-632 (Post-test only w/ statistical
controls and matching) (Blackboard)
*U.S. Dept. of State, “Outcome Assessment of the Visiting Fulbright Program.”
June 2005. Bureau of Educational and Cultural Affairs, Office of
Policy and Evaluation. One shot case study. Survey example.
See if you can find the whole thing at
http://exchanges.state.gov/education/evaluations.
**Krause and Douglas, “Does Agency Competition Improve the Quality of
Policy Analysis? Evidence from OMB and CBO Fiscal Projections.”
J. Policy Anal and Mgmt 2006, 25 (1), 53-74. (Single Interrupted
Time Series) (BB???)
**Lopoo and DeLeire, “Did Welfare Reform Influence the Fertility of Young
Teens?” J. Policy Anal and Mgmt 2006 (25), 2: 275-298. (Nat. Expt.
with statistical controls). (BB?)
**Mocan and Tekin, “Catholic Schools and Bad Behavior: A Propensity
Score Matching Analysis.” Contributions to Economic Analysis
and Policy 2006 (5), 1, Article 13. Berkeley Electronic Press.
(PTCG design) (BB)
*Carpenter, “Did Ontario’s Zero Tolerance and Graduated Licensing
Law Reduce Youth Drunk Driving?” J. Policy Anal and Mgmt.
(Time series and DID; shows need to control for trend
(find cite and add to BB)
**Berndt, Gottschalk, Philipson, Strobeck, “Assessing the Impacts of
the Prescription Drug Use Fee Acts on the FDA Approval
Process.” Frontiers in Health Policy Research 2005, 8, Article
2 (Single Interrupted Time Series).
http://www.bepress.com/fhep/8/2
**Ludwig and Miller, “Does Head Start Improve Children’s Life
Chances? Evidence from a Regression Discontinuity Design.”
Inst. for Research on Poverty, Nov. 2005.
http://www.irp.wisc.edu
**Paul Godek, "The Regulation of Fuel Economy and the Demand
for Light Trucks," Journal of Law and Economics 40(2),
Oct. 1997. Single interrupted time series with
statistical controls. (Periodical stacks and JSTOR)
**James D. King and Rodney Wambeam, "Impact of Election

7
8

Day Registration on Voter Turnout: A Quasi-
Experimental Analysis," Policy Studies Journal,
Autumn/Winter 1995/96 (14, 3/4): 263-278. Single
interrupted time series with statistical controls.
(Periodical stacks)
**Mark Schneider et al., "Institutional Arrangements and the
Creation of Social Capital: The Effects of Public
School Choice," American Political Science Review
91(1), March 1997: 82-93. Non-equivalent control
group with statistical controls, control for selection
bias. (Periodical stacks and JSTOR)
**Mark Schneider et al., "Shopping for Schools: In the Land
of the Blind, the One-Eyed Parent May Be Enough."
American Journal of Political Science 42(3), July 1998:
769-793. Non-equivalent control group with statistical
controls. (Periodical stacks and JSTOR)
Grossman and Tierney, "The Fallibility of Comparison Groups,"
Evaluation Review 17 (5), Oct. 1993: 556-571. (Optional
reading) (Periodical stacks)
James E. Veney, "Evaluation Applications of Regression
Analysis With Time Series Data." Evaluation Practice 14
(3), 1993: 259-274. (Optional reading; useful
bibliography on combining SITS with non-experimental
design)
S. Bell et al., Program Applicants as a Comparison Group in
Evaluating Training Programs. Upjohn Institute, 1995.
**Rothenberg and Sanders, "Severing the Electoral Connection:
Shirking in the Contemporary Congress," Am. J. Political
Science, 44 (2), April 2000: 310-319. (Periodical
stacks and JSTOR)
**Chay, McEwan, Urquiola, “The Central Role of Noise in
Evaluating Interventions That use Test Scores to
Rank Schools.” American Economic Review, Sept. 2005.
Regression discontinuity as solution to mean reversion.
**Klick and Tabbarok, “Using Terror Alert Levels to Estimate the
Effect of Police on Crime” (Nat. Expt) (BB)
**Angrist et al. “Long Term Educational Consequences of Secondary
School Vouchers.” AmericanEconRev. June 2006. (BB)
**Kling, “Incarceration Length, Employment and Earnings.”
American Econ Rev. June 2006.

Oct. 18 Student reports: examples from the field
Oct. 18 Distribute take-home exam (due Oct. 25 at 5:30; no late exams)

Oct. 25 Non-experimental design and analysis

Readings:

*Langbein, PPE: Guide, ch. 7
*Berry and Feldman, Multiple Regression in Practice
*Berry and Sanders, Understanding Multivariate Research
**Worrall and Pratt, “Estimation Issues Associated with Time-
Series Cross-Section Analysis in Criminology.” Western
Criminology Review (2004), 5 (1), 35-49. (Good review
Of issues in TSCS designs)
**Wholey et al., Handbook, ch. 18
*Gary Mauser and Richard Holmes, "An Evaluation of the 1977
Canadian Firearms Legislation," Evaluation Review 16
(6), Dec. 1992: 603-617. (BB)

8
9

*D. Houston, "Implications of the 65mph Speed Limit for
Traffic Safety," Eval. Review 23(3), June 99: 304-315
(BB)
*Martinez and Scicchitano, "Who Listens to Trash Talk?:
Education and Public Media Effects on Recycling
Behavior," Soc. Sci. Qtrly 79 (2), June 1998: 286-300.
(BB)
**Langbein and Snider, "The Impact of Teaching on Retention:
Some Quantitative Evidence," Soc. Sci. Qtrly 80(3),
Sept. 1999: 457-472. (BB)
**Langbein and Bess, "Sports in School: Source of Amity or
Antipathy," Soc. Sci. Qtrly (2002), 83 (2): 436-454.
(NOTE: This started as a term paper in this class.) (BB)
**Langbein and Spotswood-Bright, "The Efficiency and
Accountability of Residential Community Associations,"
Soc. Sci. Qtrly (Sept. 2004)85 (3): 640-659. (NOTE: This
also began as a term paper in this class.) (BB)
**Medoff and Magaddino, "Suicides and Firearm Control Laws,"
Evaluation Review, June 1983. (Periodical stacks)
**Nelson et al., "Media Framing and its Effect on Tolerance,"
American Political Science Review 91(3), Sept. 1997:
567-583. Example of path analysis. (Periodical stacks and
JSTOR)
**Leora Friedberg, "Did Unilateral Divorce Raise Divorce Rates?
Evidence from Panel Data." American Economic Review
88(3), June 1998: 608-627. (Periodical stacks and JSTOR)
P. Rossi, "Advances in Quantitative Evaluation, 1987-1996,"
New Directions for Evaluation 76, Winter 1997: 57-68.
**Wm. Evans et al., "Do Workplace Smoking Bans Reduce
Smoking." American Economic Review, Sept. 1999.
Great discussion of threats to internal validity of
regression estimates. (Periodical stacks and JSTOR)
**P. Ong, "Car Ownership and Welfare-to-Work," J. Policy Anal.
& Mgmt. (2002), 21 (2), 239-252. (JSTOR)
**Lykens and Jurgowski, "Medicaid Matters: Children's Health and
Medicaid Eligibility Expansions." J. Policy Anal. & Mgmt.
(2002) 21 (2): 219-238. (JSTOR)
**Buddin and Zimmer, “Is Charter School Competition in California
Improving the Performance of Traditional Public Schools?”
RAND, Sept. 2005 (BB)
**Bertelli and Richardson, “Deterring Drinking and Driving in
Theory and Practice: Evidence from the American States.”
Presented at MWPSA Conference, April 2006 (BB).
**Wright, “Campaign Contributions and Congressional Voting on
Tobacco Policy, 1980-2000. Business and Politics 6(3),
Article 1. http://www.bepress.com/bap/vol16/iss3/art1 (BB)
**Heinrich, “Demand and Supply Side Determinant of Conditional
Cash Transfer Program Effectiveness,” World Development,
2006 (forthcoming??)
**Wassmer and Baass, “Does a More Centralized Urban Form Raise
Housing Prices?” J. Policy Anal. and Mgmt. 2006, 25 (2):
439-462 (BB?)
**Cannon, Jacknowitz, and Painter, “Is Full Better than Half?
Examining the Logitudinal Effects of Full-day Kindergarten
Attendance.” J. Policy Anal. and Mgmt. 2006 (25), 2: 299-
321. (BB?)
Henry and Gordon, “Competition in the Sandbox: A Test of the
Effects of Preschool Competition on Educational Outcomes.”
J. Policy Anal and Mgmt, 2006, 25 (1):97-127. (BB??)
**Bedard and Deschenes, “The Long Term Impact of Military Service
On Health: Evidence from WWII and Korean War Veterans.”

9
10

American Econ R. 2006 (March) 96 (1). (Periodical stacks)
**Oreopoulos, “Estimating Average and Local Average Treatment
Effects of Education when Compulsory Schooling Laws
Really Matter.” American Econ R. , March 2006.
(Periodical stacks) (NE + Regression Discontinuity + IV)

Nov. 1 Non-Experimental Design: Hands-On Data Analysis

Hurst 202 Computer lab: hands on data entry, using SPSS (or
Stata) for recoding/transforming variables, running regression.

Nov. 8 Non-Experimental Design (continued)
Interpreting the Results: Do you believe them?

Nov. 15
Nov. 29 Data Collection I: The design and analysis of surveys

Reading:
*Langbein, PPE: Guide, ch. 8
*Converse and Presser, Survey Questions, all
*Mangione, Mail Surveys, all.
*Soss, Langbein and Metelko, "Why Do White Americans Support
the Death Penalty," Journal of Politics 65 (2), May 2003:
(BB)
*Brown and Potoski, “Contracting for Management: Assessing
Management Capacity Under Alternative Service Delivery
Contracts. J. Policy Anal. and Mgmt. 2006 (25) 2 : 323-
346 (BB??)
*American Customer Satisfaction Index.
http://www.theacsi.org/model.htm
*U.S. Dept. of State, “Outcome Assessment of the Visiting Fulbright Program.”
June 2005. Bureau of Educational and Cultural Affairs, Office of
Policy and Evaluation. One shot case study. Survey example.
See if you can find the whole thing at
http://exchanges.state.gov/education/evaluations.
**HHS/OIG, Office of Evaluations and Inspections, Practical
Evaluation for Public Managers.
http://oig.hhs.gov/organization/OEI/other/pepm.pdf
**Babbie, The Practice of Social Research, (5th ed.), ch. 6, 7, 9
**Wholey et al., Handbook, ch. 11, 15; skim ch. 10, 12, 14, 16
Ohls and Rosenberg, "A Building-Up Approach to Measuring
Program Costs," Jnl. Policy Anal. and Mgmt. 18(3),
1999: 473-480

Nov. 29 Data collection II: The design and analysis of case studies

Reading:
*Yin, Case Study Research, all
*M. Q. Patton, Qualitative Evaluation Methods, ch. 7 (BB)(Scan)
**J. Gerring, "What is a Case Study and What is it Good For?"
American Political Science Review 98 (2), May 2004:
341-354.
Jasjeet Sekhon, “Quality Meets Quantity: Case Studies,
Conditional Probability, and Counterfactuals, ”Perspectives
On Politics, June 2004, 2(2): 281-289.

10
11

**Wholey, Evaluation Handbook, ch. 4

Dec. 6 Hand out take-home part of Final (due Dec. 13)
Course evaluations
Dec. 6 Meta-analysis; Making Results Useful

Reading:

*Langbein, PPE: Guide, ch. 9
*Bingham and Felbinger, ch. 16
*Boyer and Langbein, "Factors Influencing the Use of Health
Evaluation Research in Congress," Evaluation Review,
Oct. 1991 (BB????)
*Ritter, “Keeping Score: What Do We Know About the Impact of
Statewide School Accountability Programs on Student
Achievement?” (Paper presented at APPAM Research
Conference, 2004) (BB???)
**U.S. Surgeon General, “The Health Consequences of Involuntary
Exposure to Tobacco Smoke.” 2006. (BB??)
**Hird, Power, Knowledge, and Politics: Policy Analysis in
the States. Georgetown Univ Press, 2005.
**Cook et al. Meta-Analysis for Explanation, ch. 1, 4, 7
**Shadish et al., pp. 425-454.
**Greenberg, Linksz, and Mandell, Social Experimentation and
Public Policymaking, ch. 3 ("Conceptual Framework") and
ch. 10 ("Lessons from the Case Studies")
**Steven Stack, "Media Impacts on Suicide: A Quantitative
Review of Findings." Soc. Sci. Qtrly 81 (4), Dec. 2000.
(on reserve)
**Rossi and Freeman, Evaluation, ch. 12
**Wholey et al., Handbook, ch. 9, 20, 23, 24, 25
Mrosek and Taylor, "What Determines the Value of Life? A
Meta-Analysis." J. Pol. Anal. & Mgmt. (2002) 21 (2):
253-270.
Cooper and Hedges, eds., The Handbook of Research Synthesis
(Russell Sage, 1994). (Library stacks)
Frederic M. Wolf, "Meta-Analysis," Sage University Paper
Series on Quantitative Applications in the Social
Sciences, #59. Especially pp. 9-18; 23 (bottom) - 26;
34-36 (end of 1st paragraph); 37-39 (end of page);
48-49; 53-56. (Periodical stacks)

Dec. 13: Take-home part of final due.
Study day. Discuss term papers.
Dec. 18 Term papers due (Final exam day for Wed. classes)
Term papers presented orally in class, as remainder of final
(5 minute presentation; use powerpoint, overheads or handouts)

11
12

OWED TO THE SELLING CHEQUER

I have a spelling chequer
It came with my pea see
It plainly marks for my revue
Miss steaks eye cannot sea.

Eye strike a quay and right a word
And weight for it two say
Weather I am wrong or write—
It shows me strait a way.

As soon as a mist ache is maid
It nose be fore two late
And I can put the error rite
It’s rarely, rarely grate.

I’ve run this poem threw it
I’m shore yore pleased to no
It’s letter perfect in it’s weight
My chequer tolled me sew.