Systematic Screening &
Update and Use in Two Projects
The Systematic Screening & Assessment Method:
Finding Innovations Worth Evaluating.
New Directions in Evaluation, 2010 No. 125
Leviton, Kettel Khan & Dawkins, Editors
Evaluations are Usually Handed to Us
“Here, evaluate this.”
Some of the Programs Just Aren’t Very Good
“Here, evaluate this.”
If You Evaluate Lots of Them…
It’s not very cost-effective!
The SSA Model
1. Get theme or topic from stakeholders.
2. Solicit nominations or scan for innovations.
3. Expert panel review –
sift for plausibly effective innovations
4. Evaluability assessments
5. Expert panel review –
Which are likely to have significant population
6. Use of information
Identify a High Volume of Programs
Then Screen Them
Yield from Screening
128 (28%) presented to expert panel to assess
• plausibility of impact
• likely reach to cover target population
• likely feasibility, generalizability, etc.
Then Assess Them
Evaluability Assessment – Steps:
• Involve the intended users
• Clarify the intended program
• Explore program reality
• Reach agreement on needed changes in activities or
• Explore alternative evaluation designs
• Agree on evaluation priorities and intended uses of
Reference: Leviton, Kettel Khan, Rog, Dawkins & Cotton (2010) Evaluability
assessment to improve public health policies, programs, and practices.
Ann. Rev. Pub. Health, 31:213–33
Yield from Assessment
53 (11.5%) deemed worth of evaluability assessment*
• 48 assessed
• 33 ready for evaluation
• 20 (4%) both ready AND likely to have population
• 6 priorities, 5 evaluations under way or approved.
*using nominations as the denominator.
Use of Information (so far)
1. Identify the most promising and ready for
2. More in-depth look at a field of practice
3 syntheses on DNPAO website
3. Feedback and TA to innovators:
• on logic model
• on fit with resources and activities
• on evaluation
• most innovators appreciated it (survey)
Example: NYC Day Care Policy
• No junk food or sugar sweetened bev. on premises
• Age appropriate physical activity
• Water available
• Limited TV or screen time
Enforced – inspections and citations
TA on nutrition and physical activity
Questions: How much implementation?
What effect on kids’ behavior?
SSA identified plausible innovations for a test:
• in a cost-effective way
SSA helped innovators
SSA developed understanding of the field
SSA now being used for:
• Nursing education innovations
• Intimate Partner Violence prevention in immigrant