Systematic Screening _ Assessment Method

Document Sample
Systematic Screening _ Assessment Method Powered By Docstoc
					Systematic Screening &
 Assessment Method
  Update and Use in Two Projects
SSA Model

The Systematic Screening & Assessment Method:
Finding Innovations Worth Evaluating.
New Directions in Evaluation, 2010 No. 125
Leviton, Kettel Khan & Dawkins, Editors

                                                SSA Model
Evaluations are Usually Handed to Us

“Here, evaluate this.”
Some of the Programs Just Aren’t Very Good

 “Here, evaluate this.”
If You Evaluate Lots of Them…

            It’s not very cost-effective!
The SSA Model
1. Get theme or topic from stakeholders.
2. Solicit nominations or scan for innovations.
  Obtain documentation.
3. Expert panel review –
      sift for plausibly effective innovations
4. Evaluability assessments
5. Expert panel review –
      Which are likely to have significant population
6. Use of information
Identify a High Volume of Programs
Then Screen Them

                   Prior Probability
                   of Impact
Yield from Screening

Childhood Obesity:

458 nominations
128 (28%) presented to expert panel to assess
 • plausibility of impact
 • likely reach to cover target population
 • likely feasibility, generalizability, etc.
Then Assess Them
Evaluability Assessment – Steps:
•   Involve the intended users
•   Clarify the intended program
•   Explore program reality
•   Reach agreement on needed changes in activities or
•   Explore alternative evaluation designs
•   Agree on evaluation priorities and intended uses of

    Reference: Leviton, Kettel Khan, Rog, Dawkins & Cotton (2010) Evaluability
        assessment to improve public health policies, programs, and practices.
                                             Ann. Rev. Pub. Health, 31:213–33
Yield from Assessment

53 (11.5%) deemed worth of evaluability assessment*
 • 48 assessed
 • 33 ready for evaluation
 • 20 (4%) both ready AND likely to have population
 • 6 priorities, 5 evaluations under way or approved.

*using nominations as the denominator.
Use of Information (so far)
1. Identify the most promising and ready for
2. More in-depth look at a field of practice
      3 syntheses on DNPAO website
3. Feedback and TA to innovators:
  •     on logic model
  •     on fit with resources and activities
  •     on evaluation
  •     most innovators appreciated it (survey)
Example: NYC Day Care Policy
• No junk food or sugar sweetened bev. on premises
• Age appropriate physical activity
• Water available
• Limited TV or screen time
Enforced – inspections and citations
TA on nutrition and physical activity
Questions: How much implementation?
             What effect on kids’ behavior?
SSA identified plausible innovations for a test:
• quickly
• in a cost-effective way
SSA helped innovators
SSA developed understanding of the field
SSA now being used for:
• Nursing education innovations
• Intimate Partner Violence prevention in immigrant

Shared By: