Center of Excellence for Training and Research Translation

Document Sample
Center of Excellence for Training and Research Translation Powered By Docstoc
					                                                                                                             

            Center of Excellence for Training and Research Translation 
                     Practice‐based Intervention Review Tool 
 
                        Cover Sheet to be completed by Center TRT Staff 
 
Center TRT Staff:   _________________________________________ 
 
Reviewers:  1.        _____________________________________ 
             2.       _____________________________________ 
 
    WISEWOMAN Intervention                    Obesity Prevention Intervention 
 
Title of Intervention: 
 
 
Developer/Contact Person: 
 
 
Date Intervention and Materials were developed or last scientific update: 
 
 
Intended Population: 
 
 
Setting: 
       Home and family     Healthcare       Childcare     School      Worksite      Place of worship 
       Community         Other  ______________________ 
 
Outcomes Targeted by the Intervention: 
Physiological outcomes                          
    Improved BMI:    healthy weight gain in children    weight loss       weight maintenance after loss 
    Improved blood pressure or control of hypertension 
    Improved cholesterol or blood lipid profile 
    Improved blood glucose or Hgb A1C or control of diabetes 
 
Behavioral outcomes targeted by the intervention   
    Physical activity 
        Increased intensity 
        Increased frequency 
        Longer duration 
    Dietary Composition or behaviors that facilitate energy balance and prevent heart disease or stroke 
        Increased consumption of dietary fiber 
        Increased consumption of fruit/vegetable 
        Increased consumption of calcium/low‐fat dairy  
        Increased consumption of omega 3 or other unsaturated fatty acids 


Developed by the Center of Excellence for Training and Research Translation              April 2007
UNC Center for Health Promotion and Disease Prevention                                  Page 1 of 10
                                                                                                              

        Decreased consumption of energy‐dense, micro‐nutrient poor foods 
        Decreased consumption of saturated and trans fat 
        Decreased consumption of sodium/salt 
        Decreased consumption of sweetened beverages such as soda and fruit drinks 
        Decreased consumption of restaurant‐prepared foods including fast foods 
        Reduced portion size 
        Decreased breakfast skipping 
    Breast feeding 
        Increased initiation 
        Increased duration 
        Increased exclusivity 
    Reduced television viewing time 
   
Environmental change with the potential to: 
     Increase physical activity 
     Increase breast feeding,  
     Improve dietary composition or dietary behaviors 
     Reduce television time 
Through its effect on: 
     availability/supply 
     physical accessibility 
     social norms 
     price 
     promotion/marketing 
 
Public and/or organizational policy change with the potential to: 
     Increase physical activity 
     Increase breast feeding,  
     Improve dietary composition or dietary behaviors 
     Reduce television time 
Through its effect on: 
     availability/supply 
     physical accessibility 
     social norms 
     price 
     promotion/marketing 
     Norms and Values across society (legislation; media campaigns; large‐scale partnerships) 
 
Infrastructure Development 
     Increased capacity to develop, implement  and sustain nutrition and physical activity interventions 
 
 
 NOTE TO REVIEWERS:




Developed by the Center of Excellence for Training and Research Translation             April 2007
UNC Center for Health Promotion and Disease Prevention                                 Page 2 of 10
                                                                                                            

                                                         
                                   Quality Review by Expert Panel 
 
Description 
 
1. How well were the following categories of the intervention described? 
 
                                              Not                                      Well 
                                              well  
        Intended population                     1               2        3        4      5 
        Location/setting of the intervention    1               2        3        4      5 
        Who delivered it                        1               2        3        4      5 
        What was done                           1               2        3        4      5 
        How it was done                         1               2        3        4      5 
     
        Explanation:___________________________________________________________________________
        ______________________________________________________________________________________
        ______________________________________________________________________________________ 
 
2. What type(s) of evidence exists to support the use of the main strategy employed in this intervention? 
                 Direct evidence from the Community Guide  
                 Direct evidence from a systematic review  
                 Direct evidence from research 
                 Monitoring/surveillance or other observational data 
                 Parallel evidence  
                 Theory or logic of the intervention 
                 Expert opinion 
         
        Explanation:___________________________________________________________________________
        ______________________________________________________________________________________
        ______________________________________________________________________________________ 
 
Formative Work   
 
3. Who is the intended audience for the intervention? 
        the entire population in    the setting      the community      the state 
        women at risk for heart disease, stroke, or diabetes 
        children at risk for overweight 
        overweight adults  (specific segment of this group below if applicable) 
        a segment of the population:   ___________________________________________ 
     
     
     
     
     




Developed by the Center of Excellence for Training and Research Translation                 April 2007
UNC Center for Health Promotion and Disease Prevention                                     Page 3 of 10
                                                                                                          

4. Did the developers of the intervention conduct formative work with stakeholder groups, including 
    the intended audience, to inform the development of the intervention? 
         YES          NO        Partially       Unable to assess 
     
         Explanation:___________________________________________________________________________
         ______________________________________________________________________________________ 
         ______________________________________________________________________________________ 
          
Theory/Conceptual Underpinnings 
 
5. Are the intervention activities and intended outcomes guided by a logical conceptual framework or 
    theory? 
        YES           NO         Partially       Unable to assess 
 
    Explanation:______________________________________________________________________________
    __________________________________________________________________________________________
    __________________________________________________________________________________________ 
 
Data Collection 
 
6. Were the methods for collecting process evaluation data systematic? 
        YES           NO         Partially       Unable to assess 
 
        Explanation:___________________________________________________________________________
        ______________________________________________________________________________________
        ______________________________________________________________________________________ 
 
7. Did the developers measure the extent to which the intervention reached the intended population? 
         Yes :  percent of intended audience that was reached _______________ 
         No 
         Partially 
         Unable to assess 
 
 
8. Did the developers measure the acceptability of the intervention to stakeholders including the 
    effected population? 
        YES          NO         Partially     Unable to assess 
        Explanation:___________________________________________________________________________
        ______________________________________________________________________________________
        ______________________________________________________________________________________ 
 
 
 
 
 
 



Developed by the Center of Excellence for Training and Research Translation           April 2007
UNC Center for Health Promotion and Disease Prevention                               Page 4 of 10
                                                                                                     

9. Did the developers measure the extent to which the intervention components were delivered as 
    intended?         
         YES            NO        Partially     Unable to assess 
          Explanation:___________________________________________________________________________
          ______________________________________________________________________________________
          ______________________________________________________________________________________ 
           
10. Is a system in place for collecting outcome evaluation data? 
          YES                     NO            Partially        Unable to assess 
     
          Explanation:___________________________________________________________________________
          ______________________________________________________________________________________ 
          ______________________________________________________________________________________ 
           
11. Were the outcomes measured reasonable given resources available and potential evaluation design 
    constraints? 
          YES                     NO            Partially        Unable to assess 
                                                            
          Explanation:___________________________________________________________________________
          ______________________________________________________________________________________
          ______________________________________________________________________________________ 
 
Data Analysis and Interpretation 
     
12. Did the developers analyze and report data appropriately? 
           YES                 NO              Partially        Unable to assess 
     
          Explanation:___________________________________________________________________________
          ______________________________________________________________________________________
          ______________________________________________________________________________________ 
 
13. Did the developers interpret the data appropriately? 
           YES                 NO              Partially        Unable to assess 
     
          Explanation:___________________________________________________________________________
          ______________________________________________________________________________________
          ______________________________________________________________________________________ 
 
 
 
 
 
 
 
 
 
 

Developed by the Center of Excellence for Training and Research Translation          April 2007
UNC Center for Health Promotion and Disease Prevention                              Page 5 of 10
                                                                                                             

Effectiveness 
 
14. To what extent did the intervention achieve anticipated process outcomes? (This includes recruitment 
       and retention of participants) (with 1 = low and 5 =high)   
                                    1        2        3        4        5        Unable to assess 
                                                                        
15. To what extent did the intervention achieve anticipated outcomes? (This includes health, behaviors, 
       environmental, policy/practice outcomes) (with 1 = low and 5 =high)   
                                    1        2        3        4        5        Unable to assess 
 
16. Did the intervention benefit all subpopulations – particularly those with existing disparities?  
            YES                  NO                   Partially                 Unable to assess 
     If No, what subpopulations achieved less benefit from the intervention?  __________________ 
 
17. Were there any adverse outcomes or unintended negative outcomes? 
            YES                  NO                   Partially                 Unable to Assess 
 
Reach  
     
18. How would you rate the percentage of the intended population that actually took part in/was 
    affected by the intervention (with 1 = low and 5 = high)?   (Note: For environmental and policy 
    interventions, the intended ʺpopulationʺ denominator may be an organization, community, setting or 
    other unit.) 
                               1        2        3        4        5           Unable to Assess 
                                                                      
19. How representative of the intended population were the participants?  e.g. race/ethnicity, gender, 
    age, or socio‐economic status (with 1 = low and 5 =high)? 
 
                               1        2        3        4        5           Unable to Assess 
     
 Adoption 
 
20. How many and what percent of targeted settings (schools, communities, worksites etc) participated 
    in the intervention?  
                      ______ (Number)          ______ (Percent)         Unable to assess 
21. Did any settings agree to participate but not adopt the change in practice or policy? 
          YES                 NO             Partially        Unable to assess 
                                                         
22. Has the intervention been adopted by other states, communities or organizations? 
          YES                 NO             Partially        Unable to assess 
     
    If yes, number of other adopters: ____________   
 
 
 
 

Developed by the Center of Excellence for Training and Research Translation              April 2007
UNC Center for Health Promotion and Disease Prevention                                  Page 6 of 10
                                                                                                                        

Implementation 
 
23. To what extent were the intervention components delivered as intended (1 = delivery inconsistent 
    and 5 = delivery consistent)? 
                              1        2        3        4        5           Unable to assess 
          
24. Has the intervention been replicated in other settings? 
                        YES                  NO                       Partially         Unable to assess 
    If yes, number of settings: ____________   
     
Maintenance 
 
25. Did the implementing organization maintain or institutionalize the intervention? 
                           YES                 NO                Partially           Unable to assess 
      
26. How was the intervention adapted or modified over time?  _____________________________________ 
    __________________________________________________________________________________________
    __________________________________________________________________________________________ 
    __________________________________________________________________________________________ 




Developed by the Center of Excellence for Training and Research Translation                             April 2007
UNC Center for Health Promotion and Disease Prevention                                                 Page 7 of 10
                                                                                                           

 
                                        Summary of Quality Review 
 
Criterion:  The methodology to develop AND evaluate the intervention is of reasonable quality 
Criterion: The intervention’s positive outcomes outweighed adverse outcomes and it achieved one or 
more of the targeted outcomes 
 
27. Based on your assessment of the methodology of the intervention, what is the likelihood that this 
    intervention is responsible for the positive outcomes that were attributed to it? 
 
        Not Likely           Possible           Likely         Unable to Assess 
     
28. Do positive effects outweigh the negative effects of the intervention? 
 
         YES                  NO                Partially      Unable to Assess 
 
29. How confident are you in your assessment of this intervention? 
 
        Not confident        Neutral            Confident      Highly confident 
 
30. Did you have enough information to make this assessment? 
 
         YES          NO              Partially 
     
    What additional information would have been useful? 
    __________________________________________________________________________________________
    __________________________________________________________________________________________
    __________________________________________________________________________________________ 
 
 
Criterion:  The intervention has potential for public health impact 
 
31. Based on your assessment of this intervention across the reach, adoption, implementation and 
    maintenance of the intervention, how do you rate the potential public health impact?   
 
    ____Low     ______ Moderate   _______ High     
 
32. How confident are you in your assessment of this intervention? 
 
          Not confident          Neutral          Confident       Highly confident 
 
33. Did you have enough information to make this assessment? 
     
           YES          NO               Partially 
     
     


Developed by the Center of Excellence for Training and Research Translation               April 2007
UNC Center for Health Promotion and Disease Prevention                                   Page 8 of 10
                                                                                                 

     What additional information would have been useful? 
     __________________________________________________________________________________________
     __________________________________________________________________________________________
     __________________________________________________________________________________________ 
 

    COMMENTS:




 
 
 
 
Expert Reviewer:   ___________________________________   
Date: ______________________________ 
 




Developed by the Center of Excellence for Training and Research Translation    April 2007
UNC Center for Health Promotion and Disease Prevention                        Page 9 of 10
                                                                                                                   

            Dissemination Readiness Review by Center TRT Translation Team 
 
Criterion:  The intervention is ready for dissemination 
 
34. Is the intervention described sufficiently to allow replication? 
    YES                 NO               Unable to Assess 
 
35. Is the capacity (staff and skill) to implement this intervention generally available in similar settings? 
          YES                   NO               Partially    
         Unable to Assess 
 
36. Are intervention materials available AND of useable quality? 
          No 
          Yes:  What format? 
              Print copies available from: ____________________________________________ 
              Electronic   download  available from:  ___________________________________ 
              Other format:  ______________________available from: ____________________ 
 
37. How much do the materials cost? 
          Single copy available free of charge 
          Free in limited quantities  (Specify number: ______) 
          Materials must be purchased (Specify cost per item: __________________) 
 
38. Is a knowledgeable contact person available to provide assistance? 
          No 
          Yes   (Provide contact information) 
          ______________________________ 
          ______________________________ 
          ______________________________ 
 
39. Is training on the intervention available? 
          No 
          Yes:  What is the cost of training?  __________________ 
          Not applicable 
 
 
Based on ratings of the strength of the intervention evaluation methodology, the effectiveness of 
the intervention, its potential public health impact; and dissemination readiness, does this 
intervention meet established review criteria? 
 
         YES                 NO           unsure, need additional information       
 
 




Developed by the Center of Excellence for Training and Research Translation                April 2007
UNC Center for Health Promotion and Disease Prevention                                   Page 10 of 10