Docstoc

Assessment Tool

Document Sample
Assessment Tool Powered By Docstoc
					Canadian Diabetes Association
Identifying Best or Promising Practices in Diabetes Education
 
Assessment Tool
Intervention Name:
Source:
Reviewer:


Effectiveness Assessment 1
A) SELECTION BIAS
(Q1) Are the individuals selected to participate in the study likely to be representative of the target population?

                                                         Very Likely            Somewhat Likely           Not Likely



(Q2) What percentage of selected individuals agreed to participate?

                      80 - 100%                                 60 – 79%            Less than 60%
                      Agreement                                Agreement              Agreement       Not Reported        Not Applicable


                             Rate this section                             Strong              Moderate                Weak
                             (see dictionary)


B) STUDY DESIGN
              Indicate the study design

                                                RCT                          Quasi-Experimental       Case-control, Before/After
                                              (go to 1)                           (go to 2)            study, No control group,
                                                                                                      or Other:
                                                                                                               (go to C)



              1.(i) Is the method of random allocation stated?                                 Yes            No
              (ii) If the method of random allocation is stated                                Yes            No
              is it appropriate?
              (iii) Was the method of random allocation                                        Yes            No
              reported as blinded (concealed)?

              2. .(i) Is the method of random selection stated?                                Yes            No
              (ii) If the method of random selection is stated                                 Yes            No
              is it appropriate?
              (iii) Was the method of random selection                                         Yes            No
              reported as blinded (concealed)?


                             Rate this section                             Strong              Moderate                Weak
                             (see dictionary)

                                                            
1
   Minor adaptations from the tool created by the Effective Public Health Practices Project. Quality Assessment Tool for Quantitative Studies 
Component Ratings. Hamilton: ON. Accessed at www.myhamilton.ca/NR/rdonlyres/04A24EBE‐2C46‐411D‐AEBA‐
95A60FDEF5CA/0/QualityTool2003.pdf 
                                                                                                                                            1 
Canadian Diabetes Association
Identifying Best or Promising Practices in Diabetes Education
 

C) CONFOUNDERS (or Co-Variance)

(Q1) Prior to the intervention were there between group differences for important confounders (covariates) reported in the
paper?
                                     Yes                    No                  Can’t Tell


See the dictionary for some examples. Relevant confounders (covariates) reported in the study:




(Q2) If there were differences between groups for important confounders (covariates), were they adequately managed in
the analysis?
                                   Yes                     No               Not Applicable


(Q3) Were there important confounders (covariates) not reported in the paper?
                                             Yes                       No


Relevant confounders (covariates) NOT reported in the study:




                   Rate this section            Strong                Moderate                  Weak
                   (see dictionary)


D) BLINDING
(Q1) Was (were) the outcome assessor(s) blinded to the intervention or exposure status of participants?
                      Yes                    No                  Not Reported           Not Applicable



                   Rate this section            Strong                Moderate                  Weak
                   (see dictionary)


E) DATA COLLECTION METHODS
(Q1) Were data collection tools shown or are they known to be valid?
                                   Yes                    No                  Not Sure/Can’t Tell

    (Q2) Were data collection tools shown or are they known to be reliable?
                                       Yes                    No              Not Sure/Can’t Tell



                   Rate this section            Strong                Moderate                  Weak
                   (see dictionary)


                                                                                                                  2 
Canadian Diabetes Association
Identifying Best or Promising Practices in Diabetes Education
 

F) WITHDRAWALS AND DROP-OUTS
(Q1) Indicate the percentage of participants completing the study. (If the percentage differs by groups, record the lowest).
             80 - 100%              60 – 79%           Less than 60%           Not Reported         Not Applicable


                    Rate this section               Strong                  Moderate                  Weak
                    (see dictionary)


G) ANALYSIS
(Q1) Is there a sample size calculation or power calculation?
                                    Yes                   Partially                        No


    (Q2) Is there a statistically significant difference between groups?
                             Yes                        No               Not Reported            Not Applicable


    (Q3) Are the statistical methods appropriate?
                                       Yes                          No                 Not Reported




H) INTERVENTION INTEGRITY
(Q1) What percentage of participants received the allocated intervention or exposure of interest?

                80 - 100%               60 – 79%                 Less than 60%      Not Reported        Not Applicable


    (Q2) Was the consistency of the intervention measured?
                         Yes                      No                      Not Reported           Not Applicable



(Q3) Is it likely that subjects received an unintended intervention (contamination or cointervention) that may influence the
results?
                                       Yes                    No                   Can’t Tell




                                                                                                                         3 
Canadian Diabetes Association
Identifying Best or Promising Practices in Diabetes Education
 

SUMMARY OF EFFECTIVENESS COMPONENT RATINGS
Please transcribe the information from the gray boxes on pages 1-3 onto this page.

A) SELECTION BIAS
                Rate this section           Strong              Moderate             Weak
                (see dictionary)

B) STUDY DESIGN
                Rate this section           Strong              Moderate             Weak
                (see dictionary)

C) CONFOUNDERS
                Rate this section           Strong              Moderate             Weak
                (see dictionary)

D) BLINDING
                Rate this section           Strong              Moderate             Weak
                (see dictionary)

E) DATA COLLECTION METHODS
                Rate this section           Strong              Moderate             Weak
                (see dictionary)

F) WITHDRAWALS AND DROPOUTS
                Rate this section           Strong              Moderate             Weak
                (see dictionary)

G) ANALYSIS
Comments: ___________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

H) INTERVENTION INTEGRITY
Comments: ___________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________




                                                                                                4 
Canadian Diabetes Association
Identifying Best or Promising Practices in Diabetes Education
 
Plausibility Assessment Tool
 
A) EVALUATION 2
(Q1) Formative Evaluation/Pilot Testing
The program uses formative evaluation (e.g., focus groups, structured surveys, key informant interviews) to assess the
relevance, comprehension and acceptability of activities, materials, methods, etc. for the intended audience / population of
interest.

                                                               Yes       No    Not Sure/Can’t Tell


Comments:




(Q2) Process Evaluation
The program includes a process evaluation component. Feedback has been gathered demonstrating that intended
audiences were reached and the program was implemented as planned.

                                                               Yes       No    Not Sure/Can’t Tell


Comments:




(Q3) Outcome Evaluation
The program evaluates outcome. It evaluates the extent to which the program met stated goals and objectives as
assessed by impact and/or outcome evaluations (e.g., changes in behaviours, policies or supportive environments).

                                                               Yes       No    Not Sure/Can’t Tell


Comments:




                                                            
2
  Towards Evidence‐Informed Practice, Heart Health Resource Centre (2006). Toronto: ON. Accessed at 
teip.hhrc.net/docs/tools/b._Program_Assessment_Tools/TEIP_ProgramAssessmentWorksheet.pdf.  
                                                                                                                   5 
Canadian Diabetes Association
Identifying Best or Promising Practices in Diabetes Education
 

B) CONTENT ATTRIBUTES
(Q1) Theory and Literature Evidence 3
The program is informed by appropriate theoretical concepts (e.g., principles of behavior change, social learning, etc.),
review of literature and/or best practices.

                                                               Yes       No    Not Sure/Can’t Tell



Comments:




                                                 4
(Q2) Behavioural Objective
A specific desired behaviour change is addressed.

                                                               Yes       No    Not Sure/Can’t Tell


Comments:




                                                            
3
   Question created by the Towards Evidence‐Informed Practice, Heart Health Resource Centre (2006). Toronto: ON. Accessed at 
teip.hhrc.net/docs/tools/b._Program_Assessment_Tools/TEIP_ProgramAssessmentWorksheet.pdf.  
4
   Heart Health Resource Centre, University of Waterloo, Department of Health Studies and Gerontology & the Health Behaviour 
Research Group (1998). International Best Practices in Heart Health. Toronto: ON. Accessed at 
www.hhrc.net/skills/bestpractices.cfm.  
                                                                                                                         6 
Canadian Diabetes Association
Identifying Best or Promising Practices in Diabetes Education
 

Rating Recommendation
    Best Practice

    Promising Practice

    Insufficient Evidence




WITH BOTH REVIEWERS DISCUSSING THE RATINGS:
Is there a discrepancy between the two reviewers with respect to the component ratings?
                                              No                       Yes


If yes, indicate the reason for the discrepancy
                               Oversight            Differences in            Differences in
                                                   Interpretation of         Interpretation of
                                                        Criteria                   Study




                                                                                                     7 

				
DOCUMENT INFO