PROGRAM EVALUATION: Where to Begin by Lnyt5Byt

VIEWS: 2 PAGES: 30

									PROGRAM EVALUATION:
    Where to Begin
   Terry Buckley, BSN, RN
    Lamar Collins, Sr., BS




      Scott & White Hospital and Clinic
               Temple, Texas
    Federal Definition of Abstinence
        Education (aka A – H)
 A-Have as its exclusive         C-Teach that abstinence
  purpose, teaching the            from sexual activity is the
  social, psychological, and       only certain way to avoid
  health gains realized by         out-of-wedlock pregnancy,
  abstaining from sexual           sexually transmitted
  activity;                        diseases, and other
                                   associated health
 B-Teach that abstinence          problems;
  from sexual activity outside
  marriage as the expected        D-Teach that a mutually
  standard for all school age      faithful monogamous
  children;                        relationship in context of
                                   marriage is the expected
                                   standard of human sexual
                                   activity
    Federal Definition of Abstinence
        Education (aka A – H)
 E-Teach that sexual            G-Teach young people
  activity outside of the         how to reject sexual
  context of marriage is          advances and how alcohol
  likely to have harmful          and drug use increases
  psychological and physical      vulnerability to sexual
  effects;                        advances;

 F-Teach that bearing           H-Teach the importance of
  children out-of-wedlock is      attaining self-sufficiency
  likely to have harmful          before engaging in sexual
  consequences for the            activity.
  child, the child’s parents,
  and society;
                  RESOURCES
             Don’t reinvent the wheel!
 Curricula developers may have instruments
    – Add your own components as necessary

 The Office of Population Affairs has developed a core
  evaluation questionnaire** accessible at:
http://opa.osophs.dhhs.gov/titlexx/coreinstruments/coreinstruments.html
    AFL Prevention Programs
      Core Baseline Questionnaire
      Core Follow-up Questionnaire (no demographics)

    **No behavior questions included as many schools/programs do not
      allow them; add as able within your communities
        Basic Question Musts
 Easy to understand-5th or 6th grade reading
  level
 Vary by type
     True/False          Yes/No
     Multiple Choice     Likert Scale (1-5)
 Reword and repeat to validate answers
 Limit to as few questions necessary to gain
  needed information
 Have evaluator review
        Before You Hit PRINT--
          Approval Process
 Internal Review Board (IRB)
 School Board/Administrators
 Permission slips from parent/guardian
  – Active
  – Passive
 Field test on small group
             Design Concerns
 Confidentiality of results
  – Totally anonymous
  – Coded so as to be able to link pre to post
      Consult statistician to design coding
       Formatting the actual tool
 How many will be needed?
   – Print in office?
   – Order form from professional source (Scantron)?
   – Questions on form or separate?
 How will they be processed?
   – Hand entry on spreadsheet
   – Scanner available
 What is your budget?
   – Data entry time vs scanning time
        Hourly wage
        Scanner price (2-sided $8500+; $3300 software & training)
   – Margin of error (hand entry vs scan)
    After the Surveys are Done
 Analysis of data
  – Local colleges/universities
  – Independent evaluators
  – In-house research department
 Re-evaluate tool after initial & ongoing
  analysis complete
 Storage
  – Location/space
  – Length of time surveys need to be on file
  Program Expectations/Needs
 Internal
  –Knowledge, attitudes, behaviors
  –Local concerns
  –Concerns/Interests of Governing
   Board
  Program Expectations/Needs
 External
  –Funding source
     Federal
     State
     Private foundations
                         Internal
 Knowledge-based on what is covered in your
  program
  –   Basic anatomy
  –   Reproduction
  –   Sexually transmitted diseases
  –   Sexual abuse
  –   Legal ramifications
        Criminal laws
        Family laws
  – Other subjects included in curricula
                      Internal
 Local Concerns
  – Availability of services
      After school programs
      Mentoring
      Counseling, etc.
  – Drinking & driving
  – Drug use
  – Safe atmosphere for teen activities on
    weekends
 Concerns/Interests of Governing Board
                  External
 Funding Agency
  – Federal
     Age
     Race
     Gender
     Performance Measures
  Federal Performance Measure #1

 Proportion of program participants who
  successfully complete or remain enrolled in
  an abstinence-only program

  – Met by tracking pre- & post-survey numbers or
    sign-in sheets
 Federal Performance Measure #2

 Proportion of adolescents who understand
  that abstinence from sexual activity is the
  only certain way to avoid out-of-wedlock
  pregnancy and sexually transmitted disease.


  – Met by asking specific question(s); can be
    True/False or multiple choice/answer
             Sample Questions
 Not having sex is the only 100% effective way to
  avoid out-of-wedlock pregnancy and sexually
  transmitted diseases.
  – Yes            No             Maybe     (True / False)

 The way(s) to be certain to avoid out-of-wedlock
  pregnancy and STDs include:
  – Use a condom every time you have sex
  – Use birth control pills if you are going to have sex
  – Abstain from sexual activity
  Federal Performance Measure #3

 Proportion of adolescents who indicate an
  understanding of the social, psychological,
  and health gains to be realized by
  abstaining from premarital sexual activity.

  – Met by asking specific question(s); can be
    True/False or multiple choice/answer
              Sample Questions
 By staying abstinent, I will not have to worry about
  pregnancy, STDs or emotional problems caused
  by having sex.
      O Yes       O No       O Unsure        (True/False)
 By staying abstinent I can be certain (choose all that apply)
      O Not to become pregnant
      O Not to get infected with an STD
      O To be able to concentrate on my future goals
      O To feel better about my self-respect & respect for
        others
      O None of the above
  Federal Performance Measure #4

 Proportion of participant who report they
  have the refusal or assertiveness skills to
  resist sexual urges and advances.

  – Met by asking specific question(s); can be
    True/False or multiple choice/answer
             Sample Questions
 I feel confident in my refusal skills to resist
  sexual urges or advances.
     O Yes    O No   O Unsure       (True/False)

 If I am tempted to have sex I will
    O Give-in to the urge
    O Depend on my refusal/assertiveness skills to
      not give-in to the urge
    O Cry
    O Be embarrassed
 Federal Performance Measure #5

 Proportion of youth who commit to abstain
  from sexual activity until marriage.

  – Met by asking specific question(s); can be
    True/False or Yes/No
             Sample Questions
 I have made a pledge to remain sexually abstinent
  until I get married.
     O Yes    O No


 Some teens have made a pledge of sexual
  abstinence until marriage. I have
    O never made a pledge
    O made a pledge in church
    O made a pledge in school
    O made a pledge at home
  Federal Performance Measure #6

 Proportion of participants who intend to
  avoid situations and risk behaviors such as
  drug use and alcohol consumption, which
  make them more vulnerable to sexual
  advances and urges.

  – Met by asking specific question(s); can be
    True/False or Multiple choice
             Sample Questions
 I will avoid the use of alcohol and drugs in the
  future.
     O Yes   O No

 Some ways to be less likely to give in to sexual
  advances and urges include
    O Only dating in groups
    O Drinking alcohol to make me “fit-in”
    O Never drinking alcohol or taking illegal drugs
    O Letting my parent/guardian know where I am and who
      I am with
  State Performance Measures
 Vary from state-to-state
  – Refer to your RFP to make certain you
    have the ability to track information
    necessary to meet requirements.
  State Performance Measures
 Examples from Texas
   1. The reduction in the proportion of teens
      engaged in sex.
   2. The reduction in the rate of pregnancy in
      teens and adolescents.
   3. The reduction in the rate of STDs in teens
      and adolescents.
   4. The increase in the number of teens receiving
      abstinence education.
   5. The increase in the number of adults
      receiving abstinence education.
        Private Foundations
 Will vary
  – Refer to your RFP to make certain you
    have the ability to track information
    necessary to meet requirements.
  – May be specific
     Racial entity
     Gender group (i.e. only girls)
     Socioeconomic level
     Combination of any/all of the above
       Administering the Tool
 Ideally overseen by someone other than the
  teacher/instructor who will be implementing
  the curricula/program
 Within an institution/campus—pre-surveys
  completed on the same day
 Post-surveys completed at a designated
  time period after the completion of program
 Additional follow-up ideal when
  possible/practical (i.e. 6 mos., 1 yr., etc,)
  You’re Never Really Finished!
Each Program Year
 Re-evaluate each question
 Reword or change questions based on analysis
  results (clarify, add information, etc)
 Re-evaluate curriculum/program components
 Update as “times” change
  – Add behavior questions if allowed
  – Include new questions as climate of community permits

								
To top