PARENT-TEEN GROUP PROGRAM EVALUATION AT CRAIGWOOD YOUTH SERVICES by umsymums38

VIEWS: 0 PAGES: 24

									PARENT-TEEN GROUP PROGRAM
 EVALUATION AT CRAIGWOOD
      YOUTH SERVICES
    2006 Program Evaluation Grant # 162606-050




  Azzano, J.; Ashbourne, G.; & Ledyit, A.
    Craigwood Youth Services:
► Non-profit  Children's Mental Health Centre
► Accredited by Children's Mental Health
  Ontario
► 50 year history of serving youth and their
  families.
► Community & Residential Mental Health
  services for adolescents and families
► Rehabilitative Custody Programs
► Primarily Serving London & Middlesex
           Evaluation Focus
► To  increase internal capacity and knowledge
  in evaluation and outcome measurement
  using Brief Child & Family Phone Interview
  (BCFPI) and Child & Adolescent Functional
  Assessment Scale (CAFAS).
► Advance skill and knowledge in data
  analysis methods using BCFPI, CAFAS and
  data analysis software.
► To provide direct feedback to, and engage
  practitioners in, the evaluation of the group
  and associated course material.
          Group BCFPI & CAFAS
► 2003:  Parent Teen Group first offered.
► BCFPI for each client.
► 2006: CAFAS begins to be used pre-group
► As data size increases, CAFAS outcome data
  for this group will be analyzed to review the
  effectiveness of the group, and the
  effectiveness of the group in
  comparison/conjunction with other services.
     6 Week Parent-Teen Group
► First  as wait list support for IFSS 12 Week.
► Six-week group based on material from
  Scott P. Sells “Parenting Your Out-of-Control
  Teenager”.
► 8 youth/group, 3X/year (prior to this study)
► It is hoped that the group will fully meet the
  needs of some participants, shorten the
  intensive intervention stage for others and
  arrest the behaviour deterioration for the
  remainder while they await intensive
  services.
    Parent-Teen Group Indicators
► Currently   a stand-alone service, or, a wait-list
  support
► (>70) BCFPI scores in conduct, family activity,
  and family functioning supports referral
► Not yet “advertised” the group as an available
  service (pending), but introduced during the
  intake interview as an option.
► Currently clients holding for IFSS receive
  phone invitation from facilitator to join group.
   Service Utilization By Clients Who
Participated in Parent-Teen Group     Jan. ‟03 – Feb. ‟06.




   60%

                                In Group at
   50%                          time of
                                study

                                6-Week
   40%                          IFSS after
                                Group

                                12-Week
                                IFSS after
   30%                          Group

                                Group
                                Sufficient
   20%


   10%


    0%
             n=58
          Barriers to Service for
          PT Group Clients (n=20)
► About 25% identified that they would either have
 a lot or too much of a problem participating in
 services if they were only offered during the day.
 Group was offered in the evening.
► Norespondents identified that they would have a
 problem getting to the office for services. Group
 was primarily held at the office.
► Norespondents identified that they would have a
 problem with arranging babysitting in order to
 participate in services. Babysitting arrangements
 were not made.
 Barriers To Service For IFSS Clients
                        (n=42)

► About   1/3 would either have a lot or too much of
  a problem participating in services if services were
  only offered during the day.
► About 1/2 would have no trouble participating in
  services that were only offered in the evening.
► About 2/3 would have no trouble getting to the
  office (yet IFSS is an in-home service). Suggests
  in-office support component may be viable.
► About 20% would have a lot of trouble arranging
  babysitting in order to participate in services.
Age Group (All)   Gender (All)    Informant Type (All)   Stage (All)   Form (All)      Status (All)   City (All)   Postal/Zip Code(All)                 1
                                                                                                                                          User Geo Code(All)


                                                Average Scores in 8 Key Domains:
                                                  IFSS 12-Week Profile(n=147)
                                                    Jan 1, 2003 - Feb 38, 2006

                       110

                       100

                         90
                                                                                                                                               Agency
                                                                                                                                               Program
                         80

                         70

                         60

                         50
                                  RAIAp   COp     CDp      SPp    MAp     MMp     ChFp    GFsP
                                 Avg Scr Avg Scr Avg Scr Avg Scr Avg Scr Avg Scr Avg Scr Avg Scr

    CYS - IFSS-12 wk              71.91        74.95        83.54        56.11             53.74      68.06         76.37          95.42

                                                                                    Data
Age Group (All)    Gender (All)    Informant Type (All)   Stage (All)   Form (All)          Status (All)   City (All)   Postal/Zip Code (All)   User Geo Code 1 (All)


                                                  Average Scores in 8 Key Domains:
                                                  Parent-Teen Group Profile (n=60)
                                                     Jan. 1, 2003 - Feb. 28, 2006

              110

              100
                  90
                                                                                                                                                            Agency
                                                                                                                                                            Program
                  80
                  70
                  60
                  50       RAIAp            COp            CDp             SPp                   MAp            MMp             ChFp            GFsP
    CYS - PT Group         72.94           76.92          88.13           54.27                 53.31           66.20          78.62            92.66


                                                                                     Data
Age Group (All)   Gender (All)   Informant Type (All)   Stage (All)    Form (All)   Status (All)   City (All)   Postal/Zip Code (All)   User Geo Code 1 (All)



                                                 Comparative Average Scores:
                                              IFSS-12 Week & Parent-Teen Group
                                                  Jan. 1, 2003 - Feb. 28, 2006

                        110

                        100
                          90                                                                                                                      Agency
                                                                                                                                                  Program
                          80

                          70
                          60
                          50
                                 RAIA COp               CDp           SPp      MAp         MMp       SHp ChFp GFsP PMMp
                                 p Avg Avg              Avg           Avg      Avg         Avg       Avg Avg Avg Avg
     CYS - IFSS-12wk 71.91 74.95 83.54 56.11 53.74 68.06 72.55 76.37 95.42 65.38
     CYS - PT Group 72.94 76.92 88.13 54.27 53.31 66.20 73.25 78.62 92.66 65.93

                                                                                    Data
Gender (All)   City (All)   Postal/Zip Code (All)   User Geo Code 1 (All)




                                                                                       Average Scores:
                                                                            Before/After Parent-Teen Group (n=17)
                                                                                  Jan 1, 2003 - Feb. 28, 2006


                               110
                               100
                                                                                                                                               Agency
                                 90                                                                                                            Program
                                                                                                                                               Stage
                                 80
                                 70
                                 60
                                 50
                                         RAIA COp CDp EXp SPp MAp MMp SHp INp TMHP SocP QRel Scho ChFp FAct FcFp GFsP PMM FADp
                                         p Avg Avg Avg Avg Avg Avg Avg Avg Avg Avg artP P Avg olP Avg P Avg Avg Avg p Avg Avg
      CYS - PT Group - Before 76 80                           83 84 55          53 70 75      61 75 83 73           74 82 99   79 92 70   60
      CYS - PT Group - After 73 76                            87 80 53          52 58 56      56 70 73 71           71 77 106 75 91 60    64


                                                                                                 Data
             Pre/Post Outcomes
            (n=17) Jan. 1, ‟03 – Feb. 28, „06
► Overall improvement in Externalizing Behaviours
  (but note increase in Conduct)
► Overall improvement in Internalizing Symptoms.
  Improvement in Parent Mood was also noted.
  Parents are identifying that the depressive
  symptoms of both themselves and their children
  have reduced by the end of the group.
► Overall improvement in social functioning.
► It appears parents see their child‟s behaviour   as
  interfering more with family activities (FActP) and
  perceive less unity in family functioning (FADp) by
  the end of group. However, the parents report
  slightly less discomfort in dealing with youth‟s
  behaviours (FcFp).
Limitations to this Evaluation!!
         Evaluation Limitations
Although they have provided a framework for further
  evaluations, these results should be interpreted
  with caution for many reasons including:
► BCFPI sensitivity to change in brief service limited
► small sample size (n=58)
► Unclear what “statistical significance” is with
  respect to BCFPI t-scores
► We were unable to test assumptions using
  statistical software due to time limitations
► See next slide for additional reasons…
Rival Plausible Explanations for
Improvements Observed in One-Group Evaluations
THREAT          ALTERNATIVE EXPLANATIONS FOR APPARENT
                PROGRAM-RELATED CHANGES
Maturation      Program participants change over time on their own
                anyway
Testing         The answers on post-tests will be affected by the fact that
                project participants already did the same tests before
Regression      Extreme scores will drift toward the average as time passes
to the Mean
Selection       The people selected (or self-selected) into the program or
Bias            who finish the program are those most likely to be
                successful anyway.
Mortality       The group at the end is fundamentally different than the
                group at the beginning because some people have dropped
                out (e.g., the most problematic individuals may have
                become ineligible for service so the post-test scores look
                better because they are gone)
History         Intervening events such as changes in laws, policies or
                program context affect outcomes.

From: Cunningham, A. (2002). One step forward: Lessons learned from a randomized study of Multisystemic Therapy in
Canada. PRAXIS: Research from the Centre for Children & Families in the Justice System, London, Ontario.
          Knowledge Integration:
                Evolution
► Now  CAFAS pre-post data collected.
► CAFASinOntario helped us organize this data
  best for future within-service and between-
  service research.
► Evolution from wait list support to stand-alone
  service.
► 6-week in-home follow-up service created to
  support families when group insufficient.
           Knowledge Integration:
                 Synergy
                  (Creative Cooperation)

► Seeing   “evidence” of group outcome
  success - now offered more consistently
  with all IFSS clinicians (and students)
  rotating the facilitator role.
► Group concepts shared with St. George
  Street Treatment Home (prepares youth to
  transition from therapeutic setting) leading to
  increase in internal referrals and use of
  concepts by staff.
         Knowledge Integration:
              Enthusiasm
Receiving this grant began to generate general
  excitement and interest within the agency:
► Valued clinician enthusiasm for ongoing
  professional development & initiative to
  contribute to program development.
► Fostered excitement about research
► Validated clinician‟s efforts through
  observation and measurement (results aside).
          When Citing This
           Presentation:
Azzano, J.; Ashbourne, G.; & Ledyit, A.; (2007, May). Parent-Teen
     Group Program Evaluation at Craigwood Youth Services.
     Slideshow presented at the semi-annual Research and
     Evaluation in Child & Youth Mental Health in the South West
     Region sponsored by The Provincial Centre for Excellence in
     Child and Youth Mental Health, and, the Ministry of Children
     and Youth Services - South West Region, Windsor, ON.

								
To top