PARENT-TEEN GROUP PROGRAM EVALUATION AT CRAIGWOOD YOUTH SERVICES 2006 Program Evaluation Grant # 162606-050 Azzano, J.; Ashbourne, G.; & Ledyit, A. Craigwood Youth Services: ► Non-profit Children's Mental Health Centre ► Accredited by Children's Mental Health Ontario ► 50 year history of serving youth and their families. ► Community & Residential Mental Health services for adolescents and families ► Rehabilitative Custody Programs ► Primarily Serving London & Middlesex Evaluation Focus ► To increase internal capacity and knowledge in evaluation and outcome measurement using Brief Child & Family Phone Interview (BCFPI) and Child & Adolescent Functional Assessment Scale (CAFAS). ► Advance skill and knowledge in data analysis methods using BCFPI, CAFAS and data analysis software. ► To provide direct feedback to, and engage practitioners in, the evaluation of the group and associated course material. Group BCFPI & CAFAS ► 2003: Parent Teen Group first offered. ► BCFPI for each client. ► 2006: CAFAS begins to be used pre-group ► As data size increases, CAFAS outcome data for this group will be analyzed to review the effectiveness of the group, and the effectiveness of the group in comparison/conjunction with other services. 6 Week Parent-Teen Group ► First as wait list support for IFSS 12 Week. ► Six-week group based on material from Scott P. Sells “Parenting Your Out-of-Control Teenager”. ► 8 youth/group, 3X/year (prior to this study) ► It is hoped that the group will fully meet the needs of some participants, shorten the intensive intervention stage for others and arrest the behaviour deterioration for the remainder while they await intensive services. Parent-Teen Group Indicators ► Currently a stand-alone service, or, a wait-list support ► (>70) BCFPI scores in conduct, family activity, and family functioning supports referral ► Not yet “advertised” the group as an available service (pending), but introduced during the intake interview as an option. ► Currently clients holding for IFSS receive phone invitation from facilitator to join group. Service Utilization By Clients Who Participated in Parent-Teen Group Jan. ‟03 – Feb. ‟06. 60% In Group at 50% time of study 6-Week 40% IFSS after Group 12-Week IFSS after 30% Group Group Sufficient 20% 10% 0% n=58 Barriers to Service for PT Group Clients (n=20) ► About 25% identified that they would either have a lot or too much of a problem participating in services if they were only offered during the day. Group was offered in the evening. ► Norespondents identified that they would have a problem getting to the office for services. Group was primarily held at the office. ► Norespondents identified that they would have a problem with arranging babysitting in order to participate in services. Babysitting arrangements were not made. Barriers To Service For IFSS Clients (n=42) ► About 1/3 would either have a lot or too much of a problem participating in services if services were only offered during the day. ► About 1/2 would have no trouble participating in services that were only offered in the evening. ► About 2/3 would have no trouble getting to the office (yet IFSS is an in-home service). Suggests in-office support component may be viable. ► About 20% would have a lot of trouble arranging babysitting in order to participate in services. Age Group (All) Gender (All) Informant Type (All) Stage (All) Form (All) Status (All) City (All) Postal/Zip Code(All) 1 User Geo Code(All) Average Scores in 8 Key Domains: IFSS 12-Week Profile(n=147) Jan 1, 2003 - Feb 38, 2006 110 100 90 Agency Program 80 70 60 50 RAIAp COp CDp SPp MAp MMp ChFp GFsP Avg Scr Avg Scr Avg Scr Avg Scr Avg Scr Avg Scr Avg Scr Avg Scr CYS - IFSS-12 wk 71.91 74.95 83.54 56.11 53.74 68.06 76.37 95.42 Data Age Group (All) Gender (All) Informant Type (All) Stage (All) Form (All) Status (All) City (All) Postal/Zip Code (All) User Geo Code 1 (All) Average Scores in 8 Key Domains: Parent-Teen Group Profile (n=60) Jan. 1, 2003 - Feb. 28, 2006 110 100 90 Agency Program 80 70 60 50 RAIAp COp CDp SPp MAp MMp ChFp GFsP CYS - PT Group 72.94 76.92 88.13 54.27 53.31 66.20 78.62 92.66 Data Age Group (All) Gender (All) Informant Type (All) Stage (All) Form (All) Status (All) City (All) Postal/Zip Code (All) User Geo Code 1 (All) Comparative Average Scores: IFSS-12 Week & Parent-Teen Group Jan. 1, 2003 - Feb. 28, 2006 110 100 90 Agency Program 80 70 60 50 RAIA COp CDp SPp MAp MMp SHp ChFp GFsP PMMp p Avg Avg Avg Avg Avg Avg Avg Avg Avg Avg CYS - IFSS-12wk 71.91 74.95 83.54 56.11 53.74 68.06 72.55 76.37 95.42 65.38 CYS - PT Group 72.94 76.92 88.13 54.27 53.31 66.20 73.25 78.62 92.66 65.93 Data Gender (All) City (All) Postal/Zip Code (All) User Geo Code 1 (All) Average Scores: Before/After Parent-Teen Group (n=17) Jan 1, 2003 - Feb. 28, 2006 110 100 Agency 90 Program Stage 80 70 60 50 RAIA COp CDp EXp SPp MAp MMp SHp INp TMHP SocP QRel Scho ChFp FAct FcFp GFsP PMM FADp p Avg Avg Avg Avg Avg Avg Avg Avg Avg Avg artP P Avg olP Avg P Avg Avg Avg p Avg Avg CYS - PT Group - Before 76 80 83 84 55 53 70 75 61 75 83 73 74 82 99 79 92 70 60 CYS - PT Group - After 73 76 87 80 53 52 58 56 56 70 73 71 71 77 106 75 91 60 64 Data Pre/Post Outcomes (n=17) Jan. 1, ‟03 – Feb. 28, „06 ► Overall improvement in Externalizing Behaviours (but note increase in Conduct) ► Overall improvement in Internalizing Symptoms. Improvement in Parent Mood was also noted. Parents are identifying that the depressive symptoms of both themselves and their children have reduced by the end of the group. ► Overall improvement in social functioning. ► It appears parents see their child‟s behaviour as interfering more with family activities (FActP) and perceive less unity in family functioning (FADp) by the end of group. However, the parents report slightly less discomfort in dealing with youth‟s behaviours (FcFp). Limitations to this Evaluation!! Evaluation Limitations Although they have provided a framework for further evaluations, these results should be interpreted with caution for many reasons including: ► BCFPI sensitivity to change in brief service limited ► small sample size (n=58) ► Unclear what “statistical significance” is with respect to BCFPI t-scores ► We were unable to test assumptions using statistical software due to time limitations ► See next slide for additional reasons… Rival Plausible Explanations for Improvements Observed in One-Group Evaluations THREAT ALTERNATIVE EXPLANATIONS FOR APPARENT PROGRAM-RELATED CHANGES Maturation Program participants change over time on their own anyway Testing The answers on post-tests will be affected by the fact that project participants already did the same tests before Regression Extreme scores will drift toward the average as time passes to the Mean Selection The people selected (or self-selected) into the program or Bias who finish the program are those most likely to be successful anyway. Mortality The group at the end is fundamentally different than the group at the beginning because some people have dropped out (e.g., the most problematic individuals may have become ineligible for service so the post-test scores look better because they are gone) History Intervening events such as changes in laws, policies or program context affect outcomes. From: Cunningham, A. (2002). One step forward: Lessons learned from a randomized study of Multisystemic Therapy in Canada. PRAXIS: Research from the Centre for Children & Families in the Justice System, London, Ontario. Knowledge Integration: Evolution ► Now CAFAS pre-post data collected. ► CAFASinOntario helped us organize this data best for future within-service and between- service research. ► Evolution from wait list support to stand-alone service. ► 6-week in-home follow-up service created to support families when group insufficient. Knowledge Integration: Synergy (Creative Cooperation) ► Seeing “evidence” of group outcome success - now offered more consistently with all IFSS clinicians (and students) rotating the facilitator role. ► Group concepts shared with St. George Street Treatment Home (prepares youth to transition from therapeutic setting) leading to increase in internal referrals and use of concepts by staff. Knowledge Integration: Enthusiasm Receiving this grant began to generate general excitement and interest within the agency: ► Valued clinician enthusiasm for ongoing professional development & initiative to contribute to program development. ► Fostered excitement about research ► Validated clinician‟s efforts through observation and measurement (results aside). When Citing This Presentation: Azzano, J.; Ashbourne, G.; & Ledyit, A.; (2007, May). Parent-Teen Group Program Evaluation at Craigwood Youth Services. Slideshow presented at the semi-annual Research and Evaluation in Child & Youth Mental Health in the South West Region sponsored by The Provincial Centre for Excellence in Child and Youth Mental Health, and, the Ministry of Children and Youth Services - South West Region, Windsor, ON.
Pages to are hidden for
"PARENT-TEEN GROUP PROGRAM EVALUATION AT CRAIGWOOD YOUTH SERVICES"Please download to view full document