Evidence-Based Children’s Services: What it Might Mean for the Future of Children’s Services and the School of Social Work HOMECOMING 2006! Celebrating our Social Work Heritage: Embracing Our Future School of Social Work University of Maryland October 18, 2006 Richard P. Barth School of Social Work University of Maryland Baltimore, MD 21201 email@example.com The Call to Change • Disappointing family preservation services • Research in Children’s MH indicating that best practice is not being used (Weisz) • Fort Bragg Study: Enriched collaboration was not effective (Bickman) • FC Reentry rates are high (20% or more) – Reinvolvement rates (including reuabuse and reentry following reunification) is higher (approaching 36% in US at 3-years) Nothing Will Change… … unless we do. They say that time changes things, but you actually have to change them yourself Andy Warhol Children’s Services Reform Requires Multiple Changes • Everyone thinks of changing the world, but no one thinks of changing himself (Tolstoy) • We »the policy framework »the funder [or amount of funding] »the program manager »the service network »and the client Getting to Better Children’s Services SYSTEM CWS OUTCOMES Personnel System Reform (Policy and Demonstration projects) Interventions Professionaliism (Education, selection, experience, training) Evidence-Based Practices (ESTs, opportunity to provide, fit, supervision) Heavily adapted from Bickman & Reimer, n.d. We Must Change All of These and Our Approach to the Science of Practice EBP: Sciences-to-Services-to-Sciences • Three parts to the treatment change cycle – Research and Development – Dissemination and implementation – Monitoring and Feedback – More Research and More Development – and so on…. Source: New Freedom Commission (2005) Three types of needed supports: 1. Commitment 2. Capacity Building 3. Collaboration EBP Supports • Commitment (what we are talking about, today, can and should we commit ourselves to this process) • Capacity Building (if we commit, then we must also train, supervise, and support) • Collaboration (researchers, educators, trainers, and clinicians will all need to work together) Intended Outcomes of the EBP Cycle • Providers will deliver the most up-to-date and effective mental health treatment to the consumers who seek their help. • Yet, … ―there is an uncomfortable irony in moving forward to implement evidence-based practice in the absence of an evidence base to guide implementation practice (New Freedom Commission on Mental Health, 2005; p. 12). Unintended Outcomes of EBP Unintended Outcomes of EBP The Alphabet of EBP What is needed, it seems to me, is some course of study where an intelligent young person can ... be taught the alphabet of charitable science. Anna Dawes (1983) From a paper given at the International Congress of Charities and Correction at the Chicago World's Fair. Source: Lehninger, L. (2000). Creating a new profession: The beginnings of social work education in the United states. Washington, DC: Council on Social Work Education. EBP and ESIs and Practice Guidelines • Evidence Based Practice – Procedures and processes that result in the integration of the best research evidence with clinical expertise and client values • Evidence Supported Interventions – Interventions that have the support of the ―best research evidence‖ showing their efficacy or effectiveness • Practice Guidelines – A set of strategies, techniques, and treatment approaches that support or lead to a specific standard of care that guides systems, care, and professions in their relationships to consumers Effective & Efficacious Interventions • Effective (or well-established) treatments are those which have beneficial effects when delivered to heterogeneous samples of clinically referred individuals treated in clinical settings by clinicians other than researchers. • Efficacious (or clinical utility) studies are directed at establishing how well a particular intervention works in the environment and under the conditions in which treatment is typically offered. Source: Lonigan, C.J., Elbert, J.C., & Johnson, S.B. (1998). Empirically Supported Psychosocial Interventions for Children. Journal of Clinical Child Psychology, 27:2. 138-14 A Children’s MH ESI Sampler Priority Set of Evidence-Based Practices for Children and Families: Home-Based Crisis Intervention (HBCI) Intensive Case Management (ICM and MST) Cognitive Behavioral Therapies for Childhood Trauma Functional Family Therapy (FFT) Family Empowerment Family Education and Support Services Promising Practices Treatment Recommendations (Practice Guidelines) for the Use of Antipsychotics for Aggressive Youth (TRAAY) NYS OMH's Website Quality Practices Underlying EBPs • The nature and quality of the therapeutic relationship is critical • Quality practices are: – inclusive and continuous – individualized: matching services to the needs strengths, preferences, and values of the recipient – promoting of responsible partnerships via informed and shared decision making. – inseparable from quality practitioners – dynamic, outcome oriented, and continuous – culturally competent http://www.omh.state.ny.us/omhweb/ebp An EBP Example: Evidence Based Parent Training • About 800,000 caregivers a year receive parent training after CWS contact • There is almost no evidence of its effectiveness • Completion is as little as 20% in some programs—may be about 55%, overall (CDC) • Parents have a right to effective services and providers have a need for effective services Basic Components of Effective Parent Mediated Interventions • • • • • Social learning framework Strengthening parent-child relationship Effectively use praise and reward Sets clear and effective limits Reserves most significant consequences for targeted, limited behaviors • Strictly limits negative consequences • Parent Training + may have worse outcomes than parent training alone (CDC) • Addresses family as well as parent-child issues Hurlburt, M., Barth, R.P., Leslie, L. & Landsverk, J. (in press). Haskins, R., Wulczyn, F., & Webb, M. (Eds). Research on child protection: Findings from NSCAW. Washington, DC: Brookings. Why Should We Invest in Parent Training? • Because we have a revolving foster and group care door that lets in too much harm and too many costs • Because we are struggling to maintain quality foster and group care homes • Because children want to be with their parents, siblings, and kin and Delivering Effective Parent Training Programs • Detailed materials corresponding to specific, narrowly focused parenting skills • Specific means of monitoring changes in parenting practices (e.g., homework) • Parents take active, participatory role in learning and practicing skills • Minimum 15 hours of intervention and 25 hours for group format • Rigor of supervision processes to ensure program delivery with fidelity Three D’s: Stages to Practice Change • Discovery of new knowledge • Development of highly effective evidence based methods • Delivery of knowledge and interventions PCIT, MTFC, SafeCare 30 Years development for 30 years California Clearinghouse Criteria Scientific Rating 1. Well supported, effective practice 2. Supported - efficacious practice 3. Promising practice 4. Acceptable/emerging practice 5. Evidence fails to demonstrate effect 6. Concerning practice California Clearinghouse Criteria Child Welfare Relevance… Do The Studies Address: 1. Safety 2. Permanency 3. Well-being Parent Training Programs SO FAR ….. NO PARENT TRAINING PROGRAMS HAVE THE HIGHEST SCIENTIFIC RATING AND CW RELEVANT RESEARCH But Some are Close… • The Incredible Years • Parent-Child Interaction Therapy • Triple-P, Positive Parenting Program • Are all rated • 1 For Scientific Merit and • 2 for Relevance to Child Welfare http://www.cachildwelfareclearinghouse.org/ Movement Toward Bringing Evidence Based Parent Training to CWS • Good new ideas have been developed that could assist CWS • Their use will require deep involvement of CWS in implementation: – We cannot implement them all at once – We must allocate adequate resources to adapting them to CWS populations and practice parameters – We must also provide extensive supervision during implementation 3 Approaches to EBP Development of PMI • The Ecological Validity Approach (Homebuilders, Healthy Families America, Family 2 Family’s Team Decision Making,& FGDM) – Develop practices in the field – Disseminate them widely – Then try to study their effectiveness • The Masterplan Approach (Nurse Home Visiting and MST) – Develop interventions in the field but then continue to replicate with tight controls prior to dissemination – Okay, but they haven’t made it to CWS, yet • The Partnering Approach (Project KEEP, PCIT, SafeCare) – Adapt evidence based interventions developed by other professions in labs and artificial settings by engaging social work/child welfare collaborators in when going into the CWS field – Test in rigorous clinical trials in CWS Four Partnering Examples • PCIT in OK (Chaffin et al, CDC]) • Project KEEP in San Diego (Chamberlain, Price and Landsverk, NIMH) • SAFECARE in OK (Chaffin and all, NIMH) • (The Incredible Years: Under development for CWS testing with foster parents in NYC) Parent Child Interaction Therapy (PCIT) • The PCIT program is for children 4 to 10 and consists of: – Relationship Enhancement: Parents are taught and 'coached' how to decrease negativity and increase consistently positive communication with their child. – Discipline: parents are taught and 'coached' the elements of effective discipline and child management skills. – Parents are taught specific skills, given the opportunity to practice these skills during therapy, then continue practicing skills until mastery is acquired and the child's behavior has improved. • PCIT is now in place in NYC (used with foster parents) several CA sites and other states – Therapists provide reports of parental competency at end of PCIT!!! ^Chaffin, M., Silovsky, J., Funderburk, B., Valle, L. A., Brestan, E. V., Balachova, T., et al. (2004). Parent-child interaction therapy with physically abusive parents: Efficacy for reducing future abuse reports. Journal of Consulting and Clinical Psychology, 72, 500-510. PCIT, RCT in OK with PA Parents • Participating parents had history of engaging in severe physically abusive behavior. • Physical abuse re-report rates at a median of 850 days of follow-up were 19% for the PCIT group compared to 49% for a standard community parenting group. – Addition of individualized wrap-around services did not improve physical abuse re-report outcomes (and may have been counterproductive). – No differences in outcomes by age, gender or race/ethnicity Different therapists achieved comparable results. • PCIT cost more than standard approach, but the longterm savings were greater. Chaffin, M., Silovsky, J. F., Funderburk, B., Valle, L. A., Breston, E. V., Balachova, T., et al. (2003). Physical abuse treatment outcome project: Application of parent child interaction therapy (PCIT) to physically abusive parents. Washington, D C: U. S. Department of Health and Human Services, The Administration on Children, Youth and Families, Children's Bureau, Office on Child Abuse and Neglect. PCIT Adaptations • The age group had to be modified from 4-10 to 4-12 – Changing the age group also changes the way that time out needs to be taught and reinforced – Working with older and abused children was different than working with younger and conduct disordered children, insofar as there was less naturally occurring misbehavior by the abused children • Transportation was a major issue because the foster parents did not have the same motivation as biological parents have • PCIT required a much higher initial investment. Usual and customary parent training care in OK was $15 an hour for the therapist, with no prep time. Chaffin, M., Silovsky, J. F., Funderburk, B., Valle, L. A., Breston, E. V., Balachova, T., et al. (2003). Physical abuse treatment outcome project: Application of parent child interaction therapy (PCIT) to physically abusive parents. Washington, D C: U. S. Department of Health and Human Services, The Administration on Children, Youth and Families, Children's Bureau, Office on Child Abuse and Neglect. MTFC: Randomized Studies • 1996 application to ―regular‖ foster care: (Chamberlain, Moreland and Reid, 1996) randomly assigned MTFC group had: – Fewer placement disruptions – Fewer foster parents dropping out of providing care – Fewer child problem behaviors in follow-up Female delinquency processes and outcomes (Chamberlain 1999): 80 girls randomly assigned to MTFC or Group Care had fewer arrests. Recent development of MTFC-P (Phil Fisher’s work on MTFC for behaviorally disordered pre-schoolers) • • MTFC Emphasizes 3 Domains: Foster Parent Consultant Family Therapist ‘Daily Report’ Caller STAFF Case Manager Child Therapist Behavioral Skills Trainer Child Psychiatrist Caregiver-Child Relationship Case Management Child Needs Contexts Home Community Preschool/school MTFC to Foster Care: Project KEEP • • • MTFC with randomized treatment and control groups Foster children ages 5 to 12 who are experiencing a new placement (first time or change of placement) and their foster parents 640 sets of foster children/parents have been enrolled so far including: – 333 Experimental treatment group – 305 Control group – 40% are kinship homes A collaboration between the: • San Diego Health and Human Services Agency, • Child and Adolescent Services Research Center, • Oregon Social Learning Center, –Funded by the NIMH How is KEEP Different than MTFC or Standard Foster Care? • Uses Parental Daily Report (PDR) to repeatedly assess behavior problems. • PDR helps identify challenges to Foster/kinship parents and tailor the intervention to be relevant – Weekly support and training in behavior management. Relates problems the foster/kin parents are having in their homes to intervention strategies (>80% attendance). – Follows up with those problems • Stipend: Foster/kin parents get $15/week to cover expenses plus day care and snacks during training Project KEEP Findings So Far • Process – Foster/kin parents tolerate about as much child problem behaviors as non-system families do –5 problem behaviors a day – PDR data is feasible to collect and is well tolerated by foster and kin parents (PDR is a new tool for child welfare services to make parent training more relevant) • Outcomes – Children with foster parents who participated in Project Keep were less likely to disrupt – Project Keep children were more likely to go home to family, relatives, or adoption – These two outcomes taken together produced a statistically significant positive effect on exits (see next slide) Project KEEP: %of Exit Type by Group 15 12 Percentage 9 6 3 0 Negative Exit Positive Exit Project KEEP Control Group (Foster Care) •Children whose foster parents participated in Project KEEP were almost twice as likely to leave foster care for reunification or adoption. •Children whose foster parents were not using project KEEP were more likely to run away, disrupt, or have another negative exit from care. Project KEEP: CWS Real World Modifications • More than one child to a home, at times • Children were already in home when training was completed • Adherence to protocols was not as sharp as with highly trained and experienced OSLC MTFC providers – The intervention could be called MTFC-lite‖ Project SafeCare • Trial in Oklahoma has promising preliminary findings (Mark Chaffin, PI) – Neglecting families that get SafeCare when the parent trainer gets high levels of supervision are having fewer re-reports of neglect than: • Families getting SafeCare without intensive supervision • Families getting services as usual Lutzker, J. R., & Bigelow, K. M. (2002). Reducing child maltreatment: A guidebook for parent services. New York, NY: The Guilford Press, describes the ―SafeCare‖ Intervention Project SafeCare • RCT in Oklahoma (2X2 Design) – SafeCare vs. Usual Care – Intensive Supervision vs. Usual Supervision • Home-based set of skills-based parent training interventions • Includes a set of twelve protocols focused on building parent skills in the areas of – parent-child interaction, – self-control training – general parenting training – money management, and others. – infant and child health care, – and home safety and cleanliness. Project SafeCare in OK Time to Reabuse by Condition 100% SAFE CARE PLUS ONGOING SUPERVISION SAFE CARE ONLY USUAL CARE + ONGOING SUPERVISION 40% USUAL CARE Project SafeCare Modifications • Extensive supervision is needed following instruction • Appears to work best with ―neglect‖ cases, requiring that cases be sorted by maltreatment type prior to referral • Requires that services be in the home Expanding Evidence-Based CWS • Changing funding practices, by: – Tie funding, and reimbursement for CWS to objective outcomes rather than inputs (NOT YET) – Use differential payment structures favoring best practices delivered with fidelity (FOR PARENT TRAINING) – Targeted funding of EBP implementation projects (e.g,. EBP uptake grants), to provide agencies with the necessary start-up capital to migrate to best practice models (YES) Chaffin, M. & Friedrich, W. (2004). Evidence-based treatments in child abuse and neglect. Children & Youth Services Review, 26, 1097-1103. Expanding Evidence-Based CWS • Increase advocacy and social demand for best practices by disseminating cautiously derived (emphasis is mine) information to: – funding organizations, – governing boards, – third-party payers, – parents, – and professional organizations Chaffin, M. & Friedrich, W. (2004). Evidence-based treatments in child abuse and neglect. Children & Youth Services Review, 26, 1097-1103. Changing Social Work Education to Make it More Evidence Based • ―Be the change you want to see in the world‖ Mahatma Ghandi • ―If you want to change the world, change yourselves‖ Tom Robbins Remember What Progress We Have Made… We Can Do It! SW Education and ESIs • SW has generally not adopted the approach of using ―practice guidelines‖ (c.f., GWB website for an extensive list of them) – Few programs use therapist protocols, client workbooks, or training videos • One program (GWB) is trying to teach ESIs in all practice methods courses Source: Woody, J. D., D’Souza, H.J., & Dartman, R. (2006). Do masters in social work programs tetach empirically supported interventions? A survey of deans and directors. Research on Social Work Practice, 16, 469-479. Limitations of Evidence-Based Treatments • Long-term effects of the treatments may not be known. • Assessments of the effectiveness of a treatment may vary across studies, populations, questions, and methods (Rodwin, 2001). • Clinicians often need to be re-trained and carefully supervised by experienced practitioners • Many of the studies used in evidence-based medicine and psychosocial treatment have excluded very important variables such as training, staff turnover, minimal family involvement and co-morbidity of conditions (Burns et al., 1999). YET…. … Unlike the APA and School Psychology and Medicine: – CSWE, SSWR, and NASW have not formed any task forces to identify and disseminate information about EBP and – No school of social work has formally adopted a list of ESIs What Would EBP-Sensitive SW Education ? STUDENTS WOULD HAVE: – Understanding of EBP approaches – Commitment to ESIs – Capacity to seek and find information through the reliable use of scientific databases – Ability to choose ESIs – Ability to deliver ESIs – Capacity to evaluate intervention efforts Howard, M.O., McMillen, C.J. & Pollio, D.F. (203). Teaching evidence-based practice: Toward a new paradigm for social work education. Research on Social Work Practice, 13, 234-259. Teaching Evidence Based Practice Clinical State & Circumstances Clinical Expertise Client Preferences and Actions Research Evidence Source: Shlonsky and Wagner, 2005 Teaching Evidence Based Practice Contextual Assessment Clinical State & Circumstances Appropriate for this client? Valid Assessment? Clinical Expertise Client Preferences and Actions Research Evidence Client Preference or Willing to Try? Cultural Barriers? Effective Services Source: Shlonsky and Wagner, 2005 What Might CPE Look Like? • Clinicians understand the need to learn the latest evidence based practices, but are frustrated by one time workshops in which new skills and techniques are reviewed, but long term practice and acquisition fall short. • CU model insures active learning and processing of the new skills through bi-weekly clinical consultation calls with nationally known experts. Clinicians slowly gain proficiency as they explore application of these new intervention skills with complex cases. http://www.kidsmentalhealth.org/ Ongoing Training and Supervision • Training takes place during a two day workshop, with extensive role-plays and feedback. • Phone consultation on cases for 12 months (twice monthly) via toll-free conference calls with nationally known expert consultants and treatment developers facilitates learning and implementation. • Training certificates are granted to clinicians who complete full training (face-to-face training and follow-up consultation calls). • Rebates on costs if they also participate in evaluation research What Might UMB SSW Do? • Clarify our commitment to EBP and ESIs • Commitment means understanding and teaching the entire process of EBP, not just teaching ESIs • Commitment means NOT searching for research evidence that only supports a given premise • Commitment means searching as diligently for disconfirming evidence as for evidence that supports our hunches. Shlonsky, A. & Gibbs, L. (2004). Will the Real Evidence-Based Practice Please Stand Up? Teaching the process of evidence-based practice to the helping professions. Brief Treatment and Crisis Intervention, 4, 137-152. From Commitment to Language • Become aware of EBP and EBP processes • Become aware of commonly nominated ESIs – Teach the key features of ESIs – Teach the language of ESIs • Have all students become familiar with important EBP resources (papers and WEBSITES) Improving the teaching of evidence based practice http://www.utexas.edu/ssw/ceu/practice/articles.html From Language to Learning • Hiring of faculty who are knowledgeable about EBP and ESIs • Training fellowships for faculty to learn ESIs • Investing in training materials for social work faculty and students • Bringing information about EBP and ESIs to field instructors From Learning to Offerings • EBP process taught in foundation courses – Use of Gibbs COPES model • Use of ESIs encouraged in field – Trench-to-bench research in the field – Mutual and collaborative exploration of interventions • ESIs in CPE – Spring program is in the works – CPE offerings from visiting experts • EBP and ESIs in advanced courses • ESIs taught in short courses • Clarification of the role of EBP in macro practice Conclusions • The public expects science-based policy and programs • Social work has a tradition based on scientific charity • The science of intervention is building but slowly • We can catch up … … or fall farther behind. So Let’s Keep Going… however slowly Good must become better and better must become best … before we rest Fear the Turtle! ANY Or Comments Partial References Aarons, G. A. (2005). Measuring provider attitudes toward evidence-based practice: Consideration of organizational context and individual differences. Child and Adolescent Psychiatric Clinics of North America, 14(2), 255-+. Aos, S. Lieb, R. Mayfield, R. Miller, M. Pennucci, A. (2004) Benefits and Costs of Prevention and Early Intervention Programs for Youth. Olympia: Washington State Institute for Public Policy, available at <http://www.wsipp.wa.gov/rptfiles/04-07-3901.pdf>. Barth, R. P., Crea, T. M., John, K., Thoburn, J., & Quinton, D. (2005). Beyond attachment theory and therapy: Towards sensitive and evidence-based interventions with foster and adoptive families in distress. Child and Family Social Work, 10, 257-268. Barth, R. P., Landsverk, J., Chamberlain, P., Reid, J., Rolls, J., Hurlburt, M., & McCabe, K. (2005). Parent training in child welfare services: Planning for a more evidence based approach to serving biological parents. Research on Social Work Practice. Burns, B. J., & Hoagwood, K. (2002). Community treatment for youth: Evidence-based interventions for severe emotional and behavior disorders. New York: Oxford University Press. Chambers, D. A., Ringeisen, H., & Hickman, E. E. (2005). Federal, state, and foundation initiatives around evidence-based practices for child and adolescent mental health. Child and Adolescent Psychiatric Clinics of North America, 14(2), 307-+. Chambless, D. L., & Ollendick, T. H. (2001). Empirically supported psychological, interventions: Controversies and evidence. Annual Review of Psychology, 52, 685-716. Dawson, K., & Berry, M. (2002). Engaging families in child welfare services: An evidence-based approach to best practice. Child Welfare, 81, 293-317. Flynn, L. M. (2005). Family perspectives on evidence-based practice. Child and Adolescent Psychiatric Clinics of North America, 14(2), 217-222. Gibbs, L.(2003). Evidence-based practice for the helping professions: A practical guide with integrated media. Monterey, CA: Brookes Cole. More Partial References Hoagwood, K. E., & Burns, B. J. (2005). Evidence-based practice, part II: Effecting change. Child and Adolescent Psychiatric Clinics of North America, 14(2), XV-XVII. Littell, J. H. (2005). Lessons from a systematic review of effects of multisystemic therapy. Children and Youth Services Review, 27(4), 445-463. Pennell, J. and Burford, G. (2000). Family group decision making: Protecting women and children. Child Welfare, 79(2), 131-158. Rosen, A. & Proctor, E. K. (Eds.), Developing practice guidelines for social work intervention. New York: Columbia University Press, 2004 Saunders, B. E., Berliner, L., & Hanson, R. F. E. (2003). Child physical and sexual abuse: Guidelines for treatment (Final report: January 15, 2003). Charleston, SC: National Crime Victims Research and Treatment Center. Sundell, K., and Vinnerljung, B. (2004). Outcomes of family group conferencing in Sweden: A 3-year follow-up. Child Abuse & Neglect, 28, 267-287. Shlonsky, A. & Wagner, D. (2005). The next step : integrating actuarial risk assessment and clinical judgment into an evidence-based practice framework in CPS case management. Children And Youth Services Review, 27, 409-427. Thomlison, B. (2003). Characteristics of evidence-based child maltreatment interventions. Child Welfare, 82, 541-569. Thyer, B. (2006). evidence-based macro practice: Addressing the challenges and opportunities for social work education. Paper presented at the Improving the Teaching of Evidence Based Practice Conference. Austin, TX (October 17, 2006). Usher, C. L., & Wildfire, J. B. (2003). Evidence-based practice in community-based child welfare systems. Child Welfare, 82, 597-614. Wulczyn, F., Barth, R. P., Yuan, Y. Y., Jones Harden, B., & Landsverk, J. (in press). Evidence for child welfare policy reform. New York: Transaction De Gruyter. Appendix A: Keeping Up Effective substance abuse and mental health programs for every community. – www.modelprograms.samhsa.gov Blueprints for violence prevention—OJJDP identification of research-effective programs – www.colorado.edu/cspv/blueprints/index.html Systematic Reviews – www.campbellcollaboration.org Child Welfare – http://www.cachildwelfareclearinghouse.org Website of websites – Healthlinks.washington.edu/ebp/ebpresources.html The Coalition for Evidence-Based Policy – http://www.excelgov.org/displaycontent.asp?keyword= prppcHomePage
"Evidence-Based Children’s Services; What it Might Mean for the Future of children's Services"