Docstoc

ACE_BestBet

Document Sample
ACE_BestBet Powered By Docstoc
					                     Ramsey County




                  Promoting Resiliency in Children
         at High Risk for Serious/Violent Delinquency,
                Substance Abuse and School Failure



      “BEST BET” INTERVENTION STRATEGY

                                     JULY 2002

REPORT TO RAMSEY COUNTY BOARD OF COMMISSIONERS
                ON IMPLICATIONS OF ACE           RESEARCH


Prepared by:
Trisha Beuhring, Ph.D.                            Hope Melton, M.A., M.C.R.P.
Institute on Community Integration                ACE Program Director
College of Education and Human Development        Department of Public Health
University of Minnesota                           Ramsey County, Minnesota
ALL CHILDREN EXCEL (ACE)

Leadership Team
  Public Health                                              Rob Fulton, Director and Chair
  Human Services                                                    Monty Martin, Director
  Corrections                                                         Joan Fabian, Director
  County Attorney’s Office                                 Darwin Lookingbill, Civil Division


Program Director                                                Hope Melton, M.A., M.C.R.P.

Screening/Integrated Services Delivery Teams
  Human Services
     Child Protection Services                                 Jack Jones, Sr. Social Worker
     Child Welfare Services                                     Jody McElroy, Social Worker
     Financial Services                                     Connie McKee, Financial Worker
     Mental Health Services                                        Ed Frickson, Psychologist
  Corrections                                      Four Juvenile Probation Officers (rotating)
  County Attorney’s Office                             Leslie Norsted, Asst. County Attorney
  Public Health                                                        Hope Melton, Director


Program Design and Evaluation                                              Trisha Beuhring
                                                        Institute on Community Integration
                                             College of Education and Human Development
                                                                    University of Minnesota
Community Organizations
  Face-to-Face Health & Counseling, Inc.                 Multisystemic Therapy intervention
  Northwest Youth and Family Services                        Case management intervention
  St. Paul Youth Services                                    Case management intervention
  St. Paul YWCA                                              Case management intervention


Grant Support
  Local Collaborative Time Study                    Ramsey County Board of Commissioners
  McKnight Foundation                                       Nancy Latimer, Program Officer
  Office of Drug Policy and Violence Prevention,
  MN Department of Public Safety                              Jeri Boisvert, Program Officer

  Ramsey County Children’s Mental Health Collaborative


                    Further information may be obtained by contacting
                          Hope Melton, ACE Program Director
             Phone: 651-266-4202 E-mail: Hope.Melton@Co.Ramsey.MN.US
ABSTRACT

    The ACE program is grounded in research which indicates that 30% to 60% of very young
offenders—children under 10 who commit chargeable offenses—are likely to become chronic, serious
or violent offenders by the end of adolescence. Children who follow this path become part of a very
small group of chronic offenders that commit up to 70% of all serious and violent juvenile crimes.1,2
Most of these chronic juvenile offenders go on to become career criminals who cost society an
estimated $1.7 to $2.4 million each over the course of their lifetime.

    At the time ACE was implemented, very little was known about the characteristics of very young
offenders or about the most effective way to intervene. Consequently, research and program
evaluation have been an integral part of the program from the outset. Together, they provide the
foundation for continually improving the effectiveness—and cost-effectiveness—of the ACE program.

   This report summarizes the key lessons learned in the three years since ACE was implemented,
and how the intervention has been refined to take these lessons into account.

Ø   ACE is successfully identifying very young offenders who are on the path to chronic serious/violent
    delinquency and other problem behaviors (page 2)

Ø   Within the long term intervention group, there are meaningful differences in risk profiles. These
    differences have implications for the type, intensity and duration of services the children will need
    for success (pages 3–6).

Ø   Multisystemic Therapy may be effective as a stand alone intervention for lesser risk young
    offenders, but it is not effective for the high risk young offenders in ACE’s long term intervention
    group (pages 9–12).

Ø   The “best bet” for ACE’s high risk young offenders is a long term case management strategy that
    targets multiple outcomes, multiple risk and protective factors, includes siblings, and is closely
    supervised to ensure steady progress on multiple fronts (pages 14–17).

Ø   Principles of Multisystemic Therapy are incorporated into the “best bet” strategy to enhance the
    effectiveness of long term case management (page 12).

Ø   Cost/benefit research confirms that the most challenging cases are the best investment. The
    issue is not which children are most likely to benefit, but which children are most likely to benefit
    the community if success is achieved (page 17).




_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                              Page i
                                  TABLE OF CONTENTS

ACE LEADERSHIP, STAFF, FUNDERS                                                Inside front cover
ABSTRACT                                                                                     i
INTRODUCTION                                                                                 1
   Role of Research and Program Evaluation                                                   1
   Continuous Quality Improvement to Date                                                    1
UNDERSTANDING THE TARGET POPULATION                                                          2
   Long-term versus Short-term Intervention Groups                                           2
   Risk Subgroups                                                                            3
   Long Term Intervention—Three Risk Subgroups                                               3
   Gender and Race                                                                           6
INVESTIGATING ALTERNATIVES                                                                   7
   Multisystemic Therapy       (MST)                                                         9
   Case Management                                                                          12
INTERVENTION STRATEGY                                                                       14
   History                                                                                  14
   “Best Bet” Strategy                                                                      15
MOST CHALLENGING CASES ARE BEST INVESTMENT                                                  17
CONCLUSIONS                                                                                 18
REFERENCES                                                                                  19

                                          FIGURES


Figure 1. Key Risk Factors Among Very Young Offenders Screened by ACE                        2
Figure 2. Percent LTI with Moderate to Severe Levels of Child Risk Factors                   4
Figure 3. Percent LTI with Moderate to Severe Levels of Family Risk Factors                  4
Figure 4. Percent LTI with Key Risk Factors for Serious/Violent Delinquency                  5
Figure 5. Percent LTI with Key Risk Factors for Substance Abuse and Mental Illness           5
Figure 6. Percent LTI with Key Risk Factors for School Failure                               6
Figure 7. Original ACE Model                                                                 8
Figure 8. “Best Bet” ACE Model                                                               16
INTRODUCTION


Role of Research and Program Evaluation

    The ACE program is grounded in research which indicates that 30% to 60% of very young
offenders—children under 10 who commit chargeable offenses—are likely to become chronic, serious
or violent offenders by the end of adolescence. Children who follow this path become part of a very
small group of chronic offenders that commit up to 70% of all serious and violent juvenile crimes.1,2
Most of these chronic juvenile offenders go on to become career criminals who cost society an
estimated $1.7 to $2.4 million each over the course of their lifetime.3

   Currently, only a handful of programs exist worldwide for early onset offenders.2 Ramsey County
ACE, which was implemented in August 1999, is one of those programs. ACE offers comprehensive,
long term services to the highest risk young offenders, supports the integration of services across
government units, and promotes collaboration among police, schools, and community non-profit
organizations toward a common goal—preventing serious/violent delinquency, substance abuse and
school failure by promoting healthy development.

    At the time ACE was implemented, very little was known about the characteristics of very young
offenders or about the most effective way to intervene. Consequently, research and program
evaluation have been an integral part of the program. Research has been used to develop screening
and evaluation instruments that are developmentally appropriate for this target population, and to
better understand the role of different risk and protective factors in the development of chronic
serious/violent delinquency and other problem behaviors. Program evaluation has been used to
understand what intervention strategies work best and why. Together, research and program
evaluation provide the foundation for continually improving the effectiveness—and cost-
effectiveness—of the ACE program.

Continuous Quality Improvement

    Ramsey County ACE was initially implemented as a replication of the Hennepin County program
called “Delinquents Under 10”.4 Substantial revisions and enhancements have been made to the
model in the subsequent three years. These include:

Ø   Involving schools as a referral source
Ø   Including school staff and county case workers in screening meetings
Ø   Designing and validating a structured risk assessment instrument
Ø   Comparing intervention strategies
Ø   Providing continuing education seminars for community case managers
Ø   Modifying the role of the Integrated Services Delivery Team
Ø   Establishing protocols for integrating services within the county
Ø   Establishing procedures for tracking costs on a client-by-client basis
Ø   Implementing multiple assessments for tracking outcomes

    This report summarizes how these, and other changes, have been integrated into a coherent
intervention strategy that ACE believes is the most likely to effectively meet the needs of this new,
and unusually high risk, target population of children.


_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                            Page 1
UNDERSTANDING THE TARGET POPULATION

    At the time ACE was implemented in 1999, very little was known about the characteristics of very
young offenders as a target population. Since then, ACE has designed and validated a screening
instrument which indicates that the risk of escalating into serious/violent delinquency falls on a broad
continuum.5,6

     This new information is being used to improve the cost-effectiveness of ACE by refining the match
between services and children’s level of need. This information is also being disseminated at
professional meetings, both nationally and internationally, as part of contributing back to the research
literature on which the ACE program is grounded.7,8


Long-term versus Short-term Intervention Groups

    The single best predictor of who will become a chronic serious/violent delinquent is contact with
police before the age of 10 to 12. Up to 60% of children who commit a chargeable offense before
age 10, for example, will become chronic, serious or violent offenders by age 18.2 These very young
offenders are also at risk for other negative outcomes, including substance abuse, suicide, teen
parenting, school failure, and dependency on government services as adults.9

    Not all very young offenders will follow this path, however. ACE therefore developed a screening
instrument to identify which children are most likely to escalate during adolescence. A validation
study of the instrument confirmed that very young offenders who score 3.0 or more on a scale of 0 to
7 are sufficiently at risk to need a comprehensive, long-term intervention (LTI group).5,6 Children at
lesser risk are referred to short-term interventions in the community (STI group). The difference in
risk profiles for the LTI and STI groups is illustrated below.

        Figure 1. Key Risk Factors for Very Young Offenders Screened by ACE

         100
                                                           91
          90        LTI (78 children)
                    STI (96 children)                                                           81
          80                                                    73       76
          70
                                                                                     63              61
          60                                                                              54
                                               52                             47
          50       44
   %                             37
          40
                                                    32
          30
                        18            18
          20

          10
            0
                Mental Health Prior Police   Delinquent   Criminal    Parent Drug   Domestic     Child
                 Diagnosis       Contact     Sibling(s)   Parent(s)      Use        Violence   Protection

                         Child               Family Criminality           Other Family Risks



_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                                              Page 2
Risk Subgroups

    The likelihood of very young offenders continuing on the path to chronic serious/violent
delinquency depends on the accumulation and severity of risk factors in their lives. In other words,
risk is not “all or none” as the cut-point score for the LTI and STI groups would suggest. Rather,
the risk of escalating falls on a broad continuum.

    The validation study suggested that the match between services and children’s level of need could
be improved by dividing the risk continuum into subgroups.4 Since then, three subgroups of risk
have been identified for the STI group and three for the LTI group. The groups are defined by very
young offenders’ degree of risk of escalating into chronic serious/violent delinquency, on a scale of 0
to 7 (no risk to extreme risk). The breakdown in terms of actual scores for nearly 200 very young
offenders is as follows:

   Short-term Intervention Group
   Ø   Low risk                            Score 0.6 to 1.4              (20% of STI)
   Ø   Moderately low risk                 Score 1.5 to 2.4              (40% of STI)
   Ø   Moderate risk                       Score 2.5 to 2.9              (40% of STI)

   Long-term Intervention Group
   Ø   Moderately high risk                Score 3.0 to 3.4              (20% of LTI)
   Ø   High risk                           Score 3.5 to 4.4              (40% of LTI)
   Ø   Very high risk                      Score 4.5 to 6.2              (40% of LTI)



Long-term Intervention—Three Risk Subgroups

    Very young offenders who are at moderately high, high or very high risk of escalating into chronic
serious/violent delinquency all merit the investment in comprehensive long term services.
Nevertheless, children in the three risk subgroups have somewhat different profiles of risk.

    Some risk factors are equally common among all LTI young offenders. These are part of the
intervention focus for every child. The prevalence of other risk factors, however, increases
dramatically as the overall risk for chronic serious/violent delinquency increases. These risk factors
have implications for the type, intensity and duration of services that young offenders in the three LTI
subgroups are likely to need. Key similarities and differences are shown on the following pages.

    Note that charts in this section show the prevalence of moderate to severe levels of each risk
factor. Minor levels of risk are excluded. This underscores the intervention challenges faced by ACE
in each of the three subgroups.

    Child risk factors. Nearly all LTI children have moderately to severely volatile temperaments
based on information provided by teachers or others at screening (Figure 2). Moderate to severe
rejection by peers, however, increases dramatically as the child’s overall risk increases. Bullying and
other aggressive behaviors cause prosocial peers to avoid LTI children.


_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                            Page 3
         Figure 2. Percent LTI with Moderate to Severe Levels of Child Risk Factors
                                                  98 100                       100
                                100         88
                                                                         74
                                 80

                                 60                                 46
             Mod. High Risk      40
             High Risk
                                 20
             Very High Risk
                                  0
                                             Volatile             Peer Rejection
                                          Tem peram ent




     Family risk factors. A constant for all LTI children is a moderately to severely chaotic
lifestyle (Figure 3). Their lives are characterized by frequent moves, frequent changes of school,
varying male partners and other adults in the home, as well as a lack of bedtime or study routines.
The prevalence of moderate to severe neglect, however, increases dramatically as overall risk
increases. So, too, does the prevalence of domestic violence.


       Figure 3. Percent LTI with Moderate to Severe Levels of Family Risk Factors


              Mod. High Risk      100                 87                      89 85 94
              High Risk            80
                                                 63              58 61
              Very High Risk
                                   60      48
                                   40                       33

                                   20

                                      0
                                           Neglect         Domestic       Chaotic
                                                           Violence       Lifestyle




    Risk Factors for Serious/Violent Delinquency. Despite being only 6 to 9 years old,
30% to 42% of LTI children had prior police contact, documented by a police report, before being
referred to ACE (Figure 4). A constant for LTI children is a high rate of criminality among siblings and
parents. Roughly two thirds (61% to 70%) of LTI children with older siblings have at least one
sibling with a police record. Roughly two thirds (66% to 70%) have at least one parent who is a
career criminal (defined as 5+ arrests as an adult).

    The severity of parents’ criminal careers, however, increases dramatically as the children’s overall
risk for chronic serious/violent offending increases. Among very high risk children, for example,
55% (16 of 29) of the fathers or current male partners have two or more arrests for felony violent
crimes such as aggravated assault, robbery, kidnapping, and rape; 27% (7 of 29) have been arrested
or convicted for murder. By comparison, the rate of arrest or conviction for murder among the
fathers of all 201 very young offenders screened by ACE is only 4%.


_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                            Page 4
       Figure 4. Percent LTI with Key Risk Factors for Serious/Violent Delinquency

               Mod. High Risk
                                   100
               High Risk
               Very High Risk       80                              71 70
                                                                                  70
                                                               61                      65 68
                                    60
                                                        42
                                    40        30
                                                   25
                                    20

                                     0
                                              Child--           Older            Parent(s)--
                                          Prior Police       Sibling(s)--          Career
                                            Report           Delinquent           Criminal




    Risk Factors for Substance Abuse and Mental Illness. Roughly three quarters of LTI
children have at least one parent with a history of drug-related arrests, chemical dependency
treatment and/or treatment for mental illness (Figure 5). Substance abuse is characteristic of
fathers, while both substance use and mental illness is characteristic of the mothers. Most mothers
have symptoms of depression. A disproportionate number display features of Borderline Personality
Disorder (19% of mothers versus 2% of women in the general population). Among other things, this
disorder is related to a history of neglect and abuse during childhood.

                          Figure 5. Percent LTI with Key Risk Factors for
                             Substance Abuse and Mental Illness

                                      100
                                                                            78
                                         80                                      69 77

                                         60                  55
                                                   41 44
                                         40
              Mod. High Risk
              High Risk                  20
              Very High Risk              0
                                                   Child--       Parent(s)--
                                               Mental Health  Substance Use
                                                Diagnosis    &/or Mental Illness



   Although some LTI children are experimenting with cigarettes, alcohol, marijuana or huffing at the
time of screening, most are not yet doing so. This reflects the fact that experimentation with
substances is both infrequent and difficult to detect among 6- to 9-year-old year children.

   The prevalence of a mental health diagnosis, however, is high and increases somewhat with risk.
Diagnoses include ADHD, Oppositional Defiant Disorder, Conduct Disorder, anxiety, depression,
bipolar disorder, psychosis, and post-traumatic stress syndrome. Multiple diagnoses are common.




_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                                 Page 5
   Risk Factors for School Failure. Nearly all LTI children have been suspended or expelled
from elementary school at least once (Figure 6). However, the prevalence of school assessments for
Emotional/Behavioral Disability (EBD) increases dramatically as overall risk increases. School
referrals to the County Attorney’s Truancy Intervention Program (TIP) also increase dramatically.
These statistics underscore the pervasiveness of these young offender’s behavior problems and the
strain they place on the school system.

                Figure 6. Percent LTI with Key Risk Factors for School Failure

               Mod. High Risk
                                  100           93 100
               High Risk                   85
               Very High Risk       80                              71

                                    60                         46                  53

                                    40                    30                  33
                                                                         21
                                    20

                                     0
                                         Suspended       Emotional/      Truancy
                                          or Expelled    Behavioral      Referral
                                                         Disability




Gender and Race

   Gender. There is debate over the extent to which risk factors for serious/violent delinquency
are different for males and females.2 Although only 15% (30/201) of the children referred to ACE
are female, those who have been screened are as likely as males to be at moderately high to very
high risk of escalating into chronic serious/violent delinquency. Overall, the young girls look no
different than the young boys on a broad range of risk factors, including the prevalence of sexual
abuse, mental health diagnoses, and history of aggression.

    The only distinct pattern is found among the 14 young girls in the LTI group. Altogether, 13 have
one parent who is a violent career criminal (typically the father) plus another parent who is also a
career criminal or else seriously mentally ill (typically the mother). The one exception is a girl whose
father is a violent career criminal but whose mother has no known history of problems.

   Race. Group differences in rates of delinquency, substance use, and school dropout are poor
predictors of which individuals are likely to engage in problem behavior.10 Consequently, the Risk
Factor Profile instrument does not include gender, race/ethnicity, poverty, or family structure as risk
factors. Nevertheless, African American children are disproportionately likely to score as moderately
high to very high risk. This reflects the fact that they are more likely than Euro American children in
Ramsey County to have been exposed to an accumulation of risk factors in multiple domains (family,
neighborhood, school, peers). By intervening early, the ACE program may help reduce the
disproportionate representation of poor minority youth in county and state corrections systems.




_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                            Page 6
INVESTIGATING ALTERNATIVES

   At the time ACE was implemented, little was known about how to intervene effectively with very
young offenders.2

    ACE relied on research regarding risk and protective factors for serious and violent delinquency to
determine what to address in the intervention. ACE focuses both on reducing family, school and
neighborhood risk factors in a child’s life, and on building protective factors in the child.

    Protective factors for the child include self-control skills, social skills, structured activities outside
of school, connection with a prosocial adult, and prosocial peers. Protective factors for the family
include improved parenting skills and relationships among the child, siblings, and parent. ACE’s
ongoing research and program evaluation will help determine which risk and protective factors are
the most important to address.

   ACE conducted its own research on two alternative intervention strategies to determine how to
best address these risk and protective factors:

Ø   Case management (August 1999 to present)
Ø   Multisystemic Therapy (April 2000 through April 2002)
How these two intervention strategies fit into the overall ACE model is shown in Figure 7 on the
following page.

    Case management is a standard medium- to long-term intervention strategy. It has been the
core intervention strategy for ACE since the program’s implementation. Funding for the case
management intervention has been provided through Prevention-and-Intervention grants from the
Office of Drug Policy and Violence Prevention, Minnesota Department of Public Safety.

    Multisystemic Therapy is an intensive short-term intervention that is considered a model program
for chronic adolescent offenders who are 13 to 17 years old.11 Funding to extend Multisystemic
Therapy to very young offenders who are only 6 to 9 years old was provided through a two-year
grant from the McKnight Foundation. If the strategy could be successfully adapted, it held the
promise of significantly reducing the cost of intervening with very young offenders, and significantly
increasing the number of cases that could be served each year.

    To provide an unbiased assessment of the relative value of the two strategies for high risk young
offenders, cases were randomly assigned to Multisystemic Therapy as openings became available.
This meant assigning some cases that MST would ordinarily have excluded, such as children who
commit sexual offenses and primary caregivers who are actively using drugs or mentally ill.

    It is important to note that case management and Multisystemic Therapy have been compared
only within the LTI group of young offenders (see Figure 7).




_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                                   Page 7
                                                   Figure 7. Original ACE Model

     OVERSIGHT                                                                                                         ACE Director
                                                                                                            Ø    Manage county staff
                                                                    Leadership Teama                        Ø    Negotiate/manage contracts
        Ramsey County Board               County                                                            Ø    Manage budget
          of Commissioners                Manager              Ø    Direct service integration
                                                               Ø    Establish ACE budget                    Ø    Report progress to Board
                                                               Ø    Supervise ACE Director                  Ø    Maintain collaborations
                                                                                                            Ø    Fund raising
                                                                                                            Ø    Grant writing




     IDENTIFICATION OF                              Schools                    Police
     CHILD DELINQUENTS
                                                                                                                            University Researcher
                                                                                                                       Ø    Program design
                                                        County Attorney’s Office                                       Ø    Program evaluation
                                               Ø Review charge for legal sufficiency                                   Ø    Dissemination
                                                                                                                       Ø    Grant writing



                                                            Screening Teama
                                               Ø     Collect documentation
                                               Ø     Assess child’s level of risk
                                               Ø     Make disposition to intervention



                                                   Integrated Service Delivery Teama
                                               Ø   Quarterly review of cases
                                               Ø   Support community staff
                                               Ø   Facilitate service integration
                                               Ø   Monitor children referred to STI


     ASSIGNMENT TO                                                                                    Short-term Interventions (STI)
     INTERVENTIONS
                                                                                            Project         (intermediate         Diversion
                                                                                            Assist             options)


                                                      Long-term Intervention (LTI)




                                      Case Management                              Multisystemic Therapy
                                Ø NW Youth & Family Services                            Ø Face-to-Face
                                Ø St. Paul Youth Services                        Adapting a best practice for
                                Ø St. Paul YWCA                                 chronic adolescent offenders



     Direct management relationship
     Contractual relationship
     Referral to community program



a
    The Screening Team and Integrated Service Delivery Team include 6-7 representatives from the departments of Public Health, Human
    Services (Mental Health, Child Protection, and Financial Services), Corrections, and the County Attorney’s Office. Program oversight is
    provided by the directors of those departments.
Multisystemic Therapy (MST)

    Multisystemic Therapy is considered a national model for chronic adolescent offenders. Evaluation
of outcomes in a series of federally funded random-assignment studies indicate that Multisystemic
Therapy is effective in reducing rates of re-arrest, out-of-home placement, and substance use among
chronic juvenile delinquents, aged 11 to 17 (average age 15), compared to standard community
services or individual therapy.11

    There was a two-pronged rationale for trying Multisystemic Therapy with 6- to 9-year-old
offenders in the LTI group:

Ø   It would reach the target population before they enter the juvenile corrections system and
    antisocial behavior is entrenched.

Ø   Intervention during childhood is likely to be more effective than during adolescence because very
    young offenders are not yet as enmeshed with antisocial peers, failing in school, and
    unresponsive to parents.

    Similar to the ACE case management strategy, Multisystemic Therapy emphasizes reducing risk
factors in multiple contexts and building on the youth’s strengths. In effect, it spends a four to six
months teaching the parent or primary caregiver, through concrete examples, how to manage their
child’s behavior more effectively in multiple domains. Therein lies its cost-effectiveness. A more
detailed description of Multisystemic Therapy, the results of this study, and a discussion of its benefits
and barriers with this target population is provided elsewhere.11,12

   Although there were theoretical, empirical and developmental reasons for expecting Multisystemic
Therapy to be successful with the ACE target population, there were also some reservations at the
outset.

Ø   Parent engagement—MST headquarters cautioned that there might not be a “catalyst to change”
    when working with very young offenders. The catalyst for juvenile delinquents is the threat of
    impending incarceration.

Ø   Lack of experience with this target population elsewhere—MST had not yet been implemented
    with very young offenders or children with related behavior disorders, except for a program in
    Ottowa, Canada which was still quite new when ACE applied for its grant from the McKnight
    Foundation.

   Discussions with MST headquarters, and a site visit with the Canadian team, left open the
possibility that the technique could be effectively adapted for very young offenders, given appropriate
developmental modifications. The purpose of the McKnight Foundation grant was to determine what
modifications were necessary and whether there were any unforeseen barriers to success.

    Method. Multisystemic Therapy is implemented by a team of four to five therapists under close
clinical oversight by a local supervisor, and with regular feedback from a supervisor from MST
headquarters to ensure fidelity of implementation. Briefly, ACE procedures were as follows:




_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                              Page 9
Ø   Randomized assignment—When there was an opening with the ACE therapists on the
    Multisystemic team, the next child identified as qualifying for the long-term case management
    intervention was assigned to MST instead. No exclusionary criteria were set in order to avoid pre-
    judging which cases would, or would not, be successful.

Ø   Flexibility on the length of intervention—Although MST specifies four to six months of intervention
    based on experience with the adolescent delinquent population, the possibility of longer
    involvement was left open in case more time would be needed to successfully close a very young
    offender case.

     The expectation was that 30 moderately high to very high risk children could be served over the
course of two years if each case required the standard four to six months of intervention. In practice,
only 15 cases were served. Although the intervention period was held to between four to six months,
it often took weeks or months to engage a parent or other primary caregiver. In the absence of a
“catalyst to change”, parents were typically reluctant to engage in a very demanding intervention that
focused intensively on them, rather than their child.

    Evaluation. Due the unexpectedly small sample size, it was not meaningful to compare
outcomes for very young offenders assigned to MST and very young offenders who received standard
case management services instead. The evaluation focused instead on process evaluation,
qualitative evaluations of barriers based on discussions with the MST therapists and supervisors, and
trends in parent and child behavior.

Positive Outcomes

Ø   Initial resistance did not predict likelihood of engagement

    MST was designed with the issue of parent engagement in mind. Once a connection was
    established, and the need for intervention established, the technique was very effective in
    engaging parents in this time-intensive intervention. This was true even in cases where the ACE
    Screening team anticipated a poor fit.

Ø   Dramatic short-term improvements

    The strategies which MST teaches parents for better managing their child’s behavior are very
    effective in the short term.

Negative Outcomes

Ø   Failure to maintain treatment gains

    While some initially resistant mothers benefited from MST, it was not uncommon for them to
    return to patterns of behavior that created continuing risks for delinquency by their child (for
    example, failure to supervise the child’s activities, or accepting an abusive partner back into the
    home).




_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                              Page 10
Ø   Failure to reduce delinquent behavior in the short term

    A few young offenders whose parents successfully completed MST have since been charged in
    court with a delinquent offense. More importantly, a few children were still committing the
    equivalent of chargeable offenses at the time their case was closed (e.g., suspension from school
    for assaulting a classmate). Because of time constraints, MST focuses on identifying and
    removing immediate triggers to problem behavior, not on addressing chronic problems that play
    an important but indirect role, such as a volatile temperament or pervasive neglect.

Ø   Lack of cost-effectiveness

    At the end of two years (April 2000—May 2002), only 12 of the 15 assigned cases had
    successfully completed MST, at a cost of over $15,000 each. The same funding would provide
    approximately two years of ongoing case management.

Barriers to Effectiveness

    In brief, the reasons for MST’s lack of effectiveness with high risk young offenders under age 10
are:

Ø   Administrative—there is no established relationship, such as with a counselor or probation officer,
    to serve as a connection to this demanding intervention.

Ø   Motivational—there is no catalyst to change for the parents and no meaningful consequence for
    failure to participate.

Ø   Developmental—unlike the adolescents in traditional MST studies, very young offenders have six
    to eight years before they reach the peak age of offending and have yet to face the challenging
    developmental tasks of adolescence.

Ø   Structural—the technique is poorly suited to working with mentally ill, antisocial, substance
    abusing and/or chronically neglecting parents, which is typical of the moderately high to very
    high risk young offenders served by ACE.

    Discussion. When the McKnight Foundation awarded a grant to ACE in late 1999 to study
Multisystemic Therapy, it was at the cutting edge of research with this new target population. At the
time, no one had experience extending any of the model interventions for adolescent offenders to this
new target population of much younger offenders. Similar experiences with MST, however, have
since been reported elsewhere:

Ø   The program in Ottowa, Canada that began using MST in 1999 with young offenders under the
    age of 12 has since been discontinued.

Ø   A school-based intervention in Oregon, which recently sought to integrate MST into its program
    for children under age 9 who have serious behavior problems, reports little success in connecting
    the children’s parents with MST.




_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                           Page 11
    A re-examination of the research literature on MST also helps put the findings in perspective.
Success rates with adolescent offenders depend in part on the type of offender (violent or not) and
the fidelity with which the intervention was implemented by community providers.

    MST’s success with violent and chronic juvenile delinquents—the ACE target population—has been
modest in absolute terms. In one federally funded study, for example, 60% of the violent and
chronic delinquents who completed MST were re-arrested during the next 2½ years, compared to
over 80% of the violent and chronic offenders who received usual services (e.g., court ordered
curfew, school attendance, referral to other community agencies). While this was a significant
reduction compared to standard interventions, the rates of re-offending were not low.

    In another federally funded multi-site study, there was no significant reduction in re-arrests for
violent and chronic juvenile delinquents after 1½ years. It appears that the high risk young offenders
served by ACE may come from the portion of MST’s challenging target population that does not
readily benefit from this intensive short-term intervention.

    When the McKnight grant ended in April 2002, ACE faced a decision about whether to seek
continuing funding for this intervention or incorporate the lessons learned into its “best bet”
intervention strategy. For reasons summarized above, ACE chose the latter. The experience gained
through this study has helped ACE learn a great deal more about what works, what doesn’t work, and
why.

Ø   While Multisystemic Therapy may not be appropriate as a stand alone intervention for high risk
    young offenders, it shows promise as a stand alone intervention for the low to moderate risk
    young offenders that ACE refers to short-term interventions in the community.

Ø   MST strategies for engaging parents, setting concrete goals, and providing close clinical
    supervision improve the effectiveness of long term case management.

    These and other lessons learned with the help of the McKnight Foundation grant will be
disseminated at national and international professional meetings for the benefit of others.13


Case Management

    Goals. Promote healthy development by building protective factors in the child, and reducing
risk factors in the family, peer, school, and neighborhood contexts. The focus is primarily on the
child, although the meeting parents’ needs also plays an important role.

    Caseloads. A total of 4½ full-time-equivalent case managers in three community agencies
work with 60 children and their families on an ongoing basis. Originally, caseloads were set at 25
children per full-time-equivalent case manager. Because these multi-problem children and their
families have complex needs, however, caseloads were soon reduced to an average of 13 children
each. If children move out of the county and do not return within four months, another case is
assigned in their place.

    Clinical Supervision. Clinical oversight was originally provided by supervisors at the three
community agencies where the case managers are employed. In order to introduce more consistency

_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                            Page 12
in services and more accountability for achieving concrete results, clinical supervision was transferred
to the psychologist and social worker on ACE’s Integrated Services Delivery Team in October 2001.

    The psychologist and social worker meet with each case manager weekly to review progress and
identify next steps. These regular meetings ensure that each case is reviewed monthly (more often if
necessary). A standardized clinical assessment (the CAFAS) is conducted quarterly to guide revisions
in the delivery of services .

    Integrated Services Delivery Team (ISDT).               Originally, the ISDT provided
accountability through brief quarterly reviews of cases using summaries provided by the case
managers. Now that clinical supervision is being provided along the lines of the Multisystemic model,
the ISDT can devote its efforts entirely to encouraging the integration of services at a policy and
administrative level as well as on a case-by-case basis. The ISDT is also responsible for the
program’s continuous quality improvement and participates in program dissemination.

   Discussion. The recent outcome evaluation study, although preliminary, indicated that a case
management strategy can be effective in reducing risks, building protective factors, and reducing the
prevalence of delinquent and problem behavior.14 Several aspects of those findings are noteworthy:

Ø   There was improvement among very young offenders in all three risk subgroups.

    Although the degree of improvement varied with the initial level of risk, this was to be expected.
    Children who present a greater intervention challenge (see pages 3—6) will require more services
    over a longer period of time to achieve success. Similar findings have been reported for other
    programs.15

Ø   Improvement was seen despite widely varying lengths of service (6 to 24 months).

    Since the Delinquents-Under-10 program concluded that 18 months is the minimum intervention
    dose that is needed to see positive outcomes, on average,16 ACE can expect outcomes to get
    better as a greater percentage of the children receive services for that length of time.

Ø   Improvement happened under the original case management model.

    The results of the preliminary outcome evaluation study assess the effectiveness of the original
    case management model that was in place from August 1999 through October 2001. Effects of
    the new “best bet” case management model, which was implemented in November 2001, will
    show up in the next evaluation study. Given the progress made with the original model, it is
    reasonable to expect future results to be even better.




_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                             Page 13
INTERVENTION STRATEGY


History
    The ACE case management intervention, currently at full capacity, serves 60 very young offenders
who are at moderately high to very high risk for chronic serious and violent juvenile delinquency and
other problem behaviors (LTI group). Each year, it also serves approximately 80 low to moderate
risk young offenders through screening and referral to short term interventions in the community (STI
group). ACE county staff make sure the STI children connect with recommended services and it
monitors their completion.

Until October 2001, the intervention was structured as follows:

Ø   Weekly screenings of child delinquents under age 10 to determine their risk of becoming SVJ
    delinquents.

Ø   Two intervention groups based on risk—A long-term intervention group for the highest risk child
    delinquents (LTI) and a short-term intervention group for child delinquents at lesser risk (STI).

Ø   Referral of STI children to a existing interventions in the community (diversion, mental health
    services).

Ø   Ongoing case management for LTI children delivered through community agencies.

Ø   Focus on the target child.

Services were focused as follows:

Ø   Ultimate outcome—Prevent high-risk children from escalating into chronic, serious and violent
    delinquency during adolescence.

Ø   Intermediate outcomes focused on reducing risk factors and building protective factors—For the
    child, bonding with a caring adult, school bonding and success, development of social skills, and
    involvement in structured activities outside of school. For the parent, helping them better
    manage the child’s behavior.

Ø   Oversight and guidance was provided through quarterly reports by case managers to the ACE
    Integrated Service Delivery Team, and semi-monthly meetings of case managers with the team
    psychologist and social worker for clinical guidance and general support.

Ø   Incorporation of a two-year experimental study, funded by the McKnight Foundation, to determine
    whether a six-month intensive intervention designed for the parents and older SVJ delinquents
    (Multisystemic Therapy) would be effective for parents and child delinquents under age 10.

Ø   Continuing education workshops for case mangers and the Screening/Integrated Service Delivery
    teams.




_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                            Page 14
“Best Bet” Strategy
    In the fall of 2001, the ACE county and community staff did a comprehensive review of the
program. The result is a “best bet” intervention for moderately high to very high risk young offenders
under age 10 and their families. The revised plan was implemented immediately after the completion
of a preliminary outcome evaluation. It is shown in Figure 8 on the next page. Beginning November
2001, the intervention was structured as follows:

Ø   Expanded ultimate outcomes—ACE now addresses four outcomes: (1) preventing chronic serious
    and violent delinquency, (2) preventing substance abuse, (3) preventing school failure, and (4)
    promoting competence. The latter was added because the absence of serious problem behavior
    is not the same as healthy development.

Ø   Expanded intermediate outcomes for the child and parent—Added healthy peer associations for
    the child. Added overcoming resistance to child assessments, medications, and services for
    parents.

Ø   Expanded services to include siblings, where possible. This improves the family context for the
    target child. It also improves the cost-effectiveness of ACE by adding benefits for the siblings at
    relatively little additional cost.

Ø   Sub-divided the two major risk groups to improve the match to services—STI young offenders are
    divided into low, moderately low, and moderate risk subgroups. LTI young offenders are divided
    into moderately high, high, and very high-risk groups.

    STI subgroups. Only low risk young offenders are referred to Diversion (a group intervention for
    first time offenders). Moderate risk young offenders are referred to Project Assist (a mental
    health intervention that includes counseling, assessments, and referrals to address specific
    needs). Moderately low risk young offenders are referred to interventions that fall somewhere in
    between Diversion and Project Assist.

    LTI subgroups. Equal amounts of time are now devoted to very young offenders in each
    subgroup in order to balance the demand on case manager resources. Moreover, it recognizes
    that interventions for multi-problem families need to be paced according to their ability make
    change. The higher the child’s initial level of risk, the longer the case management intervention is
    likely to last , especially in cases of chronic neglect and abuse. However, children whose
    developmental progress is steady, and whose family and school context appear likely to remain
    stable, may have their case closed to conserve resources.

Ø   ACE is facilitating the introduction of proven practices in the community and schools. This is part
    of a long range plan to provide research-proven options for STI children, LTI children (as part of a
    package of comprehensive services), and children at risk who have not yet been referred to ACE.

    “SNAP” (Stop Now and Plan)15—This is a 3- to 6-month behavioral self-control intervention
    designed specifically for delinquent and disruptive children under age 12. It improves children’s
    peer relationships, sense of mastery, and functioning in the classroom by teaching them to control
    their aggressive behavior and identify prosocial solutions instead. SNAP reinforces school-based
    group activities with parent instruction that supports the child’s mastery of self-control skills.


_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                             Page 15
                                                         Figure 8. “Best Bet” ACE Model


  LEADERSHIP                                                                                                                        ACE Director
                                                                                                                       Ø        Manage county staff
                                                                               Leadership Teama                        Ø        Negotiate/manage contracts
           Ramsey County                                                Ø     Direct service integration               Ø        Manage budget
                                                 County                 Ø     Establish ACE budget                     Ø        Report progress to Board
              Board of
           Commissioners
                                                 Manager
                                                                        Ø     Supervise ACE Director                   Ø        Maintain collaborations




  IDENTIFICATION OF                            Schools                             Police
  HIGH RISK CHILDREN                                                                                                                    University Researcher
                                                                                                                                    Ø     Program design
                                                      County Attorney’s Office                                                      Ø     Program evaluation
                                         Ø Review delinquent act for legal sufficiency                                              Ø     Dissemination
                                             (Child Protection statutes apply under age 10)




                                                          Screening Teama
                                         Ø    Collect documentation, arrange interviews
                                         Ø    Hold confidential screening meeting
                                         Ø    Conduct structured assessment of risk
                                         Ø    Make disposition to intervention



                                             Integrated Services Delivery Team (ISDT)a
                                         Ø   Facilitate service integration for LTI children
                                         Ø   Weekly case consultation for LTI children
                                         Ø   Interagency advocacy, communication
                                         Ø   Monitor children referred to STI


                                                                                                                      Short Term Interventions (STI)
  INTERVENTION                                                                                                 Very low, Low, or Moderate Risk of Escalating

                                                                                                                                     Other
                                Long Term Intervention (LTI)                              Diversion, Fire Setters          or                       or      Project Assist
                                                                                                                                   Community
                       High, Very high, or Extreme Risk of Escalating                         (group interventions)                 options                 (individualized)




             County Directed                                          Child- & Family-focused Services
    Service Integration, Collaboration,                                through Community Agencies
              Accountability
                                                                 Ø Stabilize children and families in crisis
  Ø Child Protection Services
                                                                 Ø Ensure coordination of school, county, and
  Ø Juvenile Corrections                              plus          community services with assistance of ISDT
  Ø Financial Services                                           Ø Construct a package of programs and
                                                                    services that support healthy child                              Direct management relationship
  Ø Mental Health Services                                          development and family functioning                               Contractual relationship
                                                                                                                                     Referral to community program
  Ø Contracts with community agencies for                        Ø Facilitate the achievement and maintenance
    services that build protective factors                          of progress over the course of development




a
  The Screening and Integrated Services Delivery Teams include 7 representatives from the departments of Public Health,
Human Services (Mental Health, Child Protection, Financial Services), Corrections, and the County Attorney’s Office. Program
oversight is provided by the directors of those four departments (Leadership Team).

  _____________________________________________________________________________________________
  ACE “Best Bet” Intervention Strategy—July 2002                                                                                                   Page 16
Ø   Improved effectiveness of case management by replacing the quarterly reports and semi-monthly
    worker meetings with weekly one-on-one consultations by the ACE psychologist and social worker
    that are held at the case manager’s community agency site.

    This more highly structured approach to clinical supervision was adapted from the MST
    intervention study. The progress of each case is reviewed monthly, and five concrete goals are
    identified for the coming month. A standardized clinical assessment, the Child and Adolescent
    Functional Assessment Scale (CAFAS) is completed quarterly for each case and provides a
    foundation for individualizing the case plan, prioritizing goals, and collecting ongoing evaluation
    data for the program.




MOST CHALLENGING CASES ARE BEST INVESTMENT

    “Suppose for $1 million, you could either fund a program expected to save 4 youths from
becoming career criminals or incarcerate 40 more youths. Which alternative should be chosen?”
(Cohen, 1999). The answer depends on benefits as well as costs.3,12,17

    Chronic serious/violent offenders cost the community $1.7 to $2.7 million each over the course of
their lifetime,3 or more if you consider the costs of early onset offending.18

    Ø $1.3 to $1.5 million in costs related to property, drug, and violent crimes
    Ø $0.4 to $1.0 million in costs related to substance abuse

    Ø $0.2 to $0.4 million in lost productivity due to school dropout

    This turns the old argument of “work with those who are most likely to benefit” on its head. The
issue is not which children are most likely to benefit, but which children are most likely to benefit the
community if success is achieved.3,12,17

   Cost/benefit research confirms that success with children who present the greatest challenge—the
very young offenders who are at highest risk—will reap the greatest reward.3 The ACE program will
pay for itself if only a few of the very high risk children are diverted from this path. The remaining
successes will be a net return on the investment.

    The more moderately high to very high risk children served, the greater the benefit to the
community.3 Aggregate benefits include safer neighborhoods, a better business climate, and safer
schools with a better environment for learning. Breaking the cycle of crime, substance abuse,
poverty, and early childbearing also pays compound dividends into the next generation. Failure to
intervene compounds the problem.




_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                              Page 17
CONCLUSIONS

   There are no short term fixes for intergenerational patterns of crime, substance abuse and
poverty. Successful intervention requires the integration of services across multiple domains, and a
long term plan for reducing risks and building resiliency in the child.

    ACE’s overarching goal is to provide a package of services that meet the needs of individual
children and their families, support the integration of services across government units, and promote
collaboration among police, schools, and community non-profit organizations toward a common
goal—preventing serious/violent delinquency, substance abuse and school failure among high-risk
children by promoting healthy development.

    Integration of services is being emphasized by policy makers at every level of government and
public service, particularly for programs involving very young offenders.2,19 The Substance Abuse and
Mental Health Services Administration (SAMHSA) funds violence prevention initiatives in communities
and schools. The National Center on Education, Disability, and Juvenile Justice (EDJJ) is jointly
funded by Office of Special Education of the U.S. Department of Education and the Office of Juvenile
Justice and Delinquency Prevention of the U.S. Department of Justice. The Director of ACE was an
invited to participate in a national summit which was convened to discuss integrating the juvenile
justice and child welfare systems.20

   ACE is finding the appropriate balance between investment in multi-jurisdictional services and
benefits to the community. Next steps in continuous quality improvement include:

Ø   Developmental adaptations to the intervention as children age

Ø   Strategies for closing stable cases to make room for new ones

Ø   Closer collaborations with Child Protection Services, Corrections, and the schools




_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                           Page 18
REFERENCES


1.    Howell, J.C., Editor (1995). Guide for Implementing the Comprehensive Strategy for Serious,
      Violent and Chronic Juvenile Offenders. Washington, DC: Department of Justice, Office of
      Juvenile Justice and Delinquency Prevention.

2.    Loeber, R. and Farrington, D.P., Eds. (2001). Child delinquents: Development, Intervention,
      and Service Needs. Thousand Oaks, CA: Sage.

3.    Cohen, M.A. (1998). The monetary value of saving a high-risk youth. Journal of Quantitative
      Criminology, 14(1), 5-33.

4.    Beuhring, T. and Melton, H. (November 2000). ACE First Year Report. St. Paul, MN: Konopka
      Institute for Best Practices in Adolescent Health, University of Minnesota and Ramsey County
      Department of Public Health.

5.    Beuhring, T. (2002). The Risk Factor Profile instrument: Identifying children at risk for serious
      and violent delinquency. In R.R. Corrado, R. Roesch, and S.D. Hart (Eds.), Multi-Problem
      Violent Youth: A Foundation for Comparative Research On Needs, Interventions and Outcomes.
      NATO Science Series. Amsterdam: IOS Press.

6.    Beuhring, T. and Melton, H. (February 2002). ACE Preliminary Outcome Evaluation Study. St.
      Paul, MN: Institute on Community Integration, University of Minnesota and Ramsey County
      Department of Public Health.

7.    Beuhring, T. (under review). Risk-Factor-Profile Instrument: Identifying Children on the Path to
      Violent Delinquency. Annual meeting of the American Society of Criminology, November 13-16,
      2002. Chicago, Illinois.

8.    Beuhring, T. (accepted). Predicting Risk of Violent Offending Among Children Under 10. Annual
      Conference of the European Assn of Psychology and Law, Sept. 14-17, 2002. Leuven, Belgium.

9.    Huizinga, D., Loeber, R., Thornberry, T.P., Cothern, L. (2000 November). Co-occurrence of
      delinquency and other problem behaviors. [Juvenile Justice Bulletin] Washington, DC:
      Department of Justice, Office of Juvenile Justice and Delinquency Prevention.

10.   Blum, R.W., Beuhring, T., Rinehart, P.M. (2000). Protecting Teens: Beyond Race, Income and
      Family Structure. Center for Adolescent Health. Minneapolis, MN: University of Minnesota.

11.   Henggeler, S.W. (1998). Multisystemic Therapy. In D.S. Elliott (Ed.), Blueprints for Violence
      Prevention. Center for the Study and Prevention of Violence, Institute of Behavioral Science.
      Boulder, CO: University of Colorado.

12.   Beuhring, T., and Melton, H. (in preparation). Family Intervention for Children at Risk of
      Becoming Serious, Violent, and Chronic Offenders: Final Report to the McKnight Foundation.
      Institute on Community Integration, University of Minnesota and the Ramsey County Dept. of
      Public Health. St. Paul, MN.

_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                           Page 19
13.   Beuhring, T., Melton, H., and Frickson, E. (under review). MST—A Fit for Child Delinquents?
      Annual meeting of the American Society of Criminology, November 13-16, 2002. Chicago,
      Illinois.

14.   Beuhring, T. and Melton, H. (February 2002). ACE Preliminary Outcome Evaluation Study: A
      Report to the Ramsey County Board of Commissioners. Institute on Community Integration,
      University of Minnesota and the Ramsey County Department of Public Health. St. Paul, MN.

15. Augimeri, L. (October 2001). Responding to Offending by Very Young Children: Risk
    Identification and Risk Management. [Presentation] Under 12 Outreach Project (ORP).
    Toronto, Canada: Earlscourt Child and Family Centre.

16.   Stevens, A.B., Owen, G., Lahti-Johnson, K. (July 1999). Delinquents under 10: Targeted Early
      Intervention. Phase I Evaluation Report. Minneapolis, MN: Wilder Research Center and
      Hennepin County Attorney’s Office.

17.   Aos, S., Barnoski, R., Lieb, R. (January 1998). Watching the bottom line: Cost-effective
      interventions for reducing crime in Washington. Olympia, Washington: Washington State
      Institute for Public Policy. [http://www.wa.gov/wsipp]

18.   Beuhring, T. (May 2001). Monetary Value of Saving a Child on the Path to Criminality: Report
      to the Ramsey County Board of Commissioners. Konopka Institute for Best Practices in
      Adolescent Health, University of Minnesota, Minneapolis, MN.

19.   Farrington, D.P., Loeber, R., Kalb, L.M. (2001). Key research and policy issues. In R. Loeber
      and D.P. Farrington (Eds.), Child delinquents: Development, Intervention, and Service Needs.
      Thousand Oaks, CA: Sage.

20.   Child Welfare League of America (2002). Juvenile Justice and Child Welfare Summit.
      May 8-10, 2002. New Orleans, LA.




_____________________________________________________________________________________________
ACE “Best Bet” Intervention Strategy—July 2002                                          Page 20

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:2/16/2012
language:
pages:24