Docstoc

Evaluation of Substance Abuse Treatment Outcomes

Document Sample
Evaluation of Substance Abuse Treatment Outcomes Powered By Docstoc
					    Evaluation of
  Substance Abuse
Treatment Outcomes

                Office of Standards and Evaluation
                            March 1998




Lawton Chiles                                        Edward A. Feaver
Governor                                                    Secretary
    The Department of Children and Families and
       the Office of Standards and Evaluation
 would like to acknowledge the clients and providers
who participated in this evaluation and made it possible.




        For reprints of this report, please contact:
        FL Department of Children and Families
       Office of Standards and Evaluation (OSEV)
                  1317 Winewood Blvd.
                    Bldg. 1, Room 306
               Tallahassee, FL 32399-0700
            For additional information contact:
             Nina B. Barrios, Ph.D., Director
        Glenn Mitchell, Ph.D., Sr. Mgt. Analyst
                      (850) 922-4691

               File reference: OSEV98_2
                     Executive Summary of Department of Children and Families
                        Evaluation of Substance Abuse Treatment Outcomes

The Office of Standards and Evaluation of the Department of Children and Families evaluated substance
abuse treatment outcomes to address questions of program effectiveness and assess the validity and
reliability of measures used for performance budgeting. The office assessed client outcomes six months
after treatment using both a telephone survey of clients and state wage data. The evaluation compared
outcomes of two groups of clients, those who completed treatment free from drug and alcohol use and
clients who entered treatment with the same problems, but did not complete it. The study also analyzed
information on all clients and services in 1996-97 to identify factors related to outcomes and assess data
                             s
quality, using the program’ State Integrated Substance Abuse Reports (SISAR) database.

Findings of the Evaluation

• Florida’ alcohol and drug treatment programs produce positive outcomes for clients after they leave
          s
  treatment. Six months after discharge, clients who complete treatment have fewer substance abuse
  problems and better employment than clients who leave before completing treatment. Comparison of
  outcomes for completers versus non-completers validates the importance of the current performance
  measure of drug free completion of treatment.
• $114.7 million was appropriated for FY 1997-98, with which the department contracted with 135
  agencies. Services such as residential and outpatient treatment, detoxification, intervention assessment
  and referral, and prevention were provided to 136,300 adults and 62,000 children in FY 1996-97.
  Primary referral sources for clients are criminal justice and juvenile justice.
• Clients have different patterns of substance abuse and outcomes. Alcohol is the most common
  substance abuse problem for adults and marijuana is most common for adolescents. Clients abusing
  crack/cocaine have the lowest treatment success. Most substance abusers are male. Compared to
  other substances, a higher proportion of clients abusing crack/cocaine are female.
• The data system used for performance reporting provides useful information on client characteristics
  and services. Due to problems with reporting discharges, the system lacks outcome information on
  many clients. To correct data problems, the program is working with providers and districts to
  implement data validation and simplify reporting. Performance measures have not been developed for
  services such as detoxification and intervention. Subgroups used to report populations served have
  substantial overlap, so information about characteristics of clients is lost.

Summary of Recommendations:
The substance abuse program should:
• Implement systematic follow-up of clients after treatment to determine instances of drug-free behavior
  and employment.
• Maintain use of completion of treatment as a performance measure. Use quality improvement
  processes to increase client completion rate.
• Develop performance measures for management and accountability of all primary services and evaluate
  the current subgroup reporting structure to better communicate client characteristics and service needs.
• Continue to improve data quality and establish measurable objectives for improvements in timeliness,
  completeness and accuracy
.


    Substance Abuse Evaluation                                                                 Page   i
The Office of Standards and Evaluation of the Department of Children                                      PURPOSE OF THIS
and Families evaluated substance abuse treatment outcomes to address
                                                                                                          EVALUATION
questions of program effectiveness and assess the validity and reliability
of measures used for performance based budgeting. The Legislature and
departmental senior management questioned whether measures of
completion of treatment reflected program effectiveness. They also
questioned the reliability of data, given the low number of clients with
reported outcomes.

                  s
The department’ Office of Standards and Evaluation prepared an
evaluation study to assess client outcomes six months after treatment.
The study compared the outcomes of two groups of clients using both a
telephone survey of clients and independent employment data. One
group was clients who completed treatment free from drug and alcohol
use. The comparison group was clients who entered treatment with the
same problems, but did not complete it. The study also analyzed
information on all clients served by substance abuse programs in 1996-
97, to provide descriptive information and identify factors related to
outcomes. The primary source of these data is the program State
Integrated Substance Abuse Reports (SISAR) database of client
admission and discharge reports from each component of service.
Results from analysis of this information assess the quality of the data
and the utility for performance management.



Statutory authority for the substance abuse services program is Chapters                                  BACKGROUND:
20, 394, and 397, Florida Statutes. The legislative intent of the program
is to provide comprehensive prevention, intervention and treatment
                                                                                                          THE SUBSTANCE
services to meet the needs of individuals and reduce consequences to                                      ABUSE PROGRAM
Florida of substance abuse. The program is to provide services                                            AND ITS RESOURCES
primarily through community-based, private not-for-profit providers
(Chapter 397, F.S.). The program serves two groups: adults and
                                                                                                          The Program Provides
                                           s
children under age 18. The department’ goals for the program are for
adults with substance abuse problems to be drug free and economically
                                                                                                          Comprehensive
self-sufficient, and for children with, or at risk of, substance abuse                                    Services Through
problems to be drug free.                                                                                 Private Not-For-Profit
                                                                                                          Providers— Goals are
                                                                                                          for Clients to be Drug
The program reports over 136,300 adults and 62,000 children under age                                     Free and Economically
18 were served in Fiscal Year 1996-971. The major types of services                                       Self-sufficient
provided by the program are:




1 In this evaluation, analysis of client characteristics and services are based on admissions of 71,000
adults and 41,000 children during FY 96-97. The number served includes clients who are not yet
discharged. Because of the low reporting of discharges, the reported number served may include
clients never discharged and not receiving services. Annual admission data was determined to better
reflect client characteristics and services.


     Substance Abuse Evaluation                                                                                       Page   1
                     • Non-residential treatment: Includes assessment, counseling,
                       family therapy and substance abuse education in outpatient and day
                       treatment settings.
                     • Residential treatment: Assessment, counseling, rehabilitation,
                       family therapy and education services in an intensive, 24 hour,
                       therapeutic environment.
                     • Residential detoxification and stabilization services:Provide for
                       stabilization and withdrawal from substance abuse dependence.
                     • Intervention services: Assessment, short-term counseling, referral
                       to treatment when needed, and case management.
                     • Prevention services:
                            Targeted prevention: For children at risk of substance abuse,
                            including children of drug abusers and poor school achievers.
                            This includes classroom-based activities in schools to increase
                            achievement and reduce substance abuse risk in grades 4
                            through 8.
                            Primary prevention: Information and education for the general
                            population to increase public awareness and reduce the
                            incidence of substance abuse in the community.
                     • Licensing: In addition to funding direct services, the department
                       licenses all providers of substance abuse treatment services, both
                       public and private.

                     The Department of Children and Families contracts with 135 community
                     substance abuse agencies to provide services. The department’ 15  s
                     service districts manage contracts with providers. Statewide, there are
                     78 district and central office positions to manage the system of services
                     in FY 1997-98. Funds are not appropriated for staff to provide direct
                     services. In addition to managing contracted services, administrative
                     staff are responsible for routine monitoring and licensure activities.

$114.7 Million was   The 1997-98 Legislature appropriated $114.7 million to the Department
Appropriated for     of Children and Families for substance abuse services, up $3.2 million
FY 97-98             from 1996-97. Sources and dispositions of funds include:

                                Program Appropriation for Services in Millions

                                                        1997-98
                                                        Adults Children                Total
                              General Revenue            $26.5   $27.3                 $53.8
                              Trust Funds*               $40.5   $20.4                 $60.9
                              Total                      $67.0   $47.7                $114.7

                                                        1996-97
                                                        Adults Children                Total
                              General Revenue            $28.3   $22.0                 $50.3
                              Trust Funds* **            $39.8   $21.4                 $61.2
                              Total                      $68.1   $43.4                $111.5
                                     * Includes Federal funding
                                     ** Excludes $3.5 unfunded budget




    Page   2                                                      Substance Abuse Evaluation
Funding for substance abuse services gradually increased during the
1980s and then decreased through the 1990s. There were large
increases in 1988-89 and 1989-90 with increased Federal Block Grant
funds and the state trust fund for children and adolescents established by
the Wheels Bill. Budget reductions in the 1990s have resulted in cuts in
four of the last seven years, so that current funding is lower than 1990-
91.

                                                       s
Federal funds provide almost 50% of the Legislature’ appropriation.
According to the US Government Accounting Office, combined Federal,
state and local public resources represent the majority of funding
available for substance abuse treatment and prevention. Nationwide
estimates based on limited private sector data indicate private insurance
and client fees may have accounted for as low as 15% of resources for
treatment and prevention in 1993 (GAO, 1996)i.


                                       s
To implement requirements of Florida’ Government Performance and             Measurable Outcomes
Accountability Act of 1994, the department developed performance             are Required for
measures of outcomes for clients of the substance abuse treatment            Performance Based
programs. The Legislature approved the measures in the 1997-98               Budgeting
General Appropriations Act. Both the Legislature and the department
continue to refine the measures for budgeting and contract management.

Performance Measures Approved by the Legislature for 1997-
98:
      For Adults With Substance Abuse Problems:
         • Percent of discharges completing treatment with no alcohol
           or other drug use during the month prior to discharge.
         • Percent reduction in frequency of substance use for
           unsuccessful discharges during the month prior to discharge
           compared to the month prior to admission.
         • Percent of clients successfully completing treatment who are
           readmitted for substance abuse services during the 12
           months following discharge.
         • Percent of clients employed at discharge from substance
           abuse treatment services.
         • Percent of women pregnant during substance abuse
           treatment who give birth to substance free newborns.
         • Percent of pregnant women receiving substance abuse
           treatment who deliver infants with normal birth weights.
         • Change in percent of clients with arrests within 90 days
           following discharge compared to arrests 90 days prior to
           admission.
         • Average level of satisfaction on the Behavioral Healthcare
           Rating of Satisfaction scale.
         • Percent of community partners satisfied based on survey.




    Substance Abuse Evaluation                                                          Page   3
                               For Children Abusing Substances:
                               • Percent of discharges completing treatment with no alcohol
                                 or drug use during the month prior to discharge.
                               • Percent reduction in frequency of substance use by those
                                 who do not successfully complete treatment.
                               • Percent of clients successfully completing treatment who are
                                 readmitted for substance abuse treatment services during the
                                 12 months following discharge.
                               • Percent of children committed or recommitted to Juvenile
                                 Justice during the 12 months following treatment
                                 completion.
                               • Percent of families reporting average or above level of
                                 satisfaction on the Family Centered Behavior Scale for
                                 parents of children receiving treatment.
                               • Percent of clients satisfied.
                               • Percent of community partners satisfied based upon survey.

                               For Children At Risk of Substance Abuse:
                               • Percentage of children in targeted prevention programs who
                                 achieve expected level of improvement in math.
                               • Percentage of children in targeted prevention programs who
                                 achieve expected level of improvement in reading.
                               • Percentage of children in targeted prevention programs who
                                 are admitted to substance abuse services during the 12
                                 months following completion of prevention services.
                               • Percentage of children in targeted prevention programs who
                                 perceive substance use to be harmful at the time of
                                 discharge when compared to admission.
                               • Percentage of children in the general population who
                                 perceive substance use as harmful.
                               • Percent reduction in monthly or more use of alcohol and
                                 other drugs by middle and high school students as measured
                                 on a statewide survey of students.


Proposed Changes In    The department has recommended a new measure for both adults and
Measures Include Six   children for FY 1999-2000 based on experience with the first year of
Month Follow-up        budgeting and management with these measures. For increased validity,
                       the “percentage drug free for six months following completion of
                       treatment” will replace the current measures of completion of treatment,
                       which will be reported as output measures. Because of questions of
                       validity and reliability, the department will defer until 1999-2000 the
                                                    s
                       two measures of children’ perceptions and use of substances based on a
                       survey of the general population.




    Page   4                                                 Substance Abuse Evaluation
With increased government accountability and the shift to managed care                s
                                                                              Florida’ Measures
systems for behavioral health care, substance abuse treatment                 Reflect National
nationwide is beginning to use measurable indicators to demonstrate           Findings of Treatment
effectiveness and improve client outcomes. National studies                   Effects
demonstrate the cost benefit of treatment to society. Treatment reduces
criminal activity, alcohol and drug use, and health costs, and increases
employment (CALDATA 1994; NTIES, 1996; Oregon, 1966; DATOS,
1997) ii . These studies support Florida’ use of measures of substance
                                         s
use, employment, arrests and other desired outcomes for performance
management.

The consensus of existing research and clinical experience is that alcohol
and drug abuse and dependence are complex disorders involving
biological, psychological and social factors (Treatment Protocol
Effectiveness Study by Office of National Drug Control Policy, 1996).
Substance abuse is generally a chronic, long term condition, not a brief
episode. Abusers tend to relapse, especially during high risk events in
their lives. People referred by the criminal justice system or employers
may be in denial about their problems. Their attempts to become drug
free may fail several times. Hard-core drug users, such as cocaine users
involved in criminal activity, are difficult to reach. At the current level
of development of substance abuse treatment, it continues to be difficult
to identify the most effective type of treatment and for whom it might be
most effective. What intensive studies show is that regardless of the
substance abused or the treatment provided, treatment works when:
• those who abuse drugs are engaged in their treatment;
• they remain in treatment to completion; and
• other needed services such as job training and placement, housing,
     health care and education, are integrated with drug treatment to help
     clients resolve problems that accompany their drug use.




I. How effective is substance abuse treatment in                                FINDINGS
Florida?


     Substance abuse treatment results in long term
     benefits of reduced substance abuse and other
     problems, and better employment.




    Substance Abuse Evaluation                                                            Page   5
                        I. A. Adult Substance Use Six Months After
                        Treatment

Clients Were Surveyed   The study surveyed a random sample of adult clients who completed
Six Months After        treatment, and clients who did not, to assess the effectiveness of
Treatment               treatment from their perspectives and to learn whether treatment resulted
                        in reduced substance abuse problems. The survey asked former clients
                        general questions about their conditions since leaving treatment. The
                        questions included whether clients’lives and personal relationships were
                        better or worse six months after treatment, and whether they were
                        employed. Clients were asked if they had a problem with substance
                        abuse in the last 30 days. The question was worded to prevent self
                        incrimination. When asked directly about frequency and amount of
                        substance use, fewer persons indicated a problem. Answers about
                        substance abuse problems were consistent with those to questions about
                        life, relationships, and employment. The survey also asked former
                        clients if they were satisfied with services, and whether they had
                        suggestions to improve services.

                        Any methodology involving self report has the potential for bias.
                        However, studies have found a close relationship between self reported
                        drug use and drug tests (NTIES, 1996). In addition, use of a
                        comparison group controls for potential bias in self reported drug use
                        since it is the difference between the groups, not the absolute level of
                        response, that is examined.

Clients Who             Findings of the survey:
Completed Treatment     • Both adults who completed treatment and those who did not report
                            they were treated with courtesy (93%) and satisfied with services
Reported Fewer
                            (81%).
Substance Abuse
Problems and Better     • Six months after leaving treatment, less than half as many clients
Employment                who completed treatment reported having a substance abuse
Outcomes                  problem as clients who did not complete treatment. Six months
                          later, 11% of completers reported they had a substance abuse
                          problem in the last 30 days compared to 26% of non-completers.
                          Only 3% of completers said they had a major problem with
                          substance abuse within the last 30 days compared to 9% of non-
                          completers.
                        • More than three-fourths (77%) of clients who completed treatment
                          reported being employed compared to 58% of non-completers.
                          About two-thirds (66%) of completers said they were employed full-
                          time (35 or more hours per week), compared to half (51%) of non-
                          completers.
                        • Only one percent of clients who completed treatment said their lives
                          were more difficult compared to seven percent of clients who did not
                          complete treatment.




    Page   6                                                   Substance Abuse Evaluation
 • Half as many clients who successfully completed treatment (6%)
   said they had re-entered a treatment program, compared to clients
   who did not complete (12%).

                    Self Reported Substance Abuse Problems
              Six Months Following Treatment, by Completion Status
100%

80%

60%

40%                                             26%

20%                 11%                                          Some or Major
                                                                 Substance Abuse
 0%                                                              Problem in Last 30 Days
                 Completed                    Did Not
               Treatment Drug              Complete (Non-
                    Free                    compliant or
                                               left)



                           Self Reported Employment
              Six Months Following Treatment, by Completion Status
100%
                    77%
80%
                                                58%
60%

40%
                                                                  Employed Full or Part
20%                                                               Time

 0%
                 Completed                    Did Not
               Treatment Drug              Complete (Non-
                    Free                    compliant or
                                               left)
            * Source: Telephone survey of adult clients. n=148

 These differences in substance abuse problems, employment, and other
 outcomes six months following treatment provide evidence that
 substance abuse treatment is effective for clients who complete
 treatment. These survey findings are supported by state wage data and
 other information examined in this evaluation.

 In addition to providing information about client outcomes, the survey
 served as a pilot test of a proposed follow-up measure for performance
 budgeting. The lessons of the pilot suggest that such follow-up of
 substance abuse clients is difficult. All existing records and analysis use
 only identification numbers to ensure confidentiality. The department
 had to obtain names and telephone numbers of clients in the sample from
 providers to be able to contact respondents after treatment. Some clients
 move frequently and do not want to have contact with the treatment
 system.




       Substance Abuse Evaluation                                                          Page   7
Survey Results of      The findings reported above are from a comparison group survey. The
Completers and Non-    department contracted with the Florida International University Institute
completers were        for Public Opinion Research (IPOR) to administer a short telephone
                       survey to a random sample of adult completers and non-completers of
Compared
                       treatment. The department chose a telephone survey over other options
                       to provide objectivity. Findings from a carefully constructed sample are
                       representative because the sample reflects characteristics of the
                       population already known from department data. The sample included
                       adults discharged from residential treatment and outpatient treatment
                       between April and June 1997. Treatment was for alcohol, cocaine and
                       marijuana use. The sample included an equal number of completers and
                       non-completers from each provider to control for variations in treatment.
                       The telephone survey resulted in information from 148 people (91
                       completers and 57 non-completers) out of a random sample of 780
                       adults.

                       The primary reasons surveys were not completed:
                       • 22% were wrong numbers.
                       • 16% had no answer after 6 attempts.
                       • 12% phones were not in service.
                       • 8% refused to participate.

                       Other reasons surveys were not completed include phone numbers that
                       were for businesses, beepers and voice mail. To protect client
                       confidentiality, the surveyors did not use these numbers.

                       We did not survey children for two reasons. Children cannot legally
                       give consent to answer confidential substance use questions over the
                       phone. Also, it would be difficult to get sufficient participation because
                       of the small number of children in residential or outpatient treatment.


                       I. B. Adult Employment Six Months After
                       Treatment

State Wage Data        To assess client outcomes after treatment, employment was also
Shows Higher           examined using state wage data. Employment outcomes were compared
Employment for         for the 3,263 completers and 2,763 non-completers discharged from
Clients Who Complete   treatment in the first quarter of 1996-97. Wage data support findings of
                       the telephone survey. The relative difference between the completers
Treatment
                       and non-completers is in the same direction and of a similar amount.
                       Both sources of information indicate the program is achieving the
                       desired outcome of increased employment. Because the employment
                       information from the Florida Education and Training Placement
                       Information Program (FETPIP) includes all clients, not just those able to
                       be reached by telephone, it indicates a lower rate of employment overall.
                       The employment rate is 29% less for non-completers than for
                       completers. State wage data indicate that six months later 49% of the
                       completers successfully discharged from treatment in the first quarter


    Page   8                                                  Substance Abuse Evaluation
 were employed in the third quarter of 96-97, compared to 35% of non-
 completers. Earnings of those employed in both groups differed by
 35%. Completers who were employed earned on average $3,663 during
 the quarter compared to $2,391 earning during the quarter by non-
 completers. Full-time minimum wage earnings per quarter at the time
 were $2,375.
               Percent of Adults Employed Six Months Follow ing Treatment,
                                  by Completion Status
100%
 80%

 60%                        49%
                                                                        35%
 40%

 20%
  0%
                        Completed                                Did Not Complete
                      Treatment Drug                             (Non-compliant or
                           Free                                        left)



               Average Quarterly Earnings of Adults Six Months Following Treatment,
                                      by Completion Status

$5,000
                           $3,663
$4,000
$3,000                                                                  $2,391

$2,000

$1,000
   $0
                     Completed Treatment                           Did Not Complete
                         Drug Free                               (Non-compliant or left)


              * Full-time quarterly minimum wage earnings at the time were $2,375.
              ** Source: FETPIP Employment Data 1-3/97. n=6,025

 The findings support Department use of drug free completion of
 treatment as a program performance measure. Use of these data
 demonstrates the feasibility of obtaining follow-up employment
 outcomes from FETPIP on a routine basis.


 The findings reported above resulted from comparison of two groups                        Employment Data for
 who entered treatment with similar problems. One group had the full                       Completers and Non-
 benefit of treatment and the other did not. We compared 3,263 clients                     Completers were
 who completed treatment and were discharged drug free to 2,763 clients                    Compared
 who left before completing treatment or were discharged for non-
 compliance. Employment after treatment cannot be compared to
 employment before treatment because of differences in criteria and data
 between the databases used.



         Substance Abuse Evaluation                                                                   Page   9
            To assess employment outcomes, we matched clients discharged from
            treatment July-September 1996, to Florida Education and Training
            Placement Information Program (FETPIP) employment data for
            January-March 1997. We compared employment rates of all 6,025
            completers and non-completers reported. FETPIP obtains the
            employment data from the Department of Labor and Employment
                      s
            Security’ Unemployment Insurance wage record data to assess the
            outcomes of vocational education and training programs. The data are
            unduplicated, cleaned and validated by FETPIP and are available for
            analysis.



            II. Who are our clients and how well do we
            serve them?


                Clients abuse primarily alcohol, crack/cocaine or
                marijuana depending on their age and sex. Most adults
                are males, age 25 to 44, who abuse alcohol or
                crack/cocaine. Most children are high school males who
                abuse marijuana. Clients with alcohol problems have
                higher rates of successful treatment. Clients with crack or
                cocaine problems tend to receive more intensive and
                expensive residential treatment and have lower success
                rates.




            Patterns of substance abuse are as varied as the people of Florida. The
            primary drugs abused by those served are alcohol, crack or cocaine, and
            marijuana. While the predominant substance abuse problem continues
            to be alcohol, substance abuse patterns continue to change as the
            availability of substances change. In 1969-73, heroin was the major
            new problem in the country. In 1979-81, the problem was heroin and
            psychoactive drugs. In the 1990s, cocaine became the most significant
            new substance abuse problem. Drug abuse patterns vary by age and
            sex. Adults and children have different substance abuse problems and
            legal status, and require different treatment approaches.




Page   10                                         Substance Abuse Evaluation
  II. A. Adults Served
  The program reports over 136,300 adults were served in 1996-97. The                                     71,000 Adults Entered
  analysis of characteristics of clients and the services they receive is                                 Services in 1996-97
                                                          2
  based on 71,095 admitted to services during the year. Non-residential                                   with $68.1 Million in
  and residential treatment Comprise the largest expense of substance                                     Expenditures
  abuse services provided to clients. In 1996-97 118 providers receiving
  state funds admitted from one to 8,452 clients. Most clients (63%)
  received non-residential or residential treatment that represented 69% of
  expenditures. Forty-seven percent received non-residential treatment.
  Twenty-nine percent of admissions were for intervention services,
  primarily Treatment Alternatives to Street Crime (TASC) for males and
  community intervention for females. Intervention includes assessment,
  brief counseling and referral to treatment when appropriate.

        Expenditures and Adults Admitted by Service, 1996-97

                                                 Expenditures
Services                                           (Millions)      Admitted
                                               Amount Percent Number Percent
Non-residential Treatment                        $16.2        24% 32,955    47%
Residential Treatment                            $30.5        45% 11,485    16%
Detoxification                                   $12.5        18%  4,875     7%
Intervention (assessment &                        $5.6         8% 20,692    29%
 referral, & case management)
Prevention                                           $3.0               4%             134           0%
Special Overlay                                      $0.1               0%             954           1%
Other                                                $0.1               0%
Total                                              $68.1            100%          71,095         100%
             * Source: Department cost center allocations and SISAR client information.



  The main referral source into substance abuse services is the criminal                                  Adults Primarily Enter
  justice system (57%), primarily from probation and parole and driving                                   Services From
  under the influence (DUI). Twenty-six percent of admissions are self                                    Criminal Justice or
  referrals. Generally adult clients do not have a pattern of multiple                                    Self Referral
  arrests prior to treatment. About one-fourth had two or more arrests
  prior to treatment while three-fourths had one arrest (39%) or none
  (35%).




  2 The program reports 136,325 adults were served in 1996-97, with 79,248 receiving non-residential
  treatment and 15,174 receiving residential treatment. This report uses admission counts rather than
  number served for analysis of client characteristics and services. The number served is calculated to
  include those admitted prior to the year and not yet discharged.


        Substance Abuse Evaluation                                                                                    Page   11
                           Referrals of Adults to Substance Abuse Services
                                                                    Cases Admitted in FY 1996-97 (n=71,095)
                        Criminal Justice                                               Community            Dept. Children &          Other Referrals
                         (Inc. Probation, DUI and                                     Social Service
                           Other Court and Law
                                                          Self Referral                                        Families               (Inc. SubAbuse MHealth,

                               Enforcement)
                                                                 26%                    Agencies            Family Safety Preserv.        Employers & Unk)
                                                                18,314                         4%                    3%                       10%
                                  57%
                                                                                              3,009                 2,098                     6,948
                                 40,726


                                                                            Detox
                                                                              7%
                                                                             4,875
                    Intervention
                (Assessment, Counseling,
                    Referral, & Case
                     Management)
                         29%
                        20,692                                                                                                        Other
                                                                                                                                       1%
                                                                                                                                      1,088




                                             Non-Residential                    Residential
                                             Outpatient & Day                   Treatment
                                                Treatment                             16%
                                                       47%                           11,485
                                                      32,955

                                                               * Source: Dept. of Children and Families SISAR Admissions
                                                               ** Numbers are admissions, not total number served. The program reports 136,300
                                                               served, including 79,248 in non-residential and 15,174 in residential treatment.



Adults Primarily                                    The three primary substances abused by clients receiving services are
Abuse Alcohol,                                      alcohol, crack/cocaine and marijuana.
Crack/Cocaine and
Marijuana                                                                   Predominant Substances Abused
                                                                         Adults Admitted to All Services, 1996-97

                                                                                                            Number
                                                        Alcohol and Drug Use                                Admitted                 Percent
                                                        Alcohol                                                26,966                    38%
                                                        Crack/Cocaine                                          19,199                    27%
                                                        Marijuana/Hashish                                        9,954                   14%
                                                        Heroin                                                   1,548                    2%
                                                        Other Opiates                                              908                    1%
                                                        Benzodiazepine & Other                                     361                    1%
                                                           Tranquilizers & Sedatives
                                                        Methamphetamine & other                                             334               0%
                                                           Amphetamines
                                                        Hallucinogens                                                    89               0%
                                                        Other Drugs                                                     139               0%
                                                        Unknown/Denies                                                2,934               4%
                                                        None (inc. spouse receiving support)                          3,379               5%
                                                        No Data                                                       5,284               7%
                                                        Total                                                        71,095             100%
                                                                         * Source: Dept. of Children and Families SISAR Admissions



    Page   12                                                                                            Substance Abuse Evaluation
The majority of both alcohol and marijuana users admitted to treatment             Most Clients are
are male (76% and 80% respectively). A higher percentage of                        Males— Crack/Cocaine
crack/cocaine users are female (37%) than for other drugs.                         is the Drug with the
                                                                                   Largest Proportion of
                   Primary Substances Abused by Sex
                                                                                   Female Clients
                 Adults Admitted to All Services, 1996-97

     Major                  Total                 Males              Females
   Substances              Number                Percent             Percent
 Alcohol                      26,966                  76%                   24%
 Crack/cocaine                19,199                  63%                   37%
 Marijuana                     9,954                  80%                   19%
 Heroin                        1,548                  68%                   32%
 Total                        57,667                  72%                   28%
         * Source: Dept. of Children and Families SISAR Admissions



Marijuana use is most common among young adults. Alcohol is most      Substance Abuse
common among older clients. The age group with the highest percentage Varies by Age Group
of crack/cocaine use is 25-34.

               Primary Substances Abused by Age Group,
                Adults Admitted to All Services, 1996-97

              Total                           Crack/
 Age          Number          Alcohol         Cocaine         Marijuana   Heroin
18-24          11,138             32%             23%              43%       2%
25-34          20,968             42%             41%              15%       2%
35-44          18,406             52%             36%               9%       3%
45+             7,155             73%             19%               5%       3%
Total          57,667             47%             33%              17%       3%
         * Source: Dept. of Children and Families SISAR Admissions



While the majority of clients are in outpatient, non-residential treatment,        Most Clients Receive
a high percentage of crack/cocaine and heroin users are in more                    Outpatient Treatment:
intensive and more costly residential treatment. Crack/cocaine users are           A Higher Percentage
the second largest group of clients who enter treatment. They have the             of Crack/Cocaine
worst characteristics for successful completion and long term outcomes.
                                                                                   Abusers Receive More
The majority, 78%, use crack. Clients using crack/cocaine are admitted
with higher rates of multiple arrests (35% two or more) and prior                  Intensive and Costly
admissions (67%), and lower employment rates, than other clients.                  Residential Treatment




    Substance Abuse Evaluation                                                                Page   13
                                 Type of Treatment by Primary Substances Abused
                                      Adults Admitted to Treatment, 1996-97

                            Major              Admitted to              Residential             Non-residential
                          Substances           Treatment                Treatment                 Treatment
                                                Number                   Percent                   Percent
                        Alcohol                     18,490                      19%                        81%
                        Crack/cocaine               14,371                      39%                        61%
                        Marijuana                     6,777                     18%                        82%
                        Heroin                          684                     49%                        51%
                        Total                       40,322                      27%                        73%
                                * Source: Dept. of Children and Families SISAR Admissions




Abusers of Alcohol                              s
                        Differences in a client’ drug use and source of referral to services have
Have Higher Success     the strongest relation to differences in successful completion of
Rates than Abusers of   treatment. Treatment is most successful for clients with alcohol
Crack/Cocaine           problems. Comparing the ratio of successful completers to unsuccessful
                        discharges, as used for the follow-up studies, 64% of those in alcohol
                        treatment are successful, compared to 45% of crack/cocaine users.

                           Ratio of Successful Completion by Primary Substances Abused
                                    Adults Discharged from Treatment, 1996-97

                                                                     Successful:            Unsuccessful:
                                                 Total               Completed               Discharged
                            Major             Successful &           Treatment             Non-Compliant
                          Substances          Unsuccessful           Drug Free            or Left Treatment
                         Alcohol                     7,023                  64%                        36%
                         Crack/cocaine               5,469                  45%                        55%
                         Marijuana                   2,587                  52%                        48%
                         Heroin                        234                  52%                        48%
                         Total **                  15, 313                  55%                        45%
                                * Source: Dept. of Children and Families SISAR Discharges
                                ** Not including 548 completing treatment not drug free and 5851 discharges to other
                                services, deaths, etc. Does not include adults in treatment for other drugs.



                        Clients referred from criminal justice sources such as probation or DUI
                        have a higher ratio of successful completion of treatment (65%)
                        compared to self referrals (43%). For the largest number of clients,
                        those in treatment for alcohol problems, residential and non-residential
                        treatment have similar success rates. This is not true for crack/cocaine
                        treatment, where the ratio of success for non-residential treatment is
                        35% successful to 65% unsuccessful, compared to a ratio for residential
                        treatment of 57% successful to 43% unsuccessful.




    Page   14                                                            Substance Abuse Evaluation
                       Ratio of Successful Completion by
                   Treatment and Primary Substances Abused
                   Adults Discharged from Treatment, 1996-97

                        Residential Treatment                      Non-residential Treatment
    Major             Successful &     Percent                     Successful &     Percent
 Substances           Unsuccessful    Successful                   Unsuccessful Successful
Alcohol                      1,383          66%                           5,640         64%
Crack/cocaine                2,402          57%                           3,067         35%
Marijuana                      454          57%                           2,133         51%
Heroin                         108          65%                             126         41%
Total                        4,347          60%                          10,966         53%
  * Source: Dept. of Children and Families SISAR Discharges



  The analysis of the characteristics of adults served and their services                                   Reporting of
  identified several areas where improvements can be made in                                                Performance Omits
  performance reporting. Department performance measures assess                                             Some Programs
  outcomes for clients in non-residential and residential treatment services
                                                                                                            Serving Adults
  that receive the majority (69%) of resources. Detoxification services
  with 18% of resources does not have performance measures.
  Intervention services receiving 8% of resources, but serving 29% of
  clients, do not have measures other than client satisfaction.

  The program loses information about the characteristics of the                                            Subgroups Have
  population it serves by the procedure it uses to disaggregate clients into                                Substantial Overlap
  four subgroups related to service needs. It uses a mixture of clinical,
  demographic and legal criteria that do not distinguish unique groups.
  The four groups are intravenous drug users, clients in need of mental
  health services, parents putting children at risk by their substance use,
  and clients involved in the criminal justice system. One third of the
  population, 34%, are both parents and criminal justice involved.
  Because criminal justice involvement, which is a characteristic of 84%
  of the population, is disaggregated last, it appears to represent less than
  half of the population. For planning and management purposes it would
  be more useful to report all these treatment related characteristics.


  II. B. Children Served
  The program reports over 62,000 children under age 18 served in 1996-                                     41,000 Children Entered
  97. The analysis of characteristics of clients and the services they                                      Services in 1996-97
  receive is based on 41,079 children admitted to services in 1996-97. 3
                                                                                                            with $43.4 Million in
  Services provided to children under age 18 include prevention,
                                                                                                            Expenditures
  intervention and treatment. The major prevention program called Alpha-


  3 The program reports 62,005 children under age 18 were served in 1996-97, with 19,414 receiving
  non-residential treatment and 2,551 receiving residential treatment. As indicated for adults, for the
  purpose of this report, current year admissions was determined to be a better source of information for
  analysis of the characteristics of clients and the services they receive.


       Substance Abuse Evaluation                                                                                      Page   15
                      Beta targets younger children in elementary and middle schools who are
                      at risk of substance abuse. It is provided in 100 classrooms across the
                      state by community agencies in partnership with local school boards, to
                      improve educational achievement, parent involvement and behavior.
                      Intervention, which includes assessment, brief counseling, referral and
                      case management, is provided to all children involved in the juvenile
                      justice system. Treatment includes both residential and non-residential
                      outpatient and day treatment.

                      Non-residential and residential treatment are the largest expenditures
                      (50%) for services provided although they involve less than one-fifth
                      (16%) of children reported admitted to services. Intervention is provided
                      to 69% of children admitted.

                         Expenditures and Children Admitted by Service, 1996-97

                                                                 Expenditures
                                                                   (Millions)       Admitted
                      Services                                  Amount Percent Number Percent
                      Non-Residential Treatment                     $8.2      19%  5,101     12%
                      Residential Treatment                       $14.1       33%  1,467      4%
                      Detoxification                                $3.0       7%  2,050      5%
                      Intervention (assessment &                    $6.5      15% 28,294     69%
                         referral, case management)
                      Targeted Prevention                            $8.1         19%          2,400        6%
                      Special Overlay & Aftercare                    $2.6          6%          1,767        4%
                      Other                                          $0.9          2%
                      Total                                         $43.4       100%         41,079        100%
                              * Source: Department cost center allocations and SISAR client information.




Children Enter        The primary referral source to substance abuse services is the
Services Primarily    Department of Juvenile Justice, 67%. The majority receive intervention
From the Department   services for assessment, brief counseling and referral to treatment if
of Juvenile Justice   needed.




    Page   16                                                          Substance Abuse Evaluation
     Referrals of Children to Substance Abuse Services
                                                Cases Admitted in FY 1996-1997 (n=41,079)

                                      Other Criminal                                                          Other Referrals
      Dept of Juvenile                                                                      Self and Family    (Inc. Substance Abuse,
                                      Justice (Inc. Courts,           Schools
          Justice                                                      10%
                                                                                                Referral        Other Agencies, Dept.
                                         Probation & TASC)                                                    Children Families & Unkn)
              67%                                                                                 4%
                                              19%                      4,010                                            4%
             25,738                                                                              1,767
                                              7,764                                                                    1,800

                                                                         Targeted
                                                                        Prevention
                                                                                6%
 Special                     Intervention                                      2,400
 Services                 (Assessment, Counseling,            Detox
(Inc Follow-up)         Referral, & Case Management)           5%
      4%                           69%                        2,050
     1,767                        28,294




       Residential                   Non-Residential
                                     Outpatient and
       Treatment
             4%                      Day Treatment
            1,467                            12%
                                             5,101

                     * Source: Dept. of Children and Families SISAR Admissions.
                     ** Numbers are admissions, not total number served. The program reports 62,005 served,
                     including 19,414 in non-residential and 2,551 in residential treatment



     Marijuana is the most common drug problem among children admitted                                        Children Served
     to services abusing substances, followed by alcohol.                                                     Primarily Abuse
                                                                                                              Marijuana
                                 Predominant Substances Abused
                            Children Admitted to All Services, 1996-97

                                                                        Number
                       Alcohol and Drug Use                             Admitted            Percent
                  Marijuana/Hashish                                       13,722                33%
                  Alcohol                                                  4,190                10%
                  Crack/Cocaine                                              697                 2%
                  Hallucinogens                                              172                 0%
                  Methamphetamine & Other                                     76                 0%
                     Amphetamines
                  Benzodiazepine & Other                                               61         0%
                     Tranquilizers & Sedatives
                  Heroin                                                            48            0%
                  Other Opiates                                                     14            0%
                  Other Drugs                                                      642            2%
                  Unknown/Denies                                                 3,971           10%
                  None (Inc. clients assessed with no drug use,                  8,016           20%
                      or in prevention programs)
                  No Data                                                       9,470            23%
                  Total                                                        41,079            58%
                              * Source: Dept. of Children and Families SISAR Admissions


            Substance Abuse Evaluation                                                                                                    Page   17
Most Children           The average age of children served is 16. Of those served, 78% are age
Served are              14-17. Seventy-four percent are males. Among clients abusing
Adolescent Males        marijuana, alcohol and crack/cocaine (95% of the total), the majority
                        (58%) are males using marijuana. The next common clients are males
                        using alcohol, and females using marijuana and alcohol.

                                          Primary Substances Used by Sex
                                     Children Admitted to All Services, 1996-97

                                       Sex and Major                                   Total
                                     Substances Abused
                                                                             Number            Percent
                                Male Marijuana                                 10,610              58%
                                Male Alcohol                                    2,990              16%
                                Female Marijuana                                2,973              16%
                                Female Alcohol                                  1,156               6%
                                Male Crack/cocaine                                429               2%
                                Female Crack/cocaine                              259               2%
                                Total **                                       18,417            100%
                                    * Source: Dept. of Children and Families SISAR Admissions
                                    ** Does not include children admitted for other substance use.




Most Do Not Have a      Similar to adults, the majority of children do not have a pattern of
Pattern of Multiple     multiple arrests. Thirty percent are reported as having no arrests in the
Arrests                 two years prior to admission, and 27% reported only one. Twenty-eight
                        percent have two or more arrests before admission to services.
                        Crack/cocaine users have the highest rates of multiple arrests.

                                    Arrests in the 24 Months Prior to Admission
                                    Children Admitted to All Services, 1996-97

                                Arrests in 24 Months                    Number                 Percent
                                 Prior to Admission
                              No Arrests                                      12,459                  30%
                              One Arrest                                      10,946                  27%
                              Two Arrests                                      5,114                  13%
                              Three or More Arrests                            6,219                  15%
                              No Data                                          6,341                  15%
                              Total                                           41,079                 100%
                                         * Source: Dept. of Children and Families SISAR Admissions




A Higher Percentage     Of the small proportion of children reported admitted to treatment
of Crack/Cocaine        services, the majority are in outpatient, non-residential treatment. A
Abusers Receive         higher percentage of crack/cocaine users are admitted to more intensive
Residential Treatment   and costly residential treatment.




    Page   18                                                           Substance Abuse Evaluation
           Type of Treatment by Primary Substances Abused
              Children Admitted to Treatment, 1996-97

                           Admitted to              Residential             Non-Residential
Major Substances           Treatment                Treatment                 Treatment
                            Number                   Percent                   Percent
Marijuana                        3,799                      18%                        82%
Alcohol                            970                      19%                        81%
Crack/cocaine                      330                      39%                        61%
Total **                         5,099                      25%                        75%
    * Source: Dept. of Children and Families SISAR Admissions
    ** Does not include children admitted for other substance use.



 Differences in the primary substance abused have a strong relation to                                 Abusers of Alcohol
 successful drug free completion of treatment. As with adults, the                                     Have Higher Success
 treatment of alcohol use has the highest success ratio (60%) and                                      Rates than Abusers of
 crack/cocaine the lowest (39%). Crack/cocaine also has the highest rate                               Crack/Cocaine
 of prior admissions.

        Ratio of Successful Completion by Substances Abused
           Children Discharged from Treatment, 1996-97

                                                Successful:             Unsuccessful:
                                                Completed                Discharged
     Major             Successful &             Treatment              Non-Compliant
   Substances          Unsuccessful             Drug Free             or Left Treatment
                         Number                  Percent                   Percent
  Marijuana                   2,144                    53%                         47%
  Alcohol                       509                    60%                         40%
  Crack/cocaine                 109                    39%                         61%
  Total **                    2,762                    54%                         46%
         * Source: Dept. of Children and Families SISAR Admissions.
         ** Not including 547 completing treatment and not drug free, and 1,112 discharges to
         other services, deaths, etc. Does not include children receiving treatment for other drugs.



                           s
 The analysis of children’ characteristics and services identified areas    Reporting of
 where performance reporting can be improved similar to adults. A           Performance Can Be
 comparatively low proportion of children were reported admitted to         Improved
 treatment programs in 1996-97 relative to the total served. Two-thirds
 (69%) were admitted to intervention services that are provided to all
 children entering the juvenile justice system. Other than client
 satisfaction, there are no performance measures to provide
 accountability for services such as detoxification and intervention. As
 with the adult population, some of the subgroups used to identify service
 needs do not clarify performance management strategies. The subgroup
 of children under the supervision of the state is the priority subgroup of
 the Department. Because this subgroup includes all children referred
 through the juvenile justice system, irrespective of their substance abuse



    Substance Abuse Evaluation                                                                                    Page   19
                      problem, it represents 86% of children served. Only a small percentage
                      of children served are not included in the subgroup.

Analysis of Clients   The above findings of both adult and child client characteristics and
and Services Are      services are based on the SISAR database of all 181,074 admission and
Based on Extensive    discharge records of clients of providers receiving state funding during
Program Data          1996-97. This study did not include clients served by private providers
                      licensed by the state that do not receive funds from state appropriations.
                      The client group studied included all clients admitted and all clients
                      discharged during the year. Client characteristics, referrals and service
                      information were analyzed using data from admission forms.
                      Completion of treatment information is based on client discharge forms.



                      III. How good are the data used for
                      performance management?


                          The data system provides useful information on the
                          characteristics of clients served and their substance
                          abuse at admission to services. Reporting problems
                          and data errors reduce outcome information. The
                          program is aggressively addressing data issues.




The Program’ Data
              s       The program office uses the State Integrated Substance Abuse Reports
System Provides       (SISAR) in the Alcohol, Drug Abuse and Mental Health data warehouse
Useful Client and     to report and manage performance. Performance information is
Service Information   available for each service district and provider. The SISAR data system
                      was built with federal funding to track characteristics of those served
                      and services provided. It includes information from admission and
                      discharge forms for each component of service. The large number of
                      clients included in the database provides good information on the
                      population served and lends itself to analysis. Security measures ensure
                      client confidentiality. No names, addresses or phone numbers of
                      individuals are included in the data system.

There are Problems    A significant problem with data used for performance measurement is
Reporting Discharge   the low ratio of reported discharges to admissions in the current
Information           database. This indicates a large loss of information on client outcomes.
                      When the department began to use the data system to report outcome
                      performance it found the number of clients reported as completing
                      treatment to be very small compared to the number admitted. For
                      example, compared to the 1,393 children admitted to residential
                      treatment in 1996-97, only 664 were reported discharged. Performance
                      outcomes can only be reported by the department for these 664 children.


    Page   20                                                Substance Abuse Evaluation
The following table illustrates the magnitude of this problem and the
extent to which clients are lost to follow-up. Compared to the number of
adults reported admitted in 1996-97, only 47% were reported
discharged. This indicates as many as 53% of adults who are treated do
                                                   s
not have discharge information in the department’ database and their
client outcomes cannot be reported. For children, only 41% of the
number admitted to treatment are reported to be discharged, so that 59%
have no outcome information. Because a low number of children are
reported to be admitted to treatment and a low percentage of those
admitted are reported discharged, accountability for treatment is based
on only 2,448 discharges.

   Follow-up Status of Persons Discharged by Treatment, 1996-97

                    Admissions              Discharges           No Discharges
                                                     Percent        Percent
                                                   Compared to    Compared to
  Treatment            Number         Number       Admissions     Admissions
Adults
Non-residential             29,643       13,528           46%             54%
Residential                 10,991        5,403           49%             51%
Total                       40,634       18,931           47%             53%
Children
Non-residential              4,520         1,784          39%             61%
Residential                  1,393           664          48%             52%
Total                        5,913         2,448          41%             59%
         * Source: SISAR Admission and Discharge Data



The program has identified the primary reason for the low ratio of
discharges to admissions as discharge forms not accepted by the
database. Discharge forms are not accepted unless they match an
admission form for the same service. The program has found reported
service dates, and coding of provider and client identification numbers,
often do not match. Reasons for these problems include formatting of
data, errors in completing and entering scan forms, and some providers’
non-compliance with reporting requirements.

The program is aggressively improving data quality. Performance                  The Program is
information is regularly reported back to districts and providers so             Aggressively
reporting problems can be identified. District staff now have on-line            Addressing Data
access to the database for analysis and contract monitoring. The                 Problems
program is checking database records against client records with
providers to identify and correct data errors and reporting problems.

The program is carrying out a data validation process with each
contracted provider to determine the accuracy of key data elements. The
process will determine if the data in the SISAR database reflects the
actual population served, is consistent with information in client files,
and whether key data elements required for performance reported are

    Substance Abuse Evaluation                                                              Page   21
            completed. The process was developed with the Florida Mental Health
            Institute. The program began implementing the process January 1,
            1998. Each district is responsible for completing the validation process
            with each one of their contracted providers and will report all
            information to the central office by April 15, 1998. The process will
            then be incorporated into the contract monitoring process.

            A program process improvement team with central office, district and
            provider members is addressing data problems. The team identified
            underlying problems that include lack of timely reporting by providers,
            provider misunderstanding about procedures and requirements, and a
            reporting process that is complex and increases the probability of
            problems and errors.

            Many of these problems reflect the large quantity of information
            reported. All providers, both those state funded and those not funded
            but licensed, are currently required to send data forms for each
            component of service for each client. This generates a very large
            number of client forms that are difficult to manage to ensure
            completeness and quality. Some information on the forms, such as client
            demographics, is redundant and some is only relevant for a small portion
            of clients and services. Much of the information is not needed or used
            for performance management and accountability. Required state
            reporting is in addition to internal provider record keeping.

            To address these underlying problems, the process improvement team is
            reengineering the data reporting system to reduce the amount of data that
            needs to be reported and improve quality. Information will only be
            reported for the entire episode during which a client receives services
            from a provider. Data are being submitted electronically through a
            system developed by the department that includes built-in data checks.
            Electronic submission from the majority of providers already
            demonstrates that it is timely and error free. Non-essential data elements
            are being eliminated. These changes to correct existing data problems
            will be phased in beginning July 1, 1998 and completed January 1,
            1999.




Page   22                                          Substance Abuse Evaluation
In conclusion, the findings of this evaluation are:                        SUMMARY AND
                                                                           CONCLUSIONS
• Completion of substance abuse treatment produces positive
  outcomes of reduced substance abuse problems and increased
  employment, based on a six month follow-up.
• Comparison of outcomes for completers versus non-completers
  validates the importance of the current performance measure of drug
  free completion of treatment.
• The substance abuse program provides an array of services to
  clients with very different characteristics, depending on age, sex and
  primary substance use.
• Alcohol, crack/cocaine and marijuana are the most common
  substances abused by clients. Alcohol is the most common problem
  for adults and marijuana is most common for adolescents.
• Clients abusing crack/cocaine, the second most common drug used
  by adults, have the lowest treatment success.
• Most substance abusers are male. A higher proportion of clients
  abusing crack/cocaine are female compared to other substances.
• Alcohol and drug treatment services are operated primarily by
  private, not-for-profit providers under contract to the Department of
  Children and Families. The department receives state General
  Revenue and substantial federal money to fund contracts.
• Due to data problems with reporting discharges, the program lacks
  outcome information on a large percentage of clients.
• Performance measures have not been developed to hold all services
  accountable, especially detoxification and intervention.
• Subgroups used to report populations served have substantial
  overlap, so information about characteristics of clients is lost.
• The program is reengineering its data reporting system to correct
  data problems. It is working with providers and districts to simplify
  reporting, and expand use of data for contract management and
  better accountability.


Based on this evaluation, the following recommendations are offered:       RECOMMENDATIONS
1. The substance abuse program should work with providers to
   develop and implement systematic follow-up of clients after
   treatment to demonstrate that desired outcomes of drug-free
   behavior and employment are met.
2. The program should maintain the use of completion of treatment as
   a performance measure. The program should apply quality
   improvement processes to increase the rate of successful drug free
   completion of treatment. This should include identifying


    Substance Abuse Evaluation                                                    Page   23
                characteristics of clients with low success rates and working with
                providers to improve treatment completion.
            3. The program should further evaluate the current subgroup reporting
               structure to better communicate client characteristics and service
               needs.
            4. The program should develop performance measures for management
               and accountability of all primary services, including detoxification
               and intervention. The program should identify desired outcomes and
               measurable indicators of how services drive program goals of drug
               free behavior and employment.
            5. The substance abuse program should establish measurable
               objectives for improvements to the data system in terms of
               timeliness of reporting, completeness and accuracy.
            6. Data validation efforts should be implemented system-wide as an
               integral function of contract monitoring and data reporting during
               the next fiscal year.




            i
               General Accounting Office,
                      1996, Drug and Alcohol Abuse: Billions Spent Annually for
                      Treatment and Prevention Activities (GAO/HEHS-97-12).
            ii
               California Department of Alcohol and Drug Programs,
                      1994, Evaluating Recovery Services: The California Drug and
                      Alcohol Treatment Assessment (CALDATA).
            US Department of Health and Human Services, Substance Abuse and Mental
            Health Services Administration,
                      1996, National Treatment Improvement Evaluation Study (NTIES).
            Oregon Department of Human Resources, Office of Alcohol and Drug Abuse
            Programs,
                      1996, Societal Outcomes and Cost Savings of Drug and Alcohol
                      Treatment in the State of Oregon.
            National Institute on Drug Abuse,
                      1997, Drug Abuse Treatment Outcome Survey (DATOS).




Page   24                                          Substance Abuse Evaluation

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:2
posted:8/16/2011
language:English
pages:28