Status Report on Compliance with Annual Performance Outcome Standards by armedman1

VIEWS: 25 PAGES: 20

									   Fiscal Year 2006-2007 Legislative Status Report




     Compliance with Annual Performance Outcome Standards
              Per Section 394.745, Florida Statutes



           Substance Abuse and Mental Health Programs
               Department of Children and Families

                         November 1, 2007




           




Bob Butterworth                                         Charlie Crist
Secretary                                                  Governor
                    Compliance with Annual Performance Outcome Standards
                        Substance Abuse and Mental Health Programs
                             Department of Children and Families
                        Fiscal Year 2006-2007 Legislative Status Report
                             Per Section 394.745, Florida Statutes


                                       Executive Summary

The purpose of this report is to demonstrate the compliance with Section 394.745, Florida Statutes. In
Fiscal Year 2006-2007, there were 280 Substance Abuse and Mental Health (SAMH) contracts with
performance data and minimum standards that permitted the calculation of success indicators. Of these 280
contracts, 259 met the level of success defined in the Method section. As footnoted in Table 2 and Table 7,
however, two (2) of these contracts were duplicate (e.g., were statewide contracts reported in multiple
districts). Therefore, the unduplicated number of contracts accountable for General Appropriation Act (GAA)
outcome measures during Fiscal Year 2006-2007 is 278. Of these 278 unduplicated contracts, 257 met the
compliance test, for a 92.5% success rate. The report describes the corrective actions for the 21 contracts
that failed the compliance test.

This report also summarizes the history of Performance Based Program Budgeting (PB)2 since 1994, and
provides a brief review of (PB)2 implementation by the Substance Abuse and Mental Health Programs,
beginning in FY 1997-98. The report also describes some major data improvement initiatives by the two
Program Office, including a data improvement workgroup, technical assistance and support, compliance
with state and federal data requirements, collaboration with other agencies, and monitoring data quality and
integrity.

This report provides tables that identify, by district, the number of contracts that require outcome measures
and have General Appropriations Act (GAA) performance standards and outcome data (n=278); the number
of clients served (n = 269,997 for mental health and 160,835 for substance abuse); the total contract
allotment amounts per district and the number of contracts within the five funding categories.

The overall contract performance is calculated by obtaining a single score for each contract. This score is
then used to determine the contract’s success or failure in meeting performance expectations. To ensure
fairness of the method used to evaluate contracted providers, this single score is based on comparisons
with other contracts serving the same target populations and using similar performance measures.

A discussion and recommendation section examines various issues, including the following: (a) the
corrective actions for 21 contracts that failed to meet the GAA performance outcome standards; (b) the
development of online exception reports to improve performance data submission; (c) the need for data
sharing agreements with other state agencies; (d) the need for identification of risk factors associated with
GAA performance outcome measures, and (e) the need for adequate data system infrastructure to meet
data analysis and reporting needs of the system users.




                                                                                                         Page ii
                          Table of Contents


Section                                                                Page
Executive Summary                                                       ii

I.      Purpose                                                         1

II.     History of Performance Based Program Budgeting (PB)2            1

III.    Implementation of (PB)2 by Substance Abuse and
        Mental Health Programs (SAMH)                                   1

IV.     Data Improvement Initiatives                                    2

V.      Overview of FY 2006-2007 Performance Contracting                4

VI.     DCF District Background On Performance Contracting              5

VII.    Method for Determining “Successful” Contract Performance        10

VIII.   Performance Results                                             13

IX.     District Corrective Actions Taken for Unsuccessful Contracts    14

X.      Discussion and Recommendations                                  16




                                                                             Page iii
                   Compliance with Annual Performance Outcome Standards
                       Substance Abuse and Mental Health Programs
                            Department of Children and Families
                       Fiscal Year 2006-2007 Legislative Status Report
                            Per Section 394.745, Florida Statutes

I. Purpose
The Department is submitting this report in compliance with Section 394.745, Florida Statutes, which
requires the Department of Children and Family Services to submit an annual report as specified in the
paragraph below.

        “By November 1 of each year, the Department of Children and Family Services shall
        submit a report to the President of the Senate and the Speaker of the House of
        Representatives which describes the compliance of providers that provide substance
        abuse treatment programs and mental health services under contract with the
        Department of Children and Family Services. The report must describe the status of
        compliance with the annual performance outcome standards established by the
        Legislature and must address the providers that meet or exceed performance standards,
        the providers that did not achieve performance standards for which corrective action
        measures were developed and the providers whose contracts were terminated due to
        failure to meet the requirements of the corrective plan”.


II. History of Performance Based Program Budgeting (PB)2
In 1994, the Florida Legislature enacted the General Appropriations Act (GAA), which required the
implementation of performance-based program budgeting (PB)2 in Florida's state agencies.
"Performance-based budgeting is a method of relating appropriations to program performance and
expected outcomes." 1 (PB)2 requires that state agencies, as part of their budget requests for the fiscal
year, establish performance outcome targets they intend to achieve on various performance measures.
Since FY 1994-1995, the following state agencies have initiated (PB)2:

        In FY 1994-95, the General Tax Administration in the Department of Revenue was the earliest
         program to participate in (PB)2;

        In FY 1995-96, the Property Tax Administration program in the Department of Revenue and the
         Facilities program in the Department of Management Services started participating;

        In FY 1996-97, Education (Community Colleges), Labor & Employment Security, Law
         Enforcement, Management Services, and the Division of Retirement had programs participating
         in the process; and

        In FY 1997-98, ten more programs in eight departments were added, including the Substance
         Abuse and Mental Health (SAMH) programs within the Department of Children and Families
         (DCF).1

III. Implementation of (PB)2 by Substance Abuse and Mental Health Programs (SAMH)
As the first human services programs to implement (PB)2, the SAMH Program Office met the challenge
of the new approach by building upon work already underway that integrated client, service, provider,
fiscal, and outcome data. This preliminary work was developed nationally as part of the Mental Health
Statistics Improvement Project (MHSIP) and funded by the U.S. Department of Health and Human
1
 A Report On Performance-Based Program Budgeting in Context: History and Comparison, Report 96-77A,
Office of Program Policy Analysis and Government Accountability, April 1997.
                                                                                                   Page 1
                Compliance with Annual Performance Outcome Standards - FY 2006-07 Legislative Status Report

Services (DHHS), Division of Substance Abuse and Mental Health Service Administration (SAMHSA),
Center for Mental Health Services (CMHS). The purpose of this project was to develop minimum
standards for implementing an information system model that sought to answer the management
question, "Who Receives What from Whom, at What Cost with What Effect?" Prior to FY 1997-98, the
SAMH Program Office was in the process of implementing and refining a new unit cost approach to
contracting with the private providers that serve SAMH clients, and a data system based on the MHSIP
model. This work eased the transition to (PB)2. By the end of FY 2006-2007, the SAMH Program
Office had completed their ninth year of (PB)2 implementation.

IV. Data Improvement Initiatives
The Substance Abuse and Mental Health (SAMH) Program Office initiated the following activities to
support the (PB) 2 process and other statutory requirements to improve the quality of the data collected
and processed in FY 2006-2007.

1. Data Improvement Workgroup (DIWG)
This is an interdisciplinary team of various SAMH stakeholders and advocacy groups, including
contracted SAMH providers, staff in central and district SAMH Program Offices, researchers from the
Florida Mental Health Institute (FMHI) at the University of South Florida, representatives of the Florida
Council for Community Mental Health (FCCMH) and Florida Alcohol and Drug Abuse Association
(FADAA). This group met in March 2007 in Orlando to discuss FY 2007-2008 data requirements
including issues related to data quality assurance and improvement, data analysis and reporting,
performance measurement and target setting.

2. District Data Liaisons
District Data Liaisons form a network of staff from each DCF district; they serve as linkages between
the SAMH central Program Office and community provider agencies. In FY 2006-2007, these
individuals met weekly through conference calls with staff in the SAMH central Program Office to
discuss ongoing technical support and training issues related to statewide implementation of data
policies and procedures, data collection and submission, and data analysis and reporting.

3. State and Federal Data Requirements
Chapter 2003-279, Laws of Florida, requires the Department of Children and Families (DCF) and the
Agency for Health Care Administration (AHCA) to establish or develop data management and reporting
systems that: (1) promote efficient use of data by the service delivery system, (2) address the
management and clinical care needs of the service providers and managing entities, and (3) provide
information needed by the Department for state and federal reporting requirements. As part of the
initiatives to continue the implementation of these state data requirements, the SAMH Program Office,
in collaboration with the DCF Office of Information Systems, successfully completed the migration of
the SAMH data system from CACHE web-based environment to Java web-based environment using
Oracle as the database. This migration has improved the processes for submitting and analyzing
statewide data related to the legislative performance outcomes, as well as to state business and
strategic plans. However, the current system is still inadequate in terms of both its accessibility to local
provider agencies and its online capacity to produce standard and ad hoc reports needed by these
agencies. To resolve these accessibility and system capacity issues, the SAMH Program Office has
identified existing resources, which will be used in FY 2007-2008 through a Request for Quote (RFQ)
process to select a third-party vendor, who will develop and implement various online reports and data
analysis packages to meet the needs of system users at the state, region, circuit, and provider levels.

Furthermore, the Substance Abuse and Mental Health Services Administration (SAMHSA) in the U.S.
Department of Health and Human Services (DHHS) requires the SAMH Program Office to collect and
report various data, including the National Outcome Measures (NOMS), as specified in various federal
grants. As a result, the SAMH Performance Management Team (PMT) met weekly throughout the fiscal
year to discuss the NOMS data requirements, including algorithms and stored procedures, which will be

                                                                                                      Page 2
                Compliance with Annual Performance Outcome Standards - FY 2006-07 Legislative Status Report

used for NOMS data analysis and production of standard reports at the state, circuit, and local levels.
The PMT members also made some recommendations to adjust current policies and procedures,
including the use of NOMS as legislative measures in FY 2008-2009.

4. Collaboration with Other Agencies
DCF-AHCA Coordination Meetings - Section 394.9082, F.S., requires DCF and the Agency for Health
Care Administration (AHCA) to develop service delivery strategies that will improve the coordination,
integration, and management of the delivery of mental health and substance abuse treatment services
to persons with emotional, mental, or addictive disorders. One of the goals of this legislation is to
improve the compliance with annual performance outcome standards. In response to this legislation,
high-level management staff members from both agencies, including the Assistant Secretary for SAMH
Programs, have held regular coordination meetings in FY 2006-2007, which will continue in FY 2007-
2008, to discuss various agenda topics, including data sharing and performance measurement issues.
Traditionally, providers have been required to provide data to DCF for clients whose services are paid
in full, or in part, by DCF contracts, Medicaid or local match. Because of Medicaid reforms and AHCA’s
contracts with managed care organizations, e.g., Health Maintenance Organizations (HMO) and pre-
paid behavioral health plans, data submission on clients served by these organizations will be
substantially reduced. AHCA’s contracts no longer require these organizations to provide service event
and performance outcome data to DCF. The impact of these changes will result in reduced DCF
capacity to account for the performance of publicly-funded services, including persons served, services
provided to these individuals, as well as the costs and outcomes of these services. DCF and AHCA
representatives will continue the collaboration to maximize mutual data sharing between the two
agencies.

DOC Aftercare Planning Web-based Application - The SAMH Program Office staff collaborated with
their counterparts in the Department of Corrections (DC) to develop a web-based application, which will
allow both agencies to track inmates released from DC for aftercare planning and service provision in
state-contracted community mental health provider agencies. This system is being piloted in DOC
Region 1 for statewide implementation beginning in January 2008.

Incident Reporting and Assessment System (IRAS) - The DCF Office of Information Systems, in
collaboration with various DCF program offices, has designed IRAS as a web-based application for
collecting and reporting critical incidents involving consumers and staff members in state-contracted,
state-licensed, or state-operated provider agencies. This system will be piloted and implemented in FY
2007-2008 to provide online information, including instant notifications and alerts, on various critical
incidents (e.g., serious injuries or deaths related to seclusions and restraints in substance abuse and
mental health facilities).

Behavioral Risk Factor Surveillance System (BRFSS): As part of the federal data infrastructure grant,
the Mental Health Program Office collaborated with the Florida’s Department of Health (DOH) and the
U.S. Center for Disease Control and Prevention to collect and analyze BRFSS data related to the
prevalence of depression and anxiety in Florida’s adult population. The manuscript containing the
BRFSS data analysis results has already been submitted for publication under the title “Self-rated and
physician-diagnosed depression and anxiety in Florida adults”.

5. Technical Assistance and Support
The SAMH Program Office has used the Performance and Resource Management Teams (PaRTs)
process to provide ongoing technical assistance and support needed by district and provider staff on
the performance measures. This process includes definitions of the measures by target population,
identification of the state standard required by the Government Accountability Act (GAA) for each
performance measure, a discussion of the algorithms used to calculate the performance on each
measure, and the reasons for any discrepancies between standards and actual performance outcomes.


                                                                                                     Page 3
                Compliance with Annual Performance Outcome Standards - FY 2006-07 Legislative Status Report

Ongoing site visits to providers and districts were also conducted throughout the year to address
various data quality and data improvement issues.

6. Monitoring Data Quality
Through collaboration with DCF Contract Oversight Unit staff, district data liaison staff, and district
contract management staff, the SAMH Program Office staff has developed several exception reports
that are used to monitor and improve data quality as part of the contract monitoring process to: (a)
ensure the conformity between information in the Department’s data system and information in the
corresponding provider client records, and (b) to verify that all the required data on enrolled clients is
submitted to the Department’s data systems. These exception reports are available and accessible
online to authorized system users at the following DCF Intranet web site:
http://ewas1.dcf.state.fl.us/PDADM/SAMHRpts/Default.asp.

7. Contract Evaluation Reporting System (CERS)
The DCF Office of Contracted Client Services, in collaboration with various DCF program offices, has
designed CERS as a web-enabled application to provide management with online information,
including contract monitoring reports and data queries, contract performance status reports, and
contract corrective action plans.

V. Overview of FY 2006-2007 Performance Contracting
Table 1 below provides a list of eleven (11) GAA performance outcome measures found in SAMH
contracts for FY 2006-07, as well as the number of contracts including each of those measures, by
target population group. Many contracts have multiple measures and target populations, which
explains why the “No. of Contracts” column adds to more than 279. The focus of this report is limited to
outcome rather than output measures, as specified in s.394.745, F.S., to wit, "The report must describe
the status of compliance with the annual performance outcome standards established by the
Legislature." Only the contracted performance outcome measures, therefore, are listed for each target
population group. Also, the same legislation limits the scope of the report to "…services under contract
with the Department of Children and Family Services." Thus no measures are listed that pertain to
state-operated or state-contracted mental health treatment facilities.

    Table 1: Substance Abuse and Mental Health Contracted GAA Outcome Measures for FY 2006-07
                No. of
   Measure
              Contracts               Target Population Group                                Performance Outcome Measure
     ID
              Involved
    M0001                 AMH: SPMI = Adults with Severe and Persistent           a. Average annual number of days spent in the
                 114
   (MH407)                Mental Illness (SPMI)                                      community (not in institutions or other facilities)
    M0003
                 107      AMH: SPMI                                               b. Average annual days worked for pay
   (MH403)
    M0010                 AMH: Forensic = Adults with Forensic                    a. Average annual number of days spent in the
                 58
   (MH410)                Involvement                                                community (not in institutions or other facilities)
                          CMH: SED = Children with Serious Emotional
    M0011
                 166      Disturbance (SED) - excluding those in juvenile         a. Annual number of days spent in the community
   (MH405)
                          justice facilities
    M0025                 CMH: ED = Children with Emotional Disturbance
                 139                                                              a. Annual number of days spent in the community
   (MH406)                (ED) - excluding those in juvenile justice facilities
                          CSA: with SA = Children with Substance Abuse
   M0045         32                                                               a. Percent of children who complete treatment
                          Problems
    M0058                 ASA: with SA = Adults with Substance Abuse              a. Percent of adults employed upon discharge from
                 52
   (SA405)                Problems                                                   treatment services
    M0062
                 57       ASA: with SA                                            b. Percent of adults who complete treatment
   (SA404)
                          AMH: in MH Crisis = Adults in Mental Health
   M0376         25                                                               c. Median length of stay in CSU/Inpatient services
                          Crisis
   M0377         130      CMH: ED                                                 b. Percent who improve their level of functioning
   M0378         161      CMH: SED                                                b. Percent who improve their level of functioning



                                                                                                                                      Page 4
               Compliance with Annual Performance Outcome Standards - FY 2006-07 Legislative Status Report

The GAA outcome measures are minimally required in each provider's contracts; however with
approval from the state program offices, the districts may include additional measures to assess
individual provider performance. Although the GAA specifies the level of performance required at the
state level for each measure, an individual contract may include one or more targets for performance
standards that are set above or below the state standard. These targets are set based on available
resources and the provider's past performance with the intent to obtain improved performance over
time. These additional performance measures are not included in this report.

VI. DCF District Background on Performance Contracting

This section provides a context for interpreting the level of effort associated with the measurement of
GAA performance outcomes. It provides information on: (a) SAMH contracts accountable for GAA
performance outcomes (Table 2); (b) SAMH contract allotment amounts per district (Table 3); (c) the
number of clients served in community mental health providers (Table 4); and (d) the number of clients
served in community substance abuse providers (Table 5)..

1. Contracts Accountable for GAA Outcome Measures.
   Column 2 of Table 2 shows a total of 498 SAMH contracts reported in the DCF Schedule of
   Allotment Balances for FY 2006-07. However, only contracts that meet the following conditions
   were considered for inclusion in further analysis of the performance compliance with GAA outcome
   measures in this report:

     The contract has Exhibit D in the SAMH contract database, which has at least one target
      population for which there is at least one GAA performance outcome rather than an output
      measure; and
     The contract has GAA performance outcome data reported in the SAMH data system.

    As shown in Column 3 of Table 2, there were 280 SAMH contracts that met the above conditions
    and, therefore, were used in this report to analyze contract success during Fiscal Year 2006-2007.
    As the footnote indicates, however, two (2) of these 280 contracts were duplicate, e.g., were
    statewide contracts reported in multiple districts. Therefore, the unduplicated number of contracts
    accountable for General Appropriation Act (GAA) outcome measures during Fiscal Year 2006-
    2007 is 278. The remaining contracts did not meet the above conditions, i.e., they neither served
    any clients (as indicated by a lack of service events records in the SAMH data system) nor had any
    client-specific outcome measures (as indicated by a lack of records in the measures table of the
    SAMH contract database). Therefore, they were not considered for further analysis of GAA
    outcome measures.




                                                                                                    Page 5
                       Compliance with Annual Performance Outcome Standards - FY 2006-07 Legislative Status Report




                 Table 2: Number of FY 2006-07 Contracts With and Without Performance Data
       (1)                           (2)                                                   (3)
                  Number of SAMH Contracts Recorded in DCF Total Number of Contracts WITH GAA Outcome Data Reported in SAMH
     District       Schedule of Allotment Balances (SAB)     Data System and Used in this Report to Analyze Contract Success



        1                            16                                                           4
        2                            18                                                           18
        3                            17                                                           8
        4                            38                                                           31
        7                            43                                                           30
        8                            36                                                           10
        9                            45                                                           26
       10                            59                                                           31
       11                            69                                                           38
       12                            21                                                           12
       13                            15                                                           5
       14                            30                                                           14
       15                            30                                                           15
       23                            61                                                           38
                                                                                                       3
    Duplicated                            2                                                      280
      Total                         498
                                                                                            (56.2% of 498)



2. Contracted Amounts
Table 3 shows the total contract allotment amounts per district and the distribution of the 498 contracts
by district, for each of five contract amount groupings for FY 2006-07. As shown in this table, the
majority of contracts (341 of the 498 or 68.4 percent) are less than $500,001.




2
  Unduplicated count is 480. Ten contracts are reported in multiple districts. One in 6 districts, four in 3 districts, and 5 in 2
districts.
3
  These 280 contracts include 2 duplicate contracts reported in multiple districts, e.g., Contract SGHG1 is reported in D07 and D14, and
Contract SIH65 is reported in D09 and D10. The unduplicated number of contracts accountable for GAA performance standards is 278.
                                                                                                                                 Page 6
                    Compliance with Annual Performance Outcome Standards - FY 2006-07 Legislative Status Report




           Table 3: Number of SAMH Contracts for FY 2006-07 and Allotment Amounts per District

                     FY 2006-2007 Amended Contract Amounts
               $0       $100,001    $500,001  $1,000,001    Greater          Total Contract
District       to          to          to          to         than             Allotment      Total Contracts
            $100,000    $500,000   $1,000,000 $3,000,000   $3,000,000          Amounts
   1            6          6           1           2            1            $20,257,502            16
   2            8          4           0           2            4            $24,271,653            18
   3           11          2           1           2            1            $17,954,444            17
   4           10          15          3           7            3            $36,701,626            38
   7           17          13          2           5            6            $61,192,748            43
   8           14          12          3           3            4            $36,654,962            36
   9           15          18          5           4            3            $35,710,247            45
   10          18          22          7           10           2            $49,034,139            59
   11          23          22          9           10           5            $75,266,065            69
   12           8          4           4           3            2            $18,319,908            21
   13           4          5           2           2            2            $23,225,645            15
   14          13          9           1           5            2            $22,448,147            30
   15           7             17          2            3            1        $21,260,409            30
   23          24             14          3           10           10        $107,901,722           61
 Total         178           163          43          68           46        $550,199,217          498



Clients Served

Clients served are persons enrolled in the various target population groups and whose services are
paid entirely or partially by SAMH contracts, Medicaid, or matching funds from local governments.
Table 4 and Table 5 show the number of clients served during FY 2006-07 in each district by Mental
Health (MH) and Substance Abuse (SA) target groups, respectively.




                                                                                                         Page 7
                         Compliance with Annual Performance Outcome Standards - FY 2006-07 Legislative Status Report



           Table 4: Number of Clients Served in Community Mental Health Providers in FY 2006-07

                                     Children's Mental Health                                             Adult Mental Health

                                        At-Risk of MH                                                        Community
    District     SED          ED                          Unknown         Total       SPMI       In Crisis             Unknown            Total
                                          Problems                                                            Forensic

       1         2,109       3,468           1,224              na        6,801       10,079      4,234          150            na       14,463
       2         3,207       1,761            31                na        4,999       7,143       7,143          190            na       14,476
       3         2,419       1,638           360                na        4,417       4,909       1,350          173            na        6,432
       4         3,903       2,576           267                na        6,746       8,715       2,230          229            na       11,174
       7         7,240       3,276           445                na       10,961       18,396      2,828          135            na       21,359
       8         2,053       1,682           272                na        4,007       6,158       1,289          143            na        7,590
       9         1,278        976             34                na        2,288       4,064       1,121           96            na        5,281
      10         3,886       1,475           225                na        5,586       6,908       1,697          396            na        9,001
      11         9,480       4,652           563                na       14,695       20,747      2,488          331            na       23,566
      12         2,180        885             86                na        3,151       2,905       2,839           86            na        5,830
      13         2,939       1,401            24                na        4,364       7,165       1,856          215            na        9,236
      14         2,090       2,544           743                na        5,377       7,880       1,144          207            na        9,231
      15         2,403       1,420           339                na        4,162       4,183        678           131            na        4,992
      23         8,698       5,879           1,449              na       16,026       28,699      4,287          800            na       33,786
                                                                     Trend Data
FY 2006-
                53,885      33,633           6,062              na       93,580      137,951      35,184        3,282           na       176,417
07 Total
FY 2005-
                58,237      32,370           5,285              na       95,892      128,705      31,711        3,819           na       164,235
06 Total4
FY 2004-
                56,375      29,069           2,801              na       88,245      102,030      40,420        3,167           na       145,617
05 Total4
FY 2003-
                49,935      28,752           4,451          3,468        86,606       78,319      48,101        2,507        11,822      140,749
04 Total4
FY 2002-
                48,785      25,667           3,355              na       77,803       75,298      62,995        1,717           na       140,010
03 Total4


The number of children served in the community mental health programs increased by 6.1 percent from
88,245 in FY 2004-05 to 93,580 in FY 2006-07. During the same period, the number of adults served
in the community mental health programs also increased by 22.1 percent from 145,617 to 176,417.
The number of adults in mental health crisis decreased by 21.5 percent from 40,420 in FY 2004-05 to
31,711 in FY 2005-06, and the number of adults with severe and persistent mental illness (SPMI)
increased by 26.1 percent, due to a change in the definition of those target population groups. That
change divided the previously defined category of “Adults in mental health crisis” into two new
categories: “Adults with serious and acute mental illness”, and “Adults with mental health problems.”
As a result, some clients previously counted as “adults in mental health crisis” were transferred to the
“SPMI” category.

The number of adults served in substance abuse programs, as shown in Table 5, decreased by 6.4
percent from FY 2004-05 to FY 2005-06, and the number of children served decreased by 8.8 percent
for the same period. This decrease is due primarily to changes in data reporting requirements, which
use service event records rather than client admission records to report the number of persons served.




4
    District numbers do not add to the statewide totals because the latter have been unduplicated for clients served in more than one district.
                                                                                                                                           Page 8
                Compliance with Annual Performance Outcome Standards - FY 2006-07 Legislative Status Report


    Table 5: Number of Clients Served in Community Substance Abuse Providers in FY 2006-07
                      District                 Adults                       Children
                        1                      6,626                         2,635
                        2                      5,836                         1,813
                        3                      2,917                         1,002
                        4                      8,838                         3,174
                        7                      13,463                        8,072
                        8                      5,952                         1,470
                        9                      6,730                         2,586
                        10                     8,691                         4,871
                        11                     13,169                        6,919
                        12                     5,688                         2,000
                        13                     4,272                         2,593
                        14                     3,016                         2,404
                        15                     2,598                         1,731
                        23                     20,015                        11,754
                                                  Trend Data
                  FY2006-07 Total             107,811                        53,024
                                                        5
                  FY2005-06 Total             102,345                        51,929
                  FY2004-05 Total             109,355                        56,941
                  FY2003-04 Total              93,541                        60,862
                  FY2002-03 Total             101,661                        62,681




Table 6 shows the percent change and direction of change in number served over a five-year and a
one-year period, at the end of FY 2006-07. For the five-year analysis, three outputs show a 44.1
percent decrease for adults in mental health crisis, mainly due to change in the definition of this target
population group as indicated above. In the children’s substance abuse program, the number served
dropped 15.4%; this is mainly due to change in the data reporting requirements as indicated above. As
a result, the overall number of persons served in substance abuse has decreased by 2.1 percent.

For the one-year analysis in Table 6, two mental health outputs show a decrease: in children’s mental
health, the number of children with serious emotional disturbance (SED) served decreased by 7.5
percent and in adult mental health, the number of community forensic clients served decreased by 14.5
percent. These decreases are related to recent Medicaid managed care reforms, which reduced the
requirement for submission of data on clients whose services are funded by Medicaid through various
Health Plans, including Health Maintenance Organizations and Prepaid Behavioral Health Plans.




5
  The number served of 102,345 does not include clients served in two programs: Access to Recovery and the
Brief Intervention and Treatment for the Elderly (BRITE) grant. If numbers served by those two programs are
included, the total number of clients served would be 108,924.
                                                                                                        Page 9
                Compliance with Annual Performance Outcome Standards - FY 2006-07 Legislative Status Report


   Table 6: Five- and One-Year Differences in Number Served Ending with FY 2006-07

                              5 Years            5 Years             1 Year              1 Year
                              Percent          Direction of          Percent          Direction of
       Program Area
                              Change             Change              Change             Change
    SED                        10.5%                                 7.5%                
    ED                         31.0%                                 3.9%                
    At-Risk                    80.1%                                14.7%                
    Total CMH                  20.3%                                 2.4%                
    SPMI                       83.2%                                 7.2%                
    In Crisis                  44.1%                                11.0%                
    Community Forensic         91.1%                                14.1%                
    Total AMH                  26.0%                                 7.4%                
    MH (CMH+AMH)               24.0%                                3.8%                 
    Total CSA                  15.4%                                2.1%                 
    Total ASA                   6.0%                                5.3%                 
    SA (CSA+ASA)               2.1%                                 4.3%                 

The section that follows explains the method that was used to define and determine whether or not
provider contracts successfully met the GAA performance outcome standards.


VII. Method for Determining “Successful” Contract Performance
Each contract which has GAA outcome measures in Exhibit D has a minimum performance standard,
i.e., target, associated with each Target Population and Performance Measure Combination
(PopMeasure). Therefore, a contract with ten PopMeasures would have ten performance standards.
In Florida’s mental health and substance abuse contracting systems for FY 2006-07, the number of
standards per contract varies widely. Since a provider can meet one standard and not another on the
same contract, it was necessary to find a way to express the overall contract performance across all the
PopMeasures in a single number. The following are statistical procedures used to accomplish this task.
1. PopMeasure Performance Ratio (PPR) - For each PopMeasure in a contract, a PPR was
   calculated by subtracting the Performance Standard from the PopMeasure and dividing the result
   by the Performance Standard. This deviation ratio measure is zero if a contract performance is
   identical to the standard. Performance scores above the standard have positive values and
   performance scores below the standard have negative values. For one measure in Mental Health
   (M0376) desired scores are equal to or below the standard, i.e., lower scores are better. In this
   case the described deviation ratio is multiplied by -1 to reverse the direction.

   For example, the SPMI (Severe and Persistently Mentally Ill) target population has a measure
   known as Days in the Community. If the minimum standard was 350 days and the actual
   performance was 360 days, this ratio would be 0.0286, as indicated in the examples shown in
   Figure 1, below.
   (PPR = [Contract Actual - Contract Standard] / Contract Standard)




                                                                                                     Page 10
               Compliance with Annual Performance Outcome Standards - FY 2006-07 Legislative Status Report

Figure 1: Sample Step 1 - Calculation of PopMeasure Performance Ratios (PPR)

      Contract XX402

       Target                                   Performance            Actual
     Population              Measure             Standard           Performance             PPR
     AMH: SPMI      Avg. Annual Work for Pay         40                  30               -0.2500
     AMH: SPMI         Days in Community            350                 360                0.0286
     AMH: Crisis       Median LOS in CSU              3                   6               -1.0000
     CMH: SED       Percent Improved Function        64                  64                0.0000
     CMH: SED          Days in Community            350                 365                0.0429




2. State Performance Ratio (SPR) – For each PopMeasure, this ratio was calculated by dividing the
   sum of all the PPRs statewide, by the number of PPRs statewide. This is simply the average PPR
   for that PopMeasure across all contracts. For example, if 126 contracts had a PopMeasure (for the
   SPMI population) of "Days in the Community,” then the State Performance Ratio (SPR) for
   contracts serving that population would be the sum of their PPRs divided by 126. If the sum of 126
   PPRs was 110, the SPR would be 0.86. Note that each PopMeasure may only have one State
   Performance Ratio.
   (SPR = Sum of PPRs / Number of PPRs)

3. The Statewide Standard Deviation (SSD) of the SPR was then calculated for each PopMeasure
   to measure the variance between PopMeasure Performance Ratio (PPR) and State Performance
   Ratio (SPR). Simply put, the SSD is the average difference between the PPRs and the SPR; in
   technical terms, it is the square root of the variance between the PPRs and the SPR. See Figure 2,
   below, for an example of steps 2 and 3.
   (SSD = Sum of [PPR - SPR] / Number of PPRs)

Figure 2. Example Steps 2 and 3 - Calculation of SPR and SSD

           PopMeasure = SPMI / Days in
                  Community
         Contract              Performance                        Actual
         Number                  Standard                      Performance                    PPR
          XX402                     350                            360                        1.03
          ZZ201                     345                            365                        0.94
          AB690                      75                             60                        1.25
          CR304                     320                            200                        1.60
                                                                                      Total   4.82

              Total of PPRs (4.82) divided by # of PPRs (4) = State Performance Ratio (SPR)   1.20
     Average difference between the PPRs and the SPR = Statewide Standard Deviation (SSD)     0.29


4. PopMeasure Z-Score (PZ). Since performance measures have different standards based on
   different measurement scales (e.g., community days, Global Assessment of Functioning (GAF)
   scores), the Z-scores are generally used to compare values from different measurement
   instruments or scales. In this case, the Z-score was calculated for each PopMeasure in each
   contract by subtracting the State Performance Ratio (SPR) from the PopMeasure Performance
   Ratio (PPR) and dividing the result by the State Standard Deviation (SSD). A Z-Score indicates the

                                                                                                     Page 11
                     Compliance with Annual Performance Outcome Standards - FY 2006-07 Legislative Status Report

     number of standard deviations the PPR is above or below the SPR. For example, a Z-score of
     +1.00 is one standard deviation above the SPR, while a Z-Score of –1.00 is one standard deviation
     below the SPR. With a normal distribution, about 68 percent of the PPR values will lie between –
     1.00 and +1.00. Furthermore, about 95 percent of the PPR values will lie between –2.00 and +2.00
     or two standard deviations. If the actual percent of PPRs in these ranges varied significantly from
     these expected values, a scaling factor was applied to the PopMeasure to better approximate a
     normal distribution.
     (PZ = [PopMeasure Performance Ratio - State Performance Ratio] / Statewide Standard Deviation).

5. Overall Contract Performance (OCP) - Next, all the Z-scores for all the PopMeasures in the
   contract were averaged to determine the overall contract performance. The OCP provides the
   single number needed to represent the contract performance across all the different combinations
   of target populations and performance measures. The distribution of these OCPs across contracts
   was examined and determined to be approximately normal, with a standard deviation of 0.7615 and
   a mean (average) of 0.0558.
   (OCP = Sum of [PZ1 thru PZn] / Number of PopMeasures in contract

6. OCPz - To better show the magnitude of differences among OCP values, OCP scores were
   converted to standard scores with a mean of 0 and a standard deviation of 1. It was decided that a
   contract with an OCPz value at or above one standard deviation below the mean (–1.00) was
   considered to have successful performance. This method of “grading to the curve” helps
   compensate for some of the variance due to unspecified variables that often unfairly influence
   outcome performance due to the providers’ inability to control them.
   (OCPz = (OCP-Average OCP)/Number of OCP’s).

     See Figure 3, below, for an example of Steps 4 and 5.

Figure 3: Example of Steps 4 and 5 - Calculation of PZ and OCP for a Single Contract with a
          Sample Data Set

      Contract XX402


     Target               Performance        Performance                         Actual
   Population               Measure           Standard                        Performance                 PPR   PZ
     SPMI             Days in Community          350                              360                        -0.429
                                                                                                         0.0286
     SPMI           Avg. Annual Work for Pay      40                               30                        -0.140
                                                                                                        -0.2500
      SED             % Improved Function         64                               64                    0.0000-1.06
      SED             Days in Community          350                              200                        -0.959
                                                                                                        -0.4286
                                                                                                        Sum -1.600
                                                              Overall Contract Performance (OCP) Average -0.400
                                                                                              Mean of OCP = 0.0527
                                                                                 Standard Deviation of OCP = 0.7446
                                                          Final Scaled Overall Contract Performance (OCPz) -0.608


Because the OCPz of –0.608 in this sample data set is above -1.00 (using 1.00 as the hypothetical standard deviation and 0 as the
hypothetical mean for z-scores), contract XX402 passed the performance compliance test.


Summary of acronyms used in this method, the terms to which they apply, and how they relate:
PopMeasure = Target Population and Performance Measure Combination (defined by the M-Code)
PPR     = PopMeasure Performance Ratio
          calculated for each PopMeasure in a contract by dividing the Actual Performance by the Target


                                                                                                                                    Page 12
                         Compliance with Annual Performance Outcome Standards - FY 2006-07 Legislative Status Report
SPR          = State Performance Ratio
               the average PPR for a specific PopMeasure across all contracts, statewide
SSD          = Statewide Standard Deviation
               the average difference between the PPR and the SPR for each PopMeasure across all contracts
PZ           = PopMeasure Z-Score
               calculated for each PopMeasure in a contract by subtracting the SPR from the PPR and dividing by the
               SSD
OCP          = Overall Contract Performance
               a single score for each contract that is the average of the PZs in the contract
OCPz = Overall Contract Performance
       represented as a standardized score with a mean of zero and a standard deviation of one.


VIII. Performance Results

As shown in Table 7, there were 280 SAMH contracts with performance data and minimum standards that permit
the calculation of success indicators in FY 06-07. Of these 280 contracts, 259 met the level of success defined
above in the Method section. As indicated in the footnote, however, two (2) of these contracts were duplicate
(e.g., were statewide contracts reported in multiple districts). Therefore, the unduplicated number of contracts
accountable for General Appropriation Act (GAA) outcome measures during Fiscal Year 2006-2007 is 278. Of
these 278 unduplicated contracts, 257 contracts met the compliance test, for a 92.5% success rate. Table 7 also
provides trend data on the number of successful and unsuccessful contracts per district in FY 2003-04 through FY
2006-07.


                                   Table 7: Contract Performance Results for FY 2006-07
                                                                 UNSUCCESSFUL                                SUCCESSFUL
                  SAMH contracts that require
                                                               Contracts with Overall                   Contracts with Overall
    District     outcome measures, have GAA
                                                         Performance that was below the –          Performance that was above or at
                  standards and outcome data
                                                              1.00 OCP Z-score cutoff                the –1.00 OCP Z-score cutoff
               FY03-04   FY04-05    FY05-06   FY0607     FY03-04   FY04-05   FY05-06    FY0607     FY03-04    FY04-05   FY05-06     FY0607
       1          4         6         11        4          0         0          1          1          4          6          10         3
       2          8        14         19        18         0         1          2          1          8          13         17         17
       3          8         8         7         8          0         0          1          0          8          8          6          8
       4         22        15         41        31         1         0          3          3          21         15         38         28
       7         20        24         27        30         2         3          1          2          18         21         26         28
       8         14         5         19        13         0         0          1          1          14         5          18         12
       9         18        28         25        23         1         1          0          2          17         27         25         21
      10         33        33         33        31         5         2          3          5          28         31         30         26
      11         30        37         65        38         1         7          6          3          29         30         59         35
      12         11        13         14        12         0         0          0          0          11         13         14         12
      13          4         7         9         5          0         0          1          0          4          7          8          5
      14         15        10         19        14         1         1          3          0          14         9          16         14
      15         28        23         19        15         0         1          0          1          28         22         19         14
      SC         38        36         45        38         0         0          3          2          38         36         42         36
                                                                                                                                            7
                                                                                                     242        243        328     259
                                                     6
     Total       253       259       353       280         11        16         25         21      (96% of    (94% of    (93% of (92.5% of
                                                                                                     253)       259)       353)     280)



6
  These 280 contracts include 2 duplicate contracts reported in multiple districts. The unduplicated number of contracts accountable for GAA
performance standards is 278.
7
  These 259 contracts include 2 duplicate contracts reported in multiple districts. The unduplicated number of successful contracts is 257.


                                                                                                                                    Page 13
                  Compliance with Annual Performance Outcome Standards - FY 2006-07 Legislative Status Report



IX. District Corrective Actions Taken for Unsuccessful Contracts

This section provides information on actions taken concerning each of the 21 contracts that did not
meet the cutoff for success in FY 2006-07. Table 8 lists the 21 unsuccessful contracts per district and
shows the number of standard deviations below the mean for each contract. The last column in this
table provides comments on contract performance for each GAA performance outcome measure.


            Table 8: District Comments for 21 Unsuccessful Contracts (FY 2006-07)

                        OCPZ -      Contract
                                               # of Pop.   # of Failed
                       Standard      Rank
            Contract                           Measures       Pop
 District
              ID
                       Deviations   (Lowest                                           District Comments
                                                   in       Measures
                       Below the       is
                                               Contract    (PZ<-1.00)
                         Mean        Worst)
                                                                         This program serves pregnant women and
                                                                         women with children under age 3, many of
  D01       AD384      -1.34283       15          2            1         whom do not or cannot work. Discharge is
                                                                         often shortly after the birth of a child, when
                                                                         the mothers are still unemployed.
  D02       BHS05      -4.08705        1          4            2         Medicaid-only contract. Terminated
                                                                         Provider is working on ways to improve %;
  D04       DD027      -1.06971       18          3            1         SA treatment traditionally more focused on
                                                                         preventing relapse, not employment.
                                                                         Contract manager is working with the
                                                                         provider to identify problems. The
                                                                         Contract Oversight Unit had a site visit
                                                                         recently and pointed out this performance
  D04       SDH22      -1.65455       11          4            2         level. Preliminary response given by the
                                                                         provider was internal problems within their
                                                                         system. Exit conference is scheduled for
                                                                         next week and provider will give more
                                                                         detailed response.
                                                                         Corrections are being made to improve
  D04       SDH26      -1.47989       12          2            1         compliance.
  D07       GHF13      -2.43829        7          4            3         Agency no longer holds a DCF contract.
                                                                         Agency is contracted for residential cost
                                                                         centers and therefore will have very low
                                                                         community days, for percent improved the
                                                                         agency has previously struggled due to
  D07       SGHG3      -1.74109       10          4            1         internal issues e.g. staff layoffs and re-
                                                                         alignment, modifications to treatment
                                                                         programs. Agency has improved monthly
                                                                         and now meets this target.
                                                                         A corrective action plan will be requested.
                                                                         A New data system conversion has
                                                                         impacted the provider's ability to enter
  D08       HH031      -1.06804       19          11           4         valid data - region is continuing to aid in
                                                                         assisting the provider, Less than 25% of
                                                                         those children served were sampled in this
                                                                         measure.
                                                                         Ongoing programmatic issues due to lack
                                                                         of provider staffing levels, problematic
                                                                         client case monitoring, follow-up and wait
  D09        IH509     -1.00343       21          2            1         list management continue to effect client
                                                                         continuity of care. A Request for Proposal
                                                                         (RFP) was issued for services and has
                                                                         been awarded to a new contractor.

                                                                                                             Page 14
                 Compliance with Annual Performance Outcome Standards - FY 2006-07 Legislative Status Report

           Table 8: District Comments for 21 Unsuccessful Contracts (FY 2006-07)

                       OCPZ -      Contract
                                              # of Pop.   # of Failed
                      Standard      Rank
           Contract                           Measures       Pop
District
             ID
                      Deviations   (Lowest                                          District Comments
                                                  in       Measures
                      Below the       is
                                              Contract    (PZ<-1.00)
                        Mean        Worst)
                                                                        Agency was acting as single point of
                                                                        access and legal guardian for severely
                                                                        difficult adolescent cases for the District.
                                                                        Pilot program for Dual Case management
                                                                        keep cases open and incorrectly reported
 D09        IH547     -1.47747       13          4            2         0 days in the community for children who
                                                                        were involved with the Department of
                                                                        Juvenile Justice (DJJ). The
                                                                        implementation of the program was
                                                                        deemed inefficient and expensive and has
                                                                        since been closed.
                                                                        Provider reports data upload/acceptance
 D10       JD214      -2.11682        8          3            3         issues. Meeting scheduled for further
                                                                        analysis.
                                                                        Clients with significant criminal justice
 D10       JD225      -2.81282        6          3            3         involvement. Meeting scheduled with
                                                                        provider for in depth review of findings
                                                                        Should be N/A -- No longer a contracted
 D10       JH202      -1.92489        9          2            2         provider
                                                                        Some of the measures for this contract
                                                                        should have been not applicable for the
 D10       JH205      -3.74444        4          4            4         target;2Q Improvement Plan to be
                                                                        developed
                                                                        Meeting scheduled with provider to
 D10       JH212       -3.1319        5          3            1         develop plan of action; Meeting set up with
                                                                        provider to address the issues
                                                                        FY 06-07 data is under continued analysis.
                                                                        Meeting to be held with provider to identify
 D11       KH153      -1.33419       16          1            0         any reporting or clinical issues, that impact
                                                                        upon outcome in order to improve FY 07-
                                                                        08 performance.
                                                                        Florida Assertive Community Treatment
                                                                        (FACT) clients have a multitude of needs
                                                                        that make them more likely to require
                                                                        acute care. Met with provider and
                                                                        discussed action plan/countermeasures.
                                                                        FACT clients face job matching issues.
                                                                        Clients decline being employed because
                                                                        they fear losing their benefits. Provider not
 D11       KH154      -1.42504       14          3            1         capturing all client employment. Provider
                                                                        data collection issues. Met with provider
                                                                        discussed identified issues and requested
                                                                        action plan. There are discrepancies in the
                                                                        data reported by provider. Individuals
                                                                        coded as "forensic" do not accurately
                                                                        reflect the target population as defined on
                                                                        the mental health outcome form.
                                                                        Sandy Pines is a State Inpatient
                                                                        Psychiatric Program (SIPP) provider and,
 D11       KH959      -3.92555        3          2            1         therefore, as a residential program they
                                                                        have no days in community.
                                                                        Fourteen (14) out of 25 clients were
                                                                        reported in July and August. No additional
 D15       ZH252      -1.06089       20          2            1         data has been submitted since August
                                                                        2006. This 60 day snapshot is not great
                                                                        enough of an interval to accurately

                                                                                                            Page 15
                  Compliance with Annual Performance Outcome Standards - FY 2006-07 Legislative Status Report

            Table 8: District Comments for 21 Unsuccessful Contracts (FY 2006-07)

                        OCPZ -      Contract
                                               # of Pop.   # of Failed
                       Standard      Rank
            Contract                           Measures       Pop
 District
              ID
                       Deviations   (Lowest                                          District Comments
                                                   in       Measures
                       Below the       is
                                               Contract    (PZ<-1.00)
                         Mean        Worst)
                                                                         represent the provider’s performance in
                                                                         this measure. The provider is aware of
                                                                         issue and is working toward correction of
                                                                         this matter.
   SC       QH6A1      -1.08153       17          4            1         No Contract for FY 07-08
                                                                         A corrective action plan will be requested.
                                                                         Technical assistance will be provided.
   SC       QH6E8       -3.9513        2          1            1         Provider entered data incorrectly into
                                                                         system. Provider performance should
                                                                         meet target.

Although these contracts were unsuccessful due to their poor performance on some outcome
measures, many of them met or exceeded the target or were within more than 90 percent of the target
on other outcome measures.

For all unmet GAA outcome measure targets, the SAMH Program Office is working with districts and
local providers to perform detailed analyses of these outcome measures to determine the root causes
and drivers needed to implement the actions necessary to improve performance. This corrective action
is being implemented and monitored as part of the Performance Management Teams process.


X. Discussion and Recommendations


1. There were 494 SAMH contracts recorded in the DCF Schedule of Allotment Balances for FY 2006-
   2007; only 280 of these contracts met the conditions for contracts that are accountable for GAA
   performance outcome measures in FY 2006-2007. However two (2) of these 280 contracts were
   duplicate, being reported in multiple districts. Of the 278 unduplicated contracts accountable for
   GAA outcome measures in Fiscal Year 2006-2007, 257 met the compliance test, for a 92.5%
   success rate.
2. As shown above in Table 8, districts provided comments and recommendations regarding the
   corrective actions for the 21 contracts that failed to meet the GAA performance standards. The
   SAMH Program Office is working with districts and local providers to implement and monitor these
   corrective action plans as part of the Performance Management Teams process, which includes
   regular conference call meetings with district data liaisons, DCF Contract Oversight Unit staff, and
   district contract managers to discuss pending performance compliance problems and pertinent
   corrective action plans.
3. Some contracts had no performance data reported in the SAMH data system, because the contract
   identification numbers were invalid or because they were not required to submit data. Other
   contracts had performance data, but these contracts did not have any outcome measures or
   standard targets in the Measures table of the SAMH contract database. As a result, the SAMH
   Program Office has developed online Exception Reports, which are available to district contract
   managers to ensure that the SAMH contract database is updated regularly to include all the GAA
   outcome measures and targets for all the contracts that have these statutory data requirements.
   These reports are also available to providers to identify and track contracts in the SAMH contract
   database that are not submitting data in the SAMH data system. These online exception reports are
   accessible statewide via the DCF Intranet at
                                                                                                           Page 16
               Compliance with Annual Performance Outcome Standards - FY 2006-07 Legislative Status Report

   http://ewqas1.dcf.state.fl.us/PDADM/SAMHRpts/frmMenuExceptRpts.asp.
   The SAMH Program Office is developing additional exception reports to monitor both the contracts,
   which do not submit performance data in accordance with their contractual requirements and
   providers, who do not have performance target data in Exhibit D contained in the contract database.
4. The need to establish linkages between performance measures and their targets to the types of
   services or programs and to the types of clients served is still a concern of many districts and
   providers. To meet this need, DCF and the Florida Mental Health Institute (FMHI) at the University
   of South Florida in Tampa have been analyzing the drivers of substance abuse and mental health
   outcomes. A preliminary analysis of some risk factors associated with completion of treatment
   outcomes has been completed for substance abuse. The plan is for the Performance Management
   Teams to continue this analysis in FY 2007-08 by examining other risk factors and by expanding
   the study to other substance abuse and mental health performance measures.
5. The Department already has data sharing agreements with the Florida Department of Juvenile
   Justice (DJJ) and the Florida Department of Law Enforcement (FDLE) to obtain data involving
   persons arrested after their discharges from substance abuse programs. The plan in FY 2007-08 is
   to continue this collaboration and to extend it to clients served in mental health programs.
6. The current data system infrastructure is very inadequate in terms of hardware and software
   needed by system users to access data for various ad hoc and standard reports. As a result, the
   SAMH Program Office submitted a Legislative Budget Request (LBR) in the amount of $1.8 million
   to meet this need in FY 2008-2009. Pending approval of this budget request, the SAMH Program
   Office will use existing resources to develop standard and ad hoc reports which will be available to
   SAMH users by June 30, 2008.




                                                                                                  Page 17

								
To top