Patient Satisfaction and Administrative Measures as Indicators of the

Document Sample
Patient Satisfaction and Administrative Measures as Indicators of the Powered By Docstoc
					Patient Satisfaction and Administrative
Measures as Indicators of the
Quality of Mental Health Care
Benjamin G. Druss, M.D., M.P.H.
Robert A. Rosenheck, M.D.
Marilyn Stolar, M.A.

Objective: Although measures of consumer satisfaction are increasingly                    abedian (3), the most widely cited au-
used to supplement administrative measures in assessing quality of                        thor on the topic of health care quali-
care, little is known about the association between these two types of in-                ty, concedes that it is not clear
dicators. This study examined the association between these measures                      whether “quality is a single attribute,
at both an individual and a hospital level. Methods: A satisfaction ques-                 a class of functionally related attributes,
tionnaire was mailed to veterans discharged during a three-month pe-                      or a heterogeneous assortment.”
riod from 121 Veterans Administration inpatient psychiatric units; 5,542                     Data for assessing health care qual-
responded, for a 37 percent response rate. These data were merged                         ity can be obtained from a variety of
with data from administrative utilization files. Random regression                        sources, including clinical charts, ad-
analysis was used to determine the association between satisfaction and                   ministrative records, direct observa-
administrative measures of quality for subsequent outpatient follow-up.                   tion of the patient-provider interac-
Results: At the patient level, satisfaction with several aspects of service               tion, outcome questionnaires, and pa-
delivery was associated with fewer readmissions and fewer days read-                      tient surveys (4). Most performance-
mitted. Better alliance with inpatient staff was associated with higher                   monitoring systems rely primarily on
administrative measures of rates of follow-up, promptness of follow-up,                   administrative measures for quality
and continuity of outpatient care, as well as with longer stay for the ini-               assessment because of the availability
tial hospitalization. At the hospital level, only one association between                 of data and low cost of data collection
satisfaction and administrative measures was statistically significant.                   (5,6). However, these data are in-
Hospitals where patients expressed greater satisfaction with their al-                    creasingly being supplemented by in-
liance with outpatient staff had higher scores on administrative mea-                     formation derived directly from con-
sures of promptness and continuity of follow-up. Conclusions: The asso-                   sumers, usually in the form of patient
ciations between patient satisfaction and administrative measures of                      satisfaction surveys. Although mea-
quality at the individual level support the idea that these measures ad-                  suring satisfaction requires primary
dress a common underlying construct. The attenuation of the associa-                      data collection and is thus more cost-
tions at the hospital level suggests that neither type can stand alone as                 ly and time consuming than obtaining
a measure of quality across institutions. (Psychiatric Services 50:1053–                  administrative measures, purchasers
1058, 1999)                                                                               increasingly regard satisfaction ques-
                                                                                          tionnaires as an essential complement
                                                                                          to administrative measures of health

      apid and sweeping changes in            quality of health care can be assessed      care quality (7,8).
      the U.S. health care system             on the basis of structure (characteris-        Both of these sources of data can
      have fueled a growing interest          tics of providers and hospitals),           describe both the process and the
among providers, purchasers, and              process (components of the en-              outcomes of care. For instance, mea-
consumers in understanding and                counter between provider and pa-            sures of satisfaction can provide infor-
measuring quality of health care (1,2).       tient), and outcomes. However, a pre-       mation on treatments as well as a con-
A general consensus exists that the           cise definition remains elusive. Don-       sumer perspective on the success of
                                                                                          those treatments. Administrative data
                                                                                          can provide information about num-
The authors are affiliated with the Veterans Administration Northeast Program Evalua-     ber of visits, which is a process mea-
tion Center, 950 Campbell Avenue, West Haven, Connecticut 06516 (e-mail,                  sure, or about readmission, which is a Dr. Druss and Dr. Rosenheck are also with the departments       common outcome measure for inpa-
of psychiatry and public health at Yale University in New Haven, Connecticut, where Ms.   tient care.
Stolar is with the department of biostatistics.                                              However, each type of measure is
PSYCHIATRIC SERVICES   o August 1999 Vol. 50 No. 8                                                                              1053
prone to certain shortcomings. Pa-          care system to examine the associa-         measures into an independent-de-
tient satisfaction surveys may be sub-      tion between two types of measures          pendent categorization, the study
ject to nonresponse bias; that is, con-     of health care quality—consumer sat-        treated each as correlative indicators
sumers who respond to health surveys        isfaction and administrative mea-           of the underlying construct of quality
may differ from those who do not.           sures. The administrative measures          of care. Patients completed the satis-
Recall bias—when consumers do not           were chosen to use existing VA elec-        faction questionnaire after the index
accurately recall information about         tronic sources of data while maximiz-       hospitalization. Data on subsequent
their care—is also a potential prob-        ing comprehensiveness, validity, and        readmissions and outpatient care
lem. Administrative measures, al-           reliability (16). A previous study iden-    were gathered after the questionnaire
though less prone to these forms of         tified a number of individual patient       was completed.
bias, may be a less sensitive measure       characteristics that were significantly
of health care process than consumer-       associated with satisfaction with men-      Sample
derived indicators (9).                     tal health care (17).                       The sample was drawn from respon-
   Despite the inclusion of both ad-           The purpose of this study was to ex-     dents to a nationwide VA satisfaction
ministrative and consumer-derived           amine three questions: What is the          survey that was sent to a random sam-
indicators in performance-monitor-          association between patient satisfac-       ple of inpatients discharged to the
ing systems, little is known about the      tion measures and administrative            community from VA medical centers
relationship between these two types        measures of plan quality for individ-       between June 1 and August 31, 1995
of measures (10). For mental health                                                     (17). Patients discharged to nursing
populations, even less information ex-                                                  homes were excluded because follow-
plicating the role of patient-based                                                     up is generally provided in those set-
measures in assessing quality of care                                                   tings rather than the VA. The subsam-
is available (11). Many satisfaction                                                    ple chosen for this study included vet-
subscales directly parallel administra-                                                 erans with psychiatric diagnoses
tive measures of plan quality. For in-                                                  (ICD-9 codes 295.00 to 302.99).
stance, most surveys ask consumers                                                         Thirty-seven percent of individuals
                                                systems rely primarily
about access to care, while simultane-                                                  who were sent questionnaires mailed
ously measuring access by examining                                                     back responses, with a range of 24
                                            on administrative measures
use of outpatient services. However,                                                    percent to 69 percent across partici-
it is not known whether the con-                                                        pating hospitals. In this sample of in-
                                                for quality assessment
structs assessed by these two types of                                                  dividuals with psychiatric diagnoses,
measures are identical, partly related,                                                 respondents were somewhat more
                                             because of the availability
or wholly distinct. A better under-                                                     likely than nonrespondents to be old-
standing of the relationship between                                                    er, female, married, and white, and
                                                 of data and the low
administrative data and satisfaction                                                    less likely than nonrespondents to
data may help guide the selection of                                                    have psychotic or substance use dis-
                                                       cost of data
measures to be included in perfor-                                                      orders (17).
mance-monitoring systems.
   A second issue that arises when as-                                                  Questionnaire
sessing quality of care is that although                                                Data on satisfaction were collected
data are collected at the level of the                                                  using a method based on a four-step
individual patient, comparisons of                                                      procedure designed to maximize re-
quality are generally conducted at the      ual patients? Are there differences in      sponse rates (18). A total of 73 ques-
level of the provider or hospital. A lit-   the relationship between the two            tions addressed ten domains of gener-
erature has emerged to relate satis-        types of measures when examined on          al quality of service delivery and four
faction to individual consumer expe-        the individual versus the hospital lev-     domains of alliance with inpatient
riences and behavior (12,13) as well        el? Is it necessary to include both         staff. The ten general-quality do-
as outcomes of care (14), but few           types of data when evaluating the per-      mains were coordination of care,
studies have examined the use of sat-       formance of health care providers?          sharing of information, timeliness and
isfaction measures to compare quality                                                   accessibility, courtesy of staff, emo-
across different hospitals or providers     Methods                                     tional support, attention to patient
(10). Report cards rate plans or            The study used a cross-sectional de-        preferences, family involvement,
providers on the basis of mean satis-       sign to assess the association between      physical comfort, transition to outpa-
faction scores across groups of pa-         data about satisfaction with inpatient      tient status, and overall quality of
tients rather than on the ratings of in-    psychiatric hospitalization, which was      care. The alliance domains were
dividual patients (15).                     obtained by a questionnaire, and ad-        sense of energy or engagement on the
   The study reported here used data        ministrative data about the index hos-      unit, practical problem orientation of
compiled for a national mental health       pitalization and care during the six        the staff, alliance with clinician, and
program monitoring system recently          months after discharge. Rather than         overall satisfaction with mental health
implemented within the VA health            fitting satisfaction and administrative     services.
1054                                                                            PSYCHIATRIC SERVICES   o August 1999 Vol. 50 No. 8
  All of the subscales had Cronbach’s         Summary components                        in 180 days of discharge and total days
alpha values of .6 and above, indicating      To better understand the underlying       readmitted within 180 days. The sec-
adequate internal reliability (17). The       relationship between the large num-       ond inpatient measure, early read-
questionnaire was developed from              ber of satisfaction and administrative    mission, included readmission within
other well-established instruments            measures, principal components            14 or 30 days. The third measure was
(19–21). The concordance between              were derived from each set of vari-       length of stay, composed of that single
the content of the subscales and those        ables using the SAS factor procedure,     measure.
used in other studies suggest appropri-       with varimax rotation. Components            The first outpatient component,
ate content validity—that is, the sub-        with eigenvalues of one or greater        promptness-continuity of outpatient
scales reflect widely accepted domains        were retained; scree plots confirmed      follow-up, comprised three measures:
of consumer satisfaction.                     this cutoff as appropriate.               days until first outpatient visit, follow-
                                                 The principal-components analysis      up within 30 days of discharge, and
Administrative measures                       of satisfaction variables revealed two    number of two-month periods in the
Demographic data, diagnostic infor-           components that explained a total of      time after discharge with at least two
mation, and other administrative              61.8 percent of the variance of the       visits. The second component, any
measures were derived from two na-            satisfaction subscales. The first,        outpatient follow-up, included the
tional VA files—the patient treatment         termed general service delivery, in-      single variable connoting any visit
file, a comprehensive discharge ab-           cluded nine of the ten subscales de-      within 180 days of discharge. Cron-
stract of all inpatient episodes of VA        rived from the general survey. The        bach’s alpha scores for these compos-
care, and the outpatient file, a nation-                                                ite administrative variables ranged
al electronic file documenting all VA                                                   from .80 to .94, again reflecting
outpatient service delivery. Each                                                       strong internal coherence of these
questionnaire contained a code that                                                     components.
could be linked to a unique patient                          More
identifier (an encrypted Social Secu-                                                   Statistical methods
rity number), which in turn was used                       satisfied                    Random regression, also known as hi-
to merge satisfaction data with the in-                                                 erarchical linear modeling, a tech-
patient and outpatient data.                           consumers may                    nique designed for models in which
   Several administrative measures                                                      individual measurements are clus-
constructed for a national VA mental                 have better outcomes               tered into larger groups sharing com-
health performance-monitoring sys-                                                      mon characteristics, such as hospitals,
tem were used for comparison with               after discharge, reducing               was used for all multivariate models
the satisfaction measure. Inpatient                                                     (24,25). This type of analysis was re-
measures included length of stay;                      the likelihood of                quired because of the lack of inde-
readmission within 14, 30, or 180                                                       pendence among observations—that
days; and time until readmission.                     rehospitalization.                is, because individuals within the
Outpatient measures included fol-                                                       same hospital cannot be considered
low-up within 30 days or 180 days af-                                                   as independent observations drawn
ter discharge, days until first outpa-                                                  from the target population. Random
tient mental health follow-up (among                                                    regression allows comparisons to be
those with ambulatory follow-up),             second component, termed alliance         made at two distinct levels—patient
and number of two-month periods af-           with inpatient staff, included all four   and hospital—without a loss of statis-
ter discharge with at least two mental        alliance subscales. These components      tical power, because all models use
health or substance abuse outpatient          parallel the technical aspects of care    the same sample size (26). Missing
visits. These indicators have been            (delivery of services) and the inter-     values were replaced with the mean
shown to identify a substantial range         personal aspects that have been           value of the hospital where the pa-
of variation across hospitals with rela-      found to underlie a number of satis-      tient was treated so that each analysis
tively little redundancy (16).                faction measures (23). Cronbach’s al-     was based on 5,542 subjects. The SAS
                                              pha values for these components           MIXED procedure was used for all
Potential confounders                         were .94 for the first and .82 for the    random regression analyses.
Demographic and diagnostic data               second, indicating strong internal co-       Two levels of analyses were con-
were obtained to control for case mix         herence of these constructs.              ducted. The first, which measured
in the multivariate analyses. These              The principal-components analysis      the associations between satisfaction
variables included age, race, gender,         of administrative variables revealed      and other performance measures at
income, marital status, severity of           five separate components—three in-        the level of the individual patient, in-
medical illness using total number of         patient and two outpatient—which          cluded a random intercept term to ac-
medical diagnoses as a proxy (22), and        explained a total of 84.9 percent of      count for potentially correlated errors
psychiatric diagnosis, which was re-          the total variance. The first inpatient   attributable to similarities among pa-
ported as one of five dichotomous             component, readmission intensity, in-     tients treated at the same hospital.
variables.                                    cluded measures of readmission with-      The second level of analyses mea-
PSYCHIATRIC SERVICES   o August 1999 Vol. 50 No. 8                                                                           1055
Table 1                                                                                                 common psychiatric diagnoses were
                                                                                                        schizophrenia (1,237 veterans, or 25.3
Association between satisfaction with inpatient care and administrative measures
of the quality of care among 5,542 veterans responding to a satisfaction survey1                        percent), affective disorder (836 vet-
                                                                                                        erans, or 17.1 percent), and posttrau-
                                  Individual level (N=5,542)           Hospital level (N=5,542)         matic stress disorder (776 veterans, or
                                                                                                        14 percent).
                                  Satisfaction measure                 Satisfaction measure
                                                                                                        Associations between satisfaction
                                  General        Alliance with         General       Alliance with
                                  service        inpatient             service       inpatient          and administrative variables
Administrative measure            delivery       staff                 delivery      staff              Table 1 presents the associations be-
                                                                                                        tween the summary satisfaction and
Inpatient care                                                                                          administrative variables. Each cell
  Readmission intensity2          –.046∗         .003                  –.007          .006
  Early readmission3
                                                                                                        contains a standardized regression co-
                                   .014          .014                   .137          .003
  Length of stay                   .027          .078∗                  .024         –.007              efficient; analyses adjusted for race,
Outpatient care                                                                                         gender, marital status, income, num-
  Promptness and continu-                                                                               ber of medical diagnoses, hospital,
     ity of follow-up              .005          .077∗                 –.002          .040∗             and psychiatric diagnosis.
  Any outpatient follow-up        –.012          .067∗                  .012         –.013
                                                                                                           At the level of the individual, satis-
1 The values are standardized regression coefficients; the analyses controlled for race, gender, mar-   faction with general quality of service
  ital status, income, number of medical diagnoses, hospital, and psychiatric diagnosis. Individual-    delivery was associated with de-
  level analyses examined whether for a given veteran, improved satisfaction was associated with        creased intensity of readmission—a
  higher ratings on administrative measures. Hospital-level analyses examined whether hospitals
  with higher satisfaction ratings also performed better on administrative measures of quality.
                                                                                                        measure derived from both the likeli-
2 Included measures of readmission within 180 days of discharge and total days readmitted within        hood of readmission and the number
  180 days                                                                                              of days readmitted after discharge.
3 Within 30 days
∗ p<.0017
                                                                                                        Better alliance with inpatient staff was
                                                                                                        significantly associated with a greater
                                                                                                        likelihood of outpatient follow-up,
                                                                                                        promptness of follow-up, and conti-
sured the associations between mean                  of magnitude across differing vari-                nuity of follow-up, as well as longer
satisfaction and other performance                   ables, represents an approximation of              length of stay for the index admission.
measures at the hospital level, with                 an r value.                                           Hospitals where patients expressed
each model using a random intercept.                    All dependent variables were                    greater satisfaction with their alliance
Thus the former set of analyses exam-                checked for normality of distribution,             with outpatient staff also had higher
ined whether for a given veteran im-                 and all variables found not to be nor-             scores for promptness and continuity
proved satisfaction would be associat-               mally distributed were appropriately               of follow-up. No other associations
ed with higher ratings on administra-                transformed. Because length of stay                between satisfaction and administra-
tive measures, such as an increased                  remained highly skewed after log                   tive measures were significant.
likelihood of follow-up after dis-                   transformation, it was converted into
charge. The latter set of analyses ex-               a five-level integer variable: less than           Discussion and conclusions
amined whether hospitals with higher                 eight days, eight to 14 days, 15 to 28             This study is the first that we are
satisfaction ratings also performed                  days, 29 to 60 days, and greater than              aware of to examine the association
better on administrative measures of                 60 days. Because of multiple compar-               between the two types of indicators
quality.                                             isons, the Bonferroni method was                   most commonly used in mental
   Each model adjusted for hetero-                   used to adjust the critical p value for            health performance-monitoring sys-
geneity of patient caseloads by enter-               statistical significance to .05 divided            tems, administrative measures and
ing terms into the model pertaining to               by 30, or .0017.                                   patient satisfaction. At the level of the
demographic and diagnostic charac-                                                                      individual patient, a number of mea-
teristics of patients or hospital case-              Results                                            sures of satisfaction with inpatient
loads. Each association controlled for               Characteristics of the sample                      care were significantly associated
age, race, gender, marital status, in-               A total of 5,542 veterans from 121                 with increased likelihood of outpa-
come, number of medical diagnoses,                   hospitals responded to the survey.                 tient follow-up, promptness of follow-
and psychiatric diagnosis. The magni-                Reflecting the veteran population                  up, and continuity of outpatient care,
tude of each association was calculat-               from which the sample was drawn,                   as well as reduced likelihood of read-
ed as a standardized regression coef-                the population was largely male                    mission. However, these relationships
ficient that represents the number of                (5,278 veterans, or 94.6 percent),                 became highly attenuated when hos-
standard deviations of change in the                 white (3,895 veterans, or 70.3 per-                pitals rather than individuals were the
outcome of interest per standard de-                 cent), and poor, with a mean±SD an-                unit of comparison, despite the use of
viation of change in the explanatory                 nual income of $9,583±$4,499. The                  an analytic method that preserved
variable. The standardized regression                mean±SD length of inpatient stay                   sample size and statistical power and
coefficient, which allows comparisons                was 13.2±52.22 days. The three most                adjusted for case mix.
1056                                                                                            PSYCHIATRIC SERVICES   o August 1999 Vol. 50 No. 8
Limitations                                   by readmission and other outpatient        hospitals were compared. With the
Several limitations exist in each             indicators—to guide our hypotheses         exception of the link between alliance
source of data. Study of patient satis-       about causality.                           with inpatient staff and promptness-
faction in any system of care should             What is the association between         continuity of mental health follow-up,
invariably raise questions about how          patient satisfaction measures and          no significant associations were found
well the findings can be generalized          administrative measures of plan            between the satisfaction components
to other populations or financing sys-        quality for individual patients? At        and the administrative components at
tems. The population with mental              the level of the individual, better re-    the hospital level. Even though the
disorders seen in VA facilities is simi-      ported alliance with staff was a signif-   use of random regression for analyses
lar to that seen in other public-sector       icant predictor of higher rates of fol-    at the hospital level preserved the
settings, although VA has a lower pro-        low-up and promptness and continu-         same sample size and statistical pow-
portion of women and of the very              ity of outpatient mental health care.      er as on the individual level, higher
poor.                                         The link between alliance with inpa-       scores on administrative measures
   Second, despite the use of an es-          tient staff and successful outpatient      were no more likely for hospitals with
tablished mail out–mail back method,          follow-up suggests that a positive pa-     more satisfied patients than for those
the survey response rate for this sam-        tient-provider relationship in inpa-       with less satisfied patients. This find-
ple was only 37 percent, a rate typical       tient psychiatric settings may be asso-    ing is of particular interest because
for mail-in questionnaires distributed        ciated with improved outcomes.             for report card systems rating quality
to seriously mentally ill subjects               Satisfaction with general service       of health care, the hospital (or health
(12,27). Past research has demon-             delivery predicted reduced likelihood      plan), rather than the individual, is
strated that telephone and in-home            of readmission and fewer days read-        the relevant unit of comparison (6,8).
surveys may help improve these re-            mitted. This latter relationship may          How can we explain the rela-
sponse rates, although only to some           ultimately be mediated by treatment        tively weak associations between
extent and at considerably increased          outcomes. More satisfied consumers         consumer-based and administra-
cost (28). Developing better methods          may have better outcomes after dis-        tive measures of quality when com-
of maximizing response rates among            charge, reducing the likelihood of re-     paring hospitals? The literature has
seriously mentally ill people is essen-       hospitalization. Whatever the mecha-       documented substantial variability in
tial both to ensure accurate measure-         nism, only the combination of satis-       quality of care not only across but also
ment of satisfaction and to better un-        faction with the general and interper-     within institutions (35). These differ-
derstand its relationship to other indi-      sonal aspects of care delivery was as-     ences are not captured when data are
cators of quality.                            sociated with higher quality as mea-       compared between hospitals. If with-
   Third, it is never possible to entire-     sured by effective use of inpatient        in a given institution, some respon-
ly adjust for differences in case mix         and outpatient services.                   dents are satisfied and give the hospi-
when comparing quality across insti-             Unexpectedly, longer stay was the       tal high performance ratings and oth-
tutions. In this study, for instance, it is   strongest positive predictor of satis-     ers are dissatisfied and give the hospi-
possible that unmeasured differences          faction with care for this sample. The     tal low performance ratings, then the
in severity of illness across institutions    finding suggests that this measure         association between satisfaction and
mediated differences both in satisfac-        may identify a point of divergence be-     administrative performance will wash
tion ratings and in quality measures          tween the consumer’s and the health        out when hospitals are compared.
across institutions. For instance, pa-        care institution’s perspective on qual-       Is it necessary to include both
tients who have more serious mental           ity of care. Rapidly declining length      types of data when evaluating the
illness might simultaneously have             of stay has been one of the hallmarks      performance of health care pro-
lower levels of satisfaction with care        of psychiatric inpatient care over the     viders? Although the relationship
(29) and worse continuity of care             past decade in both the public and         between satisfaction and other per-
(30). Developing better methods of            the private sectors (33,34). Shorter       formance measures may be attenuat-
risk adjustment for case mix is one of        stays, while valued by administrators      ed when comparisons are made at a
the most important challenges facing          and health care institutions for fiscal    hospital level, there is evidence that
performance-monitoring systems to-            reasons, may lead to dissatisfaction       satisfaction is nonetheless a valid con-
day (31).                                     for mental health consumers.               struct to assess as a measure of quali-
   Finally, causal statements about the          Are there differences in the re-        ty across institutions. Pilot data from
relationship between satisfaction and         lationship between the two types           this survey (17,36) and other similar
other quality measures must always            of measures when examined on               multidimensional satisfaction scales
be made with caution. Satisfaction            the individual level and on the            (37) have demonstrated strong psy-
can simultaneously be a cause or an           hospital level? A number of associa-       chometric properties for these mea-
outcome of health utilization, and dis-       tions existed between patient satisfac-    sures. Other studies have also found
tinguishing between the two can be            tion and administrative measures for       that these subscales can consistently
difficult (32). In this study we used         a given individual. However, these         identify differences in satisfaction
the temporal sequence of events—in-           differences largely disappeared when       across hospitals (38).
dex hospitalization, followed by satis-       mean satisfaction ratings and scores          The association between the two
faction survey completion, followed           on administrative measures across          types of measures at an individual lev-
PSYCHIATRIC SERVICES   o August 1999 Vol. 50 No. 8                                                                          1057
el suggests that consumer satisfaction               13. Roghmann KG, Hengst A, Zastowny TR:              26. Koepsell TD, Martin DC, Diehr PH, et al:
                                                         Satisfaction with medical care: its measure-         Data analysis and sample size issues in eval-
and administrative measures of quali-                    ment and relation to utilization. Medical            uations of community-based health promo-
ty go hand in hand and supports the                      Care 17:461–479, 1979                                tion and disease prevention programs: a
notion that the two are measuring re-                                                                         mixed-model analysis of variance approach.
                                                     14. Kane RL, Maciejewski M: The relationship
                                                                                                              Journal of Clinical Epidemiology 44:701–
lated underlying constructs. The at-                     of patient satisfaction with care and clinical
                                                                                                              713, 1991
tenuation of the relationships at a                      outcomes. Medical Care 35:714–730, 1997
                                                                                                          27. Lebow JL: Consumer satisfaction with
hospital level points to the potential               15. McNeil BJ, Pederson SH, Gatsonis C: Cur-
                                                                                                              mental health treatment. Psychological
                                                         rent issues in profiling quality of care. In-
difficulties in using either source of                   quiry 29:298–307, 1992
                                                                                                              Bulletin 91:244–259, 1982
data as a sole indicator of quality                                                                       28. Lebow JL: Client satisfaction with mental
                                                     16. Rosenheck R, Cicchetti D: A mental health
across institutions. o                                   program report card: a multidimensional
                                                                                                              health treatment: methodological consider-
                                                                                                              ations in assessment. Evaluation Review
                                                         approach to performance monitoring in
                                                                                                              7:729–752, 1983
Acknowledgments                                          public sector programs. Community Men-
                                                         tal Health Journal 34:85–106, 1998               29. Hermann RC, Ettner SL, Dorwart RA: The
This work was partly sponsored by grants                                                                      influence of psychiatric disorders on pa-
from the National Alliance for Research              17. Rosenheck RA, Wilson NJ, Meterko M:
                                                                                                              tients’ ratings of satisfaction with health
on Schizophrenia and Depression and the                  Consumer satisfaction with inpatient men-
                                                                                                              care. Medical Care 36:720–727, 1998
Donaghue Medical Foundation.                             tal health treatment: influence of patient
                                                         and hospital factors. Psychiatric Services       30. Druss BG, Rosenheck RA: Use of medical
References                                               48:1553–1561, 1997                                   services by veterans with mental disorders.
                                                                                                              Psychosomatics 38:451–458, 1997
                                                     18. Dillman DA: Mail and Telephone Surveys:
 1. Blumenthal D: Quality of care: part 1. what
                                                         The Total Design Method. New York, Wi-           31. Iezzoni LI: The risks of risk adjustment.
    is it? New England Journal of Medicine
                                                         ley, 1978                                            JAMA 278:1600–1607, 1997
    335:891–894, 1996
                                                     19. Cleary PD, Edgman-Levitan S, Walker JD,          32. Zastowny TR, Roghmann KJ, Cafferata
 2 Chassin MR: Quality of health care: part 3:           et al: Using patient reports to improve              GL: Patient satisfaction and the use of
   improving the quality of health care. New             medical care: a preliminary report from 10           health services: explorations in causality.
   England Journal of Medicine 335:1060–                 hospitals. Quality Management in Health              Medical Care 27:705–723, 1989
   1063, 1996                                            Care 2:31–38, 1993
                                                                                                          33. Mechanic D, McAlpine DD, Olfson M:
 3. Donabedian A: The Definition of Quality          20. Moos R: Evaluating Treatment Environ-                Changing patterns of psychiatric inpatient
    and Approaches to Its Management: Vol 1:             ments: A Social Ecological Approach. New             care in the United States, 1988–1994.
    Explorations in Quality Assessment and               York, Wiley, 1974                                    Archives of General Psychiatry 55:785–791,
    Monitoring. Ann Arbor, Mich, Health Ad-                                                                   1998
    ministration Press, 1980                         21. Horvath AO, Greenberg L: Development
                                                         and validation of the Working Alliance In-       34. Druss BG, Bruce ML, Jacobs SC, et al:
 4. Brook RH, McGlynn EA, Cleary PD: Qual-               ventory. Journal of Counseling Psychology            Trends over a decade for a general hospital
    ity of health care: part 2. measuring quality        36:223–233, 1989                                     psychiatry unit. Administration and Policy
    of care. New England Journal of Medicine                                                                  in Mental Health 25:427–435, 1998
                                                     22. Melfi C, Holleman E, Arthur D, et al: Se-
    335:966–970, 1996                                    lecting a patient characteristics index for      35. Young AS, Sullivan G, Burnam MA, et al:
 5. HEDIS 3.0: Health Plan Employer Data                 the prediction of medical outcomes using             Measuring the quality of outpatient treat-
    and Information Set. Washington, DC, Na-             administrative claims data. Journal of Clin-         ment for schizophrenia. Archives of Gener-
    tional Committee for Quality Assurance,              ical Epidemiology 48:917–926, 1995                   al Psychiatry 55:611–617, 1998
    1997                                             23. Ware J, Hays RD: Methods for measuring           36. Performance on Customer Service Stan-
                                                         patient satisfaction with specific medical en-       dards: Recently Discharged Inpatients,
 6. Kenkel PJ: Report Cards: What Every                  counters. Medical Care 26:393–402, 1988              1994. West Roxbury, Mass, Veterans Health
    Provider Needs to Know About HEDIS                                                                        Administration National Consumer Feed-
    and Other Performance Measures. Gaith-           24. Bryk AA, Raudenbush SW: Hierarchical Lin-
                                                                                                              back Center, 1994
    ersburg, Md, Aspen, 1995                             ear Models. Newbury Park, Calif, Sage, 1992
                                                                                                          37. Rubin HR, Ware JE, Hays RD: The PJHQ
 7. Allen HE, Darling H, McNeil DN, et al:           25. Gibbons RD, Hedeker D, Elkin I, et al:
                                                                                                              questionnaire: exploratory factor analysis
    The Employee Health Care Value Survey:               Some conceptual and statistical issues in
                                                                                                              and empirical scale construction. Medical
    round one. Health Affairs 13(1):25–41,               analysis of longitudinal psychiatric data: ap-
                                                                                                              Care 28 (9 suppl):S22–S29, 1990
    1994                                                 plication to the NIMH Treatment of De-
                                                         pression Collaborative Research Program          38. Hays RD, Nelson EC, Rubin HR, et al:
 8. Dickey B: The development of report cards            Dataset. Archives of General Psychiatry              Further evaluations of the PJHQ Scales.
    for mental health care, in Outcomes As-              50:739–750, 1993                                     Medical Care 28(9 suppl):S29–S39, 1990
    sessment in Clinical Practice. Edited by
    Sederer L, Dickey B. Baltimore, Williams
    & Wilkins, 1996
 9. Cleary PD, McNeil BJ: Patient satisfaction
    as an indicator of quality care. Inquiry
    25:25–36, 1988
10. Rubin HR: Patient evaluations of hospital                        Change of Address
    care. Medical Care 28(9 suppl):S3–S9,
                                                                    Authors of papers under peer review or being prepared
11. McGlynn EA, Norquist GS, Wells KB, et al:
    Quality-of-care research in mental health:
                                                                    for publication in Psychiatric Services are reminded to
    responding to the challenge. Inquiry 25:                        notify the editorial office of any changes in address.
    157–170, 1988                                                   Please call the editorial office at 202-682-6070, or send
12. Ruggieri M: Patients’ and relatives’ satisfac-                  updated information by fax to 202-682-6189 or by e-
    tion with psychiatric services: the state of                    mail to
    the art and its measurement. Social Psychi-
    atry and Psychiatric Epidemiology 29:212–
    227, 1997

1058                                                                                              PSYCHIATRIC SERVICES   o August 1999 Vol. 50 No. 8

Shared By: