Hill-Lynn by langkunxg


									                    What Do We Know About Governance?
                     An Analysis of Empirical Research

                                       Carolyn J. Hill
                                Visiting Assistant Professor
                             Georgetown Public Policy Institute
                                  Georgetown University
                                    3600 N Street, NW
                                 Washington, DC 20007

                                    Laurence E. Lynn, Jr.
                      Sydney Stein, Jr. Professor of Public Management
                       Harris Graduate School of Public Policy Studies
                                  The University of Chicago
                                     1155 E. 60th Street
                                   Chicago, IL USA 60637

                          DRAFT — DO NOT QUOTE OR CITE
                                 September 30, 2001

Prepared for the 6th National Public Management Research Conference, Bloomington, Indiana,
October 18-20, 2001.
                      What Do We Know About Governance?
                       An Analysis of Empirical Research
                              By Carolyn J. Hill and Laurence E. Lynn, Jr.

        “Big questions” among policy makers are: Does a policy or program work? Is it effective?
Does it change behavior, produce promised results, save money? To be relevant to mainstream
policy discourse, the policy research community is under pressure to answer such questions
through methods such as controlled experiments, program evaluations, and benefit_cost analyses.
Answers often produce headlines and sound bites, opportunities to testify, and financial support
for further research.

        Far less attention is attracted by research addressed to what appear to be “lesser
questions”: Why does a policy work or not work? Where and for whom does it work? How
might it be replicated in different settings? The answers to such questions are more complicated,
take longer to explain, and have more confusing qualifications. Moreover, such questions are
typically harder and more costly to address because they often require richer data and advanced
analytic methods. Answers to “lesser questions” find an audience among policy makers and
aides with an analytic turn of mind and among specialists, consultants, and scholars.

        Even farther down the food chain of political salience are the kinds of questions that
fascinate students of governance and public management. Their “big questions” are: What are
the consequences for governmental performance of how public programs are organized and
managed? Does management matter and, if so, how? Are some forms of organization better
than others for accomplishing public purposes and, if so, for whom, when, and where?
Answering these kinds of questions is even more demanding of high quality data and
discriminating analytic methods. Moreover, apart from cells of management mavens tucked
away off_line, it is seldom clear who cares about the answers, who will act on them, and whether
such questions can be answered with any authority. For these reasons, fewer concentrated
resources are devoted to addressing governance and public management questions.

        Our primary goal in the research underlying this paper is to assess the state of knowledge
in public governance and management research to discover if there is anything worth knowing in
the dispersed literature that addresses governance and public management questions. Lynn,
Heinrich and Hill (2000, 233) have argued that “well_designed, theory_driven, and data_based
governance research is more likely [than case_based research] to produce enduring knowledge
about how, why, and with what consequences public_sector activity is structured and managed.”
We aim to see if they are right. Are we learning anything — or enough — to justify efforts to
muscle our way into the discourse over governmental effectiveness?

        We have an additional goal that addresses another proposition advanced by Lynn,
Heinrich, and Hill: to overcome the compartmentalized nature of governance research on behalf
of learning across substantive and disciplinary boundaries. To what extent does research
concerning the governance of welfare reform, mental health services, economic and regulatory
policy, and other substantive domains conducted by economists, sociologists, and organizational
theorists have implications for the general problem of governance?

Analytic Approach

        Our organizing framework is a logic of governance based on a hierarchy of relationships
that links collective decisions concerning public policies — what in older parlance might be
termed concrete expressions of the public will — to the operating results, outcomes or
consequences of governmental activity (Lynn, Heinrich and Hill 2001). The interrelationships
of particular interest involve citizen preferences and interests; legislated authority; formal
organizational structures; discretionary organization, management, and administration;
field_level treatments or program activities; and observable consequences, outputs, or results.
Expressed in an explicit hierarchy, these relationships occur:

•      between        (a)    citizen preferences and interests expressed politically and
                      (b)    legislative choice;

•      between        (b)    legislator preferences expressed in enacted legislation and
                      (c)    formally authorized structures and processes of public agencies;

•      between        (c)    the structure of formal authority and
                      (d     the de facto or discretionary organization and management of
                             agencies, programs, and administrative activities;

•      between        (d)    discretionary organization, management, and administration and
                      (e)    the core technologies and primary work of public agencies;

•      between        (e)    primary work and

                      (f)    consequences, outputs, or results (e.g., the availability, quality, and
                             cost of publicly sponsored goods and services);

•      between        (f)    outputs or results and

                      (g)                          stakeholder assessments of agency or
                             program performance (i.e., judgments about whether government
                             is "working" that motivate them to political action); and
                                                                                Appendix A
                                                               Governance Studies Examined


•      between         (g)     performance assessments expressed politically and
                       (a)     public and legislative interests and preferences.

        The objective of empirical governance research is to open up "black boxes" at the
various levels of governance to identify variables that have demonstrable effects at
hierarchically subordinate (or superior) levels, e.g., the effects of legislative preferences
or the features of formal structures on discretionary choices concerning organization,
management, and administration and the results that follow upon such choices. The
general questions we shall attempt to answer in our assessment of extant research are:

                               (1) Is governance research producing findings indicating
                               that governance matters?

                               (2) Does research suggest more than that, i.e, are there
                               useful substantive findings concerning what matters?

                               (3) What are the implications for theory and for research
                               agendas of the cumulative findings from governance

        To address these questions, in the first phase of our analysis, we have surveyed
over 100 published studies (listed in Appendix A). Our selection of studies includes
those that specify causal relationships between at least two levels of variables in the logic
of governance. Further, we emphasize studies in which the independent variables have
two characteristics: they represent at least partial disaggregation of larger structures, e.g.,
of organizations, programs, policies, or strategies, and their values are at the discretion of
officials at that level. We include, for example, studies of privatization that distinguish
between for_profit and nonprofit contractors in assessing program effectiveness, studies
of bureaucratic response that distinguish between state and federal units of government,
and studies that disaggregate treatment into its constituent elements or that compare
different organizations of treatment. We exclude most experimental program impact
research and studies of undifferentiated policies such as school vouchers or welfare
reform that leave the specific mechanisms of the program in a black box. Finally, we
exclude studies that involve single cases or small numbers of observations unless their
designs incorporated the possibility of disconfirming prior notions.

        Variables at each level of the above logic of governance may serve as either
dependent or independent variables in governance research. For example, in some
studies, variables associated with discretionary organization and management — level (d)
— are hypothesized to influence lower level activity (e), (f), and (g). In other studies,
variables at level (d) are hypothesized to be influenced by factors at higher levels (a), (b),
or (c). Studies also may be designed to identify influences or feedbacks on higher levels
of governance from variables at lower levels.
                                                                             Appendix A
                                                            Governance Studies Examined


        As we shall discuss further below, using a logic of governance to evaluate these
studies enables us to piece together a larger picture of how governance matters. By its
very nature, governance implies a configuration of distinct but interrelated elements —
statutes; organizational, financial, and programmatic structures; resource levels;
administrative rules and guidelines; and institutionalized rules and norms — that
constrain and enable tasks, priorities, and values that are incorporated into regulatory,
service production, and service delivery processes. Thus, governance involves extensive
endogeneity among its constituent features rather than the mere summing of independent
influences (Ostrom 1986). 1 While acknowledging the configurational nature of
governance, we believe nevertheless that an initial foray into integrating findings across
studies may provide insights. For example, we may integrate findings from studies that
use (d) level variables as dependent variables with the findings of studies that use (d)
level variables as independent variables to gain a broad sense of how the larger system of
hierarchical governance works.

        A number of caveats must be noted. First, at this stage of the project we examine
primarily research published in journals, which introduces selection biases. For example,
published studies invariably emphasize positive findings concerning hypothesized effects.
Second, while individual studies meet minimum standards of quality (primarily internal
validity), we have not at this stage subjected them to close technical criticism. If
subjected to rigorous technical and counter_factual analysis, the external validity of
individual studies might be revealed as questionable.

        Nevertheless, we believe that the general pattern of findings constitutes a kind of
meta_analysis, suggesting patterns of findings and promising avenues for theory
development and further empirical research that transcend the narrow compartments in
which governance and public management research is found. 2 Our effort is thus broader
in scope than meta-analyses that examine particular policies, elements, or constructs
within the logic of governance (e.g., Meyers’ 1993 analyses of organizational factors that
influence coordination among human services; Wolf’s 1999 meta-analysis of case studies
of leadership; or Lee, Bryk and Smith’s 1993 review of factors that are associated with
effective schools).

        The analysis reported in this paper is drawn from concise general summaries that
we created from each of the studies. In the next phase of our project, we will undertake a
more refined analysis both of the findings and of the methods employed in these studies
in order to identify the recurring issues of research design and quality that affect the
validity of individual and cumulative findings.

1 We draw this point, as well as some others in this paper, from Lynn, Heinrich, and Hill
2 For this preliminary phase of our analysis, we focus on hypothesized governance

relationships. We do not include control variables.
                                                                               Appendix A
                                                              Governance Studies Examined


Are there Any Findings? Or, Does Governance Matter?

        Governance matters. We found that variables at virtually every level of the logic
of governance both influence and are influenced by variables at other levels, and these
relationships are statistically and substantively significant.

        Some areas of the logic of governance terrain are sparsely populated, however.
Relatively few studies in our current reference list are concerned with citizen and
legislative interests, preferences, and choices [levels (a) and (b)], with formal structures
of authority [level (c)], or with performance assessments expressed politically [level (g)].
Thus we will be unable to say much if anything about the external or political dimensions
of governance and, therefore, about the classic problem of accountability. This
shortcoming, which is in part an artifact of our preliminary strategy for selecting studies
but also reflects investigator priorities, will be remedied in the next phase of our research.

        The vast majority of the studies in our sample have a top-down perspective:
influence flows downward toward treatments and outcomes, often, as we will discuss
further below, bypassing intermediate levels. A few studies suggest that influence flows
upward. For example, citizen demands for governmental services or decisions to litigate
may be influenced by observed patterns of managerial or worker behavior or by
budgetary allocations. Legislative priorities may be influenced by structurally-
determined executive priorities. Budget allocations may be determined by discretionary
choices of managers and workers. The extent of this kind of bi-directionality of
governance is not well studied, however.

        Most studies are concerned with independent or dependent variables at levels (d),
(e), and (f), in which public management in the usual sense, viz., as involving decisions
or choices concerning organization, strategy, and resource allocation, are related to the
primary work and treatment methods of agencies and programs and to their outcomes or
results. The findings from these studies strongly imply that “management matters.”

Formal Organizational Structures

        The studies employed different classes of variables concerning formal
organizational structures: legislated policy design elements; the hierarchical level and
composition of administrative units; and mandated administrative procedures. Only one
study used a variable at this level to explain a “higher_level” construct (at the level of
legislative preferences): policy analysis procedures as an explanatory variable for
legislative influence in state clean air policy (Potoski and Woods 2001). Of the
remaining studies that used some kind of formal organizational structure as an
explanatory variable, these variables are hypothesized to explain a variety of constructs
or outcomes at subordinate levels in the logic: at the discretionary managerial level,
                                                                             Appendix A
                                                            Governance Studies Examined

primary work level, program outcome level, and also citizen or stakeholder assessments
of program performance.

       For example, Koenig and Kise (1996) found that legislative rules and actions and
regulatory requirements influenced city managers’ responsibilities for budgeting,
organizational authority, motivation and morale, and the local representative process.
Attewell and Gerstein (1979) found that federal policies that reflected multiple
stakeholder goals influenced local practices concerning methadone maintenance for
heroin addiction. Boschken (1998) found that agency autonomy of urban public transit
agencies, particularly fiscal autonomy, was related to various policy outcomes. And Reid
(1990) found that mayor_council or representative town meeting forms of local
government were associated with lower citizen perceptions of local government waste,
compared to open town meeting structures.

       In general, then, legislated policy design elements, administrative unit hierarchies,
and administrative procedures are used as explanatory variables for a broad range of
dependent variables in governance research. Our studies reported some null findings, but
most reported nonzero influences of formal organizational variables, suggesting that
formal governance does matter.

Discretionary Organization and Management

        Studies that use discretionary organization and management decisions as
independent variables are the most numerous in our study sample. These (d) level
independent variables are used primarily to explain organizational performance or
outputs. The next most frequent dependent variables are field_level treatments. A few
studies use (d) variables to examine citizen and stakeholder assessments of performance,
but no studies used these variables to explain higher levels of the logic of governance.
Whatever the reason for the higher frequency of these level (d) types of studies —
selection, data availability, or researcher preferences — the findings create a compelling
picture that public management matters.

        The independent variables at the (d) level may be further divided into
administrative structures, use of managerial tools, and management values or strategies.
Quite numerous are studies that examine the consequences of contracting out,
privatization, or the use of nonprofit or for_profit providers on organizational
performance. In general, these studies show that the type of organization providing
services has a clear influence on client and organizational outcomes for a range of
services. Continuity of care and coordination are also often examined and are also shown
to have an influence on a range of outcomes, although not necessarily in the direction of
higher service quality or effectiveness. Managerial tools, values, and strategies are used
more often to explain primary work or treatments, than they are used to explain
organizational outputs.
                                                                             Appendix A
                                                            Governance Studies Examined

        To sum up the pattern of results so far, we can say that formal governance — in
the form of legislated policy design elements, hierarchy of administrative units, and
administrative procedures — and public management — in the form of administrative
structures, use of managerial tools, and management values or strategies — have
demonstrable effects on subordinate levels of public administration and on governmental

Field_Level Interventions
        Studies that use field_level interventions as independent variables almost
exclusively use them to explain organizational outputs and outcomes. Classes of such
field_level intervention variables include program design features, field worker
discretionary activity, field worker beliefs and values, and administrative processes. As
expected, primary work variables lend themselves relatively more easily to analysis
through experiments, and in this category we see most of the experimental methods in
our sample of studies.

        As with formal organizational structures and discretionary management, the
findings for treatment variables also suggest that governance at the treatment level
matters to outcomes. The kinds of variables at the primary work level might be classified
as the organization of treatment, field-level management strategies, and administrative
procedures. The great majority of these studies are concerned with the organization of
treatment in social programs, however, so the range of findings is relatively narrow.

        A unique aspect of primary work research is the extent to which the research is
designed to evaluate particular normative theories of intervention. For example, case
management, continuity of care, and integrating specialists into an intervention team are
often favored by treatment professionals over so-called traditional approaches to
intervention, and studies are designed to confirm their intuitions. The findings appear to
be mixed, which may reflect both actual effectiveness and the difficulties of statistically
identifying the effects of particular features of treatment.

Is Governance Research Producing Useful Findings? Or, How Much
Does Governance Matter?

         Does this evidence that “governance matters” justify strong claims on behalf of its
potential for policy and political relevance? Clearly, such claims must be supported by
research that is both internally and externally valid. The next phase of our research will
involve boring into each individual study and risking losing the forest for the trees. In
this initial stage, however, we step back to assess the general patterns of explanation.

The (Gaps in) Logic in Practice
                                                                             Appendix A
                                                            Governance Studies Examined

        The logic of governance sketched at the beginning of this paper is a simplified
version of the reality of governance. A common criticism of this simplified logic is that
it omits bi-directional and feedback influences.

        Our review of this set of studies indicates that the logic fairly represents the
research orientations of governance researchers. In general, we find that studies using
independent variables at higher levels in the logic tend to skip intermediate levels in
formulating models linking these variables to dependent variables at primary work and
outcomes levels. This strategy may seem only, well, logical. Models that focus on
particular stages or processes of a governance regime may be informative building blocks
for understanding the larger processes of governance.

         However, such models are unlikely to capture adequately the configurational,
political, and loosely coupled nature of governance, and are likely to undermine the
usefulness of the findings. For example, if the characteristics of presidential
administration are shown to be correlated with primary work activity levels, or that
political party competition affects the efficiency of child support enforcement, how
should policymakers act on these findings? Unless the mediating levels or influences can
be explained away or controlled for, these relatively distant cross_level relationships may
be of little use for the practice of governance.

        The “true” model, of course, may be one in which the marginal effects are zero
for some levels of governance in a particular application. Theoretical justification may
exclude these irrelevant factors from model specifications. Indeed, like natural scientists,
social scientists frequently begin the modeling process by assuming or submitting some
reasonable evidence that the model accounts for the most influential factors. In addition,
researchers commonly assume (implicitly or explicitly) that the influence of unobserved
or omitted factors is, on average, small, random, and/or unpredictable — and of no
import to the relationships specified in the model.

        However, the more useful studies, in our view, are those that provide insights into
more proximate relationships, e.g., relating discretionary management to primary work or
primary work to organizational outcomes. While still often representing reduced-form
relationships, these types of associations have less potential “slippage” that arises from
omitted variables between the explanatory construct and the outcome of interest.

Watch Your Back

        Studies based on independent variables at discretionary management (d) and
field-level intervention (e) levels tend toward their own shortcoming: a neglect of higher
level influences on both independent and dependent variables. For example, research
reveals that higher level of governance exert numerous influences on primary work:
federal and state policies; performance, contractual, and reimbursement incentives;
organizational form; managed care; legislative, elected executive influence; bureaucratic
                                                                               Appendix A
                                                              Governance Studies Examined

factors such as centralization, professionalization, processes, size, structure, and controls;
allocation of budgetary resources; management policies and strategies; disagreement
among principals; client pathways; network stability; personnel values; agency cultures;
and external politics.

        When primary work variables instead are used as independent variables, however,
there is little acknowledgment of such actual or potential influences from higher levels.
These patterns suggest that street_ or field_level work is generally (though is not always)
isolated from its bureaucratic and political settings. Furthermore, unless the research
design controls or accounts for these higher_level factors in some way, findings
regarding primary work risk irrelevance to public management practice because
superordinate variables are ignored.

        Similarly, influences from above and below in the hierarchy are used as
explanatory variables for studying managerial factors (though these investigations are
rare). But, when managerial influences are used as independent variables, influences
from higher and lower governance levels are seldom incorporated into research designs
or interpretations. For example, production functions for student achievement seldom
contain state- or district-level political environments or financing structures that may
affect school-level governance (Lynn and Tepper 1998).

        These findings indicate that accounting for factors at higher and lower levels
should be attempted when seeking to understand the influence of managerial and primary
work factors. Managerial and practitioner roles differ with regard to the constraints that
circumscribe them, and discretion may be restricted in numerous ways: by formal or
informal rules, by guidelines, and by norms or other socialized beliefs and practices.
These constraints can be overcome — albeit at a cost in time, effort, and compromise on
other goals. Thus, contexts might be described in terms of their malleability or
susceptibility to exogenous or endogenous change, but in any case are likely to be a
influential factors.

        Individual influence in the face of constraints may be an important factor because
practitioners’ own goals and circumstances affect their inclinations to incur the costs of
overcoming rather than adapting to constraints. Thus, practitioners’ response to
constraints in a given structural context are likely to differ (Lynn 1991). The questions of
constraints and how the possibilities of overcoming them affect practitioner orientation
introduce the distinction between short- and long-run strategic behavior. Prescriptive
public management studies often finesse this distinction by appeals to a short run
inhabited by political appointees or program managers for whom contextual and
structural factors are assumed to be (or are urged to be taken as) given. Too often, such
studies give short shrift to a more malleable longer run inhabited by officials who see
themselves as confronting a wider array of variables, circumstances, and opportunities
(Knott and Hammond 2000; Lynn 2000).
                                                                             Appendix A
                                                            Governance Studies Examined

        Views of the type and extent of practitioner influence in governance systems
necessarily affect the kinds of research considered “useful” for practitioners and
policymakers. At an individual level, characteristics of practitioners — school principals,
agency heads, elected officials, or service workers — may have a significant bearing on
governance in practice. Alternatively, if organizational and social structures and roles
dominate or reduce the scope of individual attributes in accounting for governmental
performance, research that yields robust findings concerning structures and roles is more
likely to influence actual performance of government than research that is preoccupied
with leadership, managerial strategies, and best practices.

        Located between these individual and structural extremes is a perspective that
practitioner characteristics and context interact in complex ways (where “context”
encompasses the configurational, political, and loosely coupled nature of a governance
regime). An approach to research that incorporates a logic of governance assumes that
contextual influences on performance — perhaps interacting with individual
characteristics — are at least possible, if not likely. Although variation in contexts may
be insignificant in practice, this perspective holds that it is advisable to discover this
variation through empirical analysis rather than to assume it.

       For these reasons, governance research that incorporates explicit awareness of
contextual factors and allows for their causal influence on governmental performance
should have a greater claim on practitioner attention than research that disproportionately
focuses on individual-level factors and places contextual considerations into a conceptual
black box.

A Rose by Any Other Name . . . ?

       These features of the governance terrain, where empirical analyses often do not
take account of the complex workings of governance, are directly related to our ability to
draw broad conclusions about governance research even within policy areas. The
problem, of course, is that restricted models may give not merely partial accounts but
biased partial accounts of the complex, configurational workings of governance. Moe
(1985) discusses this problem in the context of regulation studies:

                              [P]opular models of regulation as well as
                              quantitative empirical work have tended to focus
                              only on very small parts of the whole–in the former
                              case for reasons for clarity and mathematical
                              tractability, and in the latter because of data
                              collection and measurement problems — and
                              because they are often guided by these same
                              models). Given the very real difficulties that
                              comprehensiveness entails, these approaches are
                                                                              Appendix A
                                                             Governance Studies Examined

                               entirely reasonable short-term research strategies.
                               But it is important to remember that they threaten to
                               yield biased inferences about the causes of
                               regulatory behavior. They clearly omit factors
                               whose causal effects may overwhelm or distort the
                               “special” relationships on which they singularly
                               focus (Moe 1985, 1095).

        In seeking to take account of other levels of governance, some research designs
may include control variables that differentially affect or qualify interpretation of
findings for a particular governance level, feature, or practice of interest. For example, a
study that concludes that there is no difference between nonprofit and for-profit firms in
terms of client outcomes may be controlling for different kinds of variables than another
study that finds differences in performance between nonprofits and for-profits

        A related problem is the operationalization of variables and our ability to
generalize from a range of studies. This is not so much a factor in studies of nonprofit
and for-profit organizations, because legal status of the organization provides a consistent
operational definition. However, other concepts of interest to governance scholars such
as discretion, leadership, context, even performance, are problematic. Concepts may be
operationalized in very different, even inconsistent, ways. In modeling complex
governance phenomena, multiple operational definitions of the same concept are not only
plausible but likely–making the replication of research models and results more
challenging, and the generalizations about a particular construct problematic.


        Most significant findings concerned hypothesized relationships are contingent and
qualified. That in itself is interesting; things work under particular circumstances, with
specific types of problems, clients, or outcomes. It follows that few if any “universal” or
generic management principles are available. Assumptions and contexts are messy but
necessary; the devil is in the details.

        One view is that the research findings from a particular study are not
generalizable to any other settings, and thus we simply cannot make generalizations
about governance (or anything else in social sciences). Mohr (1996) argues that all
empirical research is essentially historical analysis that can never be replicated. He
asserts, however, that a reasonable standard for useful empirical research is that which
generates “strong causal insights” by which he means “research that imparts (a) a
‘consummate causal understanding’ of a relation such that we are presented with (b) a
‘significant possibility’ that we can use by means of (c) ‘creative-selective
generalization’ to illuminate and inform possible iterations of this relation in other
                                                                              Appendix A
                                                             Governance Studies Examined

contexts” (Mohr 1996, 144-145). These seem reasonable standards to which governance
research may aspire.

        A related conclusion concerns the role of theory in governance research.
Theories, too, tend to be highly contextualized. The central theoretical problem in
governance research is applying theories that impose a causal ordering or a priori
structure on the logic that links context, governance, and consequences or outcomes.
This problem is extraordinarily complex. In welfare research, for example, how does the
investigator incorporate in a theoretical model the local political and cultural climate of
welfare administration, the specific design features of policies, administrative structures,
the decisions and skills of managers, the beliefs and practices of welfare workers, and the
characteristics and responses of welfare recipients to assess the relative effects of policy
designs, structures, and management on policy outcomes?

        In the face of these kinds of challenges, the temptation is strong to move to one of
the two extremes: to adduce all manner of potential interrelationships — in a process that
might be termed “taxonomic theorizing” — to the point at which the theory is vulnerable
to the charge that because it explains everything, it explains nothing or to model
governance and management problems narrowly, assuming that the governing regime is
only loosely coupled to or actually decoupled from its wider context. The latter strategy
usually reflects the investigator’s use of theory that ignores context or the investigator’s
need to restrict the scope of analysis in the light of data limitations. Often, however, the
assumptions underlying restricted approaches are unstated. In discussing modeling
problems in areas of regulation, Moe (1985, 1095) commented that “it is common in a
data-poor world for myopic models to take on lives of their own, and thus for capture or
congressional dominance or budget-maximization to attract ardent followings for reasons
that have little to do with demonstrated empirical validity.”

        A central empirical problem in governance research is obtaining data that will
enable investigators to explore causal relationships beyond a narrow perimeter of
theoretical possibilities that leaves too much out of the picture. Usually, the investigator
must resort to more limited data collection strategies or to reanalysis of data that have
been collected for other purposes and that may be inadequate for the investigator’s
purpose in many respects.

        Meeting these theoretical and empirical challenges can be daunting. As a result,
the very real possibility that broad patterns of interrelationships affect governmental
outcomes often is inadequately incorporated into research designs or even into
explanations and interpretations of research findings. For example, a study may attribute
client outcomes to client characteristics; to worker and treatment characteristics; or to
patterns of interaction between clients, treatments, and workers, but ignore the potential
significance of local or hierarchical organizational and management variables ,
systemwide incentives affecting workers and clients, or features in the local political or
administrative context that establish expectations or governing values. Thus, findings
                                                                             Appendix A
                                                            Governance Studies Examined

from specific studies may not meet the Mohr criterion of establishing sufficiently
compelling causal understanding so that they are instructive in other contexts.

        We are not suggesting, however, that governance research that is restricted in
theoretical and empirical scope is never worth the effort. Such a view flies in the face of
the realities affecting research design: small-scale research is more feasible than
comprehensive studies, and careful interpretation of findings may attenuate shortcomings
in research design. Moreover, well-designed investigations of restricted scope can
produce — and have produced — revealing insights (e.g., Mashaw 1983). We do argue,
however, that investigators whose inquiries are restricted in scope should nonetheless be
mindful of what is (necessarily) omitted from their models so they can at least speculate
on the possible implications of these omissions when they interpret their findings.


Attewell, Paul, and Dean R. Gerstein. 1979. Government policy and local practice.
American Sociological Review 44(April): 311-327.

Boschken, Herman L. 1998. “Institutionalism: Intergovernmental exchange,
administration-centered behavior, and policy outcomes in urban agencies. Journal of
Public Administration Research and Theory, 8:4:585-614.

Knott, Jack H., and Thomas H. Hammond. 2000. Congressional committees and policy
change: explaining legislative outcomes in banking, trucking, airline, and
telecommunications deregulations. In Governance and Performance: New Perspectives,
ed. Carolyn J. Heinrich and Laurence E. Lynn, Jr. Washington, DC: Georgetown
University Press.

Koenig, Heidi, and Amy Kise. 1996. Law and the city manager: beginning to
understand the sources of influence on the management of local government. Journal of
Public Administration Research and Theory 6:3: 443-459.

Lee, Valerie E. 1993. The organization of effective secondary schools. In Linda
Darling-Hammond, ed. Review of Research in Education. Washington, DC: American
Educational Research Association.

Lynn, Laurence E. Jr. 1991. The budget-maximizing bureaucrat: is there a case? In The
Budget-Maximizing Bureaucrat: Appraisals and Evidence, Ed. Andre Blais and Stephane
Dion. Pittsburgh: University of Pittsburgh Press.
                                                                           Appendix A
                                                          Governance Studies Examined

Lynn, Laurence E., Jr. 2000. A symposium: In government, does management matter? In
Advancing Public Management: New Developments in Theory, Methods, and Practice,
ed. Jeffrey L. Brudney, Laurence O’Toole, Jr., and Hal G. Rainey. Washington, DC:
Georgetown University Press.

Lynn, Laurence E. Jr., Carolyn J. Heinrich, and Carolyn J. Hill. 2000. “Studying
Governance and Public Management: Challenges and Prospects.” Journal of Public
Administration Research and Theory 10:2 (April): 233-261.

Lynn, Laurence E. Jr., Carolyn J. Heinrich, and Carolyn J. Hill. 2001. Improving
Governance: A New Logic for Empirical Research (Washington, DC: Georgetown
University Press).

Lynn, Laurence E., Jr. and Robin L. Tepper. 1998. Governing public schools: theory
and practice. Working paper, University of Chicago.

Mashaw, Jerry L. 1983. Bureaucratic Justice: Managing Social Security Disability
Claims. New Have, Conn: Yale University Press.

Meyers, Marcia K. 1993. Organizational factors in the integration of services for
children. Social Service Review December: 547-575.

Moe, Terry M. 1985. Control and feedback in economic regulation: the case of the
NLRB. American Political Science Review 79:4: 1094-1116.

Mohr, Lawrence B. The Causes of Human Behavior: Implications for Theory and
Method in the Social Sciences. Ann Arbor: University of Michigan Press.

Ostrom, Elinor. 1986. An agenda for the study of institutions. Public Choice 48:1: 3-25.

Potoski, Matthew and Neal D. Woods. 2001. Designing state clean air agencies:
Administrative procedures and bureaucratic autonomy. Journal of Public Administration
Research and Theory, 11:2:203-221.

Reid, Gary J. 1990. Perceived government waste and government structure: an empirical
examination of competing explanations Public Finance Quarterly 18:4: 395-419.

Wolf, Patrick J. 1999. Hercules, Fortuna, or Prometheus? A case meta-analysis of the
influence of leadership on agency effectiveness. Working paper, Georgetown University.

                                     Appendix A
                                                                                 Appendix A
                                                                Governance Studies Examined

                            Governance Studies Examined

Adams, Charles F., David Landsbergen, and Daniel Hecht. 1994. Organizational impediments to
paternity establishment and child support. Social Service Review (March): 109-126.

Alexander, Jeffrey A., Thomas A. D’Aunno, and Melissa J. Succi. 1996. Determinants of
Profound organizational change: Choice of conversion or closure among rural hospitals. Journal
of Health and Social Behavior 37(September): 238-251.

Arum, Richard. 1996. Do private schools force public schools to compete? American
Sociological Review. 61 (February): 29-46.

Attewell, Paul, and Dean R. Gerstein. 1979. Government policy and local practice. American
Sociological Review 44(April): 311-327.

Beierle, Thomas C. and David M. Kennedy. 2000. Values, conflict, and trust in participatory
environmental planning. Journal of Policy Analysis and Management, 19:4:587-602.

Berrens, Robert P., Alok K. Bohara, Amy Baker, and Ken Baker. 1999. Journal of Policy
Analysis and Management, 18:2:303-326.

Bickman, Leonard. 1996. A continuum of care: more is not always better. American
Psychologist 51:7: 689-701.

Bigelow, Barbara, and Melissa Middleton Stone. 1995. Why don’t they do what we want? An
exploration of organizational responses to institutional pressures in community health centers.
Public Administration Review 55:2: 183-192.

Bloom, Howard S., Carolyn J. Hill, and James Riccio. 2001. Modeling the performance of
welfare-to-work programs: The effects of program management and services, economic
environment, and client characteristics. Manpower Demonstration Research Corporation
working paper.

Bloom, Howard S. and Helen F. Ladd. 1982. Property tax revaluation and tax levy growth.
Journal of Urban Economics, 11:73-84.

Boschken, Herman L. 1992. Analyzing performance skewness in public agencies: the case of
urban mass transit. Journal of Public Administration Research and Theory 2:3:265-288;

Boschken, Herman L. 1998. Institutionalism: Intergovernmental exchange, administration-
centered behavior, and policy outcomes in urban agencies. Journal of Public Administration
Research and Theory, 8:4:585-614.

Boylan, Delia M. 1999. Bureaucratic Reform in Developing Countries: A Comparison of
Presidential and Parliamentary Rule. Harris School Working Paper 99.9, University of Chicago.
                                                                                  Appendix A
                                                                 Governance Studies Examined

Bozeman, Barry, and Stuart Bretschneider. 1994. The “publicness puzzle” in
organization theory: a test of alternative explanations of differences between public and
private organizations. Journal of Public Administration Research and Theory 4:2: 197-223.

Braddock, David. 1992. Community mental health and mental retardation services in the United
States: a comparative study of resource allocation. American Journal of Psychiatry 149:2: 175-

Brewer, Gene A. and Sally Coleman Selden. 2000. Journal of Public Administration Research
and Theory, 10:4:685-711.

Brock, Thomas and Kristen Hartnett. 1998. A comparison of two welfare-to-work case
management models. Social Service Review, Dec. 1998, 493-520.

Brown, Mary Maureen, Laurence J. O’Toole, Jr., and Jeffrey L. Brudney. 1998. Implementing
information technology in government: An empirical assessment of the role of local partnerships.
Journal of Public Administration Research and Theory, 8:4:499-525.

Brudney, Jeffrey and F. Ted Hebert, “State Agencies and Their Environments: Examining the
Influence of Important External Actors,” The Journal of Politics 49 (1987):186-206.

Carpenter, Daniel P. 1996. Adaptive signal processing, hierarchy, and budgetary control in
federal regulation. American Political Science Review 90:2: 283-302.

Chicoine, David L. And Norman Walzer. 1985. Governmental structure, service quality, and
citizens’ perceptions. Public Finance 40:3: 363-380.

Chubb, John E. 1985. The political economy of federalism. American Political Science Review
79: 994-1015.

Courty, Pascal and Gerald R. Marschke, “Do Incentives Motivate Organizations? An Empirical
Test,” in Heckman (forthcoming).

Cragg, Michael. Performance Incentives in the Public Sector: Evidence from the Job Training
Parternship Act. Journal of Law, Economics and Organization 13:1 (April 1997): 147-168.
Cuellar, Alison Evans, Anne M. Libby, and Lonnie R. Snowdon. How capitated mental health
care affects utilization by youth in the juvenile justice and child welfare systems. Mental Health
Services Research, forthcoming.

Currie, Janet and Sheena McConnell. 1991. Collective bargaining in the public sector: The
effect of legal structure on dispute costs and wages. American Economic Review, 81:4:693-718.

D’Aunno, Thomas, Robert I. Sutton, and Richard H. Price. 1991. Isomorphism and external
support in conflicting institutional environments: A study of drug abuse treatment units.
Academy of Management Journal 34:3: 636-661.
                                                                                   Appendix A
                                                                  Governance Studies Examined

Eisner, Marc Allen, and Kenneth J. Meier. 1990. Presidential control versus bureaucratic power:
Explaining the Reagan revolution in antitrust. American Journal of Political Science 34:1: 269-

Emmert, Mark A. and Michael M. Crow. 1987. Public-Private Cooperation and Hybrid
Organization. Journal. of Management, 3:1: 55-67.

Evans, William N., Sheila E. Murray, and Robert M. Schwab. 1997. Schoolhouses, Courthouses,
and Statehouses after Serrano. Journal of Policy Analysis and Management 16:1: 10-31.

Ezell, Mark, and Rino J. Patti. 1990. State human service agencies: structure and organization.
Social Service Review. March: 22-45.
Feiock, Richard C. and Jae-Hoon Kim. 2001. Form of government, administrative organization,
and local economic development policy. Journal of Public Administration Research and Theory,

Forder, Julien, “Contracts and Purchaser-Provider Relationships in Community care,” Journal of
Health Economics 16:5 (October 1997): 517-542.

Frank, Richard G., and Martin Gaynor. 1994. Organizational failure and transfers in the public
sector: evidence from an experiment in the financing of mental health care. The Journal of
Human Resources 24:1:108-125.

Frant, Howard. 1993. Rules and governance in the public sector: the case of civil service.
American Journal of Political Science 37:4: 990-1007.

Gamoran, Adam and Robert Dreeben. "Coupling and Control in Educational Organizations,"
Administrative Science Quarterly 31:4 (December 1986): 612-632.

Garfinkel, Irwin, Miller, Cynthia, McLanahan, Sara S., Hanson, Thomas L.. 1998. Deadbeat
Dads or Inept States? A Comparison of Child Support Enforcement Systems. Evaluation Review
22:4: 717-750.

Garg, Pushkal P., Kevin D. Frick, Marie Diener-West, and Neil R. Powe. 1999. Effect of the
ownership of dialysis facilities on patients’ survival and referral for transplantation. The New
England Journal of Medicine, 341:22:1653-1660.

Glisson, Charles. 1996. Judicial and Service Decisions for Children Entering State Custody: The
Limited Role of Mental Health. Social Service Review 70:2: 257-281.

Goodrick, Elizabeth and Gerald R. Salancik. 1996. Organizational Discretion in Responding to
Institutional Practices: Hospitals and Cesarean Births. Administrative Science Quarterly 41: 1-28.

Hedges, Larry V., Richard D. Laine, and Rob Greenwald. 1994. Does money matter? A meta-
analysis of studies of the effects of differential school inputs on student outcomes. Educational
Researcher VOL: 5-14.
                                                                                Appendix A
                                                               Governance Studies Examined

Heinrich, Carolyn J. 1999. Do government bureaucrats make effective use of performance
management information? Journal of Public Administration Research and Theory, 9:3:363-393.

Heinrich, Carolyn J. 2000. Organizational form and performance: an empirical investigation of
nonprofit and for-profit job-training service providers. Journal of Policy Analysis and
Management, 19:2:233-262.

Heintze, Theresa and Stuart Bretschneider. 2000. Information technology and restructuring in
public organizations: Does adoption of information technology affect organizational structures,
communications, and decision making? Journal of Public Administration Research and Theory,

Hill, Carolyn J. 2001. Casework job design and client outcomes in welfare-to-work programs.
Unpublished manuscript.
Himmelstein, David U., Steffie Woolhandler, Ida Hellander, and Sidney M. Wolfe. 1999.
Quality of care in investor-owned vs not-for-profit HMOs. JAMA: The Journal of the American
Medical Association, 282:2: 159-163.

Hollingsworth, Rogers, Robert Hanneman, Jerald Hage, and Charles Ragin. 1996. The effect of
human capital and state intervention on the performance of medical systems. Social Forces 75:2:

Hotz, V. Joseph and M. Rebecca Kilburn. 1995. Regulating child care: the effects of state
regulations on child care demand and its cost. RAND Working Paper 95-03.

Jennings, Edward T., Jr. 1994. Building bridges in the intergovernmental arena: coordinating
employment and training programs in the American states. Public Administration Review 54:1:

Jennings, Edward T., Jr. and Jo Ann G. Ewalt. 1998. Interorganizational coordination,
administrative consolidation and policy performance. Public Administration Review. 58:5: 417-

Jennings, Edward T., Jr. and Dale Krane. 1998. Interorganizational cooperation and the
implementation of welfare reform: Community service employment in welfare work programs.
Policy Studies Review, 15:2-3: 170-201.

Kakalik, James S. 1997. Just, speedy, and inexpensive? Judicial case management under the
Civil Justice Reform Act. Judicature 80:4: 184-189.

Kato, Linda Y. and James A. Riccio. 2001. Building new partnerships for employment:
Collaboration among agencies and public housing residents in the Jobs-Plus demonstration.
New York: Manpower Demonstration Research Corporation.

Keiser, Lael R. 1997. Controlling the child enforcement bureaucracy: organizational
characteristics and bureaucratic responses. Working paper. October.
                                                                                Appendix A
                                                               Governance Studies Examined

Keiser, Lael R. And Kenneth J. Meier. 1996. Policy design, bureaucratic incentives, and public
management: the case of child support enforcement. Journal of Public Management Research
and Theory 6:3: 337-364.

Kirk, Stuart A. And Herb Kutchins. 1988. Deliberate misdiagnosis in mental health practice.
Social Service Review (June): 225-237.

Koenig, Heidi, and Amy Kise. 1996. Law and the city manager: beginning to understand the
sources of influence on the management of local government. Journal of Public Administration
Research and Theory 6:3: 443-459.

Ladd, Helen F. and Jens Ludwig. 1996. Housing Vouchers, Residential Relocation, and
Educational Opportunities: Evidence from Baltimore. Paper Prepared for the session on Urban
Problems at the 1997 Conference of the American Economic Association, January 4-6, 1997.

Lambert, E. Warren and Pamela R. Guthrie. 1996. Clinical outcomes of a children’s mental
health managed care demonstration. Journal of Mental Health Administration, 23:1:51-68.

Langbein, Laura I. 2000. Privatization, empowerment, and productivity: Some empirical
evidence on the causes and consequences of employee discretion. Journal of Policy Analysis and
Management, 19:3:427-450.

Langbein, Laura I. and Cornelius M. Kerwin. 2000. Regulatory negotiation versus conventional
rule making: Claims, counterclaims, and empirical evidence. Journal of Public Administration
Research and Theory, 10:3:599-632.

Lowery, David. 1982. The Political Incentives of Government Contracting. Social Science
Quarterly 63:3: 517-29

McFadden, D. 1976. The Revealed Preferences of a Government Bureaucracy: Empirical
Evidence. Bell Journal of Economics 7:1: 55-72.

MacFarlane, Deborah R. and Kenneth J. Meier. 1998. Do different funding mechanisms produce
different results? The implications of family planning for fiscal federalism. Journal of Health
Politics, Policy, and Law 23:3: 423-454.

Marcoulides, George A. and Ronald H. Heck. 1993. Organizational culture and performance:
proposing and testing a model. Organization Science 4:2: 209-225.

Marschke, Gerald. 1993.

Martin, Lisa, Carey L. Peters and Charles Glisson. 1998. Factors affecting case management
recommendations for children entering state custody. Social Service Review, 521-544.

Maser, Steven M. 1998. Constitutions as relational contracts: Explaining procedural safeguards
in municipal charters. Journal of Public Administration Research and Theory, 8:4:527-564.
                                                                                 Appendix A
                                                                Governance Studies Examined

Maupin, James R. 1993. Control, efficiency, and the street-level bureaucrat. Journal of Public
Administration Research and Theory. 3:3:335-357.

May, Peter J. and Søren Winter. Regulatory enforcement and compliance: Examining Danish
agro-environmental policy. Journal of Policy Analysis and Management, 28:4:625-651.

Maynard, Rebecca, “Teenage Childbearing and Welfare Reform: Lessons from a Decade of
Demonstration and Evaluation Research,” Children and Youth Services Review 17:1-3 (1995):

Mead, Lawrence M. 1983. Expectations and welfare work: WIN in New York City. Policy
Studies Review 2:4:648-662.

Mead, Lawrence M. 1989. Expectations and welfare work: WIN in New York state. Polity.
Winter: 224-252.

Mead, Lawrence M. 1988. The potential for work enforcement: a study of WIN. Journal of
Policy Analysis and Management 7:2: 264-288.

Meier, Kenneth J. and Laurence J. O’Toole. 2001. Managerial strategies and behavior in
networks: a model with evidence from public education. Journal of Public Administration
Research and Theory, 11:3:271-293.

Meier, Kenneth J. and Joseph Stewart, Jr. 1992. The impact of representative bureaucracies:
educational systems and public policies. American Review of Public Administration 22:3: 157-

Meier, Kenneth J., Joseph Stewart, Jr., and Robert E. England. 1991. The politics of
bureaucratic discretion: educational access as an urban service. American Journal of Political
Science 35:1: 155-177.

Meyer, John W., Scott, W. Richard, and Strang, David. Centralization and School District
Complexity. Administrative Science Quarterly 32: 186-201.

Meyers, Marcia K., Bonnie Glaser, and Karin Mac Donald. 1998. On the front lines of welfare
delivery: Are workers implementing policy reforms? Journal of Policy Analysis and
Management, 17:1:1-22.

Meyers, Marcia K., Norma M. Riccucci, and Irene Lurie. 2001. Achieving goal congruence in
complex environments: The case of welfare reform. Journal of Public Administration Research
and Theory, 11:2:165-201.

Miranda, Rowan, and Allan Lerner. 1995. Bureaucracy, organizational redundancy, and the
privatization of public services. Public Administration Review 55:2: 193-200.

Moe, Terry M. 1985. Control and feedback in economic regulation: the case of the NLRB.
American Political Science Review 79:4: 1094-1116.
                                                                                 Appendix A
                                                                Governance Studies Examined

Morris, John R. and Suzanne W. Helburn. 2000. Child care center quality differences: The role
of profit status, client preferences, and trust. Nonprofit and Voluntary Sector Quarterly,
29:3:377-399. Regression analysis.

Northrop, Alana, and James L. Perry. 1985. A task environment approach to organizational
assessment. Public Administration Review 45:2: 275-281.

Okpaku, Samuel O., Kathryn H. Anderson, Amy E. Sibulkin, J. S. Butler, and Leonard Bickman.
The effectiveness of a multidisciplinary case management intervention on the employment of
SSDI applicants and beneficiaries. Psychiatric Rehabilitation Journal, 20:3:34-41.

Perry, James L and Timlynn T. Babitsky, “Comparative Performance in Urban Bus Transit:
Assessing Privatization Strategies,” Public Administration Review 46:1 (January/February 1986):

Perry, James L. and Theodore K. Miller. 1991. The Senior Executive Service: is it improving
managerial performance? Public Administration Review 51:6:554-563.

Potoski, Matthew. 1999. Managing uncertainty through bureaucratic design: Administrative
procedures and state air pollution control agencies. Journal of Public Administration Research
and Theory, 9:4:623-639.

Potoski, Matthew and Neal D. Woods. 2001. Designing state clean air agencies: Administrative
procedures and bureaucratic autonomy. Journal of Public Administration Research and Theory,

Provan, Keith G., and H. Brint Milward, “A Preliminary Theory of Interorganizational Network
Effectiveness: A Comparative Study of Four Community Mental Health Systems,”
Administrative Science Quarterly 40 (1995): 1-33.

Provan, Keith G. and H. Brinton Milward. 2000. Collaboration and integration of community-
based health and human services in a nonprofit managed care system. Unpublished manuscript.

Provan, Keith G., Kimberley Roussin Isett, and H. Brinton Milward. 2001. Government
financing, managed care, and services integration: The adaptation and evolution of a mental
health delivery system. Unpublished manuscript.

Provan, Keith G., and Juliann G. Sebastian. 1998. Networks within networks: service link
overlap, organizational cliques, and network effectiveness. Academy of Management Journal
41:4: 453-463.

Reid, Gary J. 1990. Perceived government waste and government structure: an empirical
examination of competing explanations Public Finance Quarterly 18:4: 395-419.

Ringquist, Evan J. 1995. Political control and policy impact in EPA’s Office of Water Quality.
American Journal of Political Science 39:2:336-363.
                                                                                 Appendix A
                                                                Governance Studies Examined

Romzek, Barbara S. and Jocelyn M. Johnston. Journal of Public Administration Research and
Theory, 9:1:107-139.

Rouse, Cecilia Elena. 1998. Do two-year colleges increase overall educational attainment?
Evidence from the states. Journal of Policy Analysis and Management, 17:4:595-620.

Rowan, B., “Organizational Structure and the Institutional Environment: The case of public
schools,” Administrative Science Quarterly 27 (1982): 259_279.

Sabatier, Paul A., John Loomis, and Catherine McCarthy. 1995. Hierarchical controls,
professional norms, local constituencies, and budget maximization: an analysis of U.S. Forest
Service planning decisions. American Journal of Political Science 39:1:204-242.

Saltzstein, Alan L., Yuan Ting, and Grace Hall Saltzstein. 2001. Work-family balance and job
satisfaction: The impact of family-friendly policies on attitudes of federal government employees.
Public Administration Review, 61:4: 452-466.

Sandfort, Jodi R. 2000. Moving beyond discretion and outcomes: Examining public
management from the front lines of the welfare system. Journal of Public Administration
Research and Theory, 10:4:729-756.

Scholz, John T., Jim Twombly, and Barbara Headrick. 1991. Street-level political controls over
federal bureaucracy. 1991. American Political Science Review 85:3:827-850.

Scott, Patrick G. 1997. Assessing determinants of bureaucratic discretion: An experiment in
street-level decision making. Journal of Public Administration Research and Theory, 7:1:35-57.

Ugboro, Isaiah O., Kofi Obeng, and Wayne K. Talley. 2001. “Motivations and Impediments to
Service Contracting, Consolidations, and Strategic Alliances in Public Transit Organizations,”
Administration and Society, 33:1:79-103.

Verma, Kiran, Barry M. Mitnick, and Alfred A. Marcus. 1999. Making incentive systems work:
Incentive regulation in the nuclear power industry. Journal of Public Administration Research
and Theory, 9:3:395-436.

Weisner, Constance and Robin Room. 1984. “Financing and Ideology in Alcohol Treatment,”
Social Problems, 32:2: 167-184.

Wheeler, John R., Nahra, Tammie A., “Private and Public Ownership in Behavioral Care,”
Working paper, University of Michigan, Department of Health Management Policy, (March

Whitford, Andy. 1997. Dissecting the political control of agencies by comparing district offices:
the EPA regions. Paper presented at the Fourth National Public Management Research
Conference, Athens, GA.
                                                                               Appendix A
                                                              Governance Studies Examined

Wolf, Patrick J. 1997. Why must we reinvent the federal government? Putting historical
development claims to the test. Journal of Public Administration Research and Theory, 7:3:353-

Wood, B. Dan. 1990. Does politics make a difference at the EEOC? American Journal of
Political Science 34:2: 503-530.

Wood, B. Dan, and Richard W. Waterman. 1991. The dynamics of political control of the
bureaucracy. American Political Science Review 85:3: 801-828.

To top