Evaluating the Outcomes of Health and Social Care Partnerships

Document Sample
Evaluating the Outcomes of Health and Social Care Partnerships Powered By Docstoc
					          Evaluating the
          Outcomes of Health
          and Social Care
          Partnerships: the
          POET Approach.

Helen Dickinson h.e.dickinson@bham.ac.uk
Health Services Management Centre, April 2007
Doubts over value of £3bn Sure Start
The first major evaluation of the government's
  flagship £3bn Sure Start programme… has
  revealed no overall improvement in the areas
  targeted by the initiative. Although some
  Sure Start schemes were successful, an
  independent study … revealed that Sure Start
  as a whole failed to boost youngsters'
  development, language and behaviour. It also
  showed children of teenage mothers did
  worse in Sure Start areas than elsewhere.
Doubts over value of £3bn Sure Start
The findings… represent only an early snapshot
  of the programme's effectiveness, and
  academics involved in the £20m evaluation
  emphasise that they do not mean the
  scheme, which varies widely around the
  country, will not succeed in helping children in
  deprived areas in the long term
Outline of session

 Policy Context
 What is partnership working?
 The evaluation challenge
 The POET approach
 Findings so far…
    The Prevalence of Partnerships
   5,500 individual partnership bodies.

   60 types of different partnerships

   Partnerships may spend around £15-20 billion per year.

   Approx 75,000 places on partnership boards

   Underestimate of the extent of partnership working?

(Sullivan and Skelcher, 2002)
The welfare state

       Five Giants         2006 equivalent
Want                 Social security

Disease              NHS

Ignorance            Education
Idleness             Employment/leisure

Squalor              Housing/regeneration
Government by hierarchy


          Health                             Social Security

 Health            Health          Social Security     Social Security
Services organised around the person



           Income             Housing
    Evidence-based policy and practice

   „What counts is what works‟

   I.e. government commitment to make policy and promote
    practices on the basis of solid evidence, rather than by
    ideological fiat.

   The Cabinet Office declares in the Magenta Book (official
    guidance notes for policy evaluation and analysis),
    „evidence-based principles are at the heart of the
    Government‟s reform agenda for better policy making
    and policy implementation‟ (Cabinet Office, 2003: 17).
The policy context (in theory)


            Better Services?

           Better Outcomes?
    A loss of confidence in the concept?
   Academic           „Partnerships also bring risks. Working
    critiques of the     across organisational boundaries
    partnership          brings complexity and ambiguity that
    concept.             can generate confusion and weaken
                         accountability. The principle of
                         accountability for public money
   Areas with           applies as much to partnerships as to
    long-lasting         corporate bodies. The public needs
    and high profile     assurance that public money is spent
    relationships        wisely in partnerships and it should
    recently             be confident that its quality of life will
    encountered          improve as a result of this form of
    difficulties.        working‟ (Audit Commission, 2005:
    Outcomes, process and partnership
   Systematic review of 491 papers      Based on an assumption
    found a dearth of evidence to         that if process is smooth
    support the notion that joint         then expected outcomes will
    working between the NHS and           be delivered?
    social services is „effective‟.
    (Cameron & Lart, 2003)               Or, selecting appropriate
                                          outcomes to evaluate
   Literature review of health and       partnerships is a
    social care partnerships              fundamentally difficult task?
    (Dowling et al, 2004) found:
-   Evaluations centred on process
    and organisational issues,
-   Little emphasis on user
The policy context questioned


              Better Services?

             Better Outcomes?

(Do they? How? For whom? In what contexts?)
What is partnership working? (1)
   Partnership is a very   Other terminology includes:
    UK-based (New
    Labour?) term for an       Joint-working, seamless-
    internationally             working, joined-up thinking,
    encountered                 interprofessional working,
    phenomenon relating         multi-professional working,
    to the relationships        integrated teams, multi-agency
    between bodies              working, interagency working,
    working together to         collaboration and
    deliver services.           interdisciplinary working.

   Not only health and
    social care
  What is partnership working? (2)

“Joint working        Number of different forms of
   arrangement of       relationships which can exist
   partners who are     between organisations:
   independent         Hierarchies
   bodies and these                       Partnerships
   co-operate to       Markets
   achieve a           Networks
   common goal”
(Audit Commission,    “Methodological anarchy and
                        definitional chaos”
                      (Ling, 2000: 83).
     Partnership map (1)


                                                  Partnership organisation

                                                  Joint management



          Health    Health    Health   Health
          & LA in   & LA in   &        & social
          civic     ‘whole’   LA       care
          society   system
     Partnership map (2)


                                     Adult           Partnership organisation
                                     services        Joint management

          Health       Health    Health    Health
          & LA in      & LA in   &        & social
          civic        ‘whole’   LA       care
          society      system
Group Exercises
    The evaluation challenge
   How is „partnership‟ defined?
   How do we know if partnerships are successful? What is it
    that defines success?
   Complex policy initiatives in complex contexts
   Multiple stakeholders and perspectives (individual,
    organisational, service user, carer, governmental…)
   Proving you‟ve prevented something?
   Long-term programmes/outcomes
   What would have happened anyway?

                                  Etc etc etc.
Aims of POET project

   Do partnerships improve outcomes for
    service users?

   Design a generic evaluation toolkit that can
    test this assumption

   Strong theoretical underpinning
POET: Partnership Outcomes
Evaluation Toolkit
1. Online staff survey      2. User survey

  - Tests the „health‟ of     - Tests whether the
  the partnership             „correct‟ outcomes have
  - Identifies                been identified.
  developmental and           - To what extent have
  problem areas               these outcomes been
  - Highlights areas for      met?
  celebration                 - Retroductive
  - Surfaces all the          techniques identify
  underlying „partnership     generative mechanisms
 Effective partnership working: the three

From Glasby (2003)

   Formative through process-based evaluation by
    staff members. Highlights areas that need
    improvement (often fairly simple) and allow
    celebration where things have gone well).
    Provides a benchmark to check back against over
    time period.

   Summative through work with service users and
    carers – what is it the partnership specifically does
    for those who use its services?
     Partnership map


                                                  Partnership organisation

                                                  Joint management



          Health    Health    Health   Health
          & LA in   & LA in   &        & social
          civic     ‘whole’   LA       care
          society   system
    Lessons so far…for staff members
 Working with different professionals, from different
  backgrounds with their associated values is difficult, but
  can be very rewarding in an individual sense through
  learning about other professions and expanding
  knowledge of client group.
 “Not sure why we are here......feel it was just because we
  were easy to move and education had to put someone
 “However there are blips - health staff have lines of
  accountability other than to integrated service
    Lessons so far…outcomes
   Conceptualising                Number of the conceptualised
    outcomes is a difficult         outcomes relate more to
    task.                           organisational and structural
                                    issues that user outcomes per
    - Culturally difficult to       se.
    do – i.e., thinking in
    such a different way.           -Integrated management
    However, a number of            system
    people did value this –         -Integrated IT system
    took arguments away
    from structures and             -Pooled budget
    relates to outcomes.            -More team meetings etc
Lessons so far…service user pathways

   „Deep‟ partnership relations strength in
    forming care pathways around the service
    user – but tensions in relating to wider
    partners (integration is always simultaneously
    a process of fragmentation).

   Internalisation of complexity within teams –
    needs quite specific support structures.
Lessons so far…impact on users

   “I am more aware of what is available and don't feel
    isolated and left to deal with it. Thank you”.

   “Received brilliant services, clearly explained and
    very helpful. We miss our key-worker”.

   “it is an early stage to provide any real evidence for it.
    However children and their parents like this
    approach. It has ability to coordinate the service
    provision and reduce duplication”.
    Key Lessons
   Where can relationships be most effective – where is the
    core business / overlap of business?
   Time periods – when would you expect to see impacts.
   Partnerships are necessary under some circumstances and
    for particular client groups as a consequence of
    organisational forms.
   However, this way of working can be complex and difficult,
    requiring quite different mechanisms of training and support.
   What about wider links? Who to include is always a
    concurrent decision about who to exclude.
   This more „nuanced‟ view of working in this way is often what
    is missing from policy guidance.
    The F word! (finance that is)

 A number of the partnerships talked about financial
 In the private sector most partnership working is
  about ££ (reducing duplication, larger budget =
  more bargaining power etc).
 Not a problem if partnerships are about saving
  money – but be more clear about this, otherwise
  risk creating expectations that can‟t deliver on.
If partnership is
   the answer –
    then what is
   the question?
Next steps

 Complete field testing and refinement of
 Write up findings of field testing for
  dissemination (journals, conferences, books)
 Launch tool nationally in conjunction with the
  Care Services Improvement Partnership
  (CSIP) – will be made freely available for
  organisations to use through

Shared By: