Paper102.ppt by zhouwenjuan


									Challenging the ‘New
      Users’ perspectives on
      evaluation in family support
                            Natasha Cortis, PhD student,
                 Political Economy, University of Sydney

              With the support of UnitingCare Burnside
                    & the Australian Research Council
Why do users matter to
performance measurement?
   Argument 1:
       Users feed essential data into evaluation

   Argument 2:
       Users have a basic right to participate in
        deciding how well services are working
  Theoretical           Argument for user
   position         involvement in evaluation
  Consumerism      Source of essential data

  Managerialism    Source of essential data

 Professionalism   Basic right

User empowerment Basic right

  Stakeholder      Source of essential data &
   pluralism       Basic right
                   Source of essential data &
                   Basic right
But limited user involvement in practice
Eg: Main Govt performance measures
                             Numeric counts of service
1. Output (turnover)
                             episodes, clients assisted

2. Resource inputs           $ funding levels or staffing levels

                             Administrative data eg change
3. Results                   in number of child protection

4. Characteristics           No. clients from target groups

5. Outcome                   **Quality of life changes**

                     Sources: PC (2005); CCQG (2003); AIHW (2001); DOCS (2002)
Data collection

   Observations
     70 hours (parenting groups & meetings)

   Service user focus groups & interviews
     52 people ( in 7 focus groups )

     8 private interviews ( 3 with translator )

     6 people in 3 pair interviews

                   = 66 service users
   9 Staff interviews (not reported here)
in the study                          Outer West

Lightest = most disadvantaged

                                                         South West


                            coast           Source: SEIFA index of
NSW                                         disadvantage, 2001 Census
Findings 1: Users’ private evaluations

Six main sources of evidence
1.   Change in legal or family status
2.   Change in self-perception as parent
3.   Self-assessed psychosocial health
4.   Gain of tangible parenting skills
5.   Feedback from peers
6.   Feedback from children
Findings 2: Users’ preferred evidence for

Main evidence sources
1.   Users’ self reports of change
2.   Observation of service user change
3.   ‘Spontaneous’ feedback gathered by
4.   Attendance data
5.   Levels of reports of abuse and neglect in
     service user families
Findings 3: Users’ perspectives on
evaluation processes

   Users thought evaluation should be based on:
       Personal relationships
       Social justice, respect and understanding
       Formative approach
Implications for performance
   Common ground- admin data
       Child protection events / Attendance data
   Differences
       Critical of counting, and detached, external, non-
        involved evaluation
       Preferred inclusive methods, personal engagement
   Contribution of research
       Shows users’ preferred ways of evaluating performance
       Challenge idea that users’ evaluations are subjective,
        anecdotal, unsystematic

To top