Spot Check Survey Report
March 2008
Department of Education,
Employment and Workplace Relations
(DEEWR)
1
RESEARCH
TABLE OF CONTENTS
EXECUTIVE SUMMARY ..................................................... 3
BACKGROUND AND PURPOSE........................................ 4
METHODOLOGY ................................................................. 4
DETAILED RESULTS .......................................................... 5
2
RESEARCH
EXECUTIVE SUMMARY
Piazza Consulting was commissioned by the Department of Education, Employment and
Workplace Relations (DEEWR) in March 2008, to conduct research on the child care quality
assurance (CCQA) Spot Check process.
A telephone survey of 40 child care services was conducted across long day care (15), outside
school hour's care (23), family day care (2) services. Commercial and community services were
sampled. The group is representative of those services that received a Spot Check between 3
and 19 March 2008.
Results were very positive regarding the Spot Check process. Key findings included:
Effectiveness - 97% of child care services agreed that the creation of Spot Checks is
effective in helping to maintain quality care for children and the principle of having a
Spot Check process is also supported. 92% of services believed that the Spot Check
process has further helped to keep industry focussed on good practice.
A learning tool - 87% of child care services consider Spot Checks as a useful learning
tool for maintaining high quality care.
Duration - The estimated average duration of Spot Check visits during March 2008 was
1 hour and 36 minutes.
Professionalism - All respondents (100%) considered their Validator professional,
courteous and constructive, and felt that they did not unduly disrupt staff from their
normal routine.
Documentation requested - Only 31% of services were required to provide
documentation to the Validator during their Spot Check process.
Follow-up actions - A third (33%) of services surveyed were required to take follow-up
actions as a result of the visit.
Continuous improvement - 82% of services believed that a service's continuous
improvement process should be reviewed during a spot check.
Suggestions for Improvement - Main suggestions for improvement to the process
include the provision of written feedback, timing of the visits to fall outside of busy
periods, provision of information about the process and Validator.
Conclusion: The Spot Check Process is considered a positive and worthwhile process by
virtually all child care services surveyed. Disruption levels were minimal with only a minority
having been asked to produce documentation. Services regarded the Validators as
professional and constructive.
3
RESEARCH
BACKGROUND AND PURPOSE
The CCQA Spot Checks established in 2006 by FaCSIA (Department of Families Community
Services and Indigenous Affairs) were created to help monitor and maintain the quality of child
care services across Australia.
DEEWR, in consultation with the National Childcare Accreditation Council (NCAC),
commissioned Piazza Consulting to conduct a short telephone survey of a small sample of
services which had received a Spot Check between the 3rd and the 19th of March 2008.
This report contains the findings of the survey.
METHODOLOGY
A telephone survey was used to achieve a sample of 41 services from a list of 134 child care
services that had received a Spot Check between the 3rd and the 19th of March 2008. Quota
sampling was used to achieve a broadly representative sample. Interviewers contacted the
services during business hours, and when necessary arranged a time convenient for the
respondent. Service directors or management most closely involved with the Spot Check
process were interviewed.
4
RESEARCH
DETAILED RESULTS
Demographic
Type of service
Family day care 5%
Long day care 37%
Out of school hours care 58%
Fifty eight percent (58%) of the services surveyed represented outside school hour's care.
Thirty seven percent (37%) were from long day care and the remaining 5% were family day
care services.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Type of service - community or commercial / private
Commercial / private 38%
Community 62%
Of the child care services surveyed, 38% were commercial or private services and 62% were
community based services. DISCLAIMER: The results of this question are represented in a
graph. The graph is not available in this format (RTF). If you would like to view the graph,
please see the PDF version of this document, which is available on the same wed page where
this document is located. If you can not access the PDF document, please contact Frank
Fernandez on frank.fernandez@deewr.gov.au.
5
RESEARCH
General Level of Support for CCQA Spot Checks
QUESTION 1.
The creation of Spot Checks is effective in helping to maintain quality care for children
overall.
Disagree 3%
Agree 47%
Strongly Agree 50%
Ninety seven percent (97%) of services agreed or strongly agreed, that Spot Checks are
effective in helping to maintain quality care for children.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
6
RESEARCH
QUESTION 2.
I support the principle of having a Spot Check process.
Disagree 3%
Agree 45%
Strongly Agree 52%
Ninety seven percent (97%) of services agreed or strongly agreed, that they supported the
principle of having a Spot Check process.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
QUESTION 3.
The Spot Check process has further helped to keep industry focused on good practice.
Disagree 8%
Agree 37%
Strongly Agree 55%
Ninety two percent (92%) of services agreed or strongly agreed that the Spot Check process
has further helped to keep industry focused on good practice.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
7
RESEARCH
QUESTION 4.
Spot checks are a useful learning tool for child care services in terms of maintaining
high quality child care.
Disagree 13%
Strongly Agree 47%
Agree 40%
Eighty seven percent (87%) of services surveyed agreed or strongly agreed, that Spot Checks
are a useful learning tool for child care services.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
The visit process
QUESTION 5.
Approximately how long did your spot check visit take?
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Thirty nine of the 41 spot checks (95%) lasted under two hours. One lasted two and a quarter
hours and one, three hours.
The average duration of the Spot Check was 95.75 minutes with standard deviation (SD) of 29
minutes. Length of visit (hours)
8
RESEARCH
QUESTION 6.
Approximately how much time was spent on:
Task Number of Average (mins) Std. Deviation (mins)
Responses
a) Observing 38 60 22
b) Looking at 32 22 21
documentation
c) Talking to staff 34 22 12
d) Other 2 10 0
On average, 60 minutes was spent by the Validator observing the activities of the service. This
represents 63% of average total time spent (see Q5). The remainder of the time was spent
looking at documentation and talking to staff.
Note "Travel": These two services were both family day care services and both reported that
the Validator's time was spent travelling from the central office to the family day care location.
QUESTION 7.
How satisfied are you that your Spot Check Validator provided you and your staff with a
useful introduction about the Spot Check process and what they would be doing?
Very Satisfied 44%
Satisfied 56%
All (100%) of the child care services surveyed were satisfied or very satisfied, with the
introduction provided to them by the Validator.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
9
RESEARCH
QUESTION 8.
How satisfied are you that your Validator observed the practice and spoke with staff
without unduly disrupting your staff's normal routine?
Satisfied 41%
Very Satisfied 59%
All respondents (100%) indicated that they were very satisfied or satisfied that the Validator did
not unduly disrupt staff.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
QUESTION 9.
How satisfied are you that your Validator was professional in their conduct?
Satisfied 31%
Very Satisfied 69%
All of the respondents (100%) were very satisfied or satisfied with their Validator's
professionalism.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
10
RESEARCH
QUESTION 10.
How satisfied are you that your Validator was courteous in their dealings with you and
your staff?
Satisfied 33%
Very Satisfied 67%
All respondents (100%) were very satisfied or satisfied that their Validator was courteous in
their dealings with the service.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
QUESTION 11.
How satisfied are you that your Validator was helpful and constructive in their dealings
with you and your staff?
Very Satisfied 54%
Satisfied 46%
All services (100%) were very satisfied, or satisfied, that the Validator was helpful and
constructive during the Spot Check visit.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
11
RESEARCH
QUESTION 12.
How satisfied are you that you had adequate opportunity to discuss the Validator's
report at the conclusion of the Spot Check?
Very Dissatisfied 2%
Dissatisfied 3%
Very Satisfied 45%
Satisfied 50%
Ninety five percent (95%) of respondents were very satisfied or satisfied with the opportunity
they had to discuss the Validator's report at the conclusion of the Spot Check. Only 5%
expressed a level of dissatisfaction.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
QUESTION 13.
Did you Validator make any adverse assessments about your practice that you did not
agree with?
Yes 13%
No 87%
The majority (87%) of respondents felt that their Validator did not make any adverse
assessments that they disagreed with. Thirteen percent (13%) did.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
12
RESEARCH
QUESTION 14.
Did your Validator ask you to provide any documentation as part of the Spot Check?
Yes 31%
No 69%
Just under a third (31%) of services were asked to provide documentation during the Spot
Check visit.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
QUESTION 15.
Can you please specify the different types of documentation you were asked to produce
at the visit?
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q15ai: Number of documents that services were asked to produce
The majority of those services asked to produce documents were required to produce 1 or 2
documents.
The average number of items that services were asked to produce was 1.58 with a standard
deviation of 0.67. Only 1 service was required to produce 3 documents.
13
RESEARCH
For the minority of service that were asked to provide documentation by the Validator, a range
of different types of documents were requested. The most common types of documents
requested were policy and procedures documents.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q15aii: Number of services that were asked to produce each type of documentation
QUESTION 15b.
Aside from documentation, can you please specify what else the Validator looked at?
Aside from documentation the Validator mainly looked at parent information, general centre
activities, wall posters, and sun safety.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q15b: Comments made by services.
14
RESEARCH
QUESTION 15c.
What, if anything would also be useful for the Validator to look at?
Services suggested that observing general centre operations and functions was the most useful
indicator of a centres performance. Checking policy documentation was also suggested.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q15c: Suggestions made by services.
15
RESEARCH
QUESTION 16.
Were any staff called away from their caring responsibilities as a result of the Spot
Check visit?
Yes 18%
No 82%
Eighteen percent (18%) of services reported that staff members were called away from their
caring responsibilities as a result of the Spot Check visit.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
If yes, can you please explain the situation where this occurred?
Situations where staff were called away from caring duties were mostly when the centre
director or coordinator were asked to discuss the report with the Validator or when the
Validators were querying centre staff.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q16: Situations where staff were called away from caring responsibilities
16
RESEARCH
QUESTION 17.
Other than providing documentation, were there any other follow up actions you were
required to take as a result of the Spot Check visit?
Yes 33%
No 67%
Thirty three percent (33%) of the services who had a Spot Check visit were required to take
follow-up actions.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
The main follow up actions required as a result of the Spot Check were improving procedures
or policy, completing evaluation forms and informing or training staff. Upgrading safety by
covering power point outlets was also required by two services.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q17: Follow up actions required
17
RESEARCH
QUESTION 18.
Considering the intended goal of the Spot Checks is to help maintain quality care for
children, do you think the level of disruption caused to your Child Care service was
acceptable or unacceptable?
Unacceptable 3%
Acceptable 97%
Virtually all (97%) of the child care services surveyed felt that the level of disruption caused to
their service was acceptable.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
18
RESEARCH
Suggestions for Improvement
QUESTION 19.
What improvements can you suggest for the Spot Check process?
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q19: Suggestions for improvement
The main suggestions for improvement of the Spot Check process were; providing written
feedback and improving the timing of the visit to avoid clashing with children pick up or drop-off
time. Also suggested was to ensure that the Validator looks at the whole service not only a few
selected areas.
19
RESEARCH
QUESTION 20.
What do you consider to be the strengths of the Spot Check system?
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q20: Perceived strengths
The main strengths of the Spot Check process suggested by services include: maintaining
standards in the industry, the process was unannounced, it provided feedback on service and
staff performance, and was helpful and constructive.
20
RESEARCH
QUESTION 21.
What feedback from the NCAC (National Childcare Accreditation Council) would be
useful?
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q21: Suggested useful feedback
Child care services suggested they would like to receive written or verbal feedback on spot
check results for their service. They would also like more information about the spot check
process (i.e. future direction, recent changes etc) and information about the Validator (e.g. their
training and what they are looking for).
21
RESEARCH
QUESTION 22.
Should child care services' continuous improvement process and progress be reviewed
during a Spot Check?
No 18%
Yes 82%
The majority (82%) believed that child care services' continuous improvement process and
progress should be reviewed during a Spot Check. Eighteen percent (18%) did not.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Why or why not?
All comments made supported the review of child care services' continuous improvement
process and progress. The main themes of the comments were that it would ensure that all
services were operating on an improvement model, would provide an overall view of the service
and complemented the Spot Check process.
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q22: Support for assessing continuous improvement processes
22
RESEARCH
QUESTION 23.
What information or material would you like NCAC to provide to child care services
about Spot Checks?
DISCLAIMER: The results of this question are represented in a graph. The graph is not
available in this format (RTF). If you would like to view the graph, please see the PDF version
of this document, which is available on the same wed page where this document is located. If
you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q23: Desired information or material
Services indicated that they would like the NCAC to provide them with information regarding the
process, what the Validator will assess, as well as providing the opportunity to give and receive
feedback about the process overall.
23