Spot Check Survey Report March 2008
Department of Education, Employment and Workplace Relations (DEEWR)
1
RESEARCH
TABLE OF CONTENTS
EXECUTIVE SUMMARY ..................................................... 3 BACKGROUND AND PURPOSE........................................ 4 METHODOLOGY ................................................................. 4 DETAILED RESULTS .......................................................... 5
2
RESEARCH
EXECUTIVE SUMMARY
Piazza Consulting was commissioned by the Department of Education, Employment and Workplace Relations (DEEWR) in March 2008, to conduct research on the child care quality assurance (CCQA) Spot Check process. A telephone survey of 40 child care services was conducted across long day care (15), outside school hour's care (23), family day care (2) services. Commercial and community services were sampled. The group is representative of those services that received a Spot Check between 3 and 19 March 2008. Results were very positive regarding the Spot Check process. Key findings included: Effectiveness - 97% of child care services agreed that the creation of Spot Checks is effective in helping to maintain quality care for children and the principle of having a Spot Check process is also supported. 92% of services believed that the Spot Check process has further helped to keep industry focussed on good practice. A learning tool - 87% of child care services consider Spot Checks as a useful learning tool for maintaining high quality care. Duration - The estimated average duration of Spot Check visits during March 2008 was 1 hour and 36 minutes. Professionalism - All respondents (100%) considered their Validator professional, courteous and constructive, and felt that they did not unduly disrupt staff from their normal routine. Documentation requested - Only 31% of services were required to provide documentation to the Validator during their Spot Check process. Follow-up actions - A third (33%) of services surveyed were required to take follow-up actions as a result of the visit. Continuous improvement - 82% of services believed that a service's continuous improvement process should be reviewed during a spot check. Suggestions for Improvement - Main suggestions for improvement to the process include the provision of written feedback, timing of the visits to fall outside of busy periods, provision of information about the process and Validator. Conclusion: The Spot Check Process is considered a positive and worthwhile process by virtually all child care services surveyed. Disruption levels were minimal with only a minority having been asked to produce documentation. Services regarded the Validators as professional and constructive.
3
RESEARCH
BACKGROUND AND PURPOSE
The CCQA Spot Checks established in 2006 by FaCSIA (Department of Families Community Services and Indigenous Affairs) were created to help monitor and maintain the quality of child care services across Australia. DEEWR, in consultation with the National Childcare Accreditation Council (NCAC), commissioned Piazza Consulting to conduct a short telephone survey of a small sample of services which had received a Spot Check between the 3rd and the 19th of March 2008. This report contains the findings of the survey.
METHODOLOGY
A telephone survey was used to achieve a sample of 41 services from a list of 134 child care services that had received a Spot Check between the 3rd and the 19th of March 2008. Quota sampling was used to achieve a broadly representative sample. Interviewers contacted the services during business hours, and when necessary arranged a time convenient for the respondent. Service directors or management most closely involved with the Spot Check process were interviewed.
4
RESEARCH
DETAILED RESULTS
Demographic
Type of service
Family day care 5% Long day care 37% Out of school hours care 58% Fifty eight percent (58%) of the services surveyed represented outside school hour's care. Thirty seven percent (37%) were from long day care and the remaining 5% were family day care services. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Type of service - community or commercial / private
Commercial / private 38% Community 62% Of the child care services surveyed, 38% were commercial or private services and 62% were community based services. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on frank.fernandez@deewr.gov.au.
5
RESEARCH
General Level of Support for CCQA Spot Checks
QUESTION 1.
The creation of Spot Checks is effective in helping to maintain quality care for children overall.
Disagree 3% Agree 47% Strongly Agree 50% Ninety seven percent (97%) of services agreed or strongly agreed, that Spot Checks are effective in helping to maintain quality care for children. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
6
RESEARCH
QUESTION 2.
I support the principle of having a Spot Check process.
Disagree 3% Agree 45% Strongly Agree 52% Ninety seven percent (97%) of services agreed or strongly agreed, that they supported the principle of having a Spot Check process. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
QUESTION 3.
The Spot Check process has further helped to keep industry focused on good practice.
Disagree 8% Agree 37% Strongly Agree 55% Ninety two percent (92%) of services agreed or strongly agreed that the Spot Check process has further helped to keep industry focused on good practice. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
7
RESEARCH
QUESTION 4.
Spot checks are a useful learning tool for child care services in terms of maintaining high quality child care.
Disagree 13% Strongly Agree 47% Agree 40% Eighty seven percent (87%) of services surveyed agreed or strongly agreed, that Spot Checks are a useful learning tool for child care services. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
The visit process
QUESTION 5.
Approximately how long did your spot check visit take?
DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Thirty nine of the 41 spot checks (95%) lasted under two hours. One lasted two and a quarter hours and one, three hours. The average duration of the Spot Check was 95.75 minutes with standard deviation (SD) of 29 minutes. Length of visit (hours)
8
RESEARCH
QUESTION 6.
Approximately how much time was spent on:
Task Number of Responses a) Observing b) Looking at documentation c) Talking to staff d) Other 34 2 22 10 12 0 38 32 60 22 22 21 Average (mins) Std. Deviation (mins)
On average, 60 minutes was spent by the Validator observing the activities of the service. This represents 63% of average total time spent (see Q5). The remainder of the time was spent looking at documentation and talking to staff. Note "Travel": These two services were both family day care services and both reported that the Validator's time was spent travelling from the central office to the family day care location.
QUESTION 7.
How satisfied are you that your Spot Check Validator provided you and your staff with a useful introduction about the Spot Check process and what they would be doing?
Very Satisfied 44% Satisfied 56% All (100%) of the child care services surveyed were satisfied or very satisfied, with the introduction provided to them by the Validator. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
9
RESEARCH
QUESTION 8.
How satisfied are you that your Validator observed the practice and spoke with staff without unduly disrupting your staff's normal routine?
Satisfied 41% Very Satisfied 59% All respondents (100%) indicated that they were very satisfied or satisfied that the Validator did not unduly disrupt staff. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
QUESTION 9.
How satisfied are you that your Validator was professional in their conduct?
Satisfied 31% Very Satisfied 69% All of the respondents (100%) were very satisfied or satisfied with their Validator's professionalism. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
10
RESEARCH
QUESTION 10.
How satisfied are you that your Validator was courteous in their dealings with you and your staff?
Satisfied 33% Very Satisfied 67% All respondents (100%) were very satisfied or satisfied that their Validator was courteous in their dealings with the service. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
QUESTION 11.
How satisfied are you that your Validator was helpful and constructive in their dealings with you and your staff?
Very Satisfied 54% Satisfied 46% All services (100%) were very satisfied, or satisfied, that the Validator was helpful and constructive during the Spot Check visit. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
11
RESEARCH
QUESTION 12.
How satisfied are you that you had adequate opportunity to discuss the Validator's report at the conclusion of the Spot Check?
Very Dissatisfied 2% Dissatisfied 3% Very Satisfied 45% Satisfied 50% Ninety five percent (95%) of respondents were very satisfied or satisfied with the opportunity they had to discuss the Validator's report at the conclusion of the Spot Check. Only 5% expressed a level of dissatisfaction. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
QUESTION 13.
Did you Validator make any adverse assessments about your practice that you did not agree with?
Yes 13% No 87% The majority (87%) of respondents felt that their Validator did not make any adverse assessments that they disagreed with. Thirteen percent (13%) did. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
12
RESEARCH
QUESTION 14.
Did your Validator ask you to provide any documentation as part of the Spot Check?
Yes 31% No 69% Just under a third (31%) of services were asked to provide documentation during the Spot Check visit. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
QUESTION 15.
Can you please specify the different types of documentation you were asked to produce at the visit?
DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q15ai: Number of documents that services were asked to produce The majority of those services asked to produce documents were required to produce 1 or 2 documents. The average number of items that services were asked to produce was 1.58 with a standard deviation of 0.67. Only 1 service was required to produce 3 documents.
13
RESEARCH
For the minority of service that were asked to provide documentation by the Validator, a range of different types of documents were requested. The most common types of documents requested were policy and procedures documents. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q15aii: Number of services that were asked to produce each type of documentation
QUESTION 15b.
Aside from documentation, can you please specify what else the Validator looked at?
Aside from documentation the Validator mainly looked at parent information, general centre activities, wall posters, and sun safety. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q15b: Comments made by services.
14
RESEARCH
QUESTION 15c.
What, if anything would also be useful for the Validator to look at?
Services suggested that observing general centre operations and functions was the most useful indicator of a centres performance. Checking policy documentation was also suggested. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q15c: Suggestions made by services.
15
RESEARCH
QUESTION 16.
Were any staff called away from their caring responsibilities as a result of the Spot Check visit?
Yes 18% No 82% Eighteen percent (18%) of services reported that staff members were called away from their caring responsibilities as a result of the Spot Check visit. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
If yes, can you please explain the situation where this occurred?
Situations where staff were called away from caring duties were mostly when the centre director or coordinator were asked to discuss the report with the Validator or when the Validators were querying centre staff. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q16: Situations where staff were called away from caring responsibilities
16
RESEARCH
QUESTION 17.
Other than providing documentation, were there any other follow up actions you were required to take as a result of the Spot Check visit?
Yes 33% No 67% Thirty three percent (33%) of the services who had a Spot Check visit were required to take follow-up actions. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
The main follow up actions required as a result of the Spot Check were improving procedures or policy, completing evaluation forms and informing or training staff. Upgrading safety by covering power point outlets was also required by two services. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q17: Follow up actions required
17
RESEARCH
QUESTION 18.
Considering the intended goal of the Spot Checks is to help maintain quality care for children, do you think the level of disruption caused to your Child Care service was acceptable or unacceptable?
Unacceptable 3% Acceptable 97% Virtually all (97%) of the child care services surveyed felt that the level of disruption caused to their service was acceptable. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
18
RESEARCH
Suggestions for Improvement
QUESTION 19.
What improvements can you suggest for the Spot Check process?
DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q19: Suggestions for improvement The main suggestions for improvement of the Spot Check process were; providing written feedback and improving the timing of the visit to avoid clashing with children pick up or drop-off time. Also suggested was to ensure that the Validator looks at the whole service not only a few selected areas.
19
RESEARCH
QUESTION 20.
What do you consider to be the strengths of the Spot Check system?
DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q20: Perceived strengths The main strengths of the Spot Check process suggested by services include: maintaining standards in the industry, the process was unannounced, it provided feedback on service and staff performance, and was helpful and constructive.
20
RESEARCH
QUESTION 21.
What feedback from the NCAC (National Childcare Accreditation Council) would be useful?
DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q21: Suggested useful feedback Child care services suggested they would like to receive written or verbal feedback on spot check results for their service. They would also like more information about the spot check process (i.e. future direction, recent changes etc) and information about the Validator (e.g. their training and what they are looking for).
21
RESEARCH
QUESTION 22.
Should child care services' continuous improvement process and progress be reviewed during a Spot Check?
No 18% Yes 82% The majority (82%) believed that child care services' continuous improvement process and progress should be reviewed during a Spot Check. Eighteen percent (18%) did not. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Why or why not?
All comments made supported the review of child care services' continuous improvement process and progress. The main themes of the comments were that it would ensure that all services were operating on an improvement model, would provide an overall view of the service and complemented the Spot Check process. DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q22: Support for assessing continuous improvement processes
22
RESEARCH
QUESTION 23.
What information or material would you like NCAC to provide to child care services about Spot Checks?
DISCLAIMER: The results of this question are represented in a graph. The graph is not available in this format (RTF). If you would like to view the graph, please see the PDF version of this document, which is available on the same wed page where this document is located. If you can not access the PDF document, please contact Frank Fernandez on
frank.fernandez@deewr.gov.au.
Figure Q23: Desired information or material Services indicated that they would like the NCAC to provide them with information regarding the process, what the Validator will assess, as well as providing the opportunity to give and receive feedback about the process overall.
23