Smart Start Evaluations
Key Studies
From Birth to School: Examining the Effects of Early Childhood Programs on Educational
Outcomes in NC (2010).
Dodge. K., Ladd, H., Mushkin, C. (2010), From Birth to School: Examining the Effects of Early
Childhood Programs on Educational Outcomes in NC. Durham, NC: Center for Child and Family
Policy.
Study Design
Researchers included all children 0-5 years in a county when Smart Start became available,
regardless of whether a child participated directly in a Smart Start service. All children born
in each county were matched to elementary school data (1993 – 2010) to determine
effects. This strategy permits estimation of total effects -- direct effects and spill-over
effects to non-participants.
This study design is well suited for an initiative like Smart Start. Because Smart Start
offers a wide variety of programs (e.g., parenting classes, screenings in doctors’
offices, improvements in the quality in child care) and children participate at
different rates, it is impossible to identify the program or combinations of programs
in which each child participated.
Key Findings
Higher grade-3 standardized math test scores. Scores are higher in counties that had
received higher allocations for Smart Start (equivalent to about 2 months of
instruction on average).
Higher grade-3 standardized reading test scores. Scores are higher in counties that
had received higher allocations for Smart Start (equivalent to about 2 months of
instruction on average).
Reduction in special education placements by grade 3 (about 10%), which amounts
to an expected savings in special education costs at least equal to the cost of the
Smart Start program.
The positive effect is highest among the group of children for whom the initiatives
were targeted -- children at risk for academic failure.
Smart Start Performance-Based Incentive System (PBIS) 2001 – 2009.
The North Carolina Partnership for Children, Inc.
Study Design
The Performance-Based Incentive System (PBIS) measures large-scale changes in factors
critical to children’s healthy development, such as the number of children enrolled in high
quality early childhood education programs, the number of child care teachers who advance
their education and the number of children who receive preventive health care.
PBIS tracks conditions of all children as well as certain vulnerable groups, including
low-income children and children with special needs.
PBIS is an annual evaluation of community indicators of healthy conditions for young
children. PBIS indicators are based in research. Studies done both in North Carolina
as well as nationally find a connection between children’s school readiness and
indicators included in the PBIS tracking system.
All data used to measure PBIS come from sources external to Smart Start.
North Carolina is the only state to institute population outcomes for young children in
every county.
Key Findings
From 2001 to 2009:
The percent of children who attend high quality childcare (4 and 5 star centers) has nearly
doubled - from 33% to 64%.
70%
64%
60%
50%
40%
30% 33%
20%
01 02 03 04 05 06 07 08 09
Year
The percent of low-income children who attend high quality childcare (4 and 5 star centers)
has more than doubled - from 30% to 74%.
80%
70% 74%
60%
50%
40%
30% 30%
20%
01 02 03 04 05 06 07 08 09
Year
Smart Start and Preschool Child Care Quality in NC: Change Over time and Relation to
Children’s Readiness (2003).
Bryant, D., Maxwell, K., Taylor, K., Poe, M., Peisner-Feinberg, E., and Bernier, K. (2003).
Smart Start and Preschool Child Care Quality in NC: Change over Time and Relation to
Children’s Readiness. Chapel Hill, NC: FPG Child Development Institute.
Study Design
This study included 512 children in 110 child care programs of varying levels of quality. Child
care facilities had participated in various Smart Start services (e.g., on-site technical
assistance, workshops, lending libraries) from 4 to 7 years. Researchers assessed the change
in quality over time as well as school readiness scores of children after their program had
participated in Smart Start services (e.g., technical assistance to improve quality and teacher
professional development and training workshops).
Key Findings
76% of children1 who had attended high
quality centers2 scored average or high on
Percent of Children Scoring
a standardized language test3, compared
Average or High
to 45% in low quality centers4.
100%
1
79% of children who had attended high 80%
quality centers1 scored average or high on high quality
60%
a standardized math test5, compared to center
55% from low quality centers. 40%
low quality
20% center
Participation in Smart Start activities was
significantly related to child care quality; 0%
Language Math
generally, the greater the participation in
Smart Start services the higher the quality.
Note - In 2002-03, the General Assembly eliminated evaluation funding for Smart Start. Therefore,
the study could not follow children into elementary school to measure sustainability of improved
learning skills because funds for evaluation were cut from Smart Start. Had researchers been able to
assess school skills over time, we likely would have seen continued sustained benefits, as multiple
national studies have shown.
1
Children had been in their child care center an average of 23 months at the time of testing
2
Score more than 5 on the Early Childhood Environmental Rating Score (ECERS)
3
Score of 85 or above on the Peabody Picture Vocabulary Test - III
4
Score less than 3 on the Early Childhood Environmental Rating Score (ECERS)
5
Score of 85 or higher on the Woodcock-Johnson Psycho-Educational Battery
Six-County Study of the Effects of Smart Start Child Care on Kindergarten Entry Skills
(1999)
Maxwell, K., Bryant, D., & Miller-Johnson, S. (1999). A Six-County Study of the Effects of
Smart Start Child Care on Kindergarten Entry Skills. Chapel Hill, NC: FPG Child Development
Institute.
Study Design
The study included 508 children attending child care from six counties, both children
receiving Smart Start support focused on improving child care quality (n=214) and a
comparison group of children enrolled in other child care centers or family child care homes
(n=294) that did not receive Smart Start support. In this study, Smart Start support included
child care quality child improvement activities such as on-site technical assistance (e.g., a
consultant visits the center and provides center-specific or classroom-specific suggestions
for improving the quality of care), health and safety training and grants, and educational
scholarships for teachers. The researchers gathered information about the cognitive,
language, and social skills of these children as they began kindergarten.
Key Findings
91% of children in centers that received Smart Start services to improve quality
(technical assistance, teacher workshops, etc,) received high scores in readiness,
language and social skills, compared to 84% of children who attended centers that
did not receive Smart Start support.
______________________________
Score above 3.71 on the Kindergarten Teacher Checklist (KTC)
Score above 87.84 on the Peabody Picture Vocabulary Test – III (PPVT-III)
Score above 85.46 on the Social Skills Rating Scale (SSRS)
The Effects of Smart Start Child Care on Kindergarten Entry Skills (1998)
Bryant, D., Bernier, K., Taylor, K., & Maxwell, K. (1998). The Effects of Smart Start Child Care on
Kindergarten Entry Skills. Chapel Hill, NC: FPG Child Development Institute.
Study Design
This study examined school readiness skills for 39 kindergarten students in one county who
attended child care centers participating intensely in Smart Start-funded quality
improvements compared to a comparison group of 272 randomly-selected kindergartners
who attended a broad range of child care or no child care. Kindergarten teachers rated
cognitive, language and social skills of both groups of kindergartners.
Key Findings
Smart Start child care quality enhancements6 resulted in higher levels of skills among
children from low-income families attending Smart Start participating centers.
Children from low-income families who attended centers that participated in Smart Start
quality enhancement services scored almost a half point (scale of 1 to 5 points) higher on
measures of school readiness7 than children from low-income families who attended
centers that that did participate in Smart Start quality enhancement services.
Children from low-income families who attended child care centers that participated
in Smart Start quality enhancement services scored an average of 4.07 on a school
readiness assessment.
Children from low-income families who attended child care centers that did not
participate in Smart Start quality enhancement services scored an average of 3.64 on
a school readiness assessment.
6
Enhancements included services such as teacher training and workshops, incentives for teachers to seek
higher education, technical assistance to improve classrooms, etc.
7
Teacher Kindergarten Checklist
The Region A Partnership for Children Longitudinal Study Final Report (2005, rev 2007).
Region A Partnership for Children: The Region A Partnership for Children Longitudinal Study
Final Report (2005, rev 2007). Conducted by Bruce Yelton, Ph D, Praxis Research, Inc. and
Paula Plonski, M.A., Praxis Research.
Study Design
This study enrolled children who had received Smart Start services (e.g., parent education
classes, use of a family resource center, newborn home visits). Children were tracked from
kindergarten to third grade. Over 400 families from seven counties participated. Data were
collected from 2002 to 2004 in 24 to 31 public schools depending upon the year of the
study.
Key Findings
Third grade end-of-grade (EOG) scores for children who had received Smart Start
services8 prior to public school were higher when compared to the third grade EOG
scores reported for all third graders in regional schools.
This finding in particularly noteworthy in that participants having contact with Smart
Start services were more likely to have multiple risk factors for academic
achievement such as low family income, behavioral or health problems, and high
family geographic mobility.
8
62% of study participants attended a child care center that had received Smart Start services such as
technical assistance to improve quality or child care subsidies. 38% of study participants had received Smart
Start services outside of child care, such as Parents as Teachers educational home visits or reading/story time
programs.