SREE 2008 Conference Structured Abstract by 6z7A81V

VIEWS: 0 PAGES: 12

									                                         Abstract Title Page



Title:

    Effectiveness of comprehensive professional development for teachers of at-risk preschoolers



Author(s):

    Susan Landry, PhD, University of Texas Health Science Center at Houston;

    Jason Anthony, PhD, University of Texas Health Science Center at Houston;

    Paul Swank, PhD, University of Texas Health Science Center at Houston;

    Pauline Monseque-Bailey, MEd, University of Texas Health Science Center at Houston




2008 SREE Conference Abstract Template
Background/context:

    Despite accumulated research that has identified the components and nature of superior
prekindergarten instruction, scaling of research-based practices in early childhood settings has
been met with much opposition and it faces major challenges like disbelief that research-based
practices can be implemented effectively by typical teachers in typical child care settings or
classrooms composed of at-risk children who need high quality instruction the most. The present
study exemplifies an effort to scale research-based practices through four overlapping
professional development programs. These programs included all three essential elements of
high quality PD programs as described by Showers, Joyce, & Bennett (1987): (1) presentation of
content knowledge and theory behind recommended practices, (2) opportunities for
demonstration and hands-on practice, and (3) prompt feedback to teachers as they engage in new
practices. Specifically, the four PD programs evaluated in this study included a year-long online
course, practice of learned material in one’s classroom, and participation in online message
boards with fellow teachers. In addition, teachers in two PD programs were provided with
frequent mentoring, and teachers in two PD programs received detailed, instructionally linked
feedback on children’s progress monitoring results.

Purpose/objective/research question/focus of study:

    The purpose of this study was twofold: One, serve as a demonstration of scaling research-
based professional development programs, and, two, test the efficacy of various components of
our professional development model. We hypothesized that teachers in all four professional
development programs would evince better teaching by the end of the school year and that their
pupils would demonstrate better language and emergent literacy by the end of the year than those
in control classrooms. We also hypothesized that teachers who received mentoring would evince
better teaching by the end of the school year and that their pupils would demonstrate better
language and emergent literacy by the end of the year than those not receiving mentoring.
Teachers who received detailed, instructionally relevant feedback on children’s progress were
hypothesized to evince better teaching and have pupils with better achievement by the end of the
year than those who received only children’s scores on the progress monitoring tool.

Setting:

   This multisite project took place in Columbus, OH, Prince George, MD, Miami, FL, and
Corpus Christi, TX.

Population/Participants/Subjects:

    In total, 158 schools housed 265 prekindergarten teachers who participated. These teachers
provided child care services to children from low SES backgrounds. Eight children from each of
the 265 classrooms were randomly selected for assessment, which totaled 1,786 children.
Children ranged in age from 3- to 5-years (Mean=4.3, SD=0.5). Half of the children were boys.
The sample was 17% Caucasian, 30% African American, 21% Hispanic, and 32% other.
Seventy-three percent of children spoke English in their home, 27% spoke Spanish in their home,
and 10% spoke a language other than English or Spanish in their home.




2008 SREE Conference Abstract Template                                                          1
Intervention/Program/Practice:

    The multiple day, face-to-face training workshops developed by the Center for Improving the
Readiness of Children for Learning and Education (CIRCLE; Landry, Swank, Smith, Assel, &
Gunnewig, 2006) were adapted for an online application. The nine courses cover language and
early literacy instruction, classroom management, responsive teaching practices, and early math.
The on-line program promotes an appropriate balance between developmentally appropriate,
teacher-directed activities designed to foster development of specific skills, and child-directed
activities designed to allow children to enhance mastery and breadth of skills through active
exploration. Consistent with research on adult learning, the online course included (a) small-
group interactive learning facilitated by a trainer with opportunity for independent review of all
course contents; (b) each course content included extensive videotaped modeling and expert
commentaries, (c) active engagement of learning with online assessments of knowledge, (d)
opportunities for practicing specific skills within the small group coursework (e.g., role playing,
development of lesson plans), and (e) teachers’ posting of experience with specific instructional
activities in their own classroom with trainer review and feedback.

    Some teachers received two hours of in-classroom mentoring every other week by mentors
trained by CIRCLE. Mentoring focused on application of the content covered in the online
course.

    While all teachers in the four PD groups administered CPALS three times per year to
monitor children’s academic progress, some teachers used the Personal Digital Assistant (PDA)
version of the screener, called mCLASS. mCLASS provides teachers with immediate feedback
about individual children’s growth in specific skill areas, how to group children for more
effective learning, and specific activities to use with different small groups of children. Teachers
who administered the paper and pencil version, CPALLS, were only provided children’s scores.

Research Design:

    A randomized experimental design was used to compare the effectiveness of the four
professional development programs relative to “business as usual” control group. Schools were
randomly assigned to experimental condition. A 2x2 (Mentoring x Feedback) design yielded four
PD conditions: (1) mentored with detailed feedback, (2) nonmentored with detailed feedback, (3)
mentored with limited feedback, and (4) nonmentored with limited feedback. All outcomes were
assessed in the beginning of the school year (Fall of 2004 or Fall of 2005) and again at the end of
the school year (Spring of 2005 or Spring of 2006), so that change in relevant outcomes as a
function of experimental condition could be tested.

Data Collection and Analysis:

    The outcomes on which professional development programs were compared were changes in
children’s achievement, changes in teachers’ behavior, and changes in the instructional
environment. Children’s achievement outcomes included scores on tests of oral language,
vocabulary, phonological awareness, letter knowledge, and print awareness (see Table 1).
Children’s achievement was tested in the language of instruction. Specifically, 90% of children
were tested in English, and 10% were tested with parallel Spanish tests.



2008 SREE Conference Abstract Template                                                             2
    Teacher outcomes included indices of frequency and quality of each of the following: Shared
reading, lesson planning, oral language instruction, print & letter knowledge instruction,
phonological awareness instruction, writing instruction, center-based instruction, team teaching,
and use of assessment. A composite score of all of the positive teaching behaviors above was
also calculated. The 21 indicators of teaching quality were based on ratings of trained observers
who were blind to teachers’ treatment condition. Raters observed 60 of the teachers at the
beginning and end of the school year. Teachers were randomly selected to be observed, and they
were evenly distributed across the 5 experimental conditions.

Findings/Results:

Children’s Achievement.
        To examine the effectiveness of the professional development programs on children’s
achievement, we used mixed modeling to account for the lack of independence among
observations, given that children were nested in classrooms. Mixed modeling ANCOVA
analyses additionally controlled for children’s chronological age, pretest achievement scores, site
differences, and any significant interactions among these covariates when predicting children’s
end-of-year achievement. Only children who provided both pretest data and posttest data were
included in analyses of PD effects on child outcomes, ns =1607-1678.
        Analysis of children’s vocabulary data found no significant interactions between group
and site. A priori group contrasts revealed the two Detailed Feedback groups had significantly
higher posttest scores on the EOWPVT than the two Limited Feedback groups, t = 2.18, p < .05.
The interaction between Mentoring and Feedback conditions approached significance, t = 1.81, p
= .07, and favored the Mentored with Detailed Feedback condition, which had significantly
higher posttest scores than the control group, t = 2.08, p < .05 (see Figure 1).
        Analysis of children’s phonological awareness performances found no significant
interactions between group and site. Collectively, children in the four PD groups had
significantly higher posttest cores on the DSC/La Lista than children in the control group, t =
2.12, p < .05 (see Figure 2). Individually, only the Nonmentored with Detailed Feedback group
had reliably higher posttest scores than the control group, t = 1.96, p = .05.
        Analysis of children’s oral language revealed a significant Group by Site by Pretest
interaction, t = 3.97, p < .0001, which indicated that the effects of the PD programs on children’s
oral language were moderated by children’s prior oral language and site differences. Across
Groups, the moderating effects of children’s pretest oral language were stronger in Texas than all
other sites, t = 2.11, p <.05 (see Figure 3a). In contrast, the moderating effects of pretest oral
language were weakest in Florida, t = -4.27, p <.0001. The weaker effect of pretest oral language
in Florida was because pretest scores were less related to children’s end-of-year oral language if
children’s teaches were in one of the PD programs, t = -3.29, p < .001 (see Figure 3b). In
Maryland, there was a significant Mentoring by Feedback by Pretest interaction, t = 302, p < .01.
This interaction demonstrated that children’s oral language benefited most if they had limited
oral language at the beginning of the year and had teachers who were mentored and received
detailed feedback from the palm (see Figure 3c). Also in Maryland, children with teachers who
received Detailed Feedback generally had higher end-of-year PLS scores than children with
teachers who received only Limited Feedback, t = 3.68, p < .001. In Ohio, the Mentored group
had higher end-of-year PLS scores than the Nonmentored group, t = 2.89, p < .01 (see Figure
3d). However, the Control group had higher PLS scores than the 4 PD groups combined, t =



2008 SREE Conference Abstract Template                                                           3
3.33, p = .001, which was largely due to the Control group having higher PLS scores than the 2
Nonmentored groups, ts = 3.89-4.02, ps < .0001.
    Analysis of children’s end-of-year Print Awareness revealed a significant Group by Site
interaction, F = 2.54, p < .01 (see Figure 4). In Texas, the two groups that received Detailed
Feedback achieved higher end-of-year PCTOPP scores than the two groups that received Limited
Feedback, t = 4.38, p < .0001. Likewise, the two groups that received Detailed Feedback
individually achieved higher end-of-year PCTOPP scores than the Control group, ts = 2.17-3.54,
ps < .05. In Florida, a significant Mentoring by Feedback interaction favored the group who
received both mentoring and detailed feedback, t = 3.18, p < .01. In Ohio, the two groups that
received Mentoring achieved higher end-of-year PCTOPP scores than the two groups that did
not receive mentoring, t = 2.45, p = .01.

Teacher behavior and classroom environment.
        To further exam the effectiveness of the professional development programs, we also
performed ANCOVA analyses using treatment condition to predict teachers’ end-of-year scores
on 21 indicators of teaching quality, controlling for teachers’ beginning-of-year scores on the
same indicator. A priori group contrasts revealed that teachers in the Mentored with Detailed
Feedback condition scored significantly higher than teachers in the control group on 8 of the 21
posttest instructional indicators, ps < .05 (see Tables 2 and 3). Teachers in the Nonmentored with
Detailed Feedback condition scored significantly higher than teachers in the control group on 7
of the 21 posttest instructional scores, ps < .05 (see Tables 2 and 3). Teachers in the Mentored
with Limited Feedback condition scored significantly higher than control teachers on 2 of the 21
posttest scores, and teachers in the Nonmentored with Limited Feedback condition scored
significantly higher than control teachers on 4 of the 21 posttest instructional indicators (see
Tables 2 and 3).
        There were significant main effects of Feedback on 5 of the 21 indicators of instructional
quality. These main effects suggested that teachers who were provided detailed, instructionally
relevant feedback from palm-based progress monitoring conducted more frequent shared
reading, conducted better print and letter knowledge instruction (see Figure 5), implemented
better lesson planning (see Figure 6), made better use of progress monitoring results, and were
generally better teachers (see Figure 7) than teachers who were only provided with children’s
scores on the progress monitoring measure (see Tables 2 and 3).
        Finally, significant Mentoring by Feedback interactions on 4 instructional indicators, ps <
.05, revealed that teachers who received ongoing mentoring and also received detailed,
instructionally relevant feedback on children’s progress monitoring results conducted higher
quality shared reading instruction, conducted more frequent and higher quality center-based
instruction, and in general were better teachers at the end of the school year, controlling for
pretest scores on the same instructional indicator (see Tables 2 and 3).


Conclusions:

        The first take home message from this demonstration project is that research-based best
practices can be scaled in typical child care and early childhood education settings via high
quality, ongoing professional development (PD). The second take home message is that PD
programs are most effective when they are comprehensive and well integrated. This study
provided clear evidence for such a PD package for teachers of at-risk prekindergarten children.


2008 SREE Conference Abstract Template                                                           4
         The most powerful of the four PD programs evaluated was the most comprehensive and
well integrated program that involved having teachers participate in a year-long online PD
course, practice what they learned in their classrooms, communicate regularly with other
participating teachers via online message boards, receive mentoring every other week over the
course of a year, and be provided with detailed feedback on individual children’s academic
progress along with recommended instructional activities. Teachers who received this
comprehensive PD package became better teachers. That is, they improved the quality of their
writing instruction, phonological awareness instruction, letter knowledge instruction, shared
reading instruction, and center-based instruction, and they kept more detailed and more useful
portfolios on children.
         Not only was the effectiveness of the most comprehensive PD package evident on quality
of teaching and classroom environments, but it was also evident in how much children learned.
Specifically, teachers who received the most comprehensive PD package graduated children who
had larger vocabularies, more highly developed phonological awareness, and more knowledge of
letters and print concepts at the end of the year than teachers who received no supplemental PD.
The positive effects of the PD programs on children’s oral language was most noticeable with
children who had low levels of oral language when they first entered the programs.
         Although regular mentoring and detailed, instructionally relevant feedback on children’s
progress were somewhat helpful in terms of improving children’s outcomes, it was the
combination of year-long coursework, hands-on practice in classrooms, communication and
accountability among peer teachers and mentors, and linking children’s progress monitoring
results with instruction that yielded the most impressive changes in quality of teachers
instruction and amount of children’s learning.
         The findings from this multi-site study have the potential to inform professional
development practices in the field of early childhood education by demonstrating the need for a
comprehensive set of supports for teachers in order to promote effective instructional practices in
language and literacy. In addition, the comprehensive and well integrated professional
development approach was effective across a diverse range of early childhood education settings
where teachers varied greatly in their level of education. This finding has the potential to inform
state education policy regarding decisions concerning finding a balance between teacher
education requirements and necessary professional development supports.




2008 SREE Conference Abstract Template                                                           5
                                        Appendix B. Tables and Figures
 Table 1. Child outcome measures.
          Construct                                                  Test
 Vocabulary             English and Spanish versions of the Expressive One-Word Picture
                         Vocabulary Test – Third Edition (EOWPVT)
 Phonological awareness Developmental Skills Checklist (DSC; English) or La Lista (Spanish)

 Oral language                  English and Spanish versions of the Preschool Language Scales – Fourth
                                 Edition (PLS)
 Letter knowledge &             English and Spanish versions of Preschool Comprehensive Test of
  print awareness                Phonological and Print Processing (PCTOPPP)


 Table 2. Effects of Professional Development Programs on Teaching Quality.
                                        Control v. Control v. Control v. Control v.
                                        Mentored Mentored No mentor No mentor Palm v. Mentor v. Interaction
                                          Palm      Pencil      Palm      Pencil    Pencil No Mentor
             Indicator            Df        F         F          F           F         F       F       F
Total quality                     53     5.35*       3.08       5.45*      0.91      1.88     0.33   0.36
Total Quantity                    53     7.08**      1.32       4.30*      0.22      5.14*    0.85   0.01
Book read quality                 51     1.84        2.18       2.75       0.29      0.46     0.27   0.95
Book read quantity                47     6.37**      1.68       4.91*      8.71**    0.15     2.60   4.62*
General teaching quality          53     0.15        0.73       2.07       0.47      1.51     0.16   3.74*
General teaching quantity         53     0.02        0.02       0.73       0.47      0.89     0.01   1.89
Lesson plans quality              53     1.09        0.51       3.90*      0.65      5.28*    0.21   3.33
Lesson plans quantity             49     1.18        0.74       4.69*      8.11**    0.12     4.65*  0.87
Oral Language quality             49     0.02        0.01       0.10       2.48      1.27     3.81*  2.53
Oral Language quantity            53     0.47        1.05       0.32       0.02      0.07     0.88   0.61
Print & letter knowledge quality 53      9.24**      0.69       4.76*      0.59     6.86**    0.44   0.36
Print & letter knowledge          53     9.54**      3.18       3.59       2.23      1.41     1.16   0.48
quantity
Team Teaching quality             38     0.00       1.81      0.37     0.62      0.70     4.24*     1.32
Team Teaching quantity            31     0.23       3.48      0.69     0.61      2.81     4.64*     2.86
Centers quality                   49     3.03       0.22      0.09     5.67      0.21     0.07      5.85**
 * p < .05. ** p < .01. *** p < .001.



 Table 3. Effects of Professional Development Programs on Teaching Quality.
                               Control v. Control v. Control v. Control v.
                               Mentored Mentored No mentor No mentor Palm v.          Mentor v. Interaction
                                 Palm       Pencil     Palm       Pencil     Pencil   No Mentor
           Indicator           Chi-square Chi-square Chi-square Chi-square Chi-square Chi-square Chi-square
 Centers quantity                 1.29       1.94       0.08        2.92      0.16      2.33      13.19***
 Assessment use                   0.95       0.04       1.69        0.43      5.26*     0.05        0.36
 Phonological quality             7.67**     6.73**     2.38        3.91*     0.14      2.06        0.26
 Portfolio quantity               6.05**     0.67       2.21        3.12      0.98      0.00        2.36
 Written expression quality     13.34***     7.26**     7.52**      9.56**    0.14      0.40        1.25
 Written expression quantity      2.56       1.32       0.75        0.88      0.05      0.44        0.14
 * p < .05. ** p < .01. *** p < .001.



 2008 SREE Conference Abstract Template                                                                B–1
Figure 1.
                  Predicted Standard Scores on EOWPVT Vocabulary by Intervention Group


  95



  90



  85



  80



  75



  70
             Control          Mentor_Palm      Mentor_Pencil     Nomentor_Palm      Nomentor_Pencil


Figure 2.

            Predicted Raw Scores on DSC Phological Awareness by Intervention Group




  10



    8



    6



    4



    2



    0
             Control        Mentor_Palm       Mentor_Pencil    Nomentor_Palm     Nomentor_Pencil




2008 SREE Conference Abstract Template                                                             B–2
                   Figure 3a (Texas).                               Figure 3b (Florida).



 60
                                                      60


 55                                                   55


 50                                                   50


 45                                                   45


 40                                                   40
      35      40         45         50     55              35       40        45        50    55
                     PLS pretest
                                                                          PLS Pretest


                   Figure 3c (Maryland).                                  Figure 3d (Ohio).


 60                                                       60


 55                                                       55


 50                                                       50


 45                                                       45


 40                                                       40
      35      40          45         50     55                 35    40        45      50      55
                      PLS Pretest                                          PLS pretest




                          Control
                          Mentor + Detailed Feedback
                          Mentor Limited Feedback
                          Nonmentor + Detailed Feedback
                          Nonmentor Limited Feedback




2008 SREE Conference Abstract Template                                                         B–3
Figure 4.

                         Predicted Scores on PCTOPP Print Awareness by
                                   Intervention Group and Site

          30



          25



          20
  Score




          15



          10



          5



          0

                    OH                MD                 FL              TX


               Control                   Mentored Palm          Mentored Pencil
               Nonmentored Palm          Nonmentored Pencil




2008 SREE Conference Abstract Template                                            B–4
Figure 5.
                      Predicted Quality of Print Instruction by Intervention Group

   4

 3.5

   3

 2.5

   2

 1.5

   1

 0.5

   0
            Control         Mentored Palm     Mentored Pencil     Nonmentored        Nonmentored
                                                                     Palm               Pencil




Figure 6.
                      Predicted Quality of Lesson Planning by Intervention Group

   4

 3.5

   3

 2.5

   2

 1.5

   1

 0.5

   0
            Control         Mentored Palm     Mentored Pencil     Nonmentored        Nonmentored
                                                                     Palm               Pencil




2008 SREE Conference Abstract Template
Figure 7.
                      Predicted Quality of Overall Teaching by Intervention Group

   4

 3.5

   3

 2.5

   2

 1.5

   1

 0.5

   0
            Control         Mentored Palm     Mentored Pencil    Nonmentored        Nonmentored
                                                                    Palm               Pencil




2008 SREE Conference Abstract Template

								
To top