What Works In Teaching Science: by ray95mLl

VIEWS: 4 PAGES: 29

									What Works In
Teaching Science:
A Meta-Analysis of Current
Research
Carolyn Schroeder, Ph.D.
Center for Math & Science Education
Texas A&M University
Texas A&M University Project Staff

 • Timothy P. Scott, Ph.D., Project Director
 • Carolyn Schroeder, Ph.D., Senior
   Research Associate

 • Homer Tolson, Ph.D., Senior Analyst
 • Yi-Hsuan Lee, Ph.D., Analyst
 • Tse-Yang Huang, Ph.D., Analyst
Advisory Board

  • Carol L. Fletcher, Ph.D., Texas
    Regional Collaboratives, UT Austin
  • Ginny Heilman, Region VI ESC
  • Anna McClane, Region IV ESC
  • Sandra S. West, Ph.D., Texas State
    University
  • Jo Ann Wheeler, Region IV ESC
What teaching strategies
 have been shown to
 improve student
 achievement in
 science???
Criteria for Selection of Studies
• Dates: 01/01/1980 – 12/31/2004
• Dealt with K-12 science education in the U.S.
• Used student achievement (success, performance,
  etc.) as dependent variable
• Used science education teaching strategies as
  independent variables
• Was experimental or quasi-experimental
• Reported effect size (ES) or statistics necessary to
  calculate it
• Could not be totally correlational
• Could not deal exclusively with special populations
• Could not be included more than once (e.g., same
  study reported in a dissertation and journal article)
Acquisition of Studies
• Broad search conducted
• Over 400 potential sources identified
  – Journal articles
  – Conference papers
  – Books
  – Dissertations
  – Government reports
  – Unpublished papers
Search Methods
 • Electronic searches
   –   Web of Science
   –   ERIC (EBSCO, First Search, CSA)
   –   Academic Search Premier
   –   PsycInfo
   –   ProQuest Dissertations and Theses

 • Reference lists from previous meta-
   analyses, books & other articles, electronic
   sources (e.g., government sites)
 • Request to NARST listserve
 • Requests to specific developers of
   instructional packages for product studies
Coding of Studies
• Study attributes coded:
  – Citation
  – Publication type (refereed journal, dissertation, etc.)
  – Study type (experimental, quasi-experimental, correlational)
  – Dependent variable (describe test used to measure
    achievement)
  – Independent variable (describe treatment & control or
    alternate treatment)
  – Length of treatment/study
  – Setting & characteristics
      • Schools (#, how selected, public/private, rural/urban, size, %
        free lunch)
      • Students (#, how selected, how assigned, gender, grade,
        ethnicity, SES)
      • Teachers ( #, how selected, experience, gender, certification)
  – Study results (ES, p, t, F, eta squared, omega squared)
Intercoder Objectivity

 • 3 randomly selected articles were coded
   independently by senior analyst and 2
   researchers
 • Degree of objectivity was 90% for two
   articles
 • Third article was identified as correlational
   therefore was not coded
 • Senior analyst read & coded all articles,
   resolved any differences in coding values
Study Design Classification
• True random assignment of schools/students to
  treatment and control groups
• Quasi-experimental with match of schools/students to
  achievement and demographics of comparison
  school/group
• Quasi-experimental with covariate adjustment for prior
  achievement differences
• Quasi-experimental comparison of schools/subjects
  based a claim of “similarity”
• Quasi-experimental comparison of schools/subjects to
  region, state, or national data
• Quasi-experimental single group pre-post comparison
• Quasi-experimental treatment vs. control pre-posttest
• Quasi-experimental multiple group ANOVA
Treatment Category Classification
Modified from Wise, 1996
  •Questioning strategies
  •Manipulation strategies
  •Enhanced materials strategies
  •Testing strategies (changed to Assessment
      strategies)
  •Inquiry strategies
  •Enhanced context strategies
  •Instructional media strategies (changed
      to Instructional technology strategies)
  •Focusing strategies (not used)
  •Collaborative learning strategies (added)
Table 1. Frequencies of Characteristics of Included Studies
    Independent Variable                        Frequencies   Percent (%)
      Publication Year
      1980 – 1984                                   6             9.7
      1985 – 1989                                   7            11.3
      1990 – 1994                                   4             6.5
      1995 – 1999                                   15           24.2
      2000 – 2004                                   30           48.4
      Publication Type
      Refereed Journal Article                      40           64.5
      Dissertation                                  18           29.0
      Unpublished Report                            4             6.5
      Type of Study
      Experimental (Complete Randomization)         3             4.8
      Quasi-Experimental (Randomization Used)       33           53.2
      Quasi-Experimental (No Randomization)         26           41.9
      Correlational                                 0             0.0
Table 1. Frequencies of Characteristics of Included Studies
   Independent Variable                     Frequencies   Percent (%)
     Test Content Area
     Biology                                     17          27.4
     Chemistry                                   12          19.4
     Physics                                     5            8.1
     Earth Science                               7           11.3
     Science                                     21          33.9
     Study Rating
     Experimental, treatment vs. control         2            3.2
     Quasi-Exp. match                            1            1.6
     Quasi-Exp. similar                          1            1.6
     Quasi-Exp. single-group pre-post            14          22.6
     Quasi-Exp. trt vs. control pre-post         27          43.5
     Quasi-Exp. ANOVA                            17          27.4
               Totals (for each variable)        62         100.0
Table 2. Dependent Variable (Test Type)

                 Test Type          Number of Cases   Percent (%)

   National Standardized-Multiple
      Science Content                     3               4.8

   National Standardized-
   Single Science Content                 6               9.7

   Local Standardized-
   Multiple Science Content               2               3.2

   Local Standardized-
   Single Science Content                 4               6.5

   Other type test                        47             75.8
                     Total                62            100.0
Effect Sizes
  • Obtained or calculated for all studies
    that met criteria
    – n = 62
    – one removed later as extreme outlier
  • Internal & external validity influences on
    effect sizes calculated
  • Regression analysis for moderator
    variables & dependent variable effect
    sizes (n = 61)
  • Failsafe N calculated for all categories
Table 3. Failsafe N for Total Data and Treatment
Description Categories

                        Data              ES       N    Nfs

   Overall                               .6696     61   756
   Questioning Strategies                .7395     3    42
   Manipulation Strategies               .5729     8    84
   Enhanced Material Strategies          .2908     12   58
   Assessment Strategies                 .5052     2    19
   Inquiry Strategies                    .6546     12   145
   Enhanced Context Strategies           1.4783    6    172
   Instructional Technology Strategies   .4840     15   130
   Collaborative Learning Strategies     .9580     3    55
Analysis of Effect Size
 • Comprehensive Meta-Analysis® software
   from BioStat
 • Outputs
    – Cohen’s d,
    – Hedges’s g,
    – Q value,
    – confidence interval etc.,
    – fixed and random effects, and
    – heterogeneity testing results.
Figure 1. Mean Effect Sizes for Treatment
Categories and Total Data
1.6

1.4
                                                             C1=Questioning
1.2                                                          C2=Manipulation
                                                             C3=Enhanced Material
 1
                                                             C4=Assessment
0.8                                                          C5=Inquiry
                                                             C6=Enhanced Context
0.6                                                          C7=Instructional Technology
                                                             C8=Collaborative Learning
0.4

0.2

 0
      C1*   C2*   C3*   C4*   C5*   C6*   C7*   C8* Total
                                                       Mean ES
Conclusions
What teaching strategies have been shown to
improve student achievement in science???
  • All of the innovative strategies have a
    positive influence on student
    achievement.
  • Innovative science instruction is a
    mixture of teaching strategies.
  • Teaching strategies are tools, and the
    right tool must be selected for the job
    at hand.
Table 4. Ranking of Teaching Strategies

            Strategies                Effect Size   Rank
Enhanced Context Strategies               1.4783     1
Collaborative Learning Strategies          .9580     2
Questioning Strategies                     .7395     3
Inquiry Strategies                         .6546     4
Manipulation Strategies                    .5729     5
Assessment Strategies                      .5052     6
Instructional Technology Strategies        .4840     7
Enhanced Material Strategies               .2908     8
Most Powerful –
    Enhanced Context Strategies
 • Make learning relevant to students
 • Use real-world examples and problems
    – Problem based learning
    – Case based learning
 • Use technology to bring real world into
   classroom
 • Take students out of classroom into real
   world
 • Use multiple contexts to teach concept
Future Research – Meta-Analysis
 • Examine studies included in MA to determine how
   many of them meet the “strong” or “possible” evidence
   of effectiveness standards of the DOE Institute of
   Education Sciences (see Identifying and
   Implementing Educational Practices Supported by
   Rigorous Evidence, available at
   http://www.ed.gov/rschstat/research/pubs/rigorousevid/rigorousevid.pdf)
 • Broaden scope of meta-analysis to include:
    – International studies
    – Correlational studies (data on two variables collected and
      summarized, showing the relationship between the variables)
    – Studies dealing with attitudinal and motivational changes in
      students and teachers
    – Studies dealing with special populations (English-language
      learners, special education, under-represented populations,
      etc.)
    – Studies dealing with teacher professional development
Products Based
on Results of
Meta-Analysis
Products

  • Research-based Teaching
    Strategies for Effective Science
    Instruction
  • Rubric for Analyzing Science
    Products
  • Combined in booklet – Effective K-
    12 Science Instruction: Elements of
    Research-based Science Education
Rubric Design Based on Meta-Analysis

 • Science content             • Effective instructional
    – Accuracy and alignment     practices
    – Safety                      – Enhanced context
                                    strategies
 • Organization and
                                  – Inquiry strategies
   structure                      – Instructional technology
    – Format of materials           strategies
    – Coherency                   – Collaborative learning
 • Meaningful assessment            strategies
    –   Alignment                 – Manipulation strategies
    –   Formative                 – Questioning strategies
    –   Summative              • Equity and practicality
    –   Metacognitive             – Equity
                                  – Practicality
Rubric Development
 • Draft created using criteria
 • Sent to advisory board and stakeholders
   for comment
 • Revision
 • Discussion with science teachers/
   supervisors
 • Further revisions, clarifications, & weighting
   of categories
 • Field test
 • Statistical validation (Interrater reliability = .945
   using Cronbach’s alpha)
Questions or Comments?
   Booklets may be ordered for
     $1.50 each + shipping

        Dr. Carolyn Schroeder
              979-458-4450
      cschroeder@science.tamu.edu


          Texas A&M University
   Center for Mathematics and Science
                Education
               3257 TAMU
    College Station, Texas 77843-3257

http://www.science.tamu.edu/cmse/tsi

								
To top