Free Science Fair Project for 2Nd Graders - PowerPoint by sup26915

VIEWS: 37 PAGES: 40

More Info
									Teaching, Learning, & Transfer of
   Experimental Procedures
               in
  Elementary School Science
               David Klahr
         Department of Psychology
     Pittsburgh Science of Learning Center (PSLC)
 Program in Interdisciplinary Education Research (PIER)
           Carnegie Mellon University
   Society for Research on Educational Effectiveness
      First Annual Conference Dec 10 - 12, 2006
Topic: Assessing different methods for
teaching experimental procedures to
middle school children
• In the lab
• In both “easy” & “challenging” classrooms
• To students of widely varying abilities

               More specifically:
               Teaching “CVS”
             What is (CVS)?
• NOT This:
   CVS: Control of Variables Strategy

• A simple procedure for designing
  unconfounded experiments:
     - Vary one thing at a time (VOTAT).

• The conceptual basis for making valid
  inferences from data:
     - isolation of causal path.
                      Why study CVS?
 Practical importance
 Topic:            Core topic in early science instruction

  Assessment: State standards
                 High stakes assessments
                NCLB to start testing science
  Best Instructional approach for teaching CVS?
          Heated controversy in profession
          Legislative battles (e.g., CA and “hands on” science)
Theoretical issues
  Surface vs deep mapping during transfer of procedures
  and concepts at different transfer “distances”.
       Goal: Compare different types of
        instruction for teaching CVS.
                                   Chen & Klahr (1999), Child Dev.
•   Participants: 60 2nd - 4th graders
•   Assessment:
    – Measure learning & transfer at different “distances”
      from initial instruction.
•   Materials: 3 different physcial domains
    – Springs
                            Between
    – Ramps                 subjects
                            design
    – Sinking objects.
                     Springs domain
 Which attributes determine how far a spring will stretch?


Materials:
  8 springs: 2 lengths x 2 widths x 2 wire sizes & 2 pair of weights

Execution:
• Select two springs
• Select two weights
• Hang springs on rack hooks
• Hang weights on springs.
• Compare amount of stretching.
 Question: does the length of a spring make a
      difference in how far it stretches?

An unconfounded test:               A       B
                          Length: short   long
                          Width:   wide   wide
                          Wire:    thin   thin
                          Weight: light   light
       Two types of instruction
                 (between subjects)

• Exploratory:
 –Hands on: work with physical materials
 – Goal provided: “find out if x makes a difference”

• Explicit = Exploratory plus:
 – Training: Explicit, good and bad examples
 – Training: Reasons why, focus on deep structure
 – Probe questions: Can you tell for sure? Why?
      Different transfer “distances”
• Near transfer (within domain):
  – CVS “tests” in same domain as training, but on a
    different dimension.
  – Time: minutes after training
  – Location, context, etc.: same as training

• Far transfer (between domain):
  – CVS tests in different domain from training.
  – Time: few days after training
  – Location, context, etc., same as training


• Remote transfer (more later)
                        Study Phases
      70%
                 Day 1             Day 2
      60%

      50%

      40%

      30%

      20%

      10%

       0%
            Exploration   Near     Far Transfer
            (Pre-test)  Transfer

Training Manipulation
                      Explicit immediately better than Exploration
                                    and remains so
                                             (4 experiments per child in each phase)
  % of unconfounded experiments



                                  70%
                                                                                Explicit
                                  60%

                                  50%

                                  40%                                           Exploratory

                                  30%

                                  20%

                                  10%

                                   0%
                                        Exploration   Near       Far Transfer
                                        (pre-test)  Transfer       (Day 2)

Training Manipulation
CVS mastery by individual children
  (at least 3 out of 4 unconfounded experiments)
                         100
% of children becoming



                          75
       Masters




                         50



                          25


                           0
                               Explicit   Exploratory
                  Extensions
1. Initial transfer measures are very close to
   training objectives.
2. Need a more “distant” ( “authentic”?) assessment
   of children’s understanding.
3. Will training effects remain with such extended
   assessments?
Procedure
  Create a more “authentic” assessment:
     •Ask children to judge science fair posters.
     • Score their comments and suggestions.
 CVS Training and Science Fair Assessments
                   (Klahr & Nigam, 2004)


1. Participants: 112 3rd & 4th graders

2. Train on CVS via Explicit or Exploration method.

3. Assess effectiveness of CVS skill.

4. Present poster evaluation task.

5. Look at how CVS skill, training condition, affect
   poster evaluation performance.
                          Study Design
                Day 1                                  1 week
      70%

      60%

      50%

      40%

      30%

      20%

      10%

       0%
       Exploration      Near transfer   Far transfer     Poster
                                                       Evaluation

Training Manipulation
Scoring Rubric for Children’s Poster Critiques
1. Adequacy of research design
2. Theoretical explanation
3. Controlling for confounds in:
        Subjects/Materials, Treatment, Experimenter bias, etc.
4. Measurement:       Reliability/Variability, Error, Data Representation

5. Statistical Inferences:      Sample size/population, effect size

6. Completeness of conclusion: Supported by data, Relate to hypothesis



                                                                          
                                             all valid, non-redundant,
 Poster Score =                              critiques about a poster

Grand Poster Score = (Pingpong Poster) + (Memory Poster)
Possible subtle effects of type of instruction
 Do the few kids who master CVS in the
   Exploratory condition do better on poster
   evaluation than the many who master
   CVS in the Explicit Instruction condition?
Possible subtle effects of type of instruction
 Do the few kids who master CVS in the
   Exploratory condition do better on poster
   evaluation than the many who master
   CVS in the Explicit Instruction condition?
  • More specifically:
    – What is the relation between Poster Scores
     and Path to CVS mastery?
  •Method:
    – Secondary analysis based on “learning paths”
           Different “paths” to mastery or
           non-mastery of CVS
           How do these children following
           these different paths perform on
           poster evaluations?
Note: following based on combining results from
two studies: original K&N plus a replication
                         Poster Assessment Score (standardized)




                                    -.8
                                    -.6
                                    -.4
                                    -.2
                                      .2
                                      .4
                                      .6
                                      .8




                                       0
                                       1
              Explicit




   n = 59
             ( Exp licit)
    Hotsh ot Masters


            Exploratory




   n = 15
              lorator y)
                                                                                       n.s.
otsh ot ( Exp Masters




                                                   p < .001
                 Experts




   n = 19
    Hotsh ot ( Na tu ral)


             Explicit




   n = 25
     tsho t (Exp licit)
n- Honon-Masters
                                                                                n.s.




        Exploratory
tsho t (Exp lora to ry)




   n = 66
      non- Masters




   irrelevant
                                                         poster scores


                            poster scores

   non-mastery is
                                                       o CVS mastery is




 o Path to mastery, or
                                                         associated with high


                          o Non-mastery with low
Question for cognitive research:
Why does training on CVS (narrow) lead
to better poster evaluations (broad)?
   Focused search for causal paths
   Decomposition (attention to detail)
   Nature of science
   Rhetorical stance
   Science as argument        Stay tuned ….
Question for applied research:
    Can CVS be taught in a normal
         classroom setting?
Procedure (in a nutshell):
  Translate experiment “script” into
  teacher lesson plan.
  Teach in “normal” science classes (in
  high SES schools).
  (Toth, Klahr, & Chen, 2000)
   Participants in Classroom Study

• 77 4th graders from 4 classrooms in two
  different private schools

• 2 different science teachers

• Neither school had participated in “lab” studies
  What to hold and what to fold?
 These are issues of “engineering design”.
     Keep                     Change & adjust
Pedagogy:                  Context:
  – Goal – teach CVS         – Lesson plan, not
  – Type of teaching:        “script”
    Explicit instruction     –Teacher, not researcher
                             –Scheduling
Assessment:
                             – Student/teacher ratio
  – Same as laboratory
                             – Group work
  – Plus, some new           – Record keeping
    assessments in
                             – Error and multiple trials
    classroom
 Results of Classroom Implementation
                              Individual students
      % unconfounded             classified as
          designs            “Experts” (8 of 9 correct)

100

80
                                Pretest   Posttest
60

40
                                  5%        91%
20

 0
       Pretest   Post Test
What about more challenging classrooms?
(“Lesson Planning Project”, w/Junlei Li, Stephanie Siler, Mandy Jabbour)




   One facet of the Lesson Planning Project:

   • Two classrooms (5th and 6th graders) in
     urban school
   • 90% eligible for free lunch.
   • Teacher is researcher (Junlei Li)
Teaching & Assessment of CVS with Urban 5th and 6th Graders
(n = 42)            (Klahr & Li, 2005)                                     Standardized
                                             Our CVS Tests                  Test Items
          100%
                      Dyads
                      Student Design
              80%
                      Mastery-based                                     Local (CTBS)
                      Formative Assessment
              60%
  % Correct




                                                                          National
                                                                            (NAEP)
              40%                                                                    International
                                                         Dyads                         (TIMSS)

                                                         Focused
                                                         Analogical
              20%
                                                         Mapping


              0%
                                                     2-Day CVS          2-Week Delay:
                    2-Day Classroom Replication of   Transfer &         Transfer to “real
                    CVS Training                     Retraining         world”, “high-
                    Domain: Ramps                    Domain: Pendulum   stakes” items
% correct for various groups on a TIMMS CVS item
Typical TIMMS CVS item


               He wants to test this idea: The heavier a
               cart is, the greater its speed at the bottom
               of a ramp. Which three trials should he
               compare?
                 Significance
Brief, theoretically grounded, focused
  instruction:
   Is highly effective for middle class students
   In the sort run & over longer durations
   On “far transfer” assessments
Path independence:
   “What” matters more than “how”.
BIG differences in effectiveness with different
  student population. Thus, current approach
  requires:
   Adaptation, Modification, & Individualization
        Questions to pursue
               (Next steps)
NCLB in “the small”:

  Goal: No child who can’t understand &
   execute CVS

  Method: Develop an “intelligent tutor” that
   can adapt to wide variability in children’s
   learning
  Wide variety of individual learning patterns
                             (From Chen & Klahr, 1999)
       4                 4                    4                  4
       3                 3                    3                  3
       2                 2                    2                  2
       1                 1                    1                  1
       0                 0                    0                  0
           Ex As T1 T2        Ex As T1 T2          Ex As T1 T2       Ex As T1 T2

TYPE       FAST GAIN         UP DOWN UP           GRADUAL GAIN   HIGH CONSTANT
Explicit          31%                 10%                  7%             7%
Socratic          0%                  0%                14%               0%

       4                 4                    4
       3                 3                    3
       2                 2                    2
                         1                    1
       1
                         0                    0
       0
                              Ex As T1 T2          Ex As T1 T2
           Ex As T1 T2

TYPE       UP & DOWN         STEADY DECLINE       LOW CONSTANT
Explicit        7%               0%                  37%
Socratic       18%               7%                  60%
Design a Tutor for Experimental Design
        w/ Mari Strand Cary, Stephanie Siler, Junlei Li
  Thanks to
 Funding $ources:
 • McDonnell Foundation, NICHD, NSF, IES


Recent & Current collaborators

   Zhe Chen, Eva Toth, Junlei Li, Mari
   Strand Cary, Stephanie Siler, Milena
   Nigam, Amy Masnick, Lara Triona
END
Extras
                Remote transfer items
A page from the 15-item test booklet

                                         Why “remote”?
    Does the amount of water           • Temporal
      affect plant growth?               – Training - test interval: 7
                                           months
                                       • Domain
                                         – Physical - biological, et al
                                       • Format
                                         – Physical materials vs. paper
                                           and pencil test booklet
               Good Test
                                       • Context
               Bad Test
                                          - One on one with
                                          Experimenter vs whole class
                                          test taking
                       Remote Transfer Results

                                                                                100                      Trained




                                                Mean % correct on 15-item far transfer test
                                                                                                         Untrained
Does the amount of water affect plant growth?                                                 75




                                                                                              50


                 Good Test

                  Bad Test                                                                    25




                                                                                               0
                                                                                                   3rd               4th
                Ramps Domain
  Question: Does the surface of a Ramp make a
        difference in how far a ball rolls?

A completely confounded test
                               Surface:     smooth
                               Run:         short
       A
                               Steepness:   high
                               Ball:        golf


                               Surface:     rough
        B                      Run:         long
                               Steepness:   low
                               Ball:        rubber

								
To top