Docstoc

Educational data mining overview

Document Sample
Educational data mining overview Powered By Docstoc
					Educational data mining overview
& Introduction to Exploratory
Data Analysis

Ken Koedinger
CMU Director of PSLC
Professor of Human-Computer Interaction &
Psychology
Carnegie Mellon University
Plan

   Because it is technical, will start with learning
    curve formulas …
   Then go to exploratory data analysis
   Return to in next session to use of formulas
    in Item Response Theory, Learning Factors
    Analysis

       (Provide some “spaced” practice for you)
Overview

   Questions on yesterday’s intro?
       Another example of learning curves
   Quantitative models of learning curves
       Power law, logistic regression
   Exercise:
       Goals:
        1) Get familiar with data,
        2) Learn/practice Excel skills
       Tasks:
        1) create a “step table”,
        2) graph learning curves using a) error rate & b)
        assistance score
Student Performance As They
Practice with the LISP Tutor
             Production Rule Analysis

             0.5


                                  Evidence for Production Rule as an
             0.4                  appropriate unit of knowledge acquisition
Error Rate




             0.3



             0.2



             0.1



             0.0
                   0   2      4           6           8          10         12   14

                           Opportunity to Apply Rule (Required Exercises)
Using learning curves to
evaluate a cognitive model
   Lisp Tutor Model
      Learning curves used to validate cognitive model

      Fit better when organized by knowledge components

       (productions) rather than surface forms (programming language
       terms)
   But, curves not smooth for some production rules
      “Blips” in leaning curves indicate the knowledge

       representation may not be right
      Corbett, Anderson, O’Brien (1995)

      Let me illustrate …
    Curve for “Declare
    Parameter” production rule
                                            What’s happening
                                            on the 6th & 10th
                                            opportunities?




   How are steps with blips different from others?
   What’s the unique feature or factor explaining these
    blips?
Can modify cognitive model using unique
factor present at “blips”
   Blips occur when to-be-written program has 2 parameters
   Split Declare-Parameter by parameter-number factor:
       Declare-first-parameter
       Declare-second-parameter
Overview

   Questions on yesterday’s intro?
       Another example of learning curves
   Quantitative models of learning curves
       Power law, logistic regression
   Exercise:
       Goals:
        1) Get familiar with data,
        2) Learn/practice Excel skills
       Tasks:
        1) create a “step table”,
        2) graph learning curves using a) error rate & b)
        assistance score
Learning curve analysis

   The Power Law of Learning
    (Newell & Rosenbloom, 1993)
    Y = a Xb
    Y – error rate
    X – opportunities to
       practice a skill
    a – error rate on 1st opportunity
    b – learning rate
    After the log transformation
    “a” is the “intercept” or starting point of the learning curve
    “b” is the “slope” or steepness of the learning curve
More sophisticated learning
curve model
   Generalized Power Law to fit learning curves
       Logistic regression (Draney, Wilson, Pirolli, 1995)

   Assumptions
       Different students may initially know more or less
         => use an intercept parameter for each student
       Students learn at the same rate
           => no slope parameters for each student
       Some productions may be more known than others
           => use an intercept parameter for each production
       Some productions are easier to learn than others
           => use a slope parameter for each production

   These assumptions are reflected in detailed math model …
More sophisticated learning curve model


    
ln p  i Xi   j Yj   j YjTj
        p
        1 p

Probability of getting a step correct (p) is proportional to:
   - if student i performed this step = Xi,

     add overall “smarts” of that student = i
    -    if skill j is needed for this step = Yj,
         add easiness of that skill = j
         add product of number of opportunities to learn = Tj
                         & amount gained for each opportunity = j

Use logistic regression because response is discrete (correct or not)
  Probability (p) is transformed by “log odds”
  “stretched out” with “s curve” to not bump up against 0 or 1

(Related to “Item Response Theory”, behind standardized tests …)
Overview

   Questions on yesterday’s intro?
       Another example of learning curves
   Quantitative models of learning curves
       Power law, logistic regression
   Exercise:
       Goals:
        1) Get familiar with data,
        2) Learn/practice Excel skills
       Tasks:
        1) create a “step table”,
        2) graph learning curves using a) error rate & b)
        assistance score
TWO_CIRCLES_IN_SQUARE problem:
Initial screen
TWO_CIRCLES_IN_SQUARE problem:
An error a few steps later
TWO_CIRCLES_IN_SQUARE problem:
Student follows hint & completes prob
Exported File Loaded into Excel




                    QuickTime™ and a
                TIFF (LZW) decomp resso r
             are neede d to see this picture.
DataShop Export & Using Excel

   Get files from
       Go to Learnlab.org
       Click on “Enabling Technologies”
       Click on “Meetings”
       Click on “Documents”


   Don’t do yet …
   Demo …
Demo: Export Step Roll
Up from Data Shop …
Now try it yourself …

   Follow instructions in download from two
    slides ago …
END

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:217
posted:3/20/2008
language:English
pages:21