Leaving Cert Maths and Performance in University

Document Sample
Leaving Cert Maths and Performance in University Powered By Docstoc
					Leaving Cert Maths and Performance in
              University

             Brendan Halpin
         Department of Sociology


            December 1, 2010
What good is Maths?




      What do we want of it?
          An important part of a rounded education?
          A feed-stock for the knowledge economy
      Should we be worried about performance?
      Anxiety and resistance
Declining LC performance of UL entrants
    60
    55
    50
    45
    40
    35




         1998   2000      2002            2004          2006   2008
                           Year first observed

                       Maths Points              Pct Higher
What do we want to know about it?




   Key research question: Will encouraging LC HL maths improve
   things in university?
       Do (where do) people who perform better at LC Maths
       perform better in University?
       How much of this is due to prior ability, and how much is it
       amenable to policy intervention?
Maths bonus points as a policy lever




      Effective?
          Will bonus points raise LC learning?
          Will they raise student quality?
      Unfair?
          Further penalise disadvantaged students?
          Penalise female students?
          Penalise the non-mathematical?
Nature of the bonus

     140
            UL bonus
             Ordinary
     120       Higher
           New bonus

     100


      80


      60


      40


      20


       0
              D3   D2   D1   C3   C2   C1   B3   B2   B1   A2   A1
Nature of the bonus




      A flat 25 points for D3 plus, rather than 5-40 rising from C3
      to A1
      A good incentive for average students, but it doesn’t really
      reward excellence
      In so far as candidates respond to the incentive, the 25 points
      will be partly consumed by higher entry points
Structural disadvantage




      Different types of school have different provision
          Disadvantaged schools more likely not to offer higher level
          maths
          Single-sex female schools more likely not to offer higher level
          maths
      De-facto or innate gender differences in maths aptitude?
Gender and maths: How it Works . . .
Gender and maths: How it Works . . .




   Credit: Randall Munroe; http://xkcd.com/385/
Leaving Cert Points and gender differences

     80
                                                       Female Irish
                                                         Male Irish
     75

     70

     65

     60

     55

     50

     45

     40
      1999   2000   2001   2002   2003   2004   2005     2006    2007   2008
Leaving Cert Points and gender differences

     80
                                                  Female English
                                                    Male English
     75

     70

     65

     60

     55

     50

     45

     40
      1999   2000   2001   2002   2003   2004   2005   2006   2007   2008
Leaving Cert Points and gender differences

     80
                                                   Female Maths
                                                     Male Maths
     75

     70

     65

     60

     55

     50

     45

     40
      1999   2000   2001   2002   2003   2004   2005   2006   2007   2008
Leaving Cert Points and gender differences

     80
                                                   MAS gender 1
                                                   MAS gender 2
     75

     70

     65

     60

     55

     50

     45

     40
      1999   2000   2001   2002   2003   2004   2005   2006   2007   2008
Gender and maths


      LC performance is poor relative to boys
      University performance differs by department:
          In MAS, females do systematically better (somewhat male
          dept)
          In MAE, females do better than males recently (very male
          dept)
          In PHY, a tie (somewhat male dept)
          In ECO, a tie (close to balanced)
          In CES, females do better than males recently (somethat male
          dept)
      In other words, no evidence of a debilitating mathematical
      disability
Maths and gender, LC grades
                                 Ordinary                               Honours
              .08
              .06
    Density
              .04
              .02
              0




                    F E D3 D2 D1 C3 C2 C1 B3 B2 B1 A2 A1   F E D3 D2 D1 C3 C2 C1 B3 B2 B1 A2 A1

                                                     F           M
Analysis: the research question




       The specific question I now address is, how much extra does
       LC maths performance help us predict third level performance?
       This addresses a part of the larger question, but since LC
       performance is due in part to innate ability and in part to the
       2nd level educational experience, it is not clear that changes
       in the educational process at 2nd level will cause changes at
       3rd level
Data and method


      The data:
          From the Student Records System
          Ten years of undergraduate grade data: 1999/2000 to 2008/9
          All Autumn and Spring XY4000 grades that are normally
          graded (no pass/fail, no Co-op)
          Information about age, gender, LC performance, module size
          etc.
          Thanks to the VPAR, Student Services, QSU and Research
          Office
      The method: Multi-level regression analysis with grade QCV
      as the dependent variable
Multi-level modelling of QCV



      Why analyse the determinants of grade?
      Retention and final award level are much more significant
      outcomes
      However, the grade is the “atom” of performance – links
      directly to the the module, the department (rather than
      programme) as well as to the individual
      Repeated measurement at individual and module/department
      level is extremely useful – assess individual and structural
      variability
Nested structure of the data




                        Dataset

              G1   G2     G3      ...   Gn
Nested structure of the data




                           Dataset




         M1                  M2                  M3

    G1   ...   Gk   Gk+1     ...     Gl   Gl+1   ...   Gn
Nested structure of the data



                             Dataset



                  D1                             D2

        M              M               M              M

    G   ...   G    G   ...   G   G     ...   G    G   ...   Gn
Nested structure of the data



                                                                               Dataset




                                       F1                                                                                 FN



                   D1                                      D2
                                                                                                    D3                                          D4

         M1                  M2                  M3                  M4
                                                                                         M5                    M6                    M7                    M8
    G1        G2        G3        G4        G5        G6        G7        G8
                                                                                  G9          G10        G11        G12        G13        G14        G15        G16
Nesting as structure of teaching and learning


       Modules (within departments) are where the actual work of
       teaching and learning and assessing goes on
       Module instances share much context: average level of ability,
       the specific curriculum, module size etc.
       Departments affect standards, resources, the identity of the
       teachers, etc.
       MLM that takes this nested structure into account deals
       correctly with higher level observed variables (e.g., module
       size)
       Also deals with higher-level unobserved heterogeneity – e.g.,
       the extent to modules differ from each other (and are
       relatively homogeneous within) due to unobserved factors
Multiple observations per individual also


                                  Dataset


                             F1              F...

                                             ...

              D1                            D2

         M1         M2            M3                M4

    G1   G2   G3   G4   G5
                             G6   G7   G8    G9     G10   G11
Multiple observations per individual also


                                  Dataset


                             F1              F...

                                             ...

              D1                            D2

         M1         M2            M3                M4

    G1   G2   G3   G4   G5
                             G6   G7   G8    G9     G10   G11
MLM structure at the individual level too



       Repeated measurement at the individual level allows us to
       take account of individual variables correctly (e.g., same LC
       points for 40 different grades)
       It also accounts for individual-level unobserved heterogeneity,
       e.g., socio-economic status, motivation, aptitudes, bar bill
       This yields a “cross-classified” multi-level model
       A very important side effect: since individuals cross
       departments we get much more meaningful estimates of
       departmental effects
Excursus: A web of departments



      Comparing disparate departments is hard: how to compare
      analysing a marketing plan, and 18th century French text, or
      the vibration pattern of a car exhaust?
      Experimentally we could assign matched samples of students
      to different programmes and compare results
      Observationally, we “control for” age, gender, LC points but
      department differences could still be accounted for by
      unobserved factors
Grade pairs




      But since we observe the same students in different
      departments we can control for this too
      But to what extent is the whole university linked by students?
      We can assess this by looking at each grade and linking it to
      the same and other departments by pairing it with that
      student’s other grades
Grade Pairs – an example




      11
   LCS00
      11
      00
                           11
                           00LAW
                           11
                           00




      11
   LCS00
      11
      00
                            1
                            0
                            0
                            1PPA


   ~
Grade Pairs – an example




      11
   LCS00
      11
      00
                           11
                           00LAW
                           11
                           00




      11
   LCS00
      11
      00
                            1
                            0
                            0
                            1PPA


   ~
Grade Pairs – an example




      11
   LCS00
      11
      00
                           11
                           00LAW
                           11
                           00




      11
   LCS00
      11
      00
                            1
                            0
                            0
                            1PPA


   ~
Grade Pairs – an example




      11
   LCS00
      11
      00
                           11
                           00LAW
                           11
                           00




      11
   LCS00
      11
      00
                            1
                            0
                            0
                            1PPA


   ~
Grade Pairs – an example




      11
   LCS00
      11
      00
                           11
                           00LAW
                           11
                           00




      11
   LCS00
      11
      00
                            1
                            0
                            0
                            1PPA


   ~
Grade Pairs – an example



      11
   LCS00
      00
      11
                           00LAW
                           11
                           11
                           00




      11
   LCS00                    1
                            0
      00
      11                    1
                            0PPA
                                         LCS   LAW   PPA
                                   LCS     2     2     2
   ~                               LAW     2     0     1
                                   PPA     2     1     0
Web of grades
    LCS
    LAW           14000
    SOC
    PPA
     HIS
    ACF           12000
    MMA
    PER
    ECO           10000
     PHI
    PSY
     NMI
    PES           8000
    EPS
    LSC
    MST
    ARC           6000
     CSI
    ECE
    MOE
    PHY           4000
    MAE
    CES
    MAS           2000
    ART
     PLE

                  0
           MMA




           MOE
           LAW
           SOC




           MAE

           MAS
           MST
           PER




           PHY
           PPA




           PSY

           PES
           LCS




           LSC
           ECO




           PLE
           NMI




           ARC

           ECE



           CES
           ACF




           EPS




           ART
            HIS




            PHI




           CSI
Web of grades (rate)
                       0.25
    LCS
    LAW
    SOC
    PPA
     HIS
    ACF                0.2
    MMA
    PER
    ECO
     PHI
    PSY                0.15
     NMI
    PES
    EPS
    LSC
    MST
    ARC                0.1
     CSI
    ECE
    MOE
    PHY
    MAE                0.05
    CES
    MAS
    ART
     PLE

                       0
           MMA




           MOE
           LAW
           SOC




           MAE

           MAS
           MST
           PER




           PHY
           PPA




           PSY

           PES
           LCS




           LSC
           ECO




           PLE
           NMI




           ARC

           ECE



           CES
           ACF




           EPS




           ART
            HIS




            PHI




           CSI
Tightly coupled




      AHSS and KBS are extremely tightly linked
      SEN and EHS are tightly linked but with some structure
      MIC’s two departments are, naturally, detached
      Within UL, only ARC stands out as being isolated
Numbers in the analysis




      728,590 grades in the working data set, with LC points data
      Where Irish, English and Maths results are known:
          Total number of grades: 614,747
          Number of individuals: 22,125
          Number of module instances: 10,947
          Number of departments: 26
Predicting QCV
                                  Base model       Plus core LC       Plus
                                                     subjects     gender×maths
   Intercept                    1.270***
   LC points/100                0.595***




   Calendar year               -0.009***
   Male                         0.055***
   Male by year interaction    -0.022***
   Age at entry                -0.004
   Years since entry
     1                           0.113***
     2                           0.244***
     3                           0.384***
     4                           0.562***
     5                           0.718***
   Spring semester              -0.014*
   Module size/100              -0.004
   Mean module points/100       -0.300***
   Dept proportion female        0.174
   Note: ***: p < 0.005; **:   p < 0.01; *: p < 0.05
Predicting QCV
                                 Base model        Plus core LC       Plus
                                                     subjects     gender×maths
   Intercept                    1.270***          1.348***
   LC points/100                0.595***          0.536***
   Extra effect, Irish                             0.003
   Extra effect, English                          -0.062*
   Extra effect, Maths                             0.331***

   Calendar year               -0.009***         -0.006***
   Male                         0.055***          0.034***
   Male by year interaction    -0.022***         -0.022***
   Age at entry                -0.004            -0.002
   Years since entry
     1                           0.113***          0.110***
     2                           0.244***          0.238***
     3                           0.384***          0.376***
     4                           0.562***          0.550***
     5                           0.718***          0.700***
   Spring semester              -0.014*           -0.013*
   Module size/100              -0.004            -0.004
   Mean module points/100       -0.300***         -0.302***
   Dept proportion female        0.174             0.206
   Note: ***: p < 0.005; **:   p < 0.01; *: p < 0.05
Predicting QCV
                                  Base model        Plus core LC       Plus
                                                      subjects     gender×maths
   Intercept                    1.270***           1.348***         1.362***
   LC points/100                0.595***           0.536***         0.535***
   Extra effect, Irish                              0.003            0.004
   Extra effect, English                           -0.062*          -0.060*
   Extra effect, Maths                              0.331***         0.311***
   Maths/gender interaction                                         0.049
   Calendar year                -0.009***         -0.006***        -0.006***
   Male                          0.055***          0.034***         0.006
   Male by year interaction     -0.022***         -0.022***        -0.022***
   Age at entry                 -0.004            -0.002           -0.002
   Years since entry
      1                          0.113***           0.110***        0.110***
      2                          0.244***           0.238***        0.238***
      3                          0.384***           0.376***        0.376***
      4                          0.562***           0.550***        0.550***
      5                          0.718***           0.700***        0.700***
   Spring semester              -0.014*            -0.013*         -0.013*
   Module size/100              -0.004             -0.004          -0.003
   Mean module points/100       -0.300***          -0.302***       -0.302***
   Dept proportion female        0.174              0.206           0.206
   Note: ***: p < 0.005; **:   p < 0.01; *: p < 0.05
Modelling by department
                                  No random slope       Departmental random
                                                        slope
    Intercept                      1.3478***
    LC points / 100                0.5362***
    Extra effect, Irish             0.0028
    Extra effect, English          -0.0618*
    Extra effect, Maths             0.3305***
    Calendar year                 -0.0063***
    Male                           0.0339***
    Male by year interaction      -0.0218***
    Age at entry                  -0.0024
    Years since entry
       1                           0.1103***
       2                           0.2383***
       3                           0.3759***
       4                           0.5499***
       5                           0.7000***
    Spring semester               -0.0132*
    Module size / 100             -0.0035
    Mean module points / 100      -0.3021***
    Dept proportion female         0.2064
    Note: ***: p < 0.005; **: p   < 0.01; *: p < 0.05
Modelling by department
                                  No random slope       Departmental random
                                                        slope
    Intercept                      1.3478***             1.6829***
    LC points / 100                0.5362***             0.4941***
    Extra effect, Irish             0.0028                0.0235
    Extra effect, English          -0.0618*              -0.0401
    Extra effect, Maths             0.3305***             0.3792***
    Calendar year                 -0.0063***            -0.0049**
    Male                           0.0339***             0.1635***
    Male by year interaction      -0.0218***            -0.0217***
    Age at entry                  -0.0024               -0.0040
    Years since entry
       1                           0.1103***             0.1127***
       2                           0.2383***             0.2399***
       3                           0.3759***             0.3705***
       4                           0.5499***             0.5382***
       5                           0.7000***             0.6772***
    Spring semester               -0.0132*              -0.0156***
    Module size / 100             -0.0035                0.0052
    Mean module points / 100      -0.3021***            -0.3209***
    Dept proportion female         0.2064                0.0338
    Note: ***: p < 0.005; **: p   < 0.01; *: p < 0.05
Departmental difference in maths effect
    2
                   Dept random effect plus maths fixed effect
                                    99% confidence interval


  1.5




    1




  0.5



         ARC   PSY     NMI     LAW   MMA
            EPS    PHI     HIS    SOC   PPA
    0
                                                   MST   PER   MOE   LCS     ECE   CES   PHY   MAS
                                                PES   PLE   LSC   ART    CSI    ACF   ECO   MAE



  -0.5




   -1




  -1.5
Departmental difference in maths effect
    2
                   Dept random effect plus maths fixed effect
                                    99% confidence interval
                                   Protection against failure

  1.5




    1




  0.5



         ARC   PSY     NMI     LAW   MMA
            EPS    PHI     HIS    SOC   PPA
    0
                                                   MST   PER   MOE   LCS     ECE   CES   PHY   MAS
                                                PES   PLE   LSC   ART    CSI    ACF   ECO   MAE



  -0.5




   -1




  -1.5
Departmental difference in maths effect
    2
                   Dept random effect plus maths fixed effect
                                    99% confidence interval
                                   Protection against failure
                                       Promoting excellence
  1.5




    1




  0.5



         ARC   PSY     NMI     LAW   MMA
            EPS    PHI     HIS    SOC   PPA
    0
                                                   MST   PER   MOE   LCS     ECE   CES   PHY   MAS
                                                PES   PLE   LSC   ART    CSI    ACF   ECO   MAE



  -0.5




   -1




  -1.5
Conclusion



      Maths performance at LC is a very important predictor of
      performance in UL
      This varies by department, but in half the effect is positive
      and sometimes very large, and in nearly all the rest it is
      neutral in effect
      That is, maths ability makes an important contribution most
      of the time
      Whether policies that improve performance at 2nd level will
      have consequences for 3rd level is another question!

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:3
posted:10/14/2011
language:English
pages:48