Docstoc

MY-Bolton 120609

Document Sample
MY-Bolton 120609 Powered By Docstoc
					      Reading the (survey) runes
      Students’ experience of Art & Design


                       Mantz Yorke
                     Lancaster University
                    mantzyorke@mantzyorke.plus.com



Expectations, Experiences and Encounters in HE and HE in FE – The 3Es

                      The University of Bolton
                           12 June 2009
What’s the story?

• There is not a problem with student commitment
• Organisation and Management in A&D consistently
  show weaknesses relative to other subject areas
• As does ‘Overall Satisfaction’ on the NSS
• Creative Arts & Design do generally poorly on the NSS
• Feedback: the picture from different surveys is
  inconsistent
• Some HEIs do consistently better and some
  consistently worse across the NSS scales
• NSS scales vary re discrimination between HEIs
The general plan


What surveys can and can’t tell

The NSS: what it says, generally, about Art & Design

What some other surveys have shown

Some pointers from this collection of evidence
What surveys can and can’t tell
Uses

Measuring:                                 Summative
• Public information

Measuring/judging:
• Auditing institutional performance

Enhancing institutional performance        Formative



Technical requirements vary with purpose
Robustness

Population > Sample > Methodology > Responses

• Biases
• Statistical testing


What is ‘good enough’?


Raw data doesn’t necessarily tell the story
NSS analyses of ‘residuals’ have their uses
The NSS is particularly useful for highlighting areas of concern,
in other words it is good at identifying the ‘where’ in student
experience: where there are differences in experiences as rated
by students. It is not, however, designed to answer the ‘why’
of student experience

Surridge 2009, p.29



The same goes for quantitative surveys in general
Art & Design in the NSS
The NSS


                     2005                  2006                  2007
                   A&D All               A&D All               A&D All
T&L                3.93     3.96         3.94      3.98         3.94     4.00
A&F                3.48     3.47         3.50      3.49         3.54     3.52
AcSup              3.60     3.71         3.65      3.76         3.70     3.82
O&M                3.39     3.69         3.39*     3.72         3.45     3.79
LRes               3.86     3.96         3.92      4.02         3.96     4.08
PDev               3.93     3.98         3.93      3.99         3.94     4.02

OvSatis            3.70     3.96         3.71      3.96         3.76     4.01


*   This figure is wrongly given as 3.90 in the report of the 2006 NSS and in the Trends 2005-07
                 0
                     20
                          40
                               60
                                    80
                                         100
                                                                                                                  0
                                                                                                                      20
                                                                                                                           40
                                                                                                                                60
                                                                                                                                     80
                                                                                                                                          100
  Ph                                                                                              Hi
     y   sS
                                                                                                     st&
                                                                                                         Ph
            ci                                                                                              il
  Ma
      th                                                                                                La
        Sc                                                                                                ng
            i
 Hi                                                                                                Ph
    st&                                                                                               y   sS
        Ph
           il                                                                                                ci
                                                                                                      Al
       La
         ng                                                                                              lM
                                                                                                            ed
     Al                                                                                               Bi
        lM                                                                                               oS
           ed                                                                                              ci

       Ed                                                                                                 La
          uc                                                                                                w
     Bi                                                                                                 Ed
        oS                                                                                                 uc
          ci
 Co                                                                                               So
    m                                                                                                c   St
        pS                                                                                                 ud
           ci
     En                                                                                            Ma
        g&                                                                                              th
           T                                                                                              Sc
                                                                                                            i
                                                                                                   Cr
         La                                                                                          eA
           w                                                                                           &D
Bu                                                                                                Ar
   s&                                                                                               ch
      Ad
        m                                                                                                 -et
                                                                                                              c
 So                                                                                              Ma
    cS                                                                                                ss
       tu                                                                                                Co
          d                                                                                                 m
  Cr                                                                                                 En
     eA
                                               Percentage agreement, Academic Support 2007
        &D                                                                                              g&
                                                                                                           T
                                                                                                                                                Percentage agreement, Teaching & Learning 2007




 Ar                                                                                              Bu
   ch                                                                                               s&
                                                                                                       Ad
         -et
             c                                                                                           m
Ma                                                                                                Co
     ss
        Co                                                                                           m
                                                                                                       pS
           m                                                                                              ci




                 0
                     20
                          40
                               60
                                    80
                                         100
                                                                                                                  0
                                                                                                                      20
                                                                                                                           40
                                                                                                                                60
                                                                                                                                     80
                                                                                                                                          100




  Ma                                                                                              Hi
                                                                                                     st&
       th                                                                                                Ph
         Sc
           i                                                                                                il
 Hi
    st&                                                                                                 La
        Ph                                                                                                ng
           il
     Bi                                                                                            Ma
        oS                                                                                              th
                                                                                                          Sc
          ci                                                                                                i
  Ph
     y   sS                                                                                             Ed
            ci                                                                                             uc
                                                                                                   Ph
         La                                                                                           y   sS
           w                                                                                                 ci
                                                                                                  So
     La                                                                                              c   St
        ng                                                                                                 ud
Bu
   s&                                                                                              Cr
      Ad                                                                                             eA
        m                                                                                              &D
 Co                                                                                              Bu
    m                                                                                               s&
      pS                                                                                               Ad
         ci                                                                                              m
 So
    cS                                                                                                    La
       tu                                                                                                   w
          d
     En                                                                                               Bi
        g&                                                                                               oS
           T                                                                                               ci
                                                                                                 Ma
     Al                                                                                               ss
        lM                                                                                               Co
           ed                                                                                               m
                                                                                                      Al
       Ed                                                                                                lM
          uc                                                                                                ed
Ma                                                                                                Co
     ss
        Co                                                                                           m
                                                                                                         pS
           m                                                                                                ci
                                               Percentage agreement, Organisation & Management
                                                                                                                                                Percentage agreement, Assessment & Feedback 2007




  Cr                                                                                                  En
    eA                                                                                                   g&
      &D                                                                                                    T
 Ar                                                                                               Ar
   ch                                                                                               ch
         -et                                                                                              -et
             c                                                                                                c
                                                                                                                    0
                                                                                                                        20
                                                                                                                             40
                                                                                                                                  60
                                                                                                                                       80
                                                                                                                                            100
                                                                                                     Ma
                                                                                                          th
                                                                                                            Sc
                                                                                                              i
                                                                                                     Ph
                                                                                                        y   sS
                                                                                                               ci
                                                                                                    Co
                                                                                                       m
                                                                                                           pS
                                                                                                              ci
                                                                                                        En
                                                                                                           g&
                                                                                                              T
                                                                                                   Bu
                                                                                                      s&
                                                                                                         Ad
                                                                                                           m

                                                                                                            La
                                                                                                              w
                                                                                                        Bi
                                                                                                           oS
                                                                                                             ci
                                                                                                        Al
                                                                                                           lM
                                                                                                              ed




                   0
                       20
                            40
                                 60
                                      80
                                           100
  Ma                                                                                                Ar
                                                                                                      ch
       th                                                                                                   -et
         Sc
           i                                                                                                    c
                                                                                                   Ma
                                                                                                        ss
                                                                                                           Co
        La
          w                                                                                                   m

     Ph                                                                                                   La
        y   Sc                                                                                              ng
               i
 Hi                                                                                                  Cr
    st&                                                                                                eA
        Ph                                                                                               &D
           il
                                                                                                    So
       La                                                                                              c   St
                                                                                                                                                  Percentage agreement, Learning Resources 2007




         ng                                                                                                  ud

     Bi                                                                                                   Ed
        oS                                                                                                   uc
          ci
                                                                                                    Hi
     Al                                                                                                st&
        lM                                                                                                 Ph
            ed                                                                                                il
 So
    c St
        ud
Bu
   s&
                                                                                                                    0
                                                                                                                        20
                                                                                                                             40
                                                                                                                                  60
                                                                                                                                       80
                                                                                                                                            100




      Ad
         m                                                                                                Ed
                                                                                                             uc
       Ed                                                                                               Al
          uc                                                                                               lM
                                                                                                              ed
     En                                                                                             Ar
        g&                                                                                            ch
           T                                                                                                -et
 Co                                                                                                             c
    m
        pS
           ci                                                                                             La
                                                                                                            w
 Ar                                                                                                Bu
   ch                                                                                                 s&
                                                 Percentage agreement, Overall satisfaction 2007




        -et
            c                                                                                            Ad
                                                                                                           m
Ma
     ss                                                                                              Ph
        Co
           m                                                                                            y   sS
                                                                                                               ci
  Cr
    eA                                                                                              So
      &D                                                                                               c   St
                                                                                                             ud

                                                                                                         La
                                                                                                            ng
                                                                                                    Hi
                                                                                                       st&
                                                                                                           Ph
                                                                                                              il
                                                                                                        Bi
                                                                                                           oS
                                                                                                             ci
                                                                                                        En
                                                                                                           g&
                                                                                                              T
                                                                                                    Co
                                                                                                       m
                                                                                                           pS
                                                                                                              ci
                                                                                                     Cr
                                                                                                       eA
                                                                                                         &D
                                                                                                                                                  Percentage agreement, Personal Development 2007




                                                                                                   Ma
                                                                                                        ss
                                                                                                           Co
                                                                                                              m
                                                                                                     Ma
                                                                                                          th
                                                                                                            Sc
                                                                                                              i
 Subject area median percentage agreement 2007, ranked for each scale
          Top 4                         Bottom 4


JACS (19) Group                   T&L   A&F    AcSupp   O&M   LrnRes   PersDev   OvSatis
Subjects allied to Medicine        4     12        5    11      8         2        7
Biological Sciences                5     10        7     3      7        10        6
Physical Sciences                  3      5        1     4      2         6        3
Mathematical Science               9      3        2     1      1        15        1
Computer Science                  15     13        8     8      3        12        12
Engineering & Technology          13     14        9    10      4        11        11
Architecture etc                  11     15        14   15      9         3        13
Social Studies                     8      6        12    9      13        7        8
Law                                6      9        10    5      6         4        2
Business & Admin Studies          14      8        11    7      5         5        9
Mass Communications etc           12     11        15   13      10       14        14
Languages                          2      2        4     6      11        8        5
Historical & Philosophical Stud    1      1        3     2      15        9        4
Creative Arts & Design            10      7        13   14      12       13        15
Education                          7      4        6    12      14        1        10
                   0
                       20
                            40
                                 60
                                      80
                                           100
                                                                                                                                   0
                                                                                                                                       20
                                                                                                                                            40
                                                                                                                                                 60
                                                                                                                                                      80
                                                                                                                                                           100
   Bi                                                                                                                Hi
      olo                                                                                                               sto
          g                                                                                                                  r
  El y                                                                                                              En y
    ec                                                                                                                  gl
        &E
             E                                                                                                      Bi
                                                                                                                           ish
   Hi                                                                                                                  olo
      sto
           ry                                                                                                               gy
     Mu                                                                                                               Dr
          si                                                                                                             am
   En c                                                                                                                       a
      gl                                                                                                              Da
         ish                                                                                                              nc
 Te                                                                                                                 Nu e
    ac                                                                                                                 rs
       hr
          T                                                                                                                ing
  Sp r
     or
        tS                                                                                                                La
 Co ci                                                                                                                        w
    m                                                                                                                 Mu
       pS                                                                                                                   s
           ci                                                                                                      Sp ic
     Da                                                                                                               or
        nc                                                                                                                tS
            e                                                                                                                ci
    Dr                                                                                                                Ps
        am                                                                                                                yc
Ac           a                                                                                                   So           h
   St                                                                                                               cio
      Ed                                                                                                                lo
          uc                                                                                                      Te        gy
  Nu                                                                                                                 ac
      rs
         ing                                                                                                             h
                                                                                                                 Ac rTr
                                                                                                                     St
        La                                                                                                              Ed
                                                                                                                            u
  Ar w                                                                                                             Ar c
     ch                                                                                                               ch
So itec                                                                                                          Ci       ite
   cio                                                                                                              ne c
       lo                                                                                                              &P
Bu gy                                                                                                                       h
   sS                                                                                                              El o
                                                                                                                      ec
       tu                                                                                                                 &E
          ds                                                                                                      So          E
     Ps                                                                                                              cW
         yc                                                                                                               o
 Me h                                                                                                             Me rk
    di                                                                                                                di
       a                                                                                                                 a
Ci Std                                                                                                           Bu Std
  ne                                                                                                                 sS
      &P                                                                                                                 tu
                                                                                                                                                                 Percentage agreement T&L 2007, selected subjects (JACS 107)




 De ho                                           Percentage agreement AcSup 2007, selected subjects (JACS 107)    De ds
    sig                                                                                                               sig
         nS                                                                                                               n
                                                                                                                  Co St
   Fi t
      ne                                                                                                              m
                                                                                                                         pS
 So Art                                                                                                                      c
    cW                                                                                                               Fi i
                                                                                                                       ne
         or                                                                                                                Ar
            k                                                                                                                  t




                   0
                       20
                            40
                                 60
                                      80
                                           100
                                                                                                                                   0
                                                                                                                                       20
                                                                                                                                            40
                                                                                                                                                 60
                                                                                                                                                      80
                                                                                                                                                           100




   Bi                                                                                                               Hi
      olo                                                                                                              sto
            gy                                                                                                               r
     Ps                                                                                                            Nu y
         yc                                                                                                            rs
              h                                                                                                            ing
    Hi                                                                                                              En
       sto                                                                                                             gl
             ry                                                                                                  Ac ish
   En                                                                                                               St
       gl                                                                                                              Ed
          ish                                                                                                     Te uc
                                                                                                                     ac
         La                                                                                                              hr
So            w                                                                                                             T
   cio                                                                                                             Sp r
       lo                                                                                                             or
 Co        gy                                                                                                    Ci tSci
     m                                                                                                             ne
        p                                                                                                              &P
Bu Sci                                                                                                            De ho
    sS                                                                                                               sig
        tu
            d                                                                                                    So nSt
  Sp s                                                                                                              cio
     or                                                                                                                 lo
         tS                                                                                                                 gy
  El ci
     ec                                                                                                               Mu
         &                                                                                                                  sic
Ac EE
   St
      Ed                                                                                                                  La
            uc                                                                                                                w
     Mu                                                                                                             Fi
                                                                                                                       ne
           sic                                                                                                             Ar
                                                                                                                               t
     Da                                                                                                               Ps
         nc                                                                                                               yc
 Me e                                                                                                              El         h
     di                                                                                                               ec
        aS                                                                                                                &
            td                                                                                                    Co EE
     Dr                                                                                                               m
        a                                                                                                                pS
Ci ma                                                                                                             So ci
  ne                                                                                                                 cW
      &P
           h                                                                                                     Bu ork
   Fi o                                                                                                             sS
     ne                                                                                                                  tu
          Ar
 Te            t                                                                                                  Me ds
    ac                                                                                                                di
        hr                                                                                                               aS
 De Tr                                                                                                                       td
     sig                                                                                                             Dr
         nS                                                                                                              am
                                                                                                                                                                 Percentage agreement A&F 2007, selected subjects (JACS 107)




                                                 Percentage agreement O&M 2007, selected subjects (JACS 107)




  Nu t                                                                                                             Ar a
      rs                                                                                                              ch
          i                                                                                                               ite
 So ng                                                                                                                        c
    cW                                                                                                                Da
         or                                                                                                               nc
  Ar k                                                                                                              Bi        e
     ch                                                                                                                olo
         ite                                                                                                                gy
              c
                                                                                                                     El




                                                                                                                                      0
                                                                                                                                          20
                                                                                                                                               40
                                                                                                                                                    60
                                                                                                                                                         80
                                                                                                                                                              100
                                                                                                                        ec
                                                                                                                           &E
                                                                                                                      Bi E
                                                                                                                         olo
                                                                                                                    Co gy
                                                                                                                       m
                                                                                                                          p
                                                                                                                   Bu Sci
                                                                                                                      sS
                                                                                                                          tu
                                                                                                                              ds
                                                                                                                        Da
                                                                                                                            nc
                                                                                                                     Sp e
                                                                                                                        or
                                                                                                                           tS
                                                                                                                               c
                                                                                                                      Fi i
                                                                                                                         ne
                                                                                                                             Ar
                                                                                                                                 t
                                                                                                                           La
                                                                                                                     Nu         w
                                                                                                                         rs
                                                                                                                             ing
                                                                                                                        Ps
                                                                                                                            y
                                                                                                                    De ch
                                                                                                                       sig
                                                                                                                            nS
                                                                                                                                 t
                                                                                                                        Mu
                                                                                                                   Ci sic
                                                                                                                     ne
                                                                                                                         &P
                                                                                                                              h




                   0
                       20
                            40
                                 60
                                      80
                                           100
    Bi
       olo                                                                                                           Ar o
           gy                                                                                                           ch
    Hi                                                                                                                      ite
       sto                                                                                                          Me c
           ry                                                                                                          di
                                                                                                                          aS
                                                                                                                               td
         La                                                                                                           En
              w                                                                                                          gl
   En                                                                                                               Te ish
       gl                                                                                                              ac
           ish                                                                                                            hr
  El
     ec                                                                                                             So Tr
         &E                                                                                                            cW
               E                                                                                                            or
     Ps                                                                                                                         k
                                                                                                                      Hi
So ych                                                                                                                   sto
   cio                                                                                                                         ry
                                                                                                                                                                    Percentage agreement LRes 2007, selected subjects (JACS 107)




        lo                                                                                                             Dr
 Te gy                                                                                                                     a
    ac                                                                                                             Ac ma
        hr                                                                                                            St
            T                                                                                                            E
  Sp r                                                                                                             So duc
     or
         tS                                                                                                           cio
             ci                                                                                                           lo
     Mu                                                                                                                       gy
Ac sic
    St
       E
Bu duc
    sS
                                                                                                                                      0
                                                                                                                                          20
                                                                                                                                               40
                                                                                                                                                    60
                                                                                                                                                         80
                                                                                                                                                              100




        tu                                                                                                            Nu
            d                                                                                                             rs
   Nu s                                                                                                                       i
       rs                                                                                                           Te ng
           ing                                                                                                         ac
                                                                                                                           hr
     Dr                                                                                                                         T
         am                                                                                                          Sp r
                                                                                                                        or
 Co a                                                                                                                       tS
     m                                                                                                              So ci
        pS                                                                                                             cW
             ci                                                                                                              o
     Da                                                                                                              Ar rk
         nc                                                                                                             ch
  Ar e                                                                                                                       i
     ch                                                                                                            Ac tec
          it                                                                                                           St
 So ec                                                                                                                    Ed
    cW                                                                                                                          uc
          o                                                                                                                 La
 Me rk
     di                                                                                                            Bu             w
        aS                                                                                                             sS
Ci           t                                                                                                             tu
   ne d                                                                                                                         ds
                                                 Percentage agreement OvSatis 2007, selected subjects (JACS 107)




      &P                                                                                                                Dr
            ho                                                                                                              am
    Fi
      ne                                                                                                                          a
           A                                                                                                           Hi
 De rt                                                                                                                    sto
     sig                                                                                                                         ry
          nS                                                                                                            Da
               t                                                                                                            nc
                                                                                                                     El e
                                                                                                                        ec
                                                                                                                            &E
                                                                                                                      En E
                                                                                                                          gl
                                                                                                                              is
                                                                                                                      Bi h
                                                                                                                         olo
                                                                                                                                gy
                                                                                                                        Mu
                                                                                                                               s
                                                                                                                    Co ic
                                                                                                                        m
                                                                                                                           pS
                                                                                                                    De ci
                                                                                                                        sig
                                                                                                                   So nSt
                                                                                                                      cio
                                                                                                                           lo
                                                                                                                               gy
                                                                                                                        Ps
                                                                                                                             y
                                                                                                                    Me ch
                                                                                                                                                                    Percentage agreement PerDev 2007, selected subjects (JACS 107)




                                                                                                                        di
                                                                                                                           aS
                                                                                                                                td
                                                                                                                       Fi
                                                                                                                         ne
                                                                                                                   Ci          Ar
                                                                                                                      ne t
                                                                                                                         &P
                                                                                                                               ho
JACS Subject   TL   AF   AS   OM   LR   PD   OvS

Nursing        6    2    12   20   9    1    13

Biology        3    22   1    1    2    14    1

SportSci       9    6    7    9    6    3     9

Psych          10   13   17   2    10   19    6    Top 5
CompSci        21   15   8    7    3    16   15

Elec&EE        16   14   2    10   1    12    5
                                                   Bottom 5
Architec       14   20   14   22   14   5    17

Sociology      11   9    15   6    22   18    7

SocWork        17   16   22   21   18   4    18

Law            7    11   13   5    8    7     3

BusStuds       19   17   16   8    4    8    12

MediaStd       18   18   18   14   15   20   19

English        2    3    5    4    16   13    4

History        1    1    3    3    19   10    2

FineArt        22   12   21   17   7    21   21

DesignSt       20   8    20   19   11   17   22

Music          8    10   4    12   12   15   10

Drama          4    19   10   15   20   9    14

Dance          5    21   9    13   5    11   16

Cine&Pho       15   7    19   16   13   22   20

TeachrTr       12   5    6    18   17   2     8

AcStEduc       13   4    11   11   21   6    11
Where the truth lies

A paradox…

Raw NSS ratings don’t tell the whole story

The ‘residuals’ make allowances for various
background characteristics of HEIs and hence
provide some evidence of ‘institutional effect’

Joe and Jill Public won’t appreciate this


With ‘residuals’ there’s no Lake Wobegon effect
    Organisation & Management, Creative Arts & Design 2007
‘Residual’
 0.60


 0.40


 0.20


 0.00


-0.20


-0.40


-0.60
     30%     40%   50%      60%       70%       80%   90%    100%
                         Percentage agreement
Overall Satisfaction, Creative Arts & Design 2007
T&L        Asst & Fdbk




Acad Sup   Org & Man
Learn Res   Pers Dev
Institutional size

Smaller institutions (N<10,000) consistently get
higher scores on the following scales:

• Teaching & Learning
• Assessment & Feedback
• Academic Support
• Personal Development
• Overall Satisfaction


So how do you make the big seem small?
Three strikes…
   JACS subject area                         T&L        A&F       AcSup       O&M        LRes       PDev       OvSatis
                                            +/ns/ns      -/-/-      -/-/-      -/-/-      +/+/+      +/+/+      ns/-/-
                  Medicine and Dentistry
                                            ns/ns/+    ns/ns/ns    +/+/+       -/-/-      +/ns/+     +/+/+     ns/ns/ns
              Subjects Allied to Medicine
                                            ns/-/ns    +/ns/ns     +/+/+      +/+/+      ns/ns+       -/-/-    ns/ns/ns
                      Biological Science
                                            ns/ns/ns     -/-/-    ns/ns/ns     -/-/-     ns/ns/ns   ns/ns/ns     -/-/-
                      Veterinary Science
                                              -/-/-    ns/ns/ns    +/ns/+      -/-/-      -/ns-       -/-/-      -/-/-
                 Agriculture and Related
                                            +/ns/ns     +/+/+      +/+/+     ns/ns/ns     +/+/+     +/ns/ns     +/+/ns
                      Physical Sciences
                                              -/-/-     +/+/+      +/+/+      +/ns/+      +/+/+       -/-/-    ns/ns/ns
                   Mathematical Science
                                              -/-/-      -/-/-    +/ns/ns      -/-/-      +/+/+       -/-/-      -/-/-
                      Computer Science
                                              -/-/-    ns/ns/-     +/+ns       -/-/-      +/+/+       -/-/-      -/-/-
            Engineering and Technology
                                              -/-/-     ns/-/-    ns/ns/ns     -/-/-     ns/ns/-    ns/ns/-      -/-/-
     Architecture, Building and Planning
                                              -/-/-     +/+/ns    ns/ns/ns     -/-/-       -/-/-      -/-/-      -/-/-
                          Social Studies
                                              -/-/-    ns/+/ns    ns/ns/ns     -/-/-      ns/+/+    ns/ns/ns     -/-/-
            Business and Administration
                                              -/-/-    ns/ns/-    ns/ns/ns     -/-/-     ns/ns/ns     -/-/-      -/-/-
                   Mass Communication
                                             +/+/+      +/+/+      +/+/+     ns/ns/ns      -/-/-    ns/ns/-    +/ns/ns
                              Languages
                                             +/+/+      +/+/+      +/+/+      +/+/+        -/-/-    ns/ns/ns    +/+/+
     Historical and Philosophical Studies
                                              -/-/-    +/ns/ns     ns/-/-      -/-/-       -/-/-      -/-/-      -/-/-
                Creative Arts and Design
                                              -/-/-     +/+/+     ns/+/ns      -/-/-       -/-/-     +/+/+       -/-/-
                              Education
                                            ns/ns/ns   +/ns/ns    +/ns/ns    ns/ns/ns    ns/ns/ns     -/-/-    ns/ns/ns
                              Combined
                                              -/-/-     +/+/+     +/ns/ns      -/-/-      -/-/ns      -/-/-      -/-/-
                           Two subjects
                 Three or more subjects       -/-/-     +/+ns     +/ns/ns      -/-/-     ns/-/ns     ns/-/-      -/-/-


   Reference subject area: Law                                                         Surridge (2009, p.18)
Art & Design in some other surveys
Interpreting surveys of ‘early leavers’


Self-selected respondents, hence possible bias

Possibility of blaming programme > self

Unclear which had done foundation programme

Items are selected, mainly focusing on programme

Responses from A&D compared with ‘the rest’:
high scores (red) are worse; low (blue) are better

Scores are percentages indicating moderate or high
influence on departure
Reactions from ‘early leavers’ (% responses)
                                      2007         1997
                                    A&D Rest     A&D Rest
                            Nmax     60    267   105   1881
Programme not as expected           52     45     48    36
Chose wrong field of study          30     44     26    40
Lack of commitment to programme     23     35     30    39
Contact with academics              68     33     29    24
Teaching quality                    60     24     39    22
Lack of staff support               52     30     33    24
Feedback quality                    52     20     -     -
Programme organisation              60     30     47    26
Lack of academic progress           48     32     38    30
Workload                            13     21     10    18
Provision of specialist equipment   39      5     16     7
Financial problems                  25     33     44    36
Selected comments from 1st year leavers, 2007

Did foundation year in art + design which I really enjoyed but
when started 1st year it was completely different, hardly any
contact with staff or students, lost motivation + interest …


I was at college for 3 years and enjoy my time there.
When I went to [univ] to study Graphic communication it was
not what I expected. The programme was unorganised and
the teaching was at a poor standard. I work hard on this course
and give my all. For one year I just hated [univ].


Badly run course with staff totally disinterested in their students.
Selected comments from 1st year leavers, 2007

There was very little actual teaching, promised classes did not
materialise until after students complained (i.e. life drawing).
Quite often we … were kept waiting for staff to arrive on time and
often what was said by one was repeated by another, usually with
a slighter different version which was confusing. The equipment
was more in line for a 6th form class rather than a degree course
and often space was at a minimum, especially in the printshop
and ceramics rooms.


The course was very disorganised. […] The content of the course
was not what I expected and at times seemed irrelevant. Constant
room changes early on was very off putting and tutors failing to
attend lectures consistently.
Selected comments from 1st year leavers, 2007

I felt [univ] graphic arts and design course was overcrowded […],
the tutors were largely unhelpful and arrogant. There was little
course structure and no group spirit or suitable work space.
I am now attending [a nearby FEC] and loving every minute.

The main reason I left [HEI] was because of the organisation of the
course and the projects. The projects were so open you could film
anything and say it has something to do with the subject matter.
I did not see one project that I thought was art when I attended.

I love photography but my experience at [univ] has put me off.
The lack of support from staff was a big part. Just finding them
took 2 hours. Then they went on strike …
The first-year experience and part-time
student surveys: same caveats as before
Self-selected respondents, hence possible bias

Possibility of blaming programme > self

Unclear which had done foundation programme

Items are selected, mainly focusing on programme

Responses from A&D compared with ‘the rest’:
low scores (red) are worse; high (blue) are better
Scale is 1 to 5, with high ratings indicating positivity
to first-year experience
Reactions from the first year (scale means)
                                             2006
                                           A&D   Rest
                                    Nmax   957   6076
Programme not as expected                  3.09   3.15
Choice of subject                          4.35   4.19
Motivation                                 3.20   2.97
Contact with academics                     3.39   3.74
Teaching quality                           3.56   3.69
Tutorial support                           3.38   3.54
Feedback quality                           3.44   3.14
Programme organisation                     3.28   3.67
Keeping up with workload                   3.19   3.14
Access to institutional computing          3.88   4.03
Financial worries                          2.36   2.46
Selected ‘best’ comments from first-year students, 2006

The tutors - great members of staff and a really interesting
vibrant course.

The tutors are great and always available if I'm having trouble
with my work.

Being able to chat to the tutors casually about your work etc.

The structure of the course is very well set up, I really like
the fact that we have specific projects and deadlines for
those projects and I find the workshop sessions very enjoyable.

The projects that we work on have been exciting.
Selected ‘worst’ comments from first-year students, 2006

Teaching! I want to learn please!
Don’t feel I'm being taught enough.

Wish I had more 1 on 1 tutorials as I have no clue what is
expected of me or how to do better!

Having no feedback on work. It's needed.
Feedback takes too long.
Not receiving enough feedback on work and staff not being present.
Lack of support, lack of contact with tutors, having to wait
6 months before having any feedback in the form of an assessment.
Last minute changes to timetable.
Selected ‘worst’ comments from first-year students, 2006

Organisation (not impressed).

The organisation of the course has been poor in respect to
sessions and lectures being changed at short notice or
even cancelled.

Not well organised course, very few teachers, no help to
develop computer skills.

Poor organisation of projects, workshops, teaching.

Confusion due to teachers saying different things for 1 brief.
Issues for practice
Issues for practice

• Organisation and management of programmes,
  and Feedback, are two aspects that seem to need
  the greatest attention

• Can programmes be structured to fit more closely
  the current situation of students?
  (E.g. ‘commuter students’, students with PT jobs)

• The general ‘message’ seems to be about being
  as considerate as possible about the whole of the
  student experience
Two messages to academic leaders

1 Target your efforts on the student experience, and
  not merely on getting higher NSS scores


2 Don’t naively demand ‘4.1 (or whatever) across
  the board’
  Look carefully at where the subject performances
  in your institution stand in relation to those in
  comparable institutions, and act appropriately

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:22
posted:9/22/2012
language:English
pages:38