Open ended by alicejenny

VIEWS: 4 PAGES: 29

									Undergraduates as Teaching
Assistants
Reducing the marks-based culture of learning
                     Tony Field
              Department of Computing

                      Joint work with
         Esat Alpay, Peter Cutler, Susan Eisenbach


     http://www.doc.ic.ac.uk/lab/firstyear/envision/
    The “marks-based” culture
    •   Use the weighting of the work to determine
        the time spent
    •   Something that carries little or no weight
        gets little or no time
    •   Concerns for marks rather than the
        knowledge/skills attained


2
    The idea
    •   Zero-weight first year tutorial work in
        Programming and Mathematics
        • 6/7 students
        • Weekly exercises (previously assessed by academic)
    •   Pay A-grade 3rd/4th year undergraduates (UTAs) to
        help run the tutorials
        • UTAs mark exercises (feedback only)
        • Feedback, problem-related and general discussion

    Academic’s role: Mentor the UTA, check the marking, help
    plan meeting and guide discussions…

3
    Some questions
    1.   Does zero weighting affect student motivation
         and participation?
    2.   How do students react to having tutorials run
         by peers?
    3.   What are the benefits for the UTA?
    4.   How do the academics perceive the change?
    5.   Is there a significant difference in student
         performance?
    6.   (Does it reduce the instances of plagiarism?)

4
    Evaluation
    • Cost: Around £60-£75 per student per term
      (one tutorial/week, 2-3hrs marking)
    • Pilot study sponsored by Envision for Maths
      tutorials (PMTs)
    • 2-year study, with qualitative and quantitative
      evaluation:
        Changing the marks-based culture of learning through peer-assisted
        tutorials, European Journal of Engineering Education,Volume 35,
        Issue 1 March 2010 , pages 17 - 32




5
Questionnaires
•   One set for first years, the other for both
    UTAs and academics
•   Five-point Likert scale
•   Open-ended questions at the end
Main theme of student
questions
   How the first years compare the UTAs
    and the academics
      ME – motivation and engagement
      SD – skills development
      ECF – explanation, contextualisation, and feedback
                    ME       Getting me to participate in the tutorial
                             Making the tutorial work interesting
                             Motivating me to want to do well in the course
                             etc.
Evaluation of PMT and UTA.                                             UTA      PMT
1) Understanding my needs as a first-year student                      12345    12345
2) Getting me to participate in the PMT                                12345    12345
3) Making the PMT work interesting                                     12345    12345
4) Making it easy for me to talk about things I don’t understand       12345    12345
5) Having a difficult concept explained to me                          12345    12345
                        SD on my helping me to…improve my logic1skills 5
6) Giving useful feedback         work                                   234    12345
                                 ...improve my communication skills 3 4 5
7) Clarifying/correcting misunderstanding                              12       12345
                                 …manage my time so that I can finish4 the exercises
8) Helping me improve my logic skills                                  123 5    12345
9) Helping me improve my communication skills                          12345    12345
                                 etc.
10) Exploring a problem/concern thoroughly                             12345    12345
11) Motivating me to want to do well on the course                     12345    12345
12) Giving me reassurance about my learning progress                   12345    12345
13) Clarifying why the material will be useful later on                12345    12345
14) Helping me to manage my time so that I can finish the exercises    12345    12345
15) Making me feel part of the department                              12345    12345
                        ECF      having a difficult concept            1 2 4 me 1 2 3 4 5
16) Exploring related topics not covered directly by the course explained3to 5
                             giving useful feedback on my work
                             clarifying why the material will be useful later on
                             etc.
Summary of results
Three key questions
   The most discriminatory questions
    (UTA>Academic):
    Getting me to participate in the PMT (ME)
    Giving useful feedback on my work (SD)
    Helping me to improve my <maths> skills (ECF)
Main theme of UTA/tutor
questions
   How UTAs and Academics judge the PMT
    system
      UPE – UTA performance and effectiveness
       UPE1 (teaching qualities)
       UPE2 (social/affective aspects as peer)
      UB – UTA personal benefits
      US – Support received by the UTA
A: Evaluation of PMT. UPE1             The UTA prepares well for the PMT meeting
1) The UTA gets good support for his/her role
                                       The UTA uses initiative in running the meeting 4 5
2) The UTA’s communication skills are improving over time
                                                                                          123
                                                                                          12345
                                       The UTA
3) The UTA prepares well for the PMT meeting has a clear idea about how to run the 4 5    1 2 3 meeting
4) The UTA is a role model for the first year students                                    12345
                                       etc.
5) The UTA has many interests in common with the students                                 12345
6) The UTA takes the lead role at the PMT meeting                                         12345
7) The UTA uses initiative in running the PMT meeting                                     12345
8) The UTA gets good support from the academic (PMT)                                      12345
                         UPE2 to the students
9) The UTA gives thorough feedback The students find it easy to talk openly to the 3 4 5  1 2 UTA
10) The UTA explains the PMT material clearly                                             12345
                                                                                          1 2 explain…
                                       The UTA is a student so it’s easier for them to 3 4 5
11) The students find it easy to talk openly to a UTA
                                       The UTA is a role model
12) The students find it easy to talk freely with the academic (PMT)                      12345
13) The students want the UTA to think well of them                                       12345
                                       etc. to think well of them
14) The students want the academic (PMT)                                                  12345
15) The students find it easy to ask a UTA when they don’t understand                     12345
16) Most students are more concerned with marks than understanding                        12345
17) I suspect that some of my students copy their PMT submissions                         12345
                         UB            The UTA seriously
18) The students take the feedback on their worklearns useful transferable skills         12345
19) The UTA has a clear idea about how to run the PMT meeting                             12345
                                       The experience is the most important motivating factor
20) The UTA and academic (PMT) discuss teaching strategy before the meeting               12345
                                       The UTA finds it useful to revisit the material2 3 4 5
21) I enjoy seeing the students develop their skills                                      1
22) The UTA is a student so it easier for him/her to explain concepts to the PMT students 12345
                                       Etc. meetings over the year
23) The students have improved in the PMT                                                 12345
24) The mentoring role of academic (PMT) is very important for a UTA                      12345
25) The UTA finds it hard to get the students to participate                              12345
26) The academic (PMT) finds it hard to get the students to participate                   12345
27) The UTA finds it helpful to revisit the first year material                           123
                         US the UTA useful advice about running the PMT from the academic 4 5
28) The academic (PMT) gives
                                       The UTA gets good support                          12345
                                       The mentoring role of
29) The UTA learns useful transferable skills by helping with the PMT the academic is important
                                                                                          12345
30) Having a UTA helps enhance the department’s intellectual culture
                                       The academic gives the UTA useful advice 1 2 3 4 5
31) The payment is the most important motivator for a UTA                                 12345
                                       etc.
32) The experience is the most important motivator for a UTA                              12345
Summary of results (17/23 returns)

             Mean   Standard deviation

    UPE1
    UTA      4.02   0.45
    PMT      4.34   0.56
    UPE2
    UTA      3.85   0.69
    PMT      3.67   0.80
    UB
    UTA      4.08   0.64
    PMT      3.90   0.57
    US
    UTA      3.42   0.72
    PMT      3.86   0.79
Four key questions
Statistically significant disagreements (UTA
vs Academic score):
  The UTA prepares well for the PMT meeting (4.06 vs
   4.59)
  The UTA takes the lead role at the PMT meeting (4.24
   vs 4.88)
  The UTA and academic discuss teaching strategy before
   the meeting (2.45 vs 3.47)
  The UTA is a student so it is easier for him/her to
   explain concepts to the students (3.88 vs 2.94)
Quantitative Measures
                        2005-2006   2006-2007     2007-2008
                           (%)         (%)           (%)
student attendance at     89.2        92.3          92.7
PMT sessions
submission rate (PMT      93.5        98.3          96.8
work)
average mark (PMT         80.0        81.4          71.2
work)
test result (Logic +      48.9        53.0          57.0
Discrete Maths)
Discrete Mathematics      64.3        59.8          57.1
exam average
Logic exam average        55.0        61.7          60.4
Reasoning exam            53.7        55.1          53.8
average

                          UTAs mark rigorously!
Open-ended questions
Observation 1: Students really like it

                                               • Excellent scheme, that a lot of other courses do not
•                                                       have, has helped a
    The UTA scheme is the best part of the whole non-programming oriented part of the course! lot in studies, and to settle in
•

•
                                               • I think it's fantastic. Great job. Really appreciate the
    Excellent scheme, that a lot of other courses do not have, has helped a lot in studies, and to settle in
    overall extremely helpful...don't know what I would have done without it!
•   Best method of study by far                         "anytime" approach.
•

•
                                               • The UTA scheme is the best part of the whole non-
    I think the PMT sessions is the most helpful hour of the week.
    very helpful, can't think of any way to gain significant improvement.
•   The UTA scheme was the most helpful learning programming oriented part of the course!
                                                        tool in my course this year.
    Excellent idea. It both encourages us to aspire to work hard and helps the UTA in furthering his/her skills in teaching.
                                               • Best method of study by far
•

•   I really appreciated that the student who is still studying at the Universrity and has very recent experience with the functioning of the course is very helpful.
•   Execllent, everybody wins.               etc.
•   I think it's fantastic. Great job. Really appreciate the "anytime" approach.
•   I think it is better than the tutorials.
•   Its an excellent system that allows people to fully grasp concepts on the course.
•   Personally I find it very helpful and my UTA is very keen to explain anything we ask. Thumbs up for UTA programme.
•   It is very helpful as unlike lectures, the UTA understands what areas we are unsure on, since he is student after all!
•   In my specfic case, my UTA is outstanding. He is a superb student, and has a lot of time and effort for me.
•   My UTA was very helpful in this course.
•   Excellent. Great help. I've spoken to first year maths degree kids and they are jealous of the amount of help I get in smaller groups. I have found the PMT very very helpful
    indeed and would like even more of this time set aside for us. We have many hours of contact time each week for programming related help, yet only this one hour PMT
    slot for maths (which I and others think is the more difficult of the sections).
•   Good idea, provides some useful feedback
Open-ended questions
Observation 2: They like students as TAs
                                       • It's a "family" atmosphere
•   It's a "family" atmosphere         • They know exactly what they found difficult when
•   The environment feels a bit more comfortable
•
                                              they did the course and guide according to that
    He is also a student, so he can give tips of what exactly is expected from us
•   Down to earth, jokes, knows the•          The we are in and can relate to have somebody who is more on
                                         situation that opportunity to it.
•   He's friendly and helpful and gives good explanations of the material
                                              our wavelength
•   The opportunity to have somebody who is more on our wavelength
•   Relaxed learning
                                       • UTA's are closer to our age and I think are better
•                                             often to communicating with us by results, can lecturers
    It is personal, small group is good for learning, getat know everyone in your group, don't feel scaredthan thediscuss work with all
    members
•   That it is mainly run by student etc. way that only person who is still studying can do
                                      - in the
•   They know exactly what they found difficult when they did the course and guide according to that.
•   Very friendly and straight to the point
•   It's very casual and he not only understands my problems, but also anticipates them.
•   That the UTA is run by students of approximately my age
•   UTA's are closer to our age and I think are better often at communicating with us than the lecturers
•   I believe its helpful, since sometimes you can approach the UTA in a more open manner than you would to with the tutor and that is
    sometimes helpful.
Open-ended questions
Observation 3: Some UTAs need
vetting/better training
                            • Train them more in terms of dealing with students
•                               tutorial if possible
    More UTA involvement in thebeforehand
•   The UTA should be commanded to make students/tutees participate in the session
                            • Possibly a more careful choice of UTA candidates
•   A more lively atmosphere would be nice
•   MAKE IT MORE FUN!!!
                            • More UTA involvement in the tutorial if possible
•   Better communication • A more lively atmosphere would be nice
•   More passionate uta's etc.
•   Train them more in terms of dealing with students beforehand
•   It is a good system, my UTA just happens to be a little too shy for the job
•   Train the UTA
•   Possibly a more careful choice of UTA candidates
•   It is good if the UTA doesn't keep canceling the meetings/ not turning up to them.
•   My UTA for the PMT course was incredibly bad at 'public speaking', incredibly nervous etc.
•   It works fine, but some people get a poor UTA, which gives them a disadvantage
Open-ended questions
Observation 4: Staff need better
training(!)
                            • PMT is not always there, and when is is usually
•
                                working and not joining in
    PMT should do more explaining, not just UTA
•
                            • involve working meeting.
    Acedemic teachers should It was more in thefine until he stopped showing up a
•   Lecturers that are assignedmonth ago
                                 to the PMT groups could be more involved with the PMT
•                           • most of my tutorial classes, which get more involved
    My PMT did not turn up toThe academic staff should was disappointing
•                           • My PMT did not turn up to most of my tutorial
    The academic staff should get more involved
•                               classes, which was disappointing
    Generally, the member of academic staff doesn't really do anything.
•                          etc.
    The main PMT tutor never participates on the scheme. Only the UTA is involved.
•   We end up with pmt sessions where UTAs do all the work while tutors are not participating.
    Open-ended questions
    Observation 5: UTAs benefit
    substantially
                                           • Enjoyable experience, learned a lot myself, good
                                                contact to PMT.
•
                                           • Other first years ask me questions in the labs - I like
     The meetings are fun and it is good talking with the students and getting them involved in the subjects.
•    Other first years ask me questions in the labs - I like helping people and it is good to know you've done it.
•
                                                helping people and it is take the sessions on board and really improve.
     The experience and confidence it has given me, and watching how some of the students good to know you've done it.
•                                          • The experience and confidence it has given me, and
     Enjoyable experience, learned a lot myself, good contact to PMT.
•    Interacting with first year students, The experience of teaching.
•
                                                watching how some of the students take the
     I enjoy working with my PMT (Dr. X) very much.
•    Good teaching experience.                  sessions on board and really improve.
•
                                           • The UTAs gain valuable experience (and some extra
     Experience gained - teaching, assessing. Seeing students develop/overcome learning barriers.
•    To try out teaching - A chance to improve my presentation skills - Be able to explain topics clearly - Watching students learn
•
                                                money which is always useful!).
     Many people benefit (student, UTA, PMT). Good atmosphere.
•                                        etc.
     The UTA program is beneficial to all of the three parties:
•    Students have the opportunity to participate in teaching roles which would be a valuable experience and it also offloaded some workload for PMTs.
•    The UTAs gain valuable experience (and some extra money which is always useful!).
Open-ended questions
Observation 6: Academics love it!


         QUIZ: Why?
Open-ended questions
Observation 6: Academics love it!
                                  • I no longer have to mark the homework
•   I no longer have to mark the homework.
Open-ended questions
Observation 6: Academics love it!
                                  • I no longer have to mark the homework
•   I no longer have to mark the homework.
•   They do the marking!
                                  • They do the marking!
Open-ended questions
Observation 6: Academics love it!
                                  • I no longer have to mark the homework
•   I no longer have to mark the homework.
•   They do the marking!
                                  • They do the marking!
•   No more marking.              • No more marking
Open-ended questions
Observation 6: Academics love it!
                                  •    I no longer have to mark the homework
•   I no longer have to mark the homework.
•   They do the marking!
                                  •    They do the marking!
•   No more marking.              •    No more marking
•   Not doing the marking.        •    Not doing the marking
Open-ended questions
Observation 6: Academics love it!
                                  •    I no longer have to mark the homework
•   I no longer have to mark the homework.
•   They do the marking!
                                  •    They do the marking!
•   No more marking.              •    No more marking
•   Not doing the marking.        •    Not doing the marking
•   Avoiding the marking!         •    Avoiding the marking!
Open-ended questions
Observation 6: Academics love it!
                                  •    I no longer have to mark the homework
•   I no longer have to mark the homework.
•   They do the marking!
                                  •    They do the marking!
•   No more marking.              •    No more marking
•   Not doing the marking.        •    Not doing the marking
•   Avoiding the marking!         •    Avoiding the marking!
    No marking.
•
                                  •    No marking
Open-ended questions
Observation 6: Academics love it!

•
                                       • I no longer have to mark the homework
    I no longer have to mark the homework.
•   They do the marking!               • They do the marking!
•   No more marking.                   • No more marking
•   Not doing the marking.
•   Avoiding the marking!
                                       • Not doing the marking
•   No marking.                        • Avoiding the marking!
•
•
                                       • No marking
    I appreciate not having to do the marking or prepare in detail for the tutorial.
    Avoid the marking of PMT on weekly basis.
•                                      • It takes some of the teaching
    It saves a lot of time that I can more profitably spend on other activities - research for example. burden away from me,
•                                           and made it easier to interact with the students on a
    More fun, less work, not so much boring exercises.
•   Not having the tedium of explaining this material for the 20th time.
•
                                            personal level
    It takes some of the teaching burden away from me, and made it easier to interact with the students on a personal level.
                                       etc.
Conclusions
   From the student perspective, the UTA performance
    exceeds that of the academic
   Despite individual problems, students value the system
    and see UTAs as role models
   The system fosters a sense of community; UTAs become
    mentors (and ‘buddies’?)
   Academics universally praise the UTA system
   There is no significant change in student exam
    performance, attendance etc.
   Needs UTAs to be carefully chosen & trained and
    academics to engage fully
   Successfully instils a culture of learning based on the
    intrinsic value of material, rather than credit

								
To top