; Open CETL Conference 2007
Learning Center
Plans & pricing Sign in
Sign Out
Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

Open CETL Conference 2007


  • pg 1
									  CETL Workshop, 17 December                  2007

               S207 eTMA projects

              Stuart Freake*, Craig McFarlane** and Jimena Gorfinkiel*,

              *Department of Physics and Astronomy
              **S207 Associate Lecturer

Title of presentation in grey 15pt
Date in grey 15pt
S207 eTMA projects

Stuart:   Background to the S207 eTMA projects

          The S207 Tablet PC trial and its evaluation

Craig:    An AL‟s experience of marking S207 eTMAs

Jimena:   Enabling S207 students to submit eTMAs

•   Use of eTMAs has grown rapidly over last few years

    Proportion of courses allowing eTMAs in 2008:
                       All courses     S       M         T
    eTMA expected         40 %        28 %     2%        58 %
    eTMA encouraged       31 %        23 %     45 %      16 %
    eTMA compulsory        5%          1%                21 %
    paper compulsory      22 %        43 %     53 %      5%
    mixed submission       4%          4%

•   Use of eTMAs has grown rapidly over last few years

    Proportion of courses allowing eTMAs in 2008:
                       All courses     S       M         T
    eTMA expected         40 %        28 %     2%        58 %
    eTMA encouraged       31 %        23 %     45 %      16 %
    eTMA compulsory        5%          1%                21 %
    paper compulsory      22 %        43 %     53 %      5%
    mixed submission       4%          4%
Science and Maths eTMAs

•   Many Science and Maths students could submit electronically
    – some students submit paper TMAs produced with a word processor,
      including equations and diagrams
    – students can scan handwritten TMA and submit electronically

•   Students who have used the eTMA system for previous courses
    may be unhappy about reverting to paper TMAs for later courses

•   We should allow the option of eTMA submission, if possible

•   But allowing eTMAs in Science and Maths requires straightforward
    methods for ALs to mark and annotate eTMA files, including
    equations and diagrams
Marking physics eTMAs
•   One focus of p CETL is e-teaching, e-learning and e-assesssment

•   π CETL approved eTMA marking project in early 2006
    – course selected: S207 The Physical World

       • 60 pts, 500 students, 25 ALs
       • 7 TMAs – with lots of equations and diagrams

    – ALs provided with Tablet PCs (Toshiba Tecra M4)
       •   enables handwritten annotation of eTMAs
       •   ALs can mark in similar way to marking paper TMAs
Criteria for software to be used

•   Students should be able to prepare eTMA using software they find
    – no requirement to purchase software
    – scanned handwritten eTMAs should be accepted

•   ALs should be able to use a single application for marking
    – application must allow straightforward insertion of equations and diagrams
    – no increase in AL workload
    – usable whatever file format student uses

•   Annotated file returned by AL must be easily readable by student

•   No loss of information – including maths content – at any stage
Software solution adopted

•   Students are allowed to submit in any file format that their tutor
    can open

•   Student‟s eTMA file is converted to pdf format
    – using PDF Creator, freeware from http://www.pdfforge.org

•   AL marks and annotates the pdf file
    – using PDF Annotator, £25 from http://www.GRAHL-software.com

•   AL „melts‟ annotation into pdf file so it cannot be changed
    – saves annotated script in pdf format
    – returns pdf file with embedded annotation to student

•   Student reads annotated pdf file using Acrobat Reader
S207 eTMA submission data

•   2006: optional participation by ALs; 19 of the 23 ALs agreed to take part
          – Tablet PCs sent out in early April, some ALs ready to go with TMA02

•   2007: „eTMA encouraged‟ course; all 24 ALs participated

                                                                                                       eTMAs        paper TMAs
    number of TMAs




                            TMA02 TMA03 TMA04 TMA05 TMA06 TMA07 TMA01 TMA02 TMA03 TMA04 TMA05 TMA06 TMA07

     % TMAs submitted




                            TMA02   TMA03   TMA04   TMA05   TMA06   TMA07   TMA01   TMA02   TMA03   TMA04   TMA05   TMA06   TMA07
S207 eTMA submission data

eTMA submission rates varied widely for
different students groups


 number of ALs







                     0-5   6-10   11-15   16-20   21-25   26-30   31-35   36-40   41-45   46-50   51-55    56-60   60-65
                                                           eTMAs marked / %
S207 eTMA submission data

eTMA submission rates varied widely for
different students groups

                                                                                                  2006      2007
 number of ALs

                     0-5   6-10   11-15   16-20   21-25   26-30   31-35   36-40   41-45   46-50   51-55   56-60    60-65
                                                          eTMAs marked / %

•   Questionnaire emailed to ALs at end of each presentation,
    plus review of messages on AL forum

In 2007
• 70% of eTMAs word processed, including equations and diagrams
• 20% mainly word processed, but some equations/figures scanned
• 10% hand written and then scanned
•   83% of eTMAs submitted as Word .doc files
•   3% in other word processor formats
•   7% .pdf files
•   6% image files
Marking procedures used in 2007

ALs fell into two groups

•   About half converted to pdf format, then marked with PDF Annotator

•   The other half marked and commented using MS Word
    – they only received Word files – perhaps they explicitly requested this format?
    – most of them used both keyboard and tablet pen entry
    – a couple of ALs opted not to use the Tablet PC – used a conventional computer
      to insert text boxes and comments

•   None of the ALs used Windows Journal
    – but they weren‟t provided with information about this software
Problems and disadvantages

•   No major problems learning how to use system

•   Some ergonomic issues
    – Comfortable writing position
    – Viewing the screen

•   Some ALs took longer to mark eTMAs than paper TMAs

•   Difficult to get an overview or to compare scripts;
    difficult to mark all Q1s as a batch
Advantages of using the Tablet PC
•   Flexible annotation – can reproduce anything done on paper, plus ….
•   Ease of erasing or modifying or moving comments, inserting extra pages,
    inserting appropriate parts of model answers
•   Ability to store and insert frequent comments
•   Time taken – eTMAs quicker to mark for some ALs

… plus all of the advantages of the standard eTMA system
    – Ease of navigation of file – can‟t lose pages or get them out of order!
    – Boxes on PT3 forms pre-filled
    – Can‟t add up question scores wrongly
    – Can write as much as you like on PT3
    – Retain a copy of TMA for reference
    – Improved turnaround time, particularly for students outside the UK
AL preferences

•   In 2007,

    – 11 ALs expressed preference for marking eTMAs

    – 8 were either equally happy with eTMAs or paper TMAs, or
      expressed no preference

    – 4 preferred paper marking (of whom 3 had received very few
Tablet PCs for P&A courses in 2008

•   S207 ALs will continue to use Tablet PCs for marking eTMAs
    – all ALs are now involved, all students have the option to submit eTMAs

•   Tablet PCs will be provided to enable ALs tutoring
    SMT359 Electromagnetism and S381 The Energetic Universe
    to mark eTMAs
    – participation by ALs optional in 2008, required in subsequent presentations
The S207 eTMA projects

 An AL’s experience of marking S207 eTMAs
               Craig McFarlane
S207 eTMA hardware and software

S207 eTMA marking needs:

•   a Tablet PC – to simulate “paper marking”
•   PDF Creator – to convert Word files etc into pdf files
•   PDF Annotator – to write directly on pdf files
•   OU eTMA Filehandler: (i) location for downloaded files
                         (ii) source of electronic PT3 form
                         (iii) zips marked files for upload
•   a wireless router – to download eTMAs direct to PC
    (not essential but preferable)
•   A USB keyboard to enable typed input in tablet mode (optional)
File conversion using PDF Creator
 – Student can submit eTMA in a variety of file formats
 – AL opens student‟s eTMA file and converts it to pdf format using PDF Creator
Marking pdf file using PDF Annotator
 – AL opens pdf file with PDF Annotator
 – marks script by writing on screen with tablet pen
Sample marked eTMA
Sample marked eTMA
Advantages of Tablet PC - AL viewpoint

•   Mimics experience of paper marking (most ALs write faster than
    they type)…
•   …but with additional advantages:
•   Comments can easily be deleted/revised
•   Extra space can be generated for extended comments
•   Model answers can be pasted in at appropriate place
•   Equations and sketches can be inserted rapidly (and copied for
    further use)
•   Time taken about the same as paper TMAs (not all ALs agree with
Disadvantages of Tablet PC

•   Awkward to enter text while in tablet mode, so…
•   …spend time toggling between tablet/keyboard modes to fill in
    PT3s (This can be overcome by using a USB keyboard at the cost
    of “ergonomic restriction”)
•   Have to convert most student files to pdf form
•   Is it a “Rolls-Royce” solution – practicable for all science courses?
Student comments
    (Although most of these comments refer to general advantages of the
     eTMA system, for S207 students it is the Tablet PC which has made
    eTMA submission possible)

•   “One benefit is getting the TMAs back so quickly. It really helps to get them back and
    realise I am doing OK. I find the system really quick and easy to use to submit the
    TMAs but I have had trouble getting them back on occasion.”

•   “I have had no problems with the system, it has been 100% reliable and the speed of the
    system means that I can work up to a few minutes before the deadline and be certain
    that it will be with the OU in time.”

•   “As you know I've been doing a bit of travelling this year, America first and now
    Afghanistan. The eTMA service for this reason has been very useful… The only down
    side is the time it takes to draw diagrams and type the equations out… At the beginning
    we were offered scanners; this in hindsight was a good idea and I should have taken it.”

•   “Although it did take me ages to work out how to use Equation Editor, I am glad I got the
    hang of it in the end.”
Thoughts on two years’ eTMA marking

 • Using the Tablet PC, it‟s possible to mark eTMAs as rapidly as
  paper TMAs, but…
 • ….speed comes with practice, and practice requires that a
  significant fraction of TMAs arrive in this form
 • This is more likely to happen if the AL actively   promotes the
  advantages of eTMA submission
 • Resistance to the system from ALs exists, but would be much
  greater in the absence of Tablet PCs
How do we get more students to submit

•   A positive attitude to eTMAs on the part of the AL can help, but…
•   Fear of MS Word Equation Editor is the biggest barrier to eTMA
•   Graphs and diagrams are an additional problem
•   Jimena has been running a project with S207 students to enable
    some of them to surmount these hurdles……
The S207 eTMA experience

     Enabling students to submit eTMAs
               Jimena Gorfinkiel
Enabling students to submit eTMAs

•   How to encourage electronic submission?
•   How to make it easier for students?
    – incorporate equations and figures
    – produce whole TMA with a single device

•   Ensure good quality for tutors to read and mark
•   Trial of electronic equipment to produce eTMAs
    – asked tutors to participate in trial and contact students from early April

•   Most students received equipment in May (in time for TMA04) and
    some after residential school (August)
•   PC Notes Taker
    – uses special pen and clip base unit attached to USB port
    – write on normal paper (produce paper and electronic version at the same time)
    – files for each page can be collated into Word document
    – jpeg files of figures and equations can be created

•   Logitech io2 Digital Writing System
    – uses special pen and special notebook
    – write on notebook (produce paper and electronic version at the same time)
    – files stored in pen and saved into computer by docking pen into penholder plugged
      into USB port
    – files for each page (converted into .gif) can be collated into Word document
    – files of figures and equations can be created
Results of evaluation
•   31 students requested equipment (not all of them used it)
•   14 students answered the questionnaire (8 Logitech)
    – 5 did whole TMA and 5 used pens for equations and figure only
    – 7 had submitted eTMAs before (2 did all eTMA with pen, 3 did EQ/F, 2 nothing)

       eTMAs             0              1              2              4
      using pen          4              6              1              3

    – Those who did 2 or more eTMAs used pens for figures and equations only

•   8 students were overall happy with the pens (5 Logitech)
•   6 would use the pen again (5 Logitech); 3 may use it (2 Logitech).
•   Most students had no problems with the installation (1 couldn‟t get
    pen to work, 1 couldn‟t save files)
Results of evaluation
•   no problems mentioned with producing single document (but did they?)
•   low accuracy of diagrams and difficulty with drawing lines with a ruler (both pens)
•   saves time over using equation editor and word processing package
•   would prefer to type/use word processor
•   would prefer to scan (Logitech)
•   it is difficult to erase / difficult to edit a downloaded page
•   mistakes in written work can be erased (Logitech)
•   process introduces mistakes
•   if more time available, better use of equipment could be made
•   pen is resource hungry (Logitech)
•   fast and easy to use
•   used for note taking (Logitech)
•   conversion to text not good (not recommended)
Future plans

•   Run project again in 2008, this time starting earlier (TMA02)
    – Allow students more time to familiarise themselves with the equipment

•   Are pens really a better option than using a scanner?
•   Canon CanoScan LIDE 25
     – write TMA normally
     – scan pages directly into pdf format
     – produce single file easily

•   More guidance on do‟s and don‟ts?
    – Guide students who can type to use them only for equations and figures?
    – More help with deleting?

•   More guidance on how to erase?

•   Jonathan Underwood who set up the project
•   24 S207 ALs who participated
•   Mike Watkins (Course Manager) and Tracey Woodcraft (Course Assistant)
•   Rukhsana Malik, Una Lawson and Asvin Savjani (Student Services)
•   Kevin Mayles (Centre Manager) and Diane Ford (Centre Assistant) of p CETL
•   Funding from p CETL

To top