Many of you will now be preparing to leave us as the academic year

Document Sample
Many of you will now be preparing to leave us as the academic year Powered By Docstoc
					                                                                                                                                               APPENDIX A: MODULE APPRAISAL QUESTIONNAIRES 2002/3


Standard                                       LSA                                        BIE                                       ENR                                       MCS, NRS, PBH
Q1 The content of the module reflected its     Q1 I was made aware of the importance      Q1 The module captured my interest        Q1 The aims and objectives of the         Q1 The material presented reflected the
aims and objectives                            of independent learning in this module     Q2 Subject matter was well directed to    module were made clear                    aims and objectives of the module
Q2 My workload on the module was               Q2 My workload on the module was           the aims of the module                    Q2 My workload on the module was          Q2 My workload on the module was
appropriate                                    appropriate                                Q3 The content was appropriate to my      appropriate                               manageable
Q3 I was made aware of the importance of       Q3 Deadlines set for the submission of     level of understanding                    Q3 The teaching sessions were clearly     Q3 My interest was stimulated during
independent learning in this module (non       work were appropriate                      Q4 Staff on the module were accessible    presented and well organised              the module
taught modules)                                Q4 Material presented reflected the aims   Q5 Practicals were worthwhile             Q4 The lecturer was approachable and      Q4 Lecturer(s) were punctual and
Q4 My interest was stimulated during the       and objectives of the module               Q6 Fieldwork was worthwhile               available to give help and guidance       reliable in their attendance
module                                         Q5 Information was presented in a clear    Q7 The module workload was about          Q5 The lecturer was knowledgeable         Q5 Materials and information were
Q5 Lecturer(s) were punctual and reliable in   and well organised way                     right                                     about the module content                  presented in a clear and well organised
their attendance (taught modules)              Q6 Lecturer(s) were approachable and       Q8 Feedback during the module was         Q6 The lecturer was enthusiastic about    way
Q6 Teaching methods were appropriate for       able to give help and advice               satisfactory                              the module subject                        Q6 In general, formal lectures were
the module                                     Q7 Teaching methods were appropriate       Q9 Assessment of the module was fair      Q7 The teaching sessions were             satisfactory
Q7 Materials and information was presented     for the module                             Q10 I was adequately informed of the      interesting and varied                    Q7 In general, small group sessions
in a clear and well organised way              Q8 Feedback on work submitted was          times and venues for this module          Q8 The support materials provided by      were satisfactory
Q8 Lecturer(s) were approachable and           returned within a reasonable time period   Q11 The complete module was               the lecturer (eg handout notes) were      Q8 A relationship was demonstrated
available to give guidance and support         Q9 The type of assessment was              delivered                                 helpful                                   between theory and practice
Q9 The type of assessment was appropriate      appropriate to the module                  Q12 The laboratories and equipment        Q9 Sufficient material for the module     Q9 Clinical skills teaching prepared me
for the module                                 Q10 Lecturer(s) made constructive and      were of an adequate standard              was available in the Learning Resource    for practice
Q10 Deadlines set for submission of work       helpful comments about submitted work      Q13 For this module, the library met my   Centre                                    Q10 Opportunities to practise skills in a
were appropriate                               Q11 Tutor support was adequate             needs                                     Q10 The accommodation was suitable        simulated environment were provided
Q11 Assessed work was returned within a        Q12 Sufficient material was available in   Q14 I would recommend this module to      for the type of activity                  Q11 Relevant and appropriate
reasonable time period (only those who         the Learning Resource Centre for the       other students                            Q11 The assessment tasks were             information was available at
have had work returned)                        module                                     Q15 Please make any further comments      appropriate to the module                 commencement of the module
Q12 Lecturer(s) made constructive and          Q13 Tutorial sessions were satisfactory    on the above, or on any other aspect of   Q12 The assessment requirements were      Q12 Lecturers were approachable and
helpful comments about assessed work           Q14 Seminar sessions were satisfactory     the module                                made clear                                able to give help and guidance
(only those who have had work returned)        Q15 Workshop sessions were                                                           Q13 Coursework was marked and             Q13 Administrative support staff were
Q13 The teaching accommodation was             satisfactory                                                                         returned in a reasonable time             available to give help and guidance
suitable                                       Q16 I would recommend this module to                                                 Q14 Feedback on coursework was            Q14 The teaching accommodation was
Q14 Sufficient material for the module was     other students                                                                       provided and was helpful                  suitable
available in the Learning Resource Centre      Q17 Please make any further comments                                                 Q15 The module is relevant to my          Q15 Learning resources were sufficient
Q15 Overall, formal lectures were              on the above, or on any other aspect of                                              programme                                 to support the module
satisfactory (taught modules)                  the module                                                                           Q16 I would recommend this module to      Q16 I would recommend this module to
Q16 Overall, small group sessions (eg                                                                                               other students                            other students
tutorials/seminars/workshop sessions) were                                                                                          Q17 Please make any further comments      Q17 Please make any further comments
satisfactory                                                                                                                        on the above, or on any other aspect of   on the above, or on any other aspect of
Q17 Please make any further comments on                                                                                             the module                                the module
the above, or on any other aspect of the
module
APPENDIX B: MODULE APPRAISAL CO-ORDINATORS (CURRENT AT SEPT 03)
Code      School                              Co-ordinator

BLW       FACULTY OF BUSINESS AND LAW
AFE       Accounting, Finance & Economics     Cathy Sindall
BSN       Business Information                Cathy Sindall
LAW       Law                                 Janet Hernandez
LNG       Languages                           Linda Archibald
MGT       Management                          Cathy Sindall

MAS       FACULTY OF MEDIA, ARTS AND SOCIAL SCIENCE
IDC       International Centre for Digital
          Content
LSA       Liverpool School of Art & Design    Dave Rees
MCA       Media, Critical & Creative Arts     Eileen McGovern
SSC       Social Science                      Liz Sperling

SCS       FACULTY OF SCIENCE
AST       Astrophysics Research Institute     Hugh Jones
BIE       Biological & Earth Sciences         Louise Higham
BML       Biomolecular Sciences
PHC       Pharmacy & Chemistry
PSY       Psychology                          Andy Tattersall
SPS       Sports & Exercise Science           Tim Cable

TAE       FACULTY OF TECHNOLOGY & ENVIRONMENT
BUE       Built Environment                    Alison Cotgrave
CMP       Computing & Mathematical Sciences Maria Morris
ENR       Engineering                          Ian Jenkinson
IUA       European Institute for Urban Affairs
LRD       Lairdside Maritime Centre
GNR       General Engineering Research
          Institute

ECL       FACULTY OF EDUCATION, COMMUNITY AND LEISURE
EDC       Education                           Pat King
OLF       Outdoors, Leisure & Food            Drew Li
PSD       PE, Sports & Dance                  Sue Hatfield

HEA       FACULTY OF HEALTH AND APPLIED SOCIAL SCIENCES
ASL       Applied Social Sciences             Janet Hernandez/Pauline Stitt
MCS       Midwifery & Child Studies           Rose Sargent
NRS       Nursing & Healthcare Studies        Pauline Stitt
PBH       Centre for Public Health            Rose Sargent
                                                             APPENDIX C: STUDENT FEEDBACK QUESTIONNAIRE 2003




          Liverpool John Moores University


                              STUDENT FEEDBACK 2003
The aim of this questionnaire is to obtain your opinions on aspects of your experience at JMU. The answers you give
are confidential and will be given serious consideration


Section 1: About you
Please use your mouse to select your responses from the drop-down lists




Please enter your full programme title in the box below (eg History BA (Hons))




Which level or programme year are you currently in?                                 Are you studying:

Undergraduates                               Postgraduates                          Full Time                  
Foundation Year                             Year One                              Full-time sandwich
                                                                                    
Level One/Year One                          Year Two                              Part-time                  
Level Two/Year Two                          Other                                 Block Release
                                                                                    
Level Three/Year Three                                                             Distance learning
                                                                                    
Other      

In which building/site do you have most of your
lectures/other forms of formal contact?
Please mark only one
Aquinas Building                      Joe H Makin Drama Centre     
Avril Robarts (Tithebarn Street)      John Foster Building         
Clarence Street                       Josephine Butler House       
Dean Walters                          Max Perutz Building          
Hahnemann Building                    2 Myrtle Street              
Henry Cotton Campus                   Peter Jost Centre            
68 Hope Street                        Redmond House                
IM Marsh                              St. Nicholas                 
James Parsons Building                Other                        
APPENDIX C: STUDENT FEEDBACK QUESTIONNAIRE 2003


Section 2: Your Programme of Study and your School
Please indicate how satisfied you are with the following:
1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral 4= Satisfied 5 = Very satisfied
If you cannot rate an item positively or negatively then mark ‘Neutral’
If you cannot answer a particular item, mark ‘Not applicable’ (N/A)
Admission and induction                                                                        1 2 345    N/A
The information you received about the programme before you applied                             
The arrangements for joining and starting your programme                                        
Enrolment at the start of each year of the programme                                            
Induction onto the programme                                                                    
Introduction to JMU’s IT facilities                                                             
Programme structure and content                                                                1 2 3 4 5 N/A
The clarity of the programme’s specification                                                    
The programme content                                                                           
The amount of choice offered within the programme                                               
The relevance of the programme to your needs                                                    
The opportunity for work-based learning (eg placements, clinical/teaching practice, visits)     
Teaching and learning                                                                          1 2 345    N/A
The delivery of the programme matches the description in the programme specification            
The teaching methods used on the programme                                                      
The availability of information to support your programme on CWIS/Blackboard                    
Your experience of work based learning (answer ONLY if experience is in the last 12             
months)
Guidance and support                                                                           1 2 345    N/A
The availability of staff to provide guidance and support (eg personal tutor, module leader,    
year tutor)
The academic guidance provided by staff                                                         
The individual (personal) support provided by staff                                             
Assessment                                                                                     1 2 345    N/A
The clarity of the assessment requirements                                                      
The return of coursework within a specified timescale                                           
Feedback on coursework (ie is it useful and informative?)                                       
The administration of exams (ie information, timetable, accommodation)                          
The notification of results by student transcripts                                             
                                                                                               
Management and organisation                                                                    1 2 345    N/A
The Student Handbook for your programme                                                         
The attention given to students' views within your School                                       
The learning environment                                                                       1 2 345    N/A
Teaching accommodation (eg lecture/seminar rooms)                                               
Specialist accommodation (eg science/language labs, sports/drama facilities )                   
Specialist equipment (eg for science, languages, sports, drama )                                
Specialist IT facilities/equipment (eg for art, media, architecture, engineering)               
APPENDIX C: STUDENT FEEDBACK QUESTIONNAIRE 2003
APPENDIX C: STUDENT FEEDBACK QUESTIONNAIRE 2003
Section 3: Learning Support Services
Please indicate how satisfied you are with the following:
1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral 4= Satisfied 5 = Very satisfied
If you cannot rate an item positively or negatively then mark ‘Neutral’
If you cannot answer a particular item, mark ‘Not applicable’ (N/A)
Learning Resource Centres (LRCs)                                                     1 2 345   N/A
The overall service provided by the LRCs                                             
                                                                                     
The helpfulness of staff                                                             
                                                                                     
Range of books                                                                       
                                                                                     
Range of journals                                                                    
                                                                                     
Availability of books                                                                
                                                                                     
Availability of journals                                                             
                                                                                     
Range of electronic information (eg CD Roms, on-line journals)                       
                                                                                     
Access to computers/IT facilities in the LRCs                                        
                                                                                     
Printing facilities in the LRCs                                                      
                                                                                     
Photocopying facilities in the LRCs                                                  
                                                                                     
Provision of space for silent study                                                  
                                                                                     
Provision of space for group study                                                   
                                                                                     
Other Learning Support Services                                                      1 2 345   N/A
Reliability of the computer network                                                  
                                                                                     
Appropriateness of core software                                                     
                                                                                     
Appropriateness of subject specific software                                         
                                                                                     
Access to computers/IT facilities in Faculty IT Centres                              
                                                                                     
                                                             APPENDIX C: STUDENT FEEDBACK QUESTIONNAIRE 2003


Section 4: The University as a Whole
Please indicate how satisfied you are with the following:
1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral 4 = Satisfied 5 = Very satisfied
If you cannot rate an item positively or negatively then mark ‘Neutral’
If you cannot answer a particular item, mark ‘Not applicable’ (N/A)
Central Services                                                                         1 2 345         N/A
The Campus Wide Information Service (CWIS)                                                
JMU’s Student Residential Accommodation Services                                          
JMU’s Catering Facilities/Services (NOT Students’ Union facilities eg the Haigh)          
JMU’s Campus Centres (based at Aquinas Building, Byrom Street & IM Marsh)                 
JMU’s Careers Advisory Service                                                            
JMU's Student Welfare Services (eg student loans, general advice)                         
JMU's Student Counselling Services                                                        
JMU's Student Medical Service                                                             
JMU's Sport and Recreation Facilities                                                     
JMU’s Nursery Provision                                                                   
JMU’s Shuttle Bus Service                                                                 
Liverpool Students’ Union (LSU) Facilities (eg LSU catering, bars, shops)                 
Services provided by LSU (eg information & advice, WorkBank, leisure, events)             


Section 5: Overall Opinions
Please indicate the extent to which the following have matched your expectations:
1 = Far below expectations 2 = Below expectations 3 = Matched expectations
4= Above expectations 5 = Far above expectations
Your expectations                                                                        1 2 345
So far, has your programme of study matched your expectations                            
So far, has JMU matched your expectations?                                               
So far, has studying in Liverpool matched your expectations?                             

Section 6: Further Comments
Please make any further comments about your experiences at JMU




Thank you for completing this questionnaire. Please click on the button below to submit your responses
APPENDIX D: POSTGRADUATE RESEARCH STUDENT QUESTIONNAIRE 2001/2




         Liverpool John Moores University


                     POSTGRADUATE RESEARCH
                     STUDENT FEEDBACK 2001/2
Use your mouse to select your responses from the options provided

Some questions will not be relevant to all students Links have therefore been included which will enable you to miss out
questions which do not apply

When you reach the end of the questionnaire, please click the ‘Submit Survey’ button to submit your responses


Section 1 : Before you chose JMU
Q1. Have you been a postgraduate research student at JMU for more than two years?
If ‘Yes’ is selected, you will be redirected to Section 3 of the questionnaire
                                              Yes                  No   

Q2. Which of the following interested you in the                    Q4. What was/were your main reason(s) for
postgraduate research opportunities at JMU?                         choosing JMU?
(mark every response which applies)                                 (mark every response which applies)
                             postgraduate prospectus                                       reputation of the University 
                                 other JMU literature                               research interests of academic staff 
               advertisement in newspaper/periodical                                                     JMU’s location 
                                       careers library                                          reputation of supervisor 
                                    World Wide Web                                                availability of funding 
                           graduate recruitment fairs                                              project based at JMU 
                                       word of mouth                                          other (please state below) 
                           other (please state below) 


Q3. If you marked ‘Other’, please give further                      Q5. If you marked ‘Other’, please give further
details below                                                       details below



Q6. Before accepting a research place at JMU, how satisfied were you with the information
you were given?
1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral 4= Satisfied 5 = Very satisfied                         1 2 3 4 5

                                                                                                       
Q7. What further information, if any, would you have found helpful before accepting a place?
                                     APPENDIX D: POSTGRADUATE RESEARCH STUDENT QUESTIONNAIRE 2001/2


Section 2 : Admission and Induction
Q8. Which of the following were made clear to you on admission to your Home School?
(mark every response which applies)
                                   enrolment procedures                               health and safety information 
                               your budget arrangements                                       teaching opportunities 
                   supervision/monitoring arrangements              the facilities/services available to you at JMU 
                        registration of research proposal                           equal opportunities information 
                          the regulations for researchers                             Postgraduate Student Charter 
                                     the Code of Practice            procedures for complaints/grievance appeals 

Q9. Please indicate how SATISFIED you are with the following:
1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral 4= Satisfied 5 = Very satisfied
If you did not attend the induction sessions, please select ‘Not Applicable                           N/A   1 2 3 4 5
                                  the information you were given on admission to your Home School
                                                                                                           
                                                                                                       
                                                                           the admissions process
                                                                                                           
                                                                                                       
                                                               the Home School induction session
                                                                                                           
                                                                                                       
                                                            the Graduate School induction session
                                                                                                           
                                                                                                       
                                              admission to the International School (if applicable)
                                                                                                           
                                                                                                       
Q10. Please make any further comments about admission and induction. We would welcome your
suggestions for how improvements could be made




Section 3 : Residential Accommodation
Q11. Did you live in Merseyside before starting your research?
If ‘Yes’ is selected, you will be redirected to Section 4
                                             Yes                  No    
Q12. What type of                        Q13. Did you have difficulty            Q14. How satisfied are you with your
accommodation do you live                in finding accommodation?               current accommodation in
in?                                                                              Liverpool/the local area?
              self-catering flat                                yes            1 = Very dissatisfied 2 = Dissatisfied
                       lodgings                                  no            3 = Neutral 4 = Satisfied 5 = Very
             family house/flat                                                  satisfied
             shared house/flat                                                                             1 2 3 4 5
  university halls of residence 
                                                                                                        
                         other 


Q15. Please make any further comments on the advice and information you received about
accommodation
APPENDIX D: POSTGRADUATE RESEARCH STUDENT QUESTIONNAIRE 2001/2
                                  APPENDIX D: POSTGRADUATE RESEARCH STUDENT QUESTIONNAIRE 2001/2


Section 4 : Supervision
Q16. Has your supervisor(s) provided a clear schedule of monitoring stages (eg                                  yes 
annual reports, conversion documents etc) through to your final oral examination?                                no 

Q17. Do you currently have a clear agreement with your supervisor(s) regarding                                  yes 
the frequency of formal meetings to discuss your progress?                                                       no 

In this section you are asked to provide separate ratings for your Director of Studies (ie main supervisor) and any
other supervisor(s) (i.e. second, third supervisors)

Q18. On average, how many times a month do                Q19. How satisfied are you with the frequency of these
you see your supervisor(s) for formal progress            formal meetings?
meetings?                                                 1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral 4 =
                                                          Satisfied 5 = Very satisfied
                       Director of   Other
                       Studies       supervisors          Director of Studies          Other supervisors
      less than once                                     1 2 3 4 5                    1 2 3 4 5
                                    
       once or twice                                                               
   more than twice                  

Q20. On average, how many times a month do                Q21. How satisfied are you with the availability of your
you see your supervisor(s) for informal progress          supervisor(s) for informal meetings?
meetings?                                                 1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral 4 =
                                                          Satisfied 5 = Very satisfied
                       Director of   Other
                       Studies       supervisors          Director of Studies          Other supervisors
           every day                                     1 2 3 4 5                    1 2 3 4 5
     several times a
                                    
                                                                                   
               week
        once a week                 
    less than once a
               week                 
    less than once a
              month                 
APPENDIX D: POSTGRADUATE RESEARCH STUDENT QUESTIONNAIRE 2001/2

Q22. How satisfied are you with your Director of Studies in the following areas?
1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral 4= Satisfied 5 = Very satisfied
                                                                    Director of Studies       Other supervisors
                                                                     N/A   1 2 3 4 5           N/A    1 2 3 4 5
                           ease with which s/he can be contacted
                                                                                              
                            giving guidance on literature sources
                                                                                              
                         helping you to design your plan of work
                                                                                              
                          giving guidance on research techniques
                                                                                              
                                        monitoring your progress
                                                                                              
               giving guidance on the standard of work expected
                                                                                              
                giving prompt feedback on work as it is produced
                                                                                              
                                          usefulness of feedback
                                                                                              
                          giving guidance when things go wrong
                                                                                              
                                        continuity of supervision
                                                                                              
                                          ability to motivate you
                                                                                              
 extent to which s/he is sympathetic and supportive of your needs
                                                                                              
Q23. How satisfied are you with your other supervisors in the following areas?

1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral 4= Satisfied 5 = Very satisfied
                                                                    Director of Studies       Other supervisors
                                                                     N/A   1 2 3 4 5           N/A    1 2 3 4 5
                           ease with which s/he can be contacted
                                                                                              
                            giving guidance on literature sources
                                                                                              
                         helping you to design your plan of work
                                                                                              
                          giving guidance on research techniques
                                                                                              
                                        monitoring your progress
                                                                                              
               giving guidance on the standard of work expected
                                                                                              
                giving prompt feedback on work as it is produced
                                                                                              
                                          usefulness of feedback
                                                                                              
                          giving guidance when things go wrong
                                                                                              
                                        continuity of supervision
                                                                                              
                                          ability to motivate you
                                                                                              
 extent to which s/he is sympathetic and supportive of your needs
                                                                                              
Q24. How satisfied are you with the consistency of advice between your
supervisors?
1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral 4= Satisfied 5 = Very satisfied        1 2 3 4 5
                                                                                          
Q25. Please make any further comments about the supervision you receive
APPENDIX D: POSTGRADUATE RESEARCH STUDENT QUESTIONNAIRE 2001/2
APPENDIX D: POSTGRADUATE RESEARCH STUDENT QUESTIONNAIRE 2001/2


Section 5 : Research Methods and Research Seminars
Q26. Have you undertaken the generic Research           Q27. How satisfied are you with the Research
Methods programme offered by the Graduate               Methods Programme?
School?
If ‘No’ or ‘Not aware of it’ is selected, you will be   1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral
redirected to questions about research methods          4 = Satisfied 5 = Very satisfied
training within your Home School
                                                                                        N/A   1 2 3 4 5
       Yes          No          Not aware of it                                    
Q28. How helpful do you feel the                        Q29. Do you feel that the amount of time you
Programme has been to your research?                    have been required to spend on the Programme
                                                        has been:
                        not at all helpful                                                   too short 
                           fairly helpful                                                 about right 
                            very helpful                                                      too long 

Q30. Overall, are you satisfied with formal             Q31. Have you attended any Home School
research training (including research seminars)         graduate research seminars?
within your Home School?
                 yes                                                        yes 
                  no                                                         no 
 N/A - there is none 
     not aware of it 

Q32. How satisfied are you with Home School                   Q33. How helpful did you find the
graduate research seminars?                                   seminars?
1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral                                  not at all helpful 
4 = Satisfied 5 = Very satisfied                                                        quite helpful 
                  1 2 3 4 5                                                              very helpful 

                 
Q34. Please make any further comments about formal training in research methods at JMU. Please
include suggestions about what would be helpful to you in a research methods programme
                                     APPENDIX D: POSTGRADUATE RESEARCH STUDENT QUESTIONNAIRE 2001/2


Section 6 : Personal Development
Q35. How satisfied are you with the                                    Q36. How satisfied are you with the extent
opportunities you have to develop the following                        to which you are being prepared for future
skills:                                                                employment?
1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral                     1 = Very dissatisfied 2 = Dissatisfied 3 =
4= Satisfied 5 = Very satisfied                                        Neutral 4= Satisfied 5 = Very satisfied
                                     1 2 3 4 5                                                      1 2 3 4 5
          communication skills
                                                                                             
                                     
                   writing skills
                                     
                                     
                  research skills
                                     
                                     
         critical appraisal skills
                                     
                                     
Q37. Please make any further comment about the opportunities you have for personal development. We
welcome your suggestions for initiatives that you feel would be useful for your personal development.




Section 7 : Postgraduate Society and Networking
Q38. Are you aware of:                                       Q39. What FIVE issues would you like to see the
                                                             Postgraduate Society address over the next year?
                                      Yes No
the JMU Postgraduate (PG)               
Society?
 the Postgraduate Society website         
 the postgraduate honorary fellow         
                          lectures


Q40. Do you feel that you have sufficient interaction with other researchers:       Yes     No
                                                              in your home school          
                                                                       within JMU          
                                                                      outside JMU          

Q41. We welcome any suggestions you may have for social events you would be interested in organising or attending




Q42. Please make any further comments about the Postgraduate Society and networking. Your suggestions for
improvements are welcomed
APPENDIX D: POSTGRADUATE RESEARCH STUDENT QUESTIONNAIRE 2001/2


Section 8 : Teaching and Demonstrating at JMU
Q43. Are you/were you a full-time lecturer at JMU?
If ‘Yes’ is selected, you will be redirected to Section 9
                                                Yes         No   
Q44. Have you provided teaching and/or helped                     Q45. What type of work was this?
with demonstrations and/or marking this year?                     (mark every response which applies)

If ‘No’ is selected, you will be redirected to the end                                       marking 
of this section                                                                   tutorials/seminars 
                                         yes (paid)                                        lecturing 
                                      yes (unpaid)                        laboratory demonstrating 
                                                 no                           studio demonstrating 

Q46. How many hours, including                                    Q47. To what extent did
preparation and marking, did this                                 this inhibit your work on
teaching/demonstrating take in a typical                          your own research?
week?
                   less than two hours                                          not at all 
            between two and six hours                                              a little 
                   more than six hours                                       considerably 
            occasional, with no pattern 

Q48. Did you receive any training before you                      Q49. If you did not receive any
helped with demonstrations or provided                            training, do you feel training would
teaching?                                                         have been beneficial?
If ‘Yes’ is selected, you will be redirected to the
end of this section
                                                 yes 
                                                  no                                            yes 
                                        not required                                             no 

Q50. Please make any further comments about teaching and demonstrating at JMU
                                   APPENDIX D: POSTGRADUATE RESEARCH STUDENT QUESTIONNAIRE 2001/2


Section 9 : Home School/Research Centre facilities and services
Q51. Please indicate how SATISFIED you are with access to the following items within your Home School and
(Q52) how IMPORTANT they are to you.
1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral 4= Satisfied 5 = Very satisfied
1 = Very unimportant 2 = Unimportant 3 = Neutral 4= Important 5 = Very important
                                                                    How satisfied       How important
                                                                      N/A     1 2 3 4 5            N/A   1 2 3 4 5
                                                       your office
                                                                                                  
                                              desk/storage space
                                                                                                  
                                                        telephone
                                                                                                  
                                                        computer
                                                                                                  
                   computer network (including access to email)
                                                                                                  
                                      tea/coffee making facilities
                                                                                                  
                                                 laboratory space
                                                                                                  
                           specialist health and safety equipment
                                                                                                  
                                                     common room
                                                                                                  
                                                  postal facilities
                                                                                                  
                                                     photocopying
                                                                                                  
                                                          printing
                                                                                                  
Q53. Do you have:                                                            Q54. If you marked ‘Other’, please give
                                                                             further details below
                        your own office 
       office shared with other students 
           office shared with other staff 
                               no office 
                                   other 

Q55. Who manages your research budget?                                      Q56. If you marked ‘Other’, please give
                                                                            further details below

                                               you 
                               your supervisor(s) 
                                director of school 
                     school research co-ordinator 
                                     outside body 
                                    not applicable 
                                             other 

Q57. How satisfied are you with the                                         Q58. Please make any further comments
arrangements for the management of your                                     about the facilities and services provided by
budget?                                                                     your Home School/Campus Office
1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral
4 = Satisfied 5 = Very satisfied
                  1 2 3 4 5
                
APPENDIX D: POSTGRADUATE RESEARCH STUDENT QUESTIONNAIRE 2001/2


Section 10 : University Facilities and Services
Q59. Please indicate how SATISFIED you are with the following aspects of the Learning Resource Centres and
(Q60.) how IMPORTANT they are to you.
1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral 4= Satisfied 5 = Very satisfied
1 = Very unimportant 2 = Unimportant 3 = Neutral 4= Important 5 = Very important
                                                                    How satisfied      How important
                                                                       N/A   1 2 3 4 5   N/A   1 2 3 4 5
                        the overall service provided by the LRCs
                                                                                        
                                           the helpfulness of staff
                                                                                        
                                                    range of books
                                                                                        
                                                  range of journals
                                                                                        
range of electronic information (e.g. on-line journals, CD ROMs)
                                                                                        
                                              availability of books
                                                                                        
                                            availability of journals
                                                                                        
                    access to computers/IT facilities in the LRCs
                                                                                        
                                     the inter-library loans service
                                                                                        
Q61. Please indicate how SATISFIED you are with the following at JMU and (Q62.) how IMPORTANT they are
to you.
1 = Very dissatisfied 2 = Dissatisfied 3 = Neutral 4= Satisfied 5 = Very satisfied
1 = Very unimportant 2 = Unimportant 3 = Neutral 4= Important 5 = Very important
                                                                    How satisfied    How important
                                                                       N/A   1 2 3 4 5   N/A   1 2 3 4 5
                                            JMU’s campus centres
                                                                                        
                                        Careers Advisory Service
                                                                                        
        JMU’s catering facilities/services (NOT Students’ Union
                                         facilities, e.g. the Haigh)
                                                                                        
                                                  nursery provision
                                                                                        
                                JMU's student counselling service
                                                                                        
                                   JMU's student medical service
                                                                                        
                                    JMU's student welfare service
                                                                                        
                                      equal opportunities practice
                                                                                        
                                environmental policy and practice
                                                                                        
                                  Institutional Student Handbook
                                                                                        
                                     Postgraduate Student Charter
                                                                                        
                 The Campus Wide Information System (CWIS)
                                                                                        
                              JMU’s sport and recreation services
                                                                                        
   Liverpool Students’ Union (LSU) facilities (eg LSU catering,
                                                     bars, shops)
                                                                                        
  Services provided by LSU (eg information & advice, Unitemp,
                                                 leisure, events)
                                                                                        
                                APPENDIX D: POSTGRADUATE RESEARCH STUDENT QUESTIONNAIRE 2001/2


Q63. Please make any further comments about JMU’s facilities and services




Section 11 : Overall Opinions
Q64. So far, have the following matched your expectations?
1 = Far below expectations 2 = Below expectations 3 = Matched expectations
4= Above expectations 5 = Far above expectations                                          1 2 3 4 5
                                                                  your research project
                                                                                          
                                                                                  JMU
                                                                                          
                                                                  studying in Liverpool
                                                                                          

Q65. This survey is a useful way for JMU to measure postgraduate research
student opinion
1 = Strongly disagree 2 = Disagree 3 = Neutral 4= Agree 5 = Strongly agree                1 2 3 4 5
                                                                                          
Q66. Do you think this survey should be conducted every year?

                                     Yes                   No   
Section 12 : About You                          The information in this section will be used to analyse the data
                                                relating to various groups of students. Your anonymity is
                                                strictly assured.

Q67. What is your age?        Q68. Are you:            Q69. Were you an             Q70. Is English
                                                       undergraduate at             your first language?
                21 - 24             female           JMU?                                             yes 
                25 - 29               male                      yes                                   no 
                30 - 34                                           no 
                   35+ 

Q71. Are you:                              Q72. Please indicate your ethnic background

              Home (UK) Student               Black African              British Asian                White Irish 
         European Union Student                 Black Asian                    Chinese      White Other European 
  International Student (Non-EU)               Black British                    Indian               White Other 
                                             Black Caribbean                  Pakistani                mixed race 
                                                 Black Other               Asian Other                      Other 
                                                 Bangladeshi                 White UK 

Q73. If you marked ‘Other’, please indicate your ethnic background below
APPENDIX D: POSTGRADUATE RESEARCH STUDENT QUESTIONNAIRE 2001/2




Q74. Do you have a health                      Q75. What is your target degree award?
condition or disability which has
existed (or is expected to exist)                                                     MPhil 
for over 12 months?                                                                    PhD 
                                yes                                                  Other 
                                 no 

Q76. Which is your Home School?                Q77. Are you enrolled:               Q78. How long have you been a
                                                                                    graduate research student at JMU?
        ART                   ENG                             full-time                         less than one year 
        BES                   HHS                             part-time                    between one and two years
                                                                                                                      
        BLT                   INT                            writing up               between two and three years 
       BMS                   LSW                                                       between three and four years 
        BUS                  MCC                                                               more than four years 
       CMS                   PAC 
       ECSS 

Q79. Who pays your fees?                                    Q80. If you marked ‘Other’, please give
                                                            further details below
                               employer 
                        JMU (employer) 
                      JMU (studentship) 
funded as part of a research programme 
           special government funding 
                                yourself 
                                   other 




Q81. Are you a salaried member of JMU staff (ie                               Q82. Are you:
not including part-time lecturing or demonstrating)?
If ‘No’ is selected, you will be redirected to section 13                                      a lecturer/senior lecturer 
                                                                                          a research assistant/associate 
                                                   yes                                              a teaching assistant 
                                                    no                                       an administrator/manager 
                                                                                                                    other 

                                                                              Q83. If you marked ‘Other’, please give
                                                                              further details below
                                 APPENDIX D: POSTGRADUATE RESEARCH STUDENT QUESTIONNAIRE 2001/2


Section 13 : Further Comments
Q84. We welcome any further comments you would like to make on issues raised in this questionnaire or on other
aspects of your experience as a student at JMU




Thank you for participating in this survey. Please now click on the ‘Submit Survey’ button to submit your
responses
Student Feedback Survey
                               No. of responses   % response

1994                           7,177              44%
1995                           7,692              46%
1996                           7,370              48%
1997                           7,680              49%
1998                           8,245              49%
1999                           8422               47.6%
2000                           8472               48.5%
2001                           7403               45.8%
2002 (electronic)              3221               20.1%
2003 (electronic)              3438               18.5%


Module Appraisal
                               No. of responses   % response

1996/7 Sem. 1                  10,663
1996/7 Sem. 2                  4,829

1997/8 Sem. 1                  14,859             43.6%
1997/8 Sem. 2                  8,050              30.4%
1998/9 Sem. 1                  15,264             43.8%
1998/9 Sem. 2                  13,728             36.1%
1999/0 Sem. 1                  19,129             28.4%
1999/0 Sem. 2                  17,294             38.8%
2000/1 Sem. 1                  21,165             45.9%
2000/1 Sem. 2                  20,035             37.9%
2001/2 Sem. 1 (paper based +   22,522             45.0%
pilot electronic in CMS)
2001/2 Sem. 2 (electronic)     8,074              12.8%
2002/3 Sem. 1 (electronic)     11,628             22.5%
2002/3 Sem. 2 (electronic)     10,501             17.8%


Postgraduate Research Student Survey

                               No. of responses   % response

1997                           171                49%
1999                           138                33%
2002                           139                37%
                                           APPENDIX F: COMMENTS RECEIVED BY EMAIL, 2002-3




The comments below were received by QUS from staff and students (mostly by email)
between November 2001 and June 2003. All comments were responded to by QUS, either
by phone or email.



From: Dave Harris, Student (edited email)
Date: 29 November 2001

There are other questions which should be asked in the module appraisal questionnaire (eg
condition of other parts of the building, value for money, behaviours of fellow students,
registration process and general administration etc), which give a better idea of student life

Edited reply from Christine Chamberlain:
Some of the questions you recommend in your email, ie learning environment, registration
and general administration, are included in the Institutional Student Feedback Survey.
__________________________________________________________________________

From: Helen Lowe (Student, BUE)
Date: 30 May 2002

I am afraid I think this is a highly unsatisfactory method of appraising modules. As a part-time
postgraduate student, who works full-time, and lives some distance from Liverpool, I am only
in the university one day per week. This day is generally filled with lectures from 9am till 5pm
and I have very little time to check my email or use library facilities. I have only just now been
able to check my email for almost a month, consequently the module questionnaire I have
been asked to complete is now closed. There are many students on my course who have
never used their university email or library facilities as it is easier for them to do so nearer to
their home. The only break we have during the day is lunchtime, if we have to hand an essay
in or see a tutor this takes up a large proportion of the time, unfortunately email is a low
priority.

I can understand the reasons for conducting module appraisals in this way, but I’m not sure
that you will get a worthwhile response for certain modules in particular. I am sorry to be so
negative and I hope you find these comments useful.

From: Christine Chamberlain (edited email)
Date: 25 June 2002

I understand that you don’t get much of a chance to access your JMU email account and that
this is a problem for students on certain types of programmes and certain modes of study (ie
part-time). It is very difficult to find a method of implementation that will suit all students. That
most students are full-time and can be urged to complete on-line questionnaires by their
lecturers who they see regularly is of no help to part-timers and distance learners.
__________________________________________________________________________

From: Tony Harvey (LAW)
Date: 29 October 2002
Subject: Module Appraisal, 2002/3 Semester 1

It beggars belief that the University has chosen to continue Module Appraisal using the
electronic medium. This process is so obviously flawed. Those who have any real contact
with students will know that few, if any, will complete the form where they are satisfied with
the quality of teaching.

Those minded to carry out the exercise will be largely unsatisfied or will have specific
complaints which they wish to be aired. This feedback is important but must be placed in a
proper context.
APPENDIX F: COMMENTS RECEIVED BY EMAIL, 2002-3



Feedback which highlights complaints but gives NO real indication of good or excellent
teaching quality is a largely useless tool for Module Leaders and those involved with the
planning of teaching and learning. Are we to assume that a nil response indicates that a
student is satisfied and, if so, at what level of satisfaction?

Staff are already demoralised enough in our mass higher education system without having to
put up with skewed statistical feedback which finds its way into University Quality Documents
such as the PSAAD.

Can we please think again?

From: Earl Selkridge (LAW)
Date: 29 October 2002

I fully endorse the views expressed by Dr Tony Harvey on this matter. While it might be
fashionable, these days, to do things 'electronically' (including teaching), I believe there is a
limit to the rationale of this development. And that limit arises where any particular exercise
has been proven to be unworkable or flawed.

Are we not in danger of designing a 'whip for our own backs', as it were, if we were to persist
with this exercise which so obviously produces distorted and, indeed, negative/adverse
statistical results IN A QAA PERIOD?

We need to rethink/revisit this matter.

From: Di Chappelle (LAW)
Sent: 30 October 2002

I agree wholeheartedly. Whilst e-mail etc are good for some things, I don't think module
appraisal forms is one of them!

From: Anne Miller
Date: 11 November 2002

Thank you for your comments which have been passed to me. We have noted them and this
year we are doing several things:

- attempting to secure a higher response rate using publicity and a cash incentive
- reviewing how module appraisal has operated since going electronic
- reviewing the nature of student surveys that operate within the University in the context of
HEFCE 02/15 information requirements.
__________________________________________________________________________

From: David Mottram (PHC)
Date 1 April 2003

I was sorry to see that the University is persisting with electronic-only versions of the
feedback. As the programme leader of a part-time postgraduate course. I noted, with alarm,
that we only received one reply from our students last year, compared with the high return
rate with paper versions.

Most of our student delivery is off-site and students do not log into the University system
frequently. It was therefore no surprise that they did not respond. I am sure that we will have a
similar experience this year.

It is just as well that we operate an extensive school-based feedback system for all elements
of our course. However, this does not cover the university-wide aspects of the student
experience.
__________________________________________________________________________
                                          APPENDIX F: COMMENTS RECEIVED BY EMAIL, 2002-3



From: Liz Sperling (SSC)
Date: 1 April 2003

I am just thinking through the possibilities for integrating module appraisal into tutorial
modules? I have not broached this topic with colleagues yet as I want to get some aspects
straight before doing so. Thus, I am writing for some advice/information from you.

1. In Social Science, the Tutorial modules incorporate some element of IT training. As more
and more students are now coming here with basic IT skills, this is likely to move more into
showing them around and familiarising them with Blackboard. Would it be possible for the on-
line module appraisals to be put into the Blackboard module areas? If so, the Tutorial
Modules could 'assess' students by requiring them to complete the module appraisal form.
This will not be a mark-accumulating assessment but a formative test if you like.

If this works for Tutorial modules, there is no reason why such an exercise should not be
incorporated into all modules!

2. A problem associated with this would be anonymity. Is there a way that your unit could
inform staff of who has/has not completed their forms. We do not need to know names of
completers, just a list of those who had not completed the on-line forms would do if possible.

We would also need this information before marking of assessments occurs at the end of
term. Is this deadline too tight?

If we extended this system to other modules, would your staff be able to cope with vast
numbers of modules requiring this information before marking starts?

3. Staff, including myself, may find this type of solution to the problem of low response rates
to the on-line module appraisals too imposing. We all know that students who do not attend
classes (and there are many) are able to fill in the forms, students with a 'grudge' can use
forms politically etc etc. Also, if module appraisals are consistently poor, there may well be a
problem with delivery, but this may equally be a problem with student commitment to the
particular module or disliking the module tutor. Is there a way of assuring staff of the value of
module appraisal? If I were to suggest that the School incorporates appraisal into Tutorial,
and other, modules, how would I go about it as a positive move and one not designed to
batter academics even further!

At the moment, I am really just musing on this as a way to improve module appraisal for an
essential core module that all Soc Sci students take, while also making it relevant to students.
I think that one problem with the appraisal system is that students are inundated with
questionnaires and don't see any 'relevance' or connection to their studies. After all, why fill in
a form that may help next years' students and not them?! By linking the process to courses
directly, some 'relevance' may be evident and that could be translated to other modules.

From: Christine Chamberlain (edited email)
Date: 3 April 2003

1. Tutorial modules (ie non taught modules) are able to take part in module appraisal as the
questions in the standard module questionnaire (used by SSC) are applicable to both taught
and non taught modules.

2. The survey software we use is owned by a company called Netweaver which also hosts the
website where the responses are collected into a dataset. It would therefore not be
compatible with ‘Blackboard’. As I understand, Blackboard has its own means of doing
assessments which could be set up as questionnaires.

The way in which we conduct module appraisal could not be handled through Blackboard.
The software we use is designed so that each student receives a questionnaire for each
module they are studying on in a particular semester. The data comes from SMIS and the
APPENDIX F: COMMENTS RECEIVED BY EMAIL, 2002-3

process is designed to be completely anonymous (ie students’ usernames don’t appear in the
dataset which is imported onto the JMU server). A ‘participants list’ is generated separately
so we can see who has responded but this is only used for the purposes of choosing at
random students who will win the prizes offered.

3. The profile of module appraisal has been raised considerably since it has gone ‘on-line’
and the response rate did increase by 10% last semester over the previous one. It is
acknowledged that response rates could be improved and only by doing so will staff will place
greater value on the results and the process. We have increased our cash prize incentives
six-fold this semester in a further attempt to increase response rates. This semester we have
also sent info. on acetates to all programme leaders so they can promote both module
appraisal and the student feedback survey among their students. Module appraisal results
are also now available in advance of MABs.

4. I agree that there is a danger of students suffering from ‘questionnaire overload’. The
surveys currently facilitated by this office are module appraisal (each semester), the student
feedback survey (annually) and the postgraduate researcher survey (every 2 years). I am
aware that there are other surveys aimed at students conducted by other central departments
and by individual programmes and modules. Unfortunately there is currently no regulation of
these.

From: Liz Sperling
Date: 3 April 2003

Tutorial modules are taught and assessed - at least in this School. They are designed as
skills modules (hence the idea that Module Appraisal could be used to test some IT skills
maybe), and also to inform students of the university's systems and processes.

The info you've sent will help us to consider whether incorporating module appraisal into
modules is worthwhile and a 'good' idea, as well as how we could go about it if we decide to.
__________________________________________________________________________

From: Dave Rees (ART)
Date: 3 April 2003

The School's Quality Committee considered the recent paper on "Changes in the
Administration of Student Surveys". They have asked me to forward their comments about
electronic module appraisal to you - or whoever the paper originated from.

1)"Changes in the Administration of Student Surveys" was felt not to address the fundamental
problem that the School has no access to networked PCs for students in some of our
buildings.

2) The impression created by offering prize incentives might be encouraging students to
complete the questionnaires for purely mercenary reasons - to get the prize - without having
any concern about making a considered judgement about the modules they took.

Perhaps we should try offering a prize to the student who manages to tick highest number of
positive boxes - i.e. indicating 100% satisfaction (or more) with all aspects of all their
modules.
__________________________________________________________________________

From: Susan Russell (NRS)
Date : 4 April 2003

I am contacting you to express my disappointment at the timing of the student feedback
survey this year as almost none of my students will be able to take part.

The awaited acetate arrived this morning (Friday 4 March) which is one week into the 2-week
period for completion of the appraisal. I have no contact with any of my group next week (w/c
                                          APPENDIX F: COMMENTS RECEIVED BY EMAIL, 2002-3

7 April) which is the second week for completion of the appraisal, as they have speakers in
and I am on holiday. I don’t feel that with such small notice that it is fair to expect another
speaker to discuss this with my students; also I won’t even be physically here to discuss it
with the speakers myself. Therefore I will have no feedback from the appraisal.

The feedback survey can be carried out over 4 weeks, however, as I have just stated, I will
not see my groups next week (7 April), all groups are off over Easter (14 and 21 April), and
the groups who are in w/c 28 April have speakers with them; also not all of the groups are
back in until the following week.

Unfortunately the best week (and it is only one week) to deliver this information to all of my
students (and there are a sizeable number, about 100) is w/c 5 May. I feel that I will miss out
on valuable information/feedback from the in comparison to the previous paper surveys.
Surely this acetate could have been sent out at least one week prior to filling in the surveys,
not a week after they have started? Unless my groups can have a special arrangement made
then I can virtually guarantee that there will be no feedback from them.
__________________________________________________________________________

From: Jyoti Vithlani (NRS)
Date: 24 April 2003

I have a group of students who are mature, come for only one semester and refuse to use
computers. This has made it very difficult for me to compile the self assessment reports in
July as I have no information apart from nominal group evaluations to relate to.

I have a 2nd group of students, who are also mature, use the computers but only at work and
home. They rarely log in at the uni. and as a result I do not get any feedback in spite of the
reminders I give them.

Whilst I strongly support an electronic Survey, I have concerns that it may not work with all
groups

From: Anne Miller
Date: 24 April 2003

I appreciate there can be resistance from some groups of students and so sympathise.
However, the decision to move to an electronic form of survey was taken 2 years ago by the
Strategic Management Group.

In addition to the arrangements for this interim year, we are reviewing the operation of the
student survey for the future. It would be helpful to know the title(s) of the programmes that
are affected in this way and the numbers of students who are involved.
__________________________________________________________________________

Comment on module appraisal questionnaire received from student, April 2003

Questions like: ‘Was the lecturer able to suggest solutions to problems?’ and ‘Were meetings
with your supervisor helpful?’ would be useful.
__________________________________________________________________________

From: Terry Craven, (Student, LAW)
Date: April 2003

Perhaps an idea to include a question on attendance levels at lectures and tutorials in the
module appraisal questionnaire. This would give an idea on how students view non
attendance etc.
__________________________________________________________________________
APPENDIX F: COMMENTS RECEIVED BY EMAIL, 2002-3



From: Norman Partington (CIS)
Date: 8 May 2003

Now that the student survey is (more or less?) done with, we ought to have a post-mortem on
the delivery process.

As of last Friday, my information was that 3280 students had (apparently) completed the
survey. This is 17.6% of the 16000 names we gave to Netweaver.

Despite all our efforts, this is still a pretty poor response, although it should be statistically
significant. Publishing the survey over Easter when students tend to be away was probably
not the best means of ensuring a good response. 5-6000 students logged in over this period.

At any given time, it is actually very difficult to know what the active population of students is.
We have 30,000 registered computer accounts but probably only 13,000 of these are active
over the year. It was very difficult to give Netweaver a 'list' of usernames to target, since
essentially this is a moving target. If the survey had been internally delivered, this problem
would not arise. Some students received a message exhorting them to complete the survey,
then couldn't.

Because the process was externally delivered, the initial email to 16000 students swamped
our email servers - each incoming email is virus checked. So we had backlogged email
institution-wide at that time. Internal email would not have produced this - it would be 1 email
to all-the-students.

A lot of time & effort went into setting this up and it appeared that Netweaver didn't
understand the sheer scale of the problem. Provided the analysis of the survey can be dealt
with OK, it must be preferable to deal with the survey internally next year (via Perception).

From: Christine Chamberlain
Date: 12 May

The raw data .csv file I downloaded last week had 3475 responses. Some of those are
invalid as about 10 students just clicked 'submit' without answering any questions (and also
included Kevin!). I have cut out these students and am currently liaising with Derek about
obtaining programme of study, level of study and mode of study information.

The survey was not live just over Easter but was live over the week before and after Easter
which were teaching weeks. It was a poor response, though, but not much different than last
year's, and last year's was done at the standard time.

Perhaps it's been a problem with the SMIS/OSS data and the response rate may have been
higher without some invalid usernames? Also, it's regrettable that you were not able to just
give the 'active' usernames to Netweaver. Will this be possible with the full move to OSS next
year?

Anyway, I do agree that we need to use some further tactics to improve the response rate
next year. If you have any suggestions, please let me know. Also, there will be a formal
review of the JMU student feedback surveys over the next few months where
recommendations will be made.

If we can iron out the problems you mention of using an external 'host' (ie Netweaver), then
we should continue using Netweaver software as we are still quite a number of responses in
credit (ie we have already paid for them).
_________________________________________________________________________
                                         APPENDIX F: COMMENTS RECEIVED BY EMAIL, 2002-3



From: John Mayhew (ASL)
Date: 18 June 2003

Some reflections on the module appraisal process:

I do not think the response rate for the social work course for which I am responsible will be
very good. This is because in the module appraisal period our first year students are about to
go out on placement and are anxiously focusing on that and our second year students are
actually on placement.

Also we have an 'in-house' process which when added to 'out-house' surveys no doubt
promotes evaluation fatigue. Our 'in-house' measures I think are robust. Each module is
evaluated after the module's completion, a report is produced for a Board of Studies and that
report appears in the following year's module handbook. What would be really helpful would
be a centrally based facility to enable our standard module proforma to be reproduced in a
form that can be optically read. This would save a lot of time and energy.

At the end of each of the two years we have a standard evaluation form for the course - again
if this could be designed for optical reading this would help. The results of this survey have
been reported in our PSAADs.
__________________________________________________________________________

From: Gary Brown (ART)
Sent: 24 June 2003

The system does not work, I wish it did work (it would save some admin work on my behalf)
however it doesn’t work, unless we start returning to a method where we can sit students in
front of us and make them fill in these forms they are not going to be done. The sooner the
university (or whoever) accepts this point the better the prospects that the various courses will
have of satisfying external examiners when the courses are quality checked.

From: Anne Miller
Date: 25 June 2003

We know it doesn't work, what we would like is solutions to the problem. For example, would
you be able to take your classes of students to a suite of PCs at the relevant point in the year
(or are they there anyway at some point?) and get them to complete the electronic surveys?
The method of collecting information was changed from a paper based one due to cost -
some time ago - we are unlikely to be given the resource to return to this approach, and yet
clearly there is a need for the response rate to be increased. Is the suggestion I make,
above, viable in your area? Would you be willing to participate in some discussions?

From: Gary Brown
Date: 26 June 2003

Yes, I’m only too well aware of the free will problem? Which was the comment concerning
things have changed since I was a student.

Ask yourself what students really care about, then you may come close to a carrot which the
students will constantly chase. What is this form filling worth to the University?????. Most of
my students work 2 or 3 days per week in order to afford to pay their fees which are a
constant burden.

Even in my day several students had their education paid for them because they were either
intelligent or were supported by corporations or the government. Why doesn’t the University
propose a series of free education positions, a serious reward which will actually cost them
nothing (for home students) as a reward for filling in the forms at least this way those who do
fill in the forms would be doing it for the right reason?
APPENDIX F: COMMENTS RECEIVED BY EMAIL, 2002-3

From: Gary Brown
Date: 26 June 2003

Actually I wasn’t putting a downer on the system, I really do wish it worked. We took all of our
24 students to our own IT suite and showed them how to access the system and how easy it
was to answer the questions etc. Out of the 24 students who were present I believe only 16
actually did answer part of the questionnaire? The rest of the students went on the internet to
entertain themselves.?

We have to sit them down at desks and they are not allowed to leave until they have filled in
the questionnaire, it is not an optional thing it is part of their course. This is the solution.

Things have changed since I was a student.

From: Anne Miller
Date: 27 June 2003

Interesting, we offered a number of incentives this year - financial and airline tickets.
Feedback in focus groups with students revealed these responses:

- I like to feel that you value my opinion and so I don't want a bribe (mature students, much to
my surprise)
- I would like to know what happened in relation to the comments we provided last time
(agreed, we need to do this)
- I should like to know who in my faculty won the prizes, because if I don't win, I'd like to think
I knew the person who did

We do ask students for their views. We amended the procedure during the second iteration of
the surveys and it still requires improvement, which we plan to do.
                   APPENDIX G: COMMENTS FROM CONSULTATION WITH STAFF, JUL-SEPT 2003




The comments below were received from staff in response to the email request from QUS (of
11.7.03) for participation in the consultation process.

Comments have been ordered within the following areas regarding surveys:

- Participation
- Frequency and timing
- Design of the questionnaires
- Implementation, publicity and incentives
- Use and exposure of results: policy and best practice.


Email from Janet Hernandez (LAW), 11 July 2003
Design of the questionnaires
If we have to persist with the electronic system (being more modern), perhaps if the
questionnaire was much shorter, it would help. A lot of questions seem irrelevant.

Also, there is only one real question about admin staff which is taken as referring to school
offices by us, but often the students use it to reply about admin matters in general.

Implementation, publicity and incentives
The main issue is the electronic format of recent surveys. With the old system of hard copies,
academics could give these out at the beginning of a tutorial and ensure that each and every
student completed them and handed them in at the end. Obviously, with the electronic system,
there's only a small response by those who can be bothered to log on and complete it.


Email from David Mottram (PHC), 14 July 2003
Design of the questionnaires
I am Programme Leader for a PgCert/PgDip/MSc, with relatively small numbers of students and
for which we have extensive feedback systems. Whilst I have no objection to the university
system, it has little impact on our PMT decisions compared with our internal feedback.


Meeting with Alice Bird (LID), 17 July 2003
Participation
Consistent feedback was needed from all students at all levels.

Frequency and timing
Two on-line surveys on the use of Blackboard have been undertaken by LID in Autumn 2001
and Autumn 2002. These clashed with module appraisal. However, LID would probably not
conduct a survey on Blackboard in 2003 as fairly similar responses have been received in the
past two surveys; instead, it is likely that LID will have discussions with student reps.

We should avoid survey overload and a co-ordinated approach was needed across the
University.

Design of the questionnaires
A question about on-line learning needed to be incorporated into the SF questionnaire.
APPENDIX G: COMMENTS FROM CONSULTATION WITH STAFF, JUL-SEPT 2003




Implementation, publicity and incentives
The on-line Blackboard surveys using Perception had been successful and user-friendly for
students. The electronic method was better: with paper questionnaires, students completed
them in class time, while staff waited to collect them, whereas students completed on-line
questionnaires because they wanted to.

Anonymity and confidentiality was guaranteed with on-line surveys and student were more likely
to make frank comments.

It would be a good idea for staff to finish the lecture early to allow students to go to PCs

Incentives were useful: in an ICT skills audit, LID offered one prize of £100 voucher and five
prizes of £20 vouchers for staff and elicited 1,000 responses from staff.

Publicity was useful: In the Blackboard surveys there was an announcement on the Blackboard
front page with a hyperlink to the survey

Use and exposure of results: policy and best practice
For the Blackboard surveys, staff and student comments provided the most useful information.


Meeting with Alex Douglas, 17 July 2003
Frequency and timing
Alex used his own paper questionnaire in his modules which he distributed at the last lecture. If
students had questionnaire fatigue from QUS and other surveys, his appraisal would suffer.

Design of the questionnaire
The comments below relate to the standard module appraisal questionnaire. However, Alex
also thought that the SF questionnaire should be rewritten.

Defining the purpose of a survey was very important. This would inform the design of the
questionnaire. If the purpose of JMU surveys was just to comply with QAA/HEFCE
requirements, then it did not really matter what the questionnaire asked. However, if the
purpose was for lecturers to improve the module, then the questionnaire needed to change.

In a recent module appraisal, half of Alex’s students completed Q11: Assessed work was
returned within a reasonable time period (only answer if you have had assessed work returned)
for a 100% exam module!

In his Quality Management module, Alex told his students that they would receive feedback on
coursework three weeks after the deadline. But what was appropriate and what was a
reasonable time period? From a staff point of view, the amount of time it would take to return
coursework would depend on the number of students on the module.

Q5: Lecturers were punctual and reliable in their attendance (taught modules) relied on student
opinion. This question could have legal implications. Also, poor attenders may complete this
question, giving unreliable answers.

Q9: The type of assessment was appropriate for the module – this was dealt with at validation.

Q10: Deadlines set for submission of work were appropriate – each module leader worked
independently to set his/her own deadlines, and deadlines were likely to occur around the same
time. Students were therefore likely to complain about the deadlines set.
                    APPENDIX G: COMMENTS FROM CONSULTATION WITH STAFF, JUL-SEPT 2003



Q13: The teaching accommodation was suitable and Q14: Sufficient material for the module
was available in the LRC. These issues were out of the control of staff, but Alex would inform
his subject group leader of the results from such questions.

Implementation, publicity and incentives
In Alex’s module handbook, he has written a learning agreement stating that students would
complete the QUS on-line module questionnaires.

Students could be directed to an IT suite but could not be made to complete questionnaires.

Staff incentives would cause problems.

The majority of Alex’s students did not use their email accounts. Perhaps an announcement or
a link on Blackboard would be useful.

Use and exposure of results: policy and best practice
Policy/best practice was not an issue; the issue is staff having time to action the results.
Dictation of ‘best practice’ by QUS was too bureaucratic.


Meeting with Liz Sperling (SSC), 17 July 2003
Participation
All JMU Schools should all be required to participate in student feedback.

Programmes run at non-standard times should be included.

Collaborative programmes should be included (although in SSC there are none).

Frequency and timing
The timing of the QUS surveys was appropriate.

Students were required to complete too many questionnaires. Selected modules only could be
appraised in different semesters, for instance, half of a School’s modules could be appraised in
one year; the other half the next.

Design of the questionnaires
Standard module appraisal questionnaire Q11: Assessed work was returned within a
reasonable time period (only answer if you have had assessed work returned) – often assessed
work was not set to come in until the end of the semester so this question was largely irrelevant
at the time of the appraisal.

The QUS module appraisal questionnaire was not appropriate for SSC. Sometimes module
leaders wished to be more ‘module specific’ in their questions, for example, they may wish to
focus on certain components of a module and the QUS questionnaire did not allow for this.
Module appraisal should be undertaken by individual module leaders as they had different
reasons for wanting to appraise their modules.

The Geography programme leader had criticised the module appraisal questionnaire for its lack
of a forward focus, for instance, it did not ask students what they would like to see happen
differently next year.

Implementation, publicity and incentives
Questionnaires could possibly become part of the requirement for module assessment. A link
could be made through Blackboard.

Geography modules conducted their own module appraisal. Also, Liz had conducted her own
appraisal on one module in 2002/3.
APPENDIX G: COMMENTS FROM CONSULTATION WITH STAFF, JUL-SEPT 2003




Liz objected to prizedraws for students as incentives.

If students were not interested in taking part, there was nothing that could be done, even if they
were led to a PC. If accessing a PC to complete the questionnaire was built into lecture time
during the survey implementation period, the PCs would be oversubscribed. Also, if students
were coerced into completing questionnaires, they would be more likely to give negative
feedback.

Staff would be insulted if staff incentives were offered.

For further publicity to improve response rates, an announcement could be made on
Blackboard. Posters could be printed (although this would cost). Survey participation could
also be made compulsory. At Liverpool University, participation in certain surveys formed part
of the assessment on some courses.

Paper questionnaires would be more effective. A colleague had asked Liz to mention that the
current system was clearly not working. The paper version was more expensive but the
University had to decide whether to spend money on something that produced data of no value
or whether to spend more money on a system that did produce important and valuable data.

Use and exposure of results: policy and best practice
All module appraisal results did was provide some basic data on the satisfaction of teaching and
the punctuality of staff.

A colleague had asked Liz to mention student comments. The students that made comments
were often the disgruntled ones, and sometimes comments were not constructive and
inappropriate – even libellous – even though staff names were removed from comments. There
was also no mechanism to enable staff to respond to student comments.

It should be ensured that all PSAADs included survey results. This was a quality issue and may
also be a way of motivating staff to improve response rates. A commitment to student feedback
could also be a requirement at the validation stage.

One programme leader in SSC was known to summarise details of his module’s feedback in his
module handbook.

Good practice was currently disseminated in SSC through the PSA process where staff
critiqued other programmes’ PSAADs before they were submitted.

Academic staff already felt ‘browbeaten’, and would feel that further requirements for valid
feedback results were just one more thing with which to harass them. To help with this, QUS
could write a briefing paper on the real purposes of the surveys and what happened to results,
and how feedback could work for the benefit of academic staff.


Email from Maxine Melling (LEA), 26 August 2003
Frequency and timing
Once per year for the SF survey is good. If this isn't possible I could live with every two years.
We are still reviewing the results of our own web-based survey. It's likely that if we continue we
will run it every two years. It would therefore be sensible for us to run your survey and ours on
alternate years IF you move to this pattern. Timing has to be dictated by the academic calendar.

Design of the questionnaires
From my point of view I need feedback from all students with the ability to identify year of study
and subject discipline. Ideally I would like responses broken down in this way. I have already
                    APPENDIX G: COMMENTS FROM CONSULTATION WITH STAFF, JUL-SEPT 2003



discussed the range of questions on LIS and would like these to stay the same. Please keep
the free text comments in the questionnaires, these are by far the most useful aspect.

Use and exposure of results: policy and best practice
The use of results has to be linked to annual planning. All areas should be expected to make a
response to the results and to say how they intend to address them, with institution-wide issues
considered in budget planning etc. Timing is obviously an issue here as the survey results are
too late for the following year's planning cycle. However, initial reactions aren't always budget
dependent.


Meeting with Sue Russell (NRS), 27 August 2003
Frequency and timing
The timing of QUS surveys was appropriate and modules should continue to be appraised each
time they ran.

It would be helpful if co-ordinators could inform staff earlier in the semester of survey dates.

Implementation, publicity and incentives
Sue preferred the paper questionnaires. The problems she had experienced (module appraisal)
for modules in her area (CPD Mentoring in Clinical Practice) was that most students just
attended one day a week for the lecture, returning to work afterwards. However, Sue felt that
these problems would improve in time as more students obtained home PCs and JMU staff
increased their use of Blackboard and email.

Sue suggested that a couple of computer terminals in the LRC be set aside during the ‘live’
survey weeks. These could be clearly signposted and students could use them to log on and
complete their questionnaire(s).

It was sensible to email questionnaires to JMU students’ usernames. The method used for
SF2002, whereby programme leaders were asked to supply email addresses for their students,
had not been successful.


Email from Cathy Sindall (BLW), 27 August 2003
Perhaps the module appraisal co-ordinator’s role should be undertaken by the Faculty Quality
Officer in the future, as they may need to look at the responses etc.


Meeting with Pauline Stitt (NRS), 3 September 2003
Participation
All programmes should be involved in surveys, but this may be problematic for programmes
which are run at non standard times and with non standard attendance patterns.

Frequency and timing
The current timing of the QUS-facilitated surveys was problematic for NRS for the reasons given
above. Pre-registration nursing students, in particular, were out on clinical placement 50% of
the time.

Generally, in NRS, there were many cohorts starting at different times of the year. Only some of
those would be surveyed by QUS and therefore a true picture was not being obtained.

Design of the questionnaires
Pauline was, in general, satisfied with the questions used in the questionnaires.
APPENDIX G: COMMENTS FROM CONSULTATION WITH STAFF, JUL-SEPT 2003




Implementation, publicity and incentives
Many NRS students did not access CWIS very often and therefore did not use their email
accounts. However, this was likely to improve in future with increased use of Blackboard by
staff.

The incentives offered by QUS had not made an impact with NRS students.

Pauline had emailed the briefing sheet (supplied by QUS) to her staff but staff generally
remained unconvinced of the usefulness of the surveys.

NRS obtained qualitative feedback from students through the nominal group evaluation exercise
(NGE). Students also completed a clinical evaluation form on completion of each clinical
placement.

Use and exposure of results: policy and best practice
Pauline had found the data supplied from the QUS surveys useful (where response rate was
good) and it had been used in PSA. Results were also discussed at Programme Boards of
Studies.

Pauline would like NRS to continue to participate in the QUS surveys as they provided data for
PSBs and the PSA process.

Students had said that no action was taken on module appraisal results. However, some issues
(such as teaching accommodation and learning resources) were beyond the control of NRS
staff.

Pauline would like to be able to use some positive feedback from the surveys for marketing
purposes. This had been done by some other universities – some had even used it in cinema
advertisements.


Meeting with Ailsa McGhie (NRS), 3 September 2003
Participation
All Schools should participate in module appraisal.

Frequency and timing
It would be useful if all modules could be appraised but in NRS some modules ran at non
standard times of the year and the QUS-facilitated appraisal was not timed to pick up all NRS
modules.

Design of the questionnaire
At the completion of each module, NRS students took part in the nominal group evaluation
(NGE) exercise. Ailsa felt this was more effective than module appraisal, which did not
sufficiently bring out issues.

It was no use having questions on the module questionnaire which were beyond the remit of the
school, eg on learning resources.

NRS students also completed a clinical placement questionnaire after each placement.

Certain questions repeatedly received a negative response, eg the catering service, the LRCs,
and the Overall Opinions section. Ailsa suggested that the Overall Opinions section be moved
to the beginning of the questionnaire.
                    APPENDIX G: COMMENTS FROM CONSULTATION WITH STAFF, JUL-SEPT 2003




Implementation, publicity and incentives
Ailsa did not think the electronic method was working in NRS as many students were not often
at JMU (50% of their time was spent doing clinical placements or working shifts) and many did
not have PCs at home. Blackboard was not widely used by NRS lecturers. It might help if
students had access to a PC at the hospital they were working at.

The paper based method was better as lecturers knew when they would be seeing their
students. A good response rate was achieved with the paper based method.

Staff could lead students to IT suites to fill in questionnaire but they would need to have pre-
booked a suite.

Use and exposure of results: policy and best practice
Ailsa was satisfied that programmes were well evaluated by the SF survey.

Ailsa used some of the results from the QUS surveys in her PSAADs. However, she would find
it too time consuming to write a précis of each module for her programme’s PSAAD.

Action taken at the institutional level was never reported back to staff.

In NRS module appraisal results were discussed at sub programme boards.


Meeting with Paulo Lisboa (RGS), 3 September 2003
Participation
For the PG research survey, all research students should be included.

Implementation, publicity and incentives
The PG survey response rates had been fairly consistent (between 33% and 49%) but needed
to be increased.

The PG Society would ideally be a good mechanism for publicity.

For the 2003 PG research survey, Paulo suggested a cash incentive of £100 and some prizes
of £20 book tokens.

Use and exposure of results: policy and best practice
The results were discussed at Research Degrees Committee and the Graduate School Board.
A summary of results was usually sent to all students and a full copy was available in each LRC.


Meeting with Llew Llewellyn (MCA), 3 September 2003
Participation
Llew would prefer that all QUS-facilitated JMU surveys remained censuses.

Frequency and timing
Students were currently surveyed on their modules before they had completed them. If
students were taking experimental drama, for example, where there was a performance at the
end, they may feel differently when they could see the outcome of the work. They needed time
to reflect before they could assess the value of what they had learned or make a judgement of
any value.

It would be useful for students, after a time of reflection, to critically evaluate their programme at
the beginning of the next semester. Surveys could easily be implemented in September
retrospectively. With a fast turn-around of results, staff could still include them in PSAADs.
APPENDIX G: COMMENTS FROM CONSULTATION WITH STAFF, JUL-SEPT 2003




Design of the questionnaires
Re. Q5 on the standard module appraisal questionnaire - Lecturer(s) were punctual and reliable
in their attendance: Llew felt that students would make it clear to the lecturer or the department
if the lecturer was tardy and so this type of question was not necessary.

The module appraisal questionnaires should be offered at the programme level, with each
programme developing its own programme-specific questionnaire and implementing and
analysing it locally. As Chair of SARC in 2002/3, Llew had proposed this. Staff in MCA felt that
the QUS questionnaires were too generic. Also, QAA did not object to locally-generated
surveys.

Arts students in general did not like form-filling. Drama students saw participation in surveys as
a bureaucratic exercise, of little or no benefit to developments on their programmes. Drama
was about social communication and it was more appropriate for drama students to give verbal
feedback.

Implementation, publicity and incentives
The electronic method was not working in the drama area and Llew was happier with the paper
method. His students did log on but used their own email addresses.

Incentives were of little use.

Use and exposure of results: policy and best practice
SF survey programme reports were viewed by the programme leader and discussed by the
Student-Staff liaison committee.

Llew used to use the survey results in his PSAAD but since electronic implementation he had
been unable to do so because of the low response rate.

PSAADs were currently read by very few people. In the old structure they were read by the
Chair of SARC, and rigorously by the DARP Chair. Student reps on Programme Boards of
Studies had the opportunity to read PSAADs.


Meeting with Dave Rees, Mads Gaardboe, Richard Gant, David Crow,
Carole Potter, Adrienne Lowy, Scott Oram, Emma Roberts, Julie Sheldon,
(all ART), 4 September 2003
Participation
Partner programmes at Southport and Wirral conducted their own surveys. At Stockport,
feedback on modules was analysed by the module leader; consequently it could not be seen as
impartial.

Design of the questionnaire
Tick-box questionnaires were contrary to Art School culture which was essentially a
qualitative/subjective culture, although the quantitative results were useful for staff. Art School
students would be happier answering a statement with a sentence.

Mads Gaardboe recounted a recent incident concerning a negative student comment about a
professor. Although the proper name had been removed, the job title was enough to identify
that the comment was aimed at one of two professors. This was unfair on the professor to
which the comment was not directed. Job titles in comments (apart from ‘lecturer’) should be
removed. It was also unfair that staff had no chance of responding to student comments.
                   APPENDIX G: COMMENTS FROM CONSULTATION WITH STAFF, JUL-SEPT 2003



Comments should not be circulated as they were unscientific and unrepresentative, and could
be embarrassing in a review. It was felt that student comments had been misinterpreted in past
QAA reviews.

Implementation, publicity and incentives
The Art School had made an effort to establish a good system with the paper-based method
which had then been replaced by the electronic method. The electronic method was not
working as students did not use PCs. There were MACs in the School but many were not
networked; only in the IT suite. It was difficult to get on a PC in the LRCs as they were
constantly in use.

With the electronic method, however, it was acknowledged that students could be encouraged
to log on and to use Blackboard, which would be increasingly utilised by the School.

The University should support students and the paper method should be re-introduced, even
though it was more expensive.

Students were not convinced of the anonymity of the electronic method.

Access was not the main problem: students just didn’t think the surveys were important.

The Art History programme had circulated its own qualitative/quantitative questionnaire but this
had only elicited a 7% response. Students just didn’t want to fill in questionnaires.

With the paper method, students were a captive audience. However, the paper method was
prone to elicit negative responses as students just filled in the questionnaires hurriedly.

Art School students were informed of the importance of surveys at the Level 1 induction stage.

It was suggested that student reps could be given responsibility for students filling in their
questionnaires. Or, student reps could gather feedback (verbal or written) which they would key
in electronically. The rep would need to be trustworthy.

Could CIS build something into the system which would suspend internet access for students
until they had completed questionnaires? Or, it could be built into the CWIS terms and
conditions, that acceptance meant that students would complete questionnaires.

Participation in surveys could be built into professional practice modules, however this would
not work for Year 1 students.

The completion of surveys could be built into the administration of the programme, perhaps into
the module learning outcomes.

As an incentive, a prize of, for example, a new MAC or kit for a department could be offered.

Richard Gant suggested the idea of a ‘feedback day’. This could be made fun and the social
aspect could be emphasised. Instead of paper or electronic questionnaires, students could
place coloured ping-pong balls in bins, denoting a response. This would have greater appeal to
Art students.

The changeover to the electronic method was also a culture-change. A transitional period was
needed and the ping-pong ball idea could be part of this transitional phase. The move to the
new Art School building (in four years’ time) could mark the change to an electronic method.
The new building would have a facility for dedicated PCs, or a ‘feedback room’.

An alternative method to module appraisal was suggested by Richard Gant. He had used the
‘student pyramid feedback method’ with his Fine Art students. He had learnt of this method
while on a staff development day at Oxford Brookes. The method operated as follows: all
APPENDIX G: COMMENTS FROM CONSULTATION WITH STAFF, JUL-SEPT 2003



students on a module were brought together by a facilitator and asked what had influenced their
learning activities on the module. They wrote their responses in bullet point form, then formed
groups of 8 and selected a ‘Chair’. Common bullet points among each group were identified
and recorded by each group. The facilitator then collected the list of bullet points from each
Chair and identified the common ones.

The whole process was expected to take about 20 minutes; 20 minutes to rationalise all the
major issues. The process was a social activity, was democratic and encouraged bonding.

Use and exposure of results: policy and best practice
Results were cited in PSAADs. It was suggested that student participation in surveys should be
voluntary, with no requirement for staff to use results in PSAADs.

Student reps wrote a short report in response to the module leader’s report.

Students did not see the results of the surveys and perhaps this was one reason why they did
not participate.


Meeting with Norman Partington (CIS) and Kevin Jump (CIS), 4 September
2003
Implementation, publicity and incentives
It was felt that last year, in particular, it had been difficult for CIS to extract an accurate list of
students because of the Oracle/SMIS changeover. The data would be ‘cleaner’ in 2003/4 but
there would always be a ‘grey area’ of students who had recently left but had yet to be removed
from the system.

There had been a confusion over roles between CIS and Netweaver. Also, CIS had felt that
QUS had not given enough notice of the tasks to be done.

Emails were slower to reach students as they had been sent from an external server. Such a
large number of emails being sent at the same time slowed down the JMU email server
considerably. This problem would not exist if internal Perception software were used.


Meeting with Simon Dowell (BIE) and Louise Higham (BIE), 4 September
2003
Participation
Sampling could be considered as students did get questionnaire fatigue.

Simon thought that St Helens College conducted its own surveys, but would need to confirm
this.

Frequency and timing
Surveys needed to be conducted at least annually as they fed into the evaluation process.
They were also required by QAA.

Module appraisal was best conducted near the end of the semester (ie Weeks 11 & 12).

Design of the questionnaires
There was a need to review the SF survey questions for 2004. Faculty targets for tutorial
modules had been included in the strategic plan and there needed to be a more specific
question about this. Also, at Developmental Engagements, QAA had been interested to know
how JMU evaluated the effectiveness of its tutorials. The questions therefore needed to reflect
QAA requirements and the strategic plan.
                   APPENDIX G: COMMENTS FROM CONSULTATION WITH STAFF, JUL-SEPT 2003




The question in Section 2 of the SF questionnaire: The availability of staff to provide guidance
and support (eg personal tutor, module leader, year tutor) should be split. Personal tutors were
a key part of university strategy.

Employability was another area which should be included in the questionnaire.

Simon and Louise were happy with the questions on the BIE module questionnaire.

Module questionnaires should be offered on a school, not faculty basis; one school’s questions
may not apply to other schools in the faculty.

The SF survey questions should be designed by the institution.

The further comments section in the SF questionnaire was of limited use for module and
programme leaders; most comments were about the institution. However, the further comments
section in the module appraisal questionnaire should be retained.

Sometimes comments led external reviewers to get the wrong impression.

Implementation, publicity and incentives
The response rate was higher with the paper method; students were a ‘captive audience’ as
questionnaires were given out in lectures.

The response rate was currently quite low and only conscientious students and those who were
disenchanted were, as a rule, completing questionnaires; this was reflected in the comments.

The one way to improve response rates was for staff to ensure that students had access to a
PC in a timetabled session. One suggestion would be to hold the final session of the module in
the computer room, but this would not be an option for schools without their own IT facilities.
The BIE computer room had 30 machines and this would work for smaller modules. However,
different modules would want to book the room at the same time! It would take some organising
but this may be the way BIE would operate for future module appraisal surveys.

For other SCS and TAE schools, part of the LRC could be used and lecturers could bring
students to PCs. In Byrom Street, the large computer room could be block booked and module
leaders could book slots in lecture time and take their students along.

Lecturers could remind students by email to complete questionnaires. School websites and
Blackboard could also display reminders.

Staff incentives would be seen as divisive and should not be offered.

Use and exposure of results: policy and best practice
Module appraisal was a useful survey. Louise produced graphs for the data supplied by QUS.
The graphs were displayed on programme notice boards. In future, the graphs would be on the
Web.

Every BIE module leader produced an end-of-semester report where they commented on the
module results. Master copies of reports were kept in the Director’s office.

Survey results were used in PSAADs. These were discussed at SMT and action was
prioritised. Any institutional level issues would be cited in the school-level report and
subsequently in the Quality Audit Report.

At the institutional level, checks should be made on whether Strategic Plan targets were being
achieved.
APPENDIX G: COMMENTS FROM CONSULTATION WITH STAFF, JUL-SEPT 2003



FQC-FLTC could organise discussion of best practice within the SCS faculty. However, this
would mean extra workload and the identification of staff to undertake it.

A minimum level of compliance re surveys should be JMU policy (ie re participation and
exposure and use of results).


Meeting with Pat Shenton (PSD), 5 September 2003
Pat Shenton oversees current and future developments for all ITE subject areas. Her
comments therefore apply to QTS students throughout ECL.

Frequency and timing
Module appraisal should continue to be conducted at the end of each semester.

Design of the questionnaires
For all QTS students, the questionnaires needed to ask specialist questions about the school
experience. 55% of PSD students were QTS and they were not being asked by QUS about
their experience in schools.

Pat suggested that for the SF survey, QTS students received the SF questionnaire plus another
with specialist questions. The questions should map to the new ITE inspection framework and
provide evidence that students had achieved the standards therein.

Implementation, publicity and incentives
Response rates had not been good as many students did not log on.

The surveys were important for Ofsted reviews. Within the new framework there was an
emphasis on quality feedback and quality review. In November and December 2003 there
would be three more inspections in ECL. The poor response rates would be queried by Ofsted.

PSD conducted its own in-house survey at the module level, the results from which were used
at a recent Ofsted inspection. This survey took the form of an evaluation session at the end of
the module. All QTS students also completed a local school experience questionnaire as part of
their module. A new type of school experience survey was currently being piloted in ECL by
Sue Hatfield.

It was not the intention to give PSD students survey fatigue, but with the poor QUS surveys
response rates, Pat did not see an alternative.

Staff could take more ownership for promoting the surveys and ensuring questionnaires were
completed. Staff could take students to the IT suite.

Use and exposure of results: policy and best practice
Feedback from local questionnaires was published and circulated within the School.


Email from Hugh Jones (AST), 11 September 2003
Participation
Should be as wide as possible but with questions omitted for different levels/modes of study.

Frequency and timing
Should be set by strong guidelines but also by local programme demands not by one date fits
all.
                   APPENDIX G: COMMENTS FROM CONSULTATION WITH STAFF, JUL-SEPT 2003




Design of the questionnaires
It would be very useful if staff could have a chance to have input/feedback on questions before
they go to students. In the past we have had quite a few comments about inappropriate
questions for level/mode of study. It is important that the surveys inform modules and
programmes in a useful and positive way.

Implementation, publicity and incentives
Students should not need to be logged into JMU or have a local username and password to
operate.

I am not advocating it for the moment but the ultimate incentive for students to feed back would
of course be to make it a condition of passing the module. In the meantime cash and holiday
incentives seem reasonable.
APPENDIX H: COMMENTS FROM STUDENT FOCUS GROUPS, SEPT 2003




The comments below were received in response to an email request to a number of students
from QUS (23.703). Students were invited to participate in one of two focus groups, held on 5 &
12 September.

Comments have been ordered within the following areas regarding surveys:

- Participation
- Frequency and timing
- Design of the questionnaires
- Implementation, publicity and incentives
- Use and exposure of results: policy and best practice.


Focus Group 1: Caroline Smith (CMP), Jim Clarke (CMP), Jack Miller
(LSU), 5 September 2003
Participation
QUS should continue with a census approach. however, with sampling it would be easier to
ensure that students completed their questionnaires.

Frequency and timing
Modules should continue to be appraised each time they ran, for consistency.

Timing would always be an issue. At the time surveys were live (ie near the end of semesters)
it was difficult to find a free PC. It was best to survey students at the beginning of the
subsequent semester/year (ie retrospectively); students would have completed all of their
assignments by then and could be more objective. The downside to this was that students’
memory could be hazy; also students leaving JMU would not complete a questionnaire on their
last semester/year.

Design of the questionnaires
Module appraisal was an annoying survey as there were several questionnaires to complete.
There were 16 closed questions per module, so if you were doing four modules, that would be
64 questions you would be required to answer. You were repeating the same questions for
each module and could get bored and there was the tendency to give a stock answer.

The questions could be reduced to 10 questions per module. For example Q2: My workload on
the module was appropriate could be deleted. Most students would be likely to say they were
dissatisfied with this as most were overloaded with work, especially at the time of the semester
when they were filling in the questionnaires.

Some questions were more module specific than others, eg books (different for each module)
and only the module specific questions should be retained.

Implementation, publicity and incentives
Many students did not use their email accounts or even log on.

Caroline did not complete her questionnaires in 2003 as she had not been clear of the purpose
of the surveys.

Many students were not completing their questionnaires as they were not interested, not
because they were apathetic.

There needed to be a clearer distinction between the SF and module appraisal questionnaires:
students could think that an email request for one was just a reminder for the other.
                          APPENDIX H: COMMENTS FROM STUDENT FOCUS GROUPS, SEPT 2003



Anonymity should be stressed further.

It could be made conditional that students completed questionnaires in order to change their
passwords.

It could be made part of the registration procedure that students completed questionnaires on
the previous semester.

The importance of the surveys could be stressed at induction, ie that they were a part of
continuous quality improvement.

The LSU website and magazine Reload could publicise the surveys. The surveys should be
seen to be approved by LSU.

As a result of the DE Computing, module programme reps and module leaders held a meeting
which was more productive than module appraisal feedback.

An alternative method to module appraisal questionnaires was the ‘inverted pyramid’ method
whereby module reps obtained feedback from students and then met with module leaders.
Students could be asked to write down 10 good and 10 bad things about the module.

Use and exposure of results: policy and best practice
It was important that students were told what action had been taken in the past as a result of the
survey.

Survey results were usually discussed at Programme Boards of Study.


Focus Group 2: Darren Mooney (LSU/BIE), Vicky Marney (LSU/BLW),
Michael Evans (MCA), 12 September 2003
Frequency and timing
It was felt that the timing of the module appraisal survey was OK. However, students might be
less inclined to complete it when they had more or less completed the module.

February would be the best time for the student feedback survey as students would not be
revising and would have more time. They would be able to answer the questionnaire as they
would have used all of the services mentioned in the first semester; also students would now be
starting the second semester earlier in January.

Design of the questionnaires
The ‘neutral’ box should be removed; this would force students to give firm answers.

The SF and module appraisal questionnaires were OK and were short, clear, and easy to
complete.

Implementation, publicity and incentives
Although a better response was achieved with the paper method, Darren preferred the on-line
method as it gave him more time to consider the questions.

Students received many emails and the email including the link to the survey was too official-
looking and not very inviting. Maybe larger text and colour could be used.

Student JMU email accounts were drab; Hotmail was more colourful.

The email should mention the time it should take for students to complete the questionnaire.
The title of the email should be something more attention-grabbing, such as ‘Win £100’.
APPENDIX H: COMMENTS FROM STUDENT FOCUS GROUPS, SEPT 2003



In Spring 2003 it was likely that students had questionnaire fatigue as they had repeatedly
received reminders re the LIS questionnaire at that time.

All students did not log on to CWIS. The LSU’s website could promote future surveys (contact:
Paul Twigger). Surveys could receive further publicity in the student magazine Reload (3-
weekly) and JMU News.

LRC PCs could carry a screen saver over survey implementation periods, advertising the survey
and incentives.

Any publicity should make students aware that the survey is going to be of benefit to them.

A suggestion was to ask for 10 volunteers out of each class to give an opinion on their module,
but the disadvantage with this method was that the volunteers were likely to be students who
had a complaint to make.

Use and exposure of results: policy and best practice
Students never saw the results of surveys.

Results and actions should be publicised clearly, ie ‘this is what you asked for and this is what
was done about it’. Posters should be displayed in Schools.

Results should be sent directly to student reps so they could ensure action was being taken,
and could lobby staff as necessary.

Boards of Studies were of no use regarding the discussion and actioning of results: lecturers
and students often did not attend as they were usually at lectures!
                                APPENDIX J: SURVEY PRACTICES IN OTHER UK HE INSTITUTIONS




The comments below were received from staff in other UK HEIs in response to an email request
from QUS (of 25.7.03) for information on current survey practices.

Comments have been ordered where possible within the following areas:

- Participation
- Frequency and timing
- Design of the questionnaires
- Implementation, publicity and incentives
- Use and exposure of results: policy and best practice.


Email from Derek Blease, Loughborough University, 29 July 2003
Loughborough Business School recently concluded a Hefce FDTL project on Teaching Quality
Systems in Business and Management Studies. The main focus was on identifying good
practice in student feedback systems.

The project investigated the student interface with teaching quality systems in Business and
Management Studies.

[Derek also sent some briefing papers by post, produced as a result of the project.]


Email from Alison Taylor, London School of Economics, 4 August 2003
Design of the questionnaires
The information on our web page (link below) will enable to you to view 'static' samples of the
questionnaires we use. http://www.lse.ac.uk/collections/TQARO/surveys/Default.htm

Frequency and timing
The Teaching Quality Assurance and Review Office conducts three surveys in each academic
year of students' opinions of teaching:

- Michaelmas Term Survey of Teaching by Part-Time Class Teachers. This survey is intended
  to cover teaching by hourly-paid class teachers, who are mostly but not exclusively
  postgraduate students and is conducted in the second half of the Michaelmas Term;
- Michaelmas Term Survey of Course Teaching. This covers all teaching on courses that do
  not run for the full academic year and so will be completed before the main survey takes
  place in the Lent Term and can include, on request, courses where there is a change of
  lecturer in the Lent term. This is also conducted in the second half of the Michaelmas Term;
- Lent Term Survey of Course Teaching. This covers all teaching on all assessed courses and
  is conducted in the second half of the Lent Term.

Implementation, publicity and incentives
At LSE we now conduct all our student surveys on-line.

In terms of publicity and incentives, we use as much as possible! In our experience, it is vital to
target publicity at teachers as well as at students, and to try to get teachers to support the
survey.

The process is as follows:

 ADVANCE INFORMATION is sent to Department Managers ('DMs'), for circulation, to give
  information on what will be happening and when
APPENDIX I: SURVEY PRACTICES IN OTHER UK HE INSTITUTIONS



 OHPs are sent to teachers for use in teaching sessions
 FURTHER MESSAGES are sent to DM's (for circulation) when a survey goes live to remind
  teachers to (a) check that their students have received it and telling them what to do if not,
  and (b) encourage their students to complete it. Also to tell teachers to let us know if they
  haven't received the OHP
 An ITEM is placed in News & Views (School newsletter) and also sent to The Beaver
  (student newspaper)
 POSTERS are sent to all departments and put up around the School
 An EMAIL is sent to Department Conveners encouraging them to write to their students.
 INCENTIVES offered to students for taking part - random prizes ( 'First prize' for each
  survey is a travel voucher worth £250.00, other prizes of Virgin vouchers and book tokens)
 LOGIN box appears regularly during survey 'live' period (pops up when anyone logs into
  any School PC)
 PLASMA SCREENS around the School regularly display version of the posters during
  survey 'live' period
 REMINDERS (approximately weekly) sent to students who have not submitted completed
  survey(s) for each of their courses

The Teaching Quality Assurance and Review Office (TQARO) sends an e-mail to students
which contains a link to the on-line survey for each course the student is registered on and
which is eligible for inclusion in the survey. At the same time a message is sent to all
Department Managers to alert them to the fact that the survey has begun and giving advice as
to the action that teachers should take to ensure that their students have received it and to
encourage students to take part.

The questionnaire incorporates opportunity for free comments and is submitted anonymously
unless the student wishes, by nature of these comments, to make their identity known.

Part-time class teachers are also asked for their own comments on the course, which are
forwarded by TQARO to the Dean, the Director of Teaching and Learning and then to the
teacher with principal responsibility for the course unless the teacher completing the form has
requested confidentiality.

Free comments submitted in the survey of students’ opinions of teaching by part-time class
teachers are read by TQARO to remove any that are defamatory or inappropriate. Free
comments submitted in the course teaching surveys are not scrutinised. These are then
returned to the teacher along with the statistical results on his/her teaching for each course.
These statistics are not produced in cases where there are fewer than 5 respondents. This is to
protect respondent anonymity and also because feedback from fewer than 5 students is of
limited use.
Free comments from the Course Teaching surveys that relate to the course as a whole (ie not
individual teachers) are sent to the teacher responsible for the course who is encouraged to
share them with other teachers as appropriate.

Use and exposure of results: policy and best practice
Access to survey results that relate to individual teachers is strictly controlled under an
agreement negotiated with the relevant School committees and the Association of University
Teachers (AUT)

Part-Time Class Teachers Survey: summary results from this survey (two scores per teacher
per course) are sent to Department Conveners prior to the Christmas break. This allows them to
identify those teachers whose performance may be below the standard required and to take
action if necessary before the Lent Term begins. The summary scores are also sent to the
Director of Teaching and Learning. Individual teachers are sent their full results prior to, or as
soon as possible after, the Christmas break. These results are also sent to the teacher's
Department Convener and may be requested by the teacher responsible for the course on
which they teach.
                               APPENDIX J: SURVEY PRACTICES IN OTHER UK HE INSTITUTIONS




A set of aggregate results, combining responses to all courses within a department, and an
aggregate for the School, is produced and sent to Department Conveners. This aggregate may
be made available to others if the numbers of respondents are sufficiently high that no teacher
can be identified.

Course Teaching Surveys: No summary results are produced. Full results are sent to the
individual teachers at the beginning of the Lent Term (from the Michaelmas Term survey) and at
the beginning of the Summer Term (from the Lent term survey). These results are also copied to
each teacher's Department Convener and may be considered by the Promotions Committee
when making decisions in promotion or review cases. No other parties are given access to
results for full-time teachers. Course conveners may request a copy of results relating to part-
time class teachers on the course for which they are responsible.

Aggregate results are again produced for each Department and for the School and sent to the
Convener. These are made generally available within the School and published on the TQARO
website and public folder. The terms of the agreement with the AUT limit the level of detail that
can be fed back to students, but we are aware that students would like to see more of the
information generated by the survey; this issue is currently under investigation.

In the 1999-2000 and subsequent surveys the Department and School aggregates were also
presented split into undergraduate and postgraduate courses but these are not generally
available as there may be problems of confidentiality.
APPENDIX J: NATIONAL STUDENT SURVEY QUESTIONNAIRE
                              The National Student Survey
In order to assist future generations of higher education students in their choices of what and where to
study, the Government has decided to introduce a regular survey of recent graduates in order to obtain
their views about the quality of the teaching and support they experienced in higher education. This is the
first such survey to be undertaken and it will act as a pilot for surveys in future years.

The following questions have been derived from comments made by recent graduates and final year
students. They should not take more than 10 minutes to complete. The questionnaire asks for your views
about a number of aspects of the teaching and support provided on your recent higher education course.
Please answer the questions in terms of the course as a whole. (By course, we are referring to the totality
of your studies. ‘Programme’ or ‘degree’ are other terms that might be used.)

We realise that you will have experienced a wide range of




                                                                                                                                         Definitely Disagree
teaching during your time in higher education. Please try to




                                                                                                     Neither Agree nor




                                                                                                                                                               Not applicable to
                                                                   Definitely Agree
reflect your typical experiences of the course.




                                                                                                                         Disagree with
                                                                                      reservations




                                                                                                                         reservations
                                                                                      Agree with




                                                                                                                                                               my course
                                                                                                     Disagree
1. The teaching on my course
Staff motivated me to work well.
Staff were good at explaining things.
Staff made the subject interesting.
Staff were enthusiastic about what they were teaching.
Staff were alert to the sort of difficulties we might encounter.
Staff seemed to have prepared their teaching thoroughly.
2. The assessment on my course
It was clear what standard of work was required.
I received prompt feedback on my work.
I received detailed feedback on my work.
Assessment criteria were clear in advance.
Assessment arrangements and marking were fair.
There was a good mix of examinations and other forms of
assessment.
Assessment tasks were always bunched towards the end of a
module/course.
Lecturers here frequently give the impression they have
nothing to learn from students.
To do well on this course all you really need is a good memory.
Feedback on student work is usually provided only in the form
of marks and grades.
It would be possible to get through this course just by working
hard around exam times.
3. Knowledge and skills
I feel confident in the subject knowledge I acquired through
the course.
There were sufficient opportunities to apply the theoretical
knowledge I acquired to practical situations.
The course developed my problem-solving skills.
The course sharpened my analytic skills.
The course helped me develop my ability to work in a team.
The course improved my communication skills.
The course helped me to develop the ability to manage my own
work.
The course helped me to present myself with confidence.
The course was intellectually stimulating.
                                            APPENDIX J: NATIONAL STUDENT SURVEY QUESTIONNAIRE
4. Course organisation and management
The course was well-organised and ran smoothly.
It was clear what was required of me.
The course enabled me to pursue my academic interests.
I found the workload was too heavy.
I was given enough time to understand the things I had to
learn.
The sheer volume of work meant that it could not all be
thoroughly comprehended.
The course was flexible enough to fit in with my other
commitments.
5. Support and advice
I received helpful feedback from staff about my progress on
the course.
During my first year, I received sufficient support and advice
with my studies.
Over the course as a whole, I received sufficient support and
advice with my studies.
I found other students helped me with my studies.
There was good advice available to ensure that I chose things
that were best suited to me.
I had as much contact with staff as I needed.
Good careers information and advice were available.
6. Learning resources
Course materials were useful.
The library resources were good enough for me.
I was always able to access IT resources when I needed to.
When it was needed, there was sufficient access to specialised
equipment or rooms.
7. Overall, I was satisfied with the quality of the course
8. Overall, I feel the course was a good investment.
9. I would recommend the course to a friend
10. Looking back on the experience of your course as a whole, what do you consider to have
    been the best and the worst aspects of it? (Please use the space below and attach an
    additional sheet if necessary.)

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:9
posted:7/4/2012
language:
pages:51