Docstoc

Virtual learning environments and student teaching evaluations

Document Sample
Virtual learning environments and student teaching evaluations Powered By Docstoc
					Virtual Learning Environments and Student Teaching Evaluations
Simon Cassidy Directorate of Psychology University of Salford, UK
PLAT 2008

Virtual Learning Environments
Virtual learning environments remain one of the most significant tools for the development of teaching and learning practices which are able to accommodate and reflect the changing landscape of education and pedagogy: Student numbers Student diversity Student expectations Integration of information technology Issues of best practice

What are they?
►

A software system comprising a collection of tools / applications to enable online communication, collaborative learning, uploading content, student assessment, feedback and administration Originally developed for distance learning but now commonly used as part of a blended learning approach
Example – Blackboard is a commercially available, offthe-shelf system to enable the creation of a VLE without web development expertise

►

►

Purpose
To support, manage, enrich, enhance teaching, learning and assessment

Anticipated benefits of VLE (1)
► Increased

communication, interactivity and incorporation of collaborative pedagogical models; synchronous and asynchronous communication; international information sharing; shared passion and deepening of knowledge from ongoing interaction (Gannon-Leary & Fontainha, 2007)

Anticipated benefits of VLE (2)
Provides a means of improving the quality of learning opportunities ► Creates learning environments / learning not dependent on / without the restriction of time or space ► Take into account individual learning needs ► Greater responsibility taken by the student in the learning process ► Improved learning in a ‘social setting’ ► Enhanced learning environment ► Feeling of connectedness ► Learner centred
►

Evaluating VLE
(how) vs. outcome (what) ► Subjective (opinion) vs. objective (performance) ► User (student) vs. expert (usability, HCI) ► Learner perceptions (evaluations) vs. learning outcomes (performance)
► Process

Student perceptions studies
Positive Negative

Helps apply learning
Helps achieve learning outcomes Increases enjoyment

Used as an ‘online textbook’; a ‘safety net’
Does not assist / is not used for self-study Taught sessions adding nothing extra to the VLE

More positive feedback for elective than required modules and disappointing overall

The Study
– 5 years ► Pre and post VLE periods ► Data from student evaluations of teaching and learning (subjective, process + product) ► Quantitative and qualitative ► Indirect approach to VLE evaluation ► Compares required (core) and elective (option) modules
► Retrospective

Aims
► Establish

whether there is any evidence that the reputed benefits of VLEs are reflected in the student experience of teaching and learning any specific contributions made by VLEs in the context of teaching and learning

► Identify

Table 1. Overall student satisfaction before and after introduction of VLE supported teaching (Required)
Very / extremely satisfied Pre VLE 2003/4 (N=23) Post VLE (2005/6) (N=30) Post VLE (2006/7) (N=48) 40% (n=12) 38% (n=18) Satisfied Neither Dissatisfied Very / extremely dissatisfied 9% (n=2) 7% (n=2) 4% (n=2)

52% (n=12) 43% (n=13) 50% (n=24)

39% (n=9) 10% (n=3) 8% (n=4)

Post VLE (2007/8) (N=27)

52% (n=14)

19% (n=5)

15% (n=4)

11% (n=3)

4% (n=1)

Table 2. Would student recommend the module to an ‘interested friend’ (Required)
V. strongly/ strongly agree Pre VLE 2003/4 (N=23) Post VLE (2005/6) (N=30) 10% (n=3) Agree 17% (n=4) 57% (n=17) Neither 57% (n=13) 23% (n=7) Disagree 17% (n=4) 7% (n=2) V. strongly/ strongly disagree 9% (n=2) 3% (n=1)

Post VLE (2006/7) (N=48)
Post VLE (2007/8) (N=27)

21% (n=10)
30% (n=8)

46% (n=22)
41% (n=11)

15% (n=7)
7% (n=2)

18% (n=9)
15% (n=4) 7% (n=2)

Table 3. Students satisfied that teaching and learning

methods assisted learning (Required)
Very / extremely satisfied Pre VLE 2003/4 (N=25) 4% (n=1) Satisfied Neither Dissatisfied Very / extremely dissatisfied 4% (n=1)

36% (n=9)

32% (n=8)

24% (n=6)

Post VLE (2005/6) (N=27)
Post VLE (2006/7) (N=48) Post VLE (2007/8) (N=27)

41% (n=11)
27% (n=13) 46% (n=12)

41% (n=11)
53% (n=25) 31% (n=8)

14% (n=4)
10% (n=5) 0% (n=0)

4% (n=1)
10% (n=5) 19% (n=5) 4% (n=1)

Table 4. Student 1pooled satisfaction with staff, assessment methods and clarity of stated module aims (Required)
Very / extremely satisfied Satisfied Neither Dissatisfied Very / extremely dissatisfied

Pre VLE 2003/4 (N=99)
Post VLE (N=119) (2005/6) Post VLE (2006/7) (N=192)

8% (n=8)
43% (n=51) 37% (n=71)

39% (n=39)
44% (n=52) 45% (n=87)

31% (n=31)
8% (n=9) 13% (n=25)

18% (n=18)
5% (n=6) 4% (n=8) 9% (n=10)

3% (N=3)
1% (n=1) 1% (n=1) 4% (n=4)

Post VLE 37% 44% 6% (2007/8) (n=40) (n=48) (n=6) (N=108) 1pooled responses from 4 separate questions

Table 5. Overall student satisfaction before and after introduction of VLE supported teaching (Elected)
Very / extremely satisfied Satisfied Neither Dissatisfied Very / extremely dissatisfied

Pre VLE 2003/4 (N=13)
Post VLE (2005/6) (N=20) Post VLE (2006/7) (N=19) Post VLE (2007/8) (N=31)

85% (n=11)
60% (n=12) 26% (n=5) 55% (n=17)

15% (n=2)
30% (n=6) 42% (n=8) 45% (n=14) 10% (n=2) 11% (n=2) 21% (n=4)

Table 6. Would student recommend the module to an ‘interested friend’ (Elected)
V. strongly/ strongly agree Pre VLE 2003/4 (N=13) 77% (n=10) Agree Neither Disagree V. strongly/ strongly disagree

23% (n=3)

VLE 1st Period (2005/6) (N=18)
VLE 2nd Period (2006/7) (N=19) VLE 3rd Period (2007/8) (N=31)

56% (n=10)
21% (n=4) 58% (n=18)

33% (n=6)
32% (n=6) 39% (n=12)

6% (n=1)
26% (n=5) 3% (n=1)

6% (n=1)
16% (n=3) 5% (n=1)

Table 7. Students satisfied that teaching and learning

methods assisted learning (Elected)
Very / extremely satisfied Satisfied Neither Dissatisfied Very / extremely dissatisfied

Pre VLE 2003/4 (N=13)
VLE 1st Period (2005/6) (N=20) VLE 2nd Period (2006/7) (N=19) VLE 3rd Period (2007/8) (N=32)

62% (N=8)
45% (n=9) 21% (n=4) 47% (n=15)

31% (n=4)
55% (n=11) 32% (n=6) 34% (n=11)

7% (n=1)

21% (n=4) 19% (n=6)

21% (n=4)

5% (n=1)

Table 8. Student 1pooled satisfaction with staff, assessment methods and clarity of stated module aims (Elected)
Very / extremely satisfied Satisfied Neither Dissatisfied Very / extremely dissatisfied

Pre VLE 2003/4 (N=52)
VLE 1st Period (2005/6) (N=76) VLE 2nd Period (2006/7) (N=76)

63% (n=33)
45% (n=34) 20% (n=15)

25% (n=13)
47% (n=36) 42% (n=32)

12% (n=6)
8% (n=6) 28% (n=21) 9% (n=7) 1% (n=1)

VLE 3rd Period 54% 43% 3% (2007/8) (n=68) (n=55) (n=4) (N=127) 1pooled responses from 4 separate questions

Table 9. Comparison of student comments regarding module ‘dislikes’ (Required)
Pre VLE ‘Dislikes’ Post VLE ‘Dislikes’

More support (e.g. assignment, missed lecture)

YES

More detail re SPSS & statistics
Room changes (e.g. suitability, disruption) Time slot Seminars (e.g. poor attendance) Communication with students NO N/A N/A N/A N/A N/A N/A N/A

YES
YES YES YES NO Need more PRINTED materials / handouts Too much emphasis on Bb Too much information on Bb Organisation of Bb Bb unreliable Lecture notes on Bb so lost motivation to take notes concentrate [during lecture/seminar] Working on own on Bb is boring Need more / all lecture notes on Bb

Table 10. Comparison of student comments regarding module ‘likes’ (Required)
Post VLE ‘Likes’ Module organisation Thorough / detailed / depth of understanding Good links between lectures and seminars Assessments and assignment support / feedback Good supporting notes Relaxed approach Applied / relevant Staff Variety of teaching methods Pre VLE ‘Likes’ YES YES YES YES YES YES YES YES NO

Developed confidence
Enjoyable / interesting Bb (content, for reference, helped understanding)

NO
NO N/A

Table 11. Comparison of student comments regarding module ‘dislikes’ (Elected)
Pre VLE ‘Dislikes’ Not enough assignment support Post VLE ‘Dislikes’ YES

Room changes (e.g. suitability, disruption)
Timing (e.g. too long - lecture plus seminar) Seminars (e.g. poor attendance) Student diversity (lack of previous knowledge) Need more variety of teaching techniques

YES
YES YES YES NO

Limited books on the subject
Cancelled / postponed lectures N/A NO NO NO NO NO

NO
NO Lecture notes not always on Bb Lectures (taking notes, involve students more, too much information, provide handouts) Some transparencies not clear Some topics covered elsewhere Seminars (not always useful) Punctuality of some tutors

Table 12. Comparison of student comments regarding

module ‘likes’ (Elected)
Post VLE ‘Likes’ Content Structure and organisation Teaching / delivery (e.g. informative, helpful) Seminar / group work (e.g. well structured, interactive, useful) Variety / range of topics Pre VLE ‘Likes’ YES YES YES YES YES

Interesting
Understandable Staff (e.g. support, students-tutor interaction) Assessment (e.g. no exams, support, timing) Good use of Blackboard

YES
YES YES YES N/A

Clear and available supporting notes and material
Variety of teaching methods / styles / approaches All relevant to course Making notes [rather than being given handouts]

NO
NO NO NO

SUMMARY (1)
►

Students reported being more satisfied in a number of areas of teaching and learning following the introduction of a VLE to support delivery, e.g. communication, variety of approaches to T&L, interest and enjoyment, availability of supporting material, relevance The effect was most marked in relation to an increase in ‘very / extremely satisfied’ responses and a decrease in ‘neither satisfied nor dissatisfied’ responses following the introduction of the VLE The effect was [in the main] limited to the required module (Research Methods) and was not evident in elected module (Health Psychology) VLEs have a universal appeal but must consider further how best to design and implement them on the basis that the benefits may be dependent upon course type and existing levels of satisfaction (and other ‘Critical Success Factors’)

►

►

►

SUMMARY (2)
►

Student perceptions are important because they are related to motivation and learning The introduction of a VLE to support teaching and learning may have contributed to an improved student experience – but many potential confounding factors Measuring the impact of VLEs indirectly through student teaching evaluations is not entirely satisfactory given the many factors which could be reflected in such general teaching evaluations The study is limited in terms of limited sample information – individual differences such as computer / internet user self-efficacy and learning style which may play a major role in the efficacy of VLEs

►



►

SUMMARY (3)
 Acknowledge

that there may be ‘Critical success factors’ in design and implementation on VLEs
and ‘agreed’ framework for evaluation

 Clearer


				
DOCUMENT INFO