Report on Qualitative Data by tar19045


									Evaluation of the PAL Scheme at
    Bournemouth University

              By Martina Johnson

    Learning and Teaching Research Assistant
          Centre for Academic Practice
               Academic Services

                December 2005
Table of Contents

1. INTRODUCTION....................................................................................................1
2. QUALITATIVE DATA...........................................................................................1
  2.1 FIRST-YEAR STUDENTS ..........................................................................................1
  2.2 PAL LEADERS .......................................................................................................3
3. QUANTITATIVE DATA.......................................................................................6
  3.1 FIRST-YEAR STUDENTS ..........................................................................................6
  3.2 PAL LEADERS ......................................................................................................9
EVALUATION ..........................................................................................................11
  4.1 FURTHER ISSUES ..................................................................................................11
1. Introduction
In order to evaluate the usefulness of the 2004/2005 PAL programme for both PAL Leaders
and first-year students, questionnaires were designed and distributed to all PAL Leaders and
first-year students. This is a report on the data gathered from the questionnaires, and the
actions taken as a consequence.

An electronic questionnaire was set up using the Bristol Online Survey Tool (BOS). It was
available on the Internet (internally and externally) from 18th March 2005 until 6th May 2005.
Course PAL contacts were requested to encourage their PAL leaders and first-year students
to complete the questionnaires. Five £10 book tokens were offered as an incentive to fill in the

13 PAL Leaders and 50 first-year students took part in the survey. This low response rate
may have been due to the fact that the questionnaire went online at the same time as the
TLAS questionnaires, and students may have been suffering from questionnaire fatigue. To
increase the response rate for the 2005/2006 questionnaire, the aim is to publish the
questionnaires online by mid-February 2006 at the latest.

2. Qualitative Data
2.1 First-year Students
2.1.1 Helpfulness of PAL Programme

38 of 50 first-year students felt the PAL programme had helped them to integrate more
quickly into university life mainly because they got to know other students in an informal
        “the activities at the start made the group bond better”,
         “It helped me to meet new people on the course, especially people that were not in
        my seminar group” and
        “helped cultivate a strong bond with others”,
         “helps you to relax and participate in other seminars and lectures”.

Other students did not feel it helped them to integrate into university life:
        “not really into uni life”,
        “[I needed] more help with library skills“,
        “it would have been more helpful to have sessions in the access centres of the library

44 of 50 students thought the PAL Programme helped them gain a clearer understanding
of course direction and expectations. Students found it helpful to be told about
expectations by someone who had been through it before:
        “Definitely, since PAL Leaders had experience with the direction and expectations of
        the course”,
        “talked about experiences from their first year”,
        “passed on her own knowledge of how best to approach things” ,
        “especially helpful (…) where the PAL leaders would show us examples of previous
        “from people who had done the course”,
        “explained what the course is like in second and third years”,
        “good to see what they are doing next year”.

Those students who did not feel it helped them get a clearer understanding of the course,
mentioned that
       “the PAL leaders had exaggerated how tough the course would be”

PAL04/05 Evaluation                            1                                  Martina Johnson
         “found out for myself”,
         “the handout at the start of term (…) outlines the direction of the course”.

33 out of 50 students felt it did help them develop their study and learning strategies. They
felt the PAL Leaders
         “were able to advise on the best ways to study and learn”,
         “help with using the library”,
         “revision tips very helpful”,
         “making essay plans and going through material for essays”.

The other 17 students felt it didn’t help, as
       “most have retained these study and learning skills from GCSE and A-level work”,
       “No real study or learning strategies were put forward to us through our time at PAL”,
       “we haven’t really concentrated on that”,
       “not so much help given here”.

38 of 50 students felt it helped to improve their understanding of the subject matter of
their course, commenting that PAL leaders are
        “someone on your level”,
        “they have a similar mindset”,
        “give us their viewpoint”,
and that they often explained things in
        “a more simplified way”, or in
        “simple plain English” and
        “in a way that I feel that the students relate to”.
Students found it helpful to
        “clarify matters which the seminars left unclear” and
        “reiteration of topic areas, detailed advice from PAL leaders who have done the same
        “doing past papers” and
         “learn and relearn information in teams”.

Those students who did not feel that the PAL programme benefited their understanding,
commented that the PAL leaders
       “didn’t teach us anything just guided and referred us to books”,
       “no new information was given (…) so my understanding did not grow”.

40 of 50 students felt that the PAL programme helped them prepare better for assessed
work and examinations. Students commented that
        “there was a lot of exam tips and assignment tips”,
        “it has encouraged everyone in our class to share ideas”,
        “seeing past papers and the way they’ve been graded”,
        “revision techniques”.

2.1.2 The best and worst things about the PAL Programme

Students were asked what the best thing about the PAL programme was. Their answers
mostly involved the “relaxed” and “informal atmosphere” in which PAL took place, and the fact
that the PAL leaders were
         “in the same position”,
         “on your level”,
         had a “student perspective”,
         and could explain things “in a more accessible language”.

When asked what the worst thing about the PAL sessions was, a lot of students felt that
student attendance was low. The time PAL sessions were held also featured heavily. Several
students also mentioned that the PAL sessions
        “weren’t planned and poorly structured”, or

PAL04/05 Evaluation                              2                                  Martina Johnson
       “not always knowing what we were doing the next week”, and
       “lack of organisation”.
One student didn’t like “not knowing quite how to behave, was it a formal seminar or a bit of a

2.1.3 Improvements

Regarding possible improvements to the PAL sessions, students’ main suggestion was for
them to be
        “more structured”,
        “some kind of agenda (…) of PAL weekly content”,
        “a clear outline”,
        “more planning”.
Other suggestions were made concerning the level of communication with proposals of
        “a separate folder on build for pal”,
        “a discussion forum on BUILD”.

2.2 PAL Leaders
2.2.1 Helpfulness of the PAL programme for first year students

12 out of 13 PAL leaders thought the PAL programme helped students integrate more
quickly into university life, as they could
        “ask questions which they wouldn’t normally be able to”,
        “became confident”, and
        “had someone to relate to who has been in the same position”.

All 13 PAL Leaders felt the programme helped students to get a clearer understanding of
course direction and expectations, as the students
        “are able to learn from our experiences”,
        “it’s a relaxed environment”,
        the PAL leaders could give the students “experience and advice from last year”, and
        “how it will be used in their second year”.

10 of 13 PAL Leaders thought that the programme helped students develop their study and
learning strategies. PAL leaders thought the programme provided
        “a good basis for the students to share how they learn with one another”, and
        advice on “using library and other info sources for references”,
        “a wide range of learning activities” were provided and
        “different ways of studying” were touched upon.
However, some leaders also said that
        “it could have been better, we concentrated more on the course content” and
        “most of the Group were happy with arranging their own study strategies” and
        “many students (…) already have study and learning strategies”.

As regards whether the PAL programme helped students improve their understanding of
the subject matter, 11 PAL leaders thought it did.
       “It gives the students (a) chance to understand issues that they may have been too
       worried to ask a lecturer” and
       the leaders “could keep explaining in different ways”, and
       “other students helped their peers a lot to improve understanding”.

12 of 13 PAL Leaders thought it helped students prepare themselves better for assessed
work and examinations. The students were able to
        “learn from our mistakes”, and
        were also told “what we did (both wrong and right)”,
        “what was expected in assignments”,
        “grading criteria”.

PAL04/05 Evaluation                            3                                  Martina Johnson
2.2.2 Personal and transferable skills

PAL leaders felt they had developed the following personal skills from their experiences as a
PAL Leader: 9 of 13 PAL leaders felt they had developed confidence (“I am much more
confident in myself and my ability to stand in front of a group”), and 7 of 13 felt their
communication skills (including listening skills, articulation of ideas) had improved, and 3 of 13
felt they were more organised (“my organisational skills have improved a lot, which has
helped me in my degree as I organised my work a lot more efficiently”). Other skills
mentioned were tolerance, assertiveness, willingness to help, approachability, and patience.

Asked about the professional and transferable skills they felt they had developed, 6 of 13
students felt they had improved their communication skills, several again mentioned
confidence, organisation and presentation skills (“I feel that I could quite easilygive
presentations or speeches now”) , others felt their leadership skills, and leading/participating
in group discussions had improved.

2.2.3 Two-day training programme (Sep 2004)

10 of the 13 PAL leaders felt the 2-day training programme prepared them effectively for the
PAL sessions. They felt the most useful things about the training were the practice PAL
sessions, and the manual (“the folder was a bible”), also
        “what our role was”, and
        “identifying exactly what PAL was so we knew we weren’t expected to be teachers”.

They felt that in addition to the training provided
        “having informal chats with previous PAL leaders” and
        “more activities with previous PAL leaders”
would have been useful.

2.2.4 Follow-up training sessions

The PAL leaders felt that the follow-up training session on “reflections on skills developed
through being a PAL leader” was the most useful, followed by note-taking, study habits, giving
presentations, essay writing, assertiveness, and the OCN portfolio. Students felt that these
sessions were not only beneficial for use in the PAL sessions, but also for their own personal

2.2.5 Support

All PAL leaders felt the support given by Hugh Fleming/Alison Green was very helpful. In
most cases the support given by course PAL contacts was felt to be helpful. The support
provided by other members of teaching and administration staff in the schools was variable
across courses; however, where provided, it was felt to be valuable.

2.2.6 Best and worst things about being a PAL Leader

Students were asked what the best thing was about being a PAL Leader. Their answers
        “making friends” and “meeting people”,
        “feeling valued” and “helping people”,
        “huge confidence boost”,
        “positive feedback from the Group”,
        “great to talk about at job interviews”.

PAL04/05 Evaluation                             4                                   Martina Johnson
PAL leaders felt the worst things about being a PAL leader were
       “poor attendance” and “trying to get the students to participate and interact”,
       “time consuming” especially “when you are really busy with your own work”,
       “trying to organise some structure for the session”.

2.2.7 Future PAL programmes

Asked about the one piece of advice they would give future PAL leaders,
       “be prepared” featured often, as well as “always have a stand by plan”
       “not take it personally if people don’t turn up”, and “don’t give up”.

Improvements that could be made to the PAL programme included
       “more structure”,
       “better communication between other PAL leaders”,
       “regular meetings with tutors” and
       “to ensure that timetabling and room changes are minimal”.

PAL04/05 Evaluation                            5                                  Martina Johnson
3. Quantitative Data
3.1 First-year Students

50 first year students from 8 courses completed the first-year student questionnaire. The PAL
scheme was run on 13 courses, but there were only respondents from 8 of the 13 courses. It
is regretted that there were no respondents from the School of Services Management from
the ‘Hospitality Management’, ‘Leisure Marketing’ and ‘Retail Management’ courses. This may
represent a problem with accessing the e-questionnaire through the Services Management
Intranet site SIM.


                           1 / 31                                 Accounting and
                                    4 / 91                        Business
                                               3 / 30             Management
                                                                  Management (PT)
                                                                  Visualisation and
                                               6 / 53             Animation
        26 / 184                                                  Multimedia
                                                        2 / 17    Communication
                                                      2 / 68      Hospitality

                                             6 / 50               Retail Management
                                                                  Leisure Marketing

Figure 3.1.1 shows the number of respondents from each course.The numbers (x / y)
represent the number of respondents (x) compared to the number of expected respondents
(y) (i.e. the number of students who took part in PAL from the particular course).

PAL04/05 Evaluation                              6                                 Martina Johnson
Figure 3.1.2 shows the respondents broken down according to schools. The numbers (x / y)
represent the number of respondents (x) compared to the number of expected respondents
(y)(i.e. the number of students who took part in PAL from the particular school). As mentioned
above, there were no respondents from the School of Services Management:

                                                   1 / 172

                                                                                          SM (0 / 132)
                                          8 / 118

                           3 / 30

                                                                      38 / 345

Figure 3.1.3 shows the attendance pattern of all students. Only 1 student attended less than 3
sessions, and about half (24) of the students attended 15 or more sessions.

                                          Student attendance (all)


 Number of students




                              3 or less          4-9              10-14      15 or more

PAL04/05 Evaluation                                                               7                Martina Johnson
Figures 3.1.4 to 3.1.8 show attendance levels broken down into schools.

                                       Student attendance: IBAL
                                                                                                                                                 Student attendance: BMS


 Number of students


                                                                                                           Number of students

                       5                                                                                                         5

                       0                                                                                                         0
                           3 or less          4-9                10-14   15 or more                                                        4-9               10-14         15 or more
                                                    sessions                                                                                               sessions

                                       Student attendance: DEC
                                                                                                                                       Student attendance: ConSci


 Number of students

                                                                                      Number of students


                       5                                                                                    5

                       0                                                                                    0
                                                    15 or more                                                                       4-9                    10-14          15 or more
                                                    sessions                                                                                             sessions

Participants were asked whether they felt that the PAL sessions helped them
        - integrate more quickly into university life
        - get a clearer understanding of course direction and expectations
        - develop your study and learning strategies
        - improve your understanding of the subject matter of your course
        - prepare yourself better for assessed work and examinations
There were significant relationships between attendance level and all the above, in other
words, those students who attended more sessions felt it helped them with all of the above
issues. However, only very few students from DEC, BMS and the School of Conservation
Sciences took part in the survey, so the results can only be considered significant for students
from IBAL.

PAL04/05 Evaluation                                                                   8                                                                                           Martina Johnson
3.2 PAL Leaders

13 PAL Leaders from 7 courses took part in the survey. None of the PAL leaders on the
following courses took part in the survey: ‘Accounting and Finance (IBAL)’, ‘Business
Information Systems Management (IBAL)’, Business Information Technology (DEC)’,
Computer Visualisation and Animation (BMS)’, ‘Multimedia Communication Systems (DEC)’,
and ‘Retail Management (SM)’.


                      1/4                                      Management (PT)
                                  1/4                          Communication
                                          1/4                  re Engineering
                                                               Leisure Marketing
          4 / 13                                               Accounting & Finance
                                                               Business Information
                                            2/4                Systems Mgment
                                                               Business Inforamtion
                                                               Computer Visualisation &

                          3/7                                  Multimedia
                                                               Communication Systems
                                                               Retail Management

Figure 3.2.1 shows which courses the respondents were PAL leaders on. The numbers (x / y)
represent the number of respondents (x) compared to the number of expected respondents
(y) (i.e. the number of PAL leaders on each course).

PAL04/05 Evaluation                         9                                  Martina Johnson
Figure 3.2.2 breaks down the respondents according to their schools. At least one PAL
Leader from each school took part in the survey. The numbers (x / y) represent the number of
respondents (x) compared to the number of expected respondents (y) (i.e. the number of PAL
leaders from each school).



                                            5 / 26

               4 / 17

                 2 / 14

Table 3.2.1 shows the usefulness of the follow-up training sessions in order of usefulness.
The numbers represent number of students out of 13.

                      Most useful                    useful                      Least useful (or
                                                                                 didn’t attend)
Reflections           6              2               2           0               3
Note-taking           5              2               3           0               3
Study habits          4              3               0           2               4
Presentations         4              2               2           0               5
Essay writing         3              3               2           1               4
Assertiveness         2              2               3           0               6
OCN Portfolio         1              2               4           1               5

PAL04/05 Evaluation                            10                                    Martina Johnson
4. Changes implemented in PAL as an outcome of this
    1. To encourage attendance PAL leaders are invited to promote their sessions via
       different media, e.g. email, notices, announcements, reminding students of the topic
       in the next session.
    2. Course PAL contacts now receive attendance sheets for PAL to monitor attendance
       and where necessary, be proactive in encouraging students to go to their PAL
    3. During their training, PAL leaders are encouraged to agree a programme of topics
       with their students and email them a reminder of the topic(s) to be covered, so
       students can come to the sessions prepared to discuss the specified topic. Leaders
       are also encouraged to plan their sessions in advance so that they are friendly but
       also structured and purposeful. They are also encouraged to reflect on their sessions
    4. It is beneficial for PAL leaders to receive teaching schemes and assignment
       deadlines from course PAL contacts at the beginning of term so that they can plan
       the PAL sessions according to the course. This will ensure that likely PAL topics can
       be anticipated and will help PAL leaders better prepare for the sessions.
    5. Course PAL contacts have been encouraged to meet more frequently with the
       leaders, fortnightly initially, moving to once per month as the Leaders confidence
       grows. Where this works well, it has ensured that PAL integrates closely with other
       course activities, optimising the benefits of the scheme for students, leaders and the
       teaching team.

4.1 Further issues

It may be beneficial to provide some space on each school’s virtual learning environment
(VLE) for students and leaders to further discuss issues outside of the actual session. It could
also be used to set tasks, which should be completed for the next PAL session, or to store
articles for students to download and read, i.e. as a resource base.

PAL04/05 Evaluation                            11                                 Martina Johnson

To top