Docstoc

Kristina Everett - Macquarie University

Document Sample
Kristina Everett - Macquarie University Powered By Docstoc
					EXAMPLES OF MACQUARIE PRACTICE
Assessment and Feedback Project
Case study: Kristina Everett
Multi-modal assessment design: Creating a space for transforming students‟
experiences in Indigenous studies


 Discipline              Warawara


 Delivery mode           D1; 3cp


 Year level              ABST200 and ABST300


 Overview of Unit: We begin this unit with a brief review of the historical, political and
 sociological context of Indigenous studies in post-colonial Australia before we move on
 to a more global perspective.
 We then begin formulating conceptual frameworks for the ways in which Indigenous
 studies can proceed. We do this by examining the ideas that have shaped
 understandings of the colonial encounter between Indigenous peoples and their various
 colonisers the world over and conduct some comparative analyses of specific cases. In
 particular we will be concerned with understanding the idea of colonialism in the 19th and
 20th centuries and post-colonialism in the 20th and 21st and how these relate to ideas of
 race and culture.
 It is highly recommended that students undertaking the unit have some familiarity with
 the concepts of imperialism, colonialism and post/neo-colonialism.


 Summary: Kristina Everett is the convenor for all Indigenous Studies units at Macquarie.
 When she began teaching at Macquarie a few years ago, it became very apparent that
 students enrolled in Indigenous Studies units were often International students, looking
 for “some kind of touristic experience of the exotic” Kristina says. The marketing of the
 unit became an important focus while they successfully attracted more local students into
 Indigenous Studies.
 The assessment of the course has undergone a revival too. After participating in a
 leadership in assessment program and the Foundations in Learning and Teaching (FILT)
 program at Macquarie, Kristina and her colleagues were motivated to redesign the
 assessment across all Indigenous Studies units. Using the assessment matrix tool,
 Kristina could see that only two types of assessment were being utilised and they were
 usually due in the middle and at the end of semester.
 Now, the assessments in the Indigenous Studies units are aligned with the graduate
 capabilities and the individual unit learning outcomes, an important part of the redesign
 process Kristina explains. Kristina can see that the Indigenous studies units are incredibly
 challenging and are successful in getting students “to stand outside the boxes that
 they‟ve always been in”. Kristina attests this success to a change in thinking around
Learning and Teaching Centre
 course design, saying “I used to think we could do it through our course content, now I
 think we‟re doing it through the assessment.”



How long have you been teaching at Macquarie?
I‟ve been here at Macquarie full-time since the middle of 2007, but I‟ve worked here
on a casual basis since 1998. While I was doing my PhD I taught here. I was
involved in writing a lot of the Indigenous studies units over the years.

What is your role in the Indigenous Studies programs at Macquarie?
I convene all of the Indigenous studies programs, including postgraduate and
honours convenor (with two honours coursework programs). I do like to be as
involved as I can be in marking and student contact and engagement – basically
student support throughout the units. I always have access to the Blackboard to see
what‟s happening, how students are responding to the readings and how the courses
are going.
I have also been recruited into the Leadership Excellence and Assessment Program
(LEAP). I‟m also on the Assessment Working Party for the University as well in
helping to put together the assessment policy for the University.

Where is Indigenous studies situated in the University?
We‟re situated in the social sciences. Until more Indigenous academics start coming
in and talking about what they actually want Indigenous studies to be in Australia, it‟s
currently very much a branch of critical cultural studies. We maintain the kinds of
rigours and standards that come from that and from other multi-disciplinary areas,
drawing from disciplines as diverse as anthropology, sociology, philosophy,
psychology and history.

Could you introduce me to the ABST units?
We‟ve been trialling a lot of different assessment tasks across all of our units, but the
assessments for both the 200 and 300 level subjects have turned out to the most
successful for lots of reasons.
We‟re hoping to turn ABST300 into our capstone unit. So, 300 is already the fulcrum
unit in an incremental learning program. It begins with the introductory ABST100, it‟s
developed in 200 and then it becomes part of this coherency into 300.
It already addresses graduate capabilities by assessing for critical, analytical and
integrative thinking, problem solving and ethical research capabilities, effective
communication, social responsibility, and action and creativity. Our stated graduate
capabilities include demonstrated ability to think and act outside the constraints of
taken for granted Western cultural assumptions in relation to issues in Indigenous
contexts.




Learning and Teaching Centre
What were some of the issues you found when you started convening
the Indigenous Studies units?
One thing that has always struck me as a real problem in Indigenous studies is that
up until quite recently, we have had an enormous percentage of our students as
overseas students – 80 percent of our students in 2004 were overseas students.
We‟ve been trying to deliberately attract more local students because from the point
of view of our political agenda, it‟s just so important to educate our local students,
Indigenous and non-Indigenous, but especially non-Indigenous students about
Indigenous studies. I‟m really happy to tell you that this year 49 percent of our
student body is local and we‟re continuing to be active in recruiting more and more
local students.
We try to keep things very standardised in Harvard referencing and the rigor of
creating an argument and supporting it and substantiating it with research. It was
really clear to us that our students were simply not doing that. They were coming into
third year without the skills they needed to be able to properly research their
assessment tasks. It was also really, really clear that when I first came here, many of
the units had two assessment tasks, and they were often 50 percent essays. When
students don‟t have these skills, they were just set up to fail the essays.
It was really clear to me that we needed to train students throughout the semester
with some tasks that they kept repeating and of course, I believe in incremental
learning.
The program is set up so that there is incremental learning across units. So from first
to second, second to third there are all kinds of progressive learning of particular
threads of knowledge. But within each unit as well, it‟s set up so that students
gradually build capabilities in certain areas. That research area was one that was
simply not happening, even though the material was providing them with incremental
learning vertically, they weren‟t able to engage in a repetitive task that provided them
with the skills they needed to do a larger task at the end.
The other thing that stuck me when I was putting together the postgraduate
coursework framework was the way postgraduates were completely left to their own
devices and undergraduates were absolutely spoon-fed with these readers and all
the resources they needed. It seemed to me that logically the undergraduates were
the ones that needed to build up their capabilities, particularly in research and then
when they got to honours or postgraduate, that‟s when they needed the support and
perhaps a little bit more spoon-feeding having already gained the skills they needed.

So when you realised that students didn‟t have the skills to do two 50
percent essays and didn‟t have the skills to find the sources that they
needed, how did you recognise that? Was it purely that they weren‟t
doing well on their assessment tasks?
It was that, but it was also in the way we‟ve structured the unit. We have an hour
lecture and a two hour seminar. There‟s also another assessment task that‟s related
now to this ongoing assessment task where one student presents a paper in class
that‟s been properly researched and prepared and then the next week presents it as
a written piece of work.
Learning and Teaching Centre
It was clearly evident from those assessment tasks and from the presentations that
the students weren‟t going beyond the reader that was provided. If there was any
further research, it was to things like newspapers and sources along those lines –
which I don‟t mind, but it seemed to me this opportunity to really become academic
researchers, or to even understand what academic research was about, was being
missed.

So the LEAP program helped you to see some of the problems with the
types of assessments being used across all Indigenous Studies units?
Yes. What was very fortunate was through the LEAP research, I had a group of
people from all levels of the University, so there were some very senior people right
down to me. We were all a group of colleagues that helped each other with
questions.
The idea was that it was an action research project based on critically reflectively
evaluating assessment in our Department. It was really clear to me we had a long
way to go. The first part of the project was to create a matrix – collect all of the unit
outlines, go through the unit outlines, look at the different kinds of assessment tasks
that were being offered and when, and to map it. So that‟s what I did and I colour
coded it.
When I looked at it, we practically only had two colours which showed that there
were only two kinds of assessment tasks happening, and they were all lumped in the
middle and at the end of semester, so that students have this huge workload in the
middle and at the end of semester. I wouldn‟t be surprised if all their other subjects
looked similar. So the poor things were probably having nervous breakdowns! We
were doing nothing to help them. So that was clear right from the beginning.

In terms of rethinking the assessment, did you work with the teaching
team?
I was very fortunate back then because we had quite a few new members of staff
and four of us did the Foundations in Learning and Teaching (FILT) Program here –
we did it all together. It was really excellent, because we all had that same
knowledge base at the same time when it came to doing things like peer review.
We sat down together and peer reviewed each other‟s unit outlines; worked together
to work on the ones that we were responsible for; did some peer review of each
other‟s teaching and lecturing; and we‟ve been going on from there to the extent that
we‟ve reviewed just about every unit outline each semester since I‟ve arrived.

In terms of the actual assessment weighting, could you describe the
weightings for each assessment part in the units?
Well, we‟re still working this out with the ongoing research task because we do
change it and refine it each semester.
Each unit has the seminar question where it has to be verbally presented and
researched and then written up, and that‟s normally around 30%.
Then we have another writing task that involves research worth 40%, and it depends
on the unit and it depends on what we‟ve read what each person in the team has

Learning and Teaching Centre
been thinking about but for example, we‟ve had things like critical analyses of
websites, just essays, or a letter to a friend – a critical reflection on what has been
learnt over the 13 weeks of the unit.
Then we have the ongoing research task where the students are assessed on their
10 best weeks of contributions in class or on the blackboard and are given a mark
out of 10 each week and they‟re given a mark out of 10 for the relevance of the
citation and the demonstration.
The mark for the citation is really money for jam, because if they put in a relevant
citation, that‟s from a database that can be proven to be relevant and it does all the
things it‟s supposed to do and is used effectively in the discussion, then that‟s marks.
The marks out of 10 for both sections of that are then scaled to become 30% of the
overall mark.

Could you explain in more detail the weekly ongoing research task?
Students are provided with citations each week for searching essential texts using a
library database. But they also need to supplement these – so they‟re not given a
reading book, they‟re not given a reader – they‟re just given a list of the texts that we
know are in the University database, and they have to find those themselves and
usually print them off. They need to use those and show evidence of having found
them in class. So they need to supplement these essential readings each week with
another one that they‟ve researched themselves from different journal databases. So
they‟ve got to be able to demonstrate that they‟ve gone to the one that we know, but
they‟ve also used other journal databases.
The reading that they source themselves must be relevant to the topic of the week, it
must be correctly cited using Harvard referencing, and posted onto the unit‟s
electronic blackboard site.
The students must use both the reading they source themselves and the prescribed
reading for that week to respond to a seminar question which is posted on the
electronic blackboard each week. So the same topic is verbally discussed during the
weekly face-to-face seminar.

So how do students respond to the seminar question based on the
reading they‟re sourced themselves?
Students have a choice of whether they want to participate entirely in-class verbally,
or whether they post their discussion of the topic using the readings that they‟ve
sourced themselves and the ones that we‟ve provided on the discussion board.
What normally happens is that during the face-to-face discussion, students often get
very interested and carried away and sometimes go off on a tangent and they feel as
though they haven‟t been able to properly discuss the article that they‟ve sourced
themselves and they want to demonstrate that they‟ve done the work. Often what
they‟ll do is, even if they have participated in the class, they‟ll go online and put up
extra information to demonstrate their literacy in that area of study in relation to the
reading that they‟ve sourced themselves and in relation to what they‟ve learnt from
their peers during the seminar.



Learning and Teaching Centre
What that also does then is stimulate discussion between students online. We‟ve still
got one discussion thread going from week one, and we‟re coming into week nine.
Students are already posting discussion for the week eleven topic. So it gives
students a chance to get ahead of the game and get marks. If they‟ve got a heavy
assessment load at the end of semester, they can actually complete this task earlier
in the semester online if they want to.

What are the benefits to students‟ learning in an assessment task such
as this?
This task has proven to be highly effective in developing students‟ library research
skills and for reviewing and critically comparing material as it is presented each
week. It‟s also effectively created a database of readings for the unit which is
collaborative contributed to by all the members of the group. The task is currently set
for 200 and 300 units, and it will be introduced to 100 next semester.
As a capstone research project, it‟s envisaged that the students engaged in an
Indigenous studies major could archive their databases from 100, 200 and 300 level
units and use these databases to critically reflect and synthesise all they‟ve learnt
over the core units of the major.
They‟ll then be able to review the database and use it to answer a research question,
hopefully reflecting an authentic problem for research in Indigenous contexts. So this
project equips students for future careers as researchers and workers in Indigenous
contexts regardless of what that work might be. And it helps them to understand how
research, research products and research questions fit together and how they can
operate as lifelong independent learners.

What is the workload like for the
teaching staff?
Some of our students, particularly our Indigenous students who are in remote areas,
have taken some of these units and have done them entirely online. But because of
the amount of discussion that is generated online by the weekly assessment
requirement, there‟s actually no great amount of extra support needed for the remote
students. And the students found that they were interacting with each other to such a
level, with the Indigenous students leading so much of the discussion that they were
surprised at how independent their learning was. The other thing is that it this
approach is very inexpensive. Students don‟t need to spend money on readers and
they really don‟t need to enter a library. But at the end, they‟ve got this amazing
database of readings. And that‟s very useful for me in setting the next course.

How do you assess their contributions
in class?
Contributions in class are assessed with the same criteria that we use to measure
their contribution on the discussion board. All the criteria are written in the study
guides, so the students know before they complete the task.




Learning and Teaching Centre
So it‟s a formal assessment situation in class when they choose
something to discuss and they go out the front of the class and talk
about it?
Yes. Even if it‟s an open discussion, again, that‟s still a formal part of it where they
can either put in an entry onto the discussion board or talk in class, they have to
identify when they‟re talking that it is their contribution to the discussion. They flag
that.

So the tutor would have a stack of those assessment criteria papers
ready to start marking if someone says „this is my contribution for the
week‟?
That‟s exactly right. One thing I‟ve found that has been really important is to keep
reiterating to the students to read the criteria. They get so carried away – they get
incredibly passionate about the subject material, and if they do flag it that they are
speaking for their weekly assessment, then I have to remind them that they have to
stick to the aspects of the criteria in their response.

It seems that to have 10 of those per student that it would be an
enormous workload?
It isn‟t for most of them. Only some of them will choose to do the weekly assessment
verbally, and many of those who flag a verbal contribution often contribute to the
blackboard as well.
What we‟ve got at the end of the course is this huge amount of data and evidence of
the ways in which the students have engaged with the task. And the thing is, when
we average it all out, it‟s only 30% of the mark, yeah, it does seem like a lot of work
for the students. But not so much for the teaching team, because the way we mark it
up, we do it like a rubric in class so it is more ticks and things like that. Whereas
when we‟re marking an essay, we write comments under each of the criteria as
feedback.
I must admit, even on Blackboard, when the students are discussing, we will often
put in a comment in response to their discussion especially if they‟re off the track, but
often what they say is incredibly productive and generative. As teachers, we almost
can‟t help but to engage with it ourselves. Yeah, maybe it is work, but it‟s not the kind
of work that is tedious and not something I identify as an enormous amount of work.
Our tutors haven‟t complained at all, in fact, I think they find it very productive.




Learning and Teaching Centre
Can you talk about the underlying principles you‟ve drawn on to re-
design the assessment? What philosophy of teaching and learning have
you been guided by?
The number one thing for me was that it needed to be student centred - flexibility for
students was important.
From an intuitive point of view and from the passion that comes with teaching the
subject, I wanted the students to be able to experience the huge pleasure that comes
from suddenly realising that there are different ways of thinking, and different ways of
being in the world.
I think that innovative assessment is just the most wonderful thing for allowing
students to experiment with the ways in which they think. And being involved and
being an active participant in that, regardless of whatever other problems they might
have.

Are you happy overall with the assessment and how it‟s going?
Every now and again, you see something not working. Something I realised was that
the students contributing online are getting constant feedback all the time from us,
while the ones contributing in class aren‟t – we‟re just ticking the rubric. What we
need to do is give them a rolling mark. We should give them a mark in week 4, week
8 and week 12 so that they know how they‟re going.

Why do you think it works?
I think what we are achieving is independent learning. People are taking
responsibility for their own learning and because of that they are invested in the
subject matter.
Can you sum up how your assessment has contributed to your goals for
student learning outcomes in Indigenous Studies?
Students may come to our subject with the idea that they are going to get some kind
of touristic experience of the exotic or something. But when they suddenly realise that
our program is actually more about post-colonial studies, and I often think rather than
Indigenous studies it‟s more about non-Indigenous studies – we‟re actually looking at
ourselves.
I think some of the ideas that students are confronted with, particularly white
students, are incredibly challenging. And once they start to think about that, they start
to reflect on other things and other aspects of so called reality.
I think we achieve our objective of getting students to stand outside the boxes that
they‟ve always been in. I used to think we could do it through our course content,
now I think we‟re doing it through the assessment.




Learning and Teaching Centre
In terms of assessing your assessment approach, have you done any
formal review?
Yes, a lot of review. We‟ve reviewed through various kinds of processes, the main
one being peer review.
Certainly we‟ve also drawn on critical reflection, TEDS – early TEDS as well as end of
semester TEDS. I‟ve found that TEDS can be so much after the fact – I worry more
about the students I‟m actually teaching than the ones I‟m going to teach, so I prefer to
have early TEDS.
I don‟t want to bother students with TEDS too much, but at the same time, I‟ve found
the early TEDS to be really, really useful in adjusting some of the things that they
were having problems with.

Did you design specific questions to ask students in the early TEDS?
We‟ve just used the standard questions provided by the Learning and Teaching
Centre. As well as that, our own critical reflection has helped us to see that students
need to know how they are going sooner and we need feedback on how they are
travelling sooner, that‟s why we‟re going to start giving them a rolling mark for their
research component.

Was there anything else that stood out in the TEDS and other self
assessment?
Not really, the 200 and 300 level TEDS we received were overwhelmingly positive.
We had more criticism at the 100 level, but it tended to me more things that were
related to conceptual issues.
In the 100 level, because the material that we‟re working with is so confronting,
sometimes students do resist it. And I think the key to that will be reworking 100 level
again to align it even more carefully, so that students can access what it is we‟re
talking about. I‟ll give you an example – we‟re talking about engaging with Indigenous
writers and authors, if we do that as a critique some students will see that as racist.
And without realising that by not engaging with Indigenous writers and authors, that‟s
the racist thing. So I think we need to be much more clear about the intentions and
clear about giving students access to the things we‟re doing. And more careful
alignment might do that.
Another thing that came out in the TEDS was that we had way too many learning
outcomes in all of our units, so we‟ve been honing those.

And was that affecting the way you were designing assessment?
Yes, it was affecting the criteria. Just the amount of marks we had to allocate to
particular parts of an assessment task – it was impossible because with too many
learning outcomes, if you‟re trying to assess for all of them, you end up giving
miniscule amounts for way too many things, when some things needed to be
weighted more than others. So basically the best and fairest way to deal with that is
to limit the number of learning outcomes.



Learning and Teaching Centre
With the curriculum renewal process when we were defending Indigenous Studies as
a major, we looked at all of them together and started comparing criteria across all of
our units, and we saw that way too many of our learning outcomes were the same at
all the different levels. But even though they were the same but they showed
incremental learning across them, they were still way too similar. So we needed to
change those.
That comparing across units was a very, very useful exercise as well. I‟d never really
done that since I‟d designed the program, had I really looked. Because what
happens every time somebody teaches a unit, they rewrite it to suit their area of
expertise and change it to suit updates in research, and if you don‟t keep an eye on it
overall, sometimes the units end up evolving in all different ways and they‟re not
actually achieving the kinds of incremental learning that we‟re looking for. And that
had happened to an extent, so that was a really useful exercise.

For other Macquarie teachers who are looking to rethink their
assessment, what advice, tips, tricks or advice would you give?
Certainly for us, one of the main things was working together. Sit down as a group of
teachers and think about things collaboratively. For us, we‟ve always been very
much standards based as far as our assessment is concerned. We‟ve always
resisted this idea of marking according to a bell-shaped curve.
So I think you need to sit down as a Department and think about things like the
standards, what your graduate capabilities are, and what are your core values as a
Department are. For us, we already had very clear political and ethical
considerations and frameworks that not only wanted to work within but are absolutely
core to how we understand what we‟re doing here.
After that, think about how you are going to achieve that with the student body that
you‟ve got. And what are the strengths and weaknesses that you‟ve noticed over
time in your student body. And that will be different with every Department depending
on the students that they are drawing from. It‟s very specific for us because we don‟t
count as a qualification for anything. So the students that come to us, come to us for
all kind of reasons. What was clear, particularly when we had such a huge group of
overseas students, was that tourism was a major reason that people were coming to
us – academic tourism. We needed to address that and think about whether that was
our agenda. We decided it wasn‟t! We decided that if that‟s what they want, we‟re not
going to provide it, but we need to tell them why and what we‟re going to give them
instead.




Learning and Teaching Centre
Did you use any other resources to help you when you were re-
designing the assessment?
Again, I‟ve been so lucky to be part of this research project. It was a Carrick project
called Leadership in Assessment – Strengthening the Nexus. And through that, I had
access to not only the peer support and mentoring from people like Sherman Young
(Dean of Learning and Teaching for SCMP) and prior to that Judy Lattas (she was
the prior Dean). They were partners with me in the projects, so basically they
mentored me throughout the process. The team leader, Marina Harvey, would send
me literature.
Being involved in the research project has provided us with all kinds of material. The
FILT provided us with a huge amount of material and support. And it was also really
good that four of us did it at the same time, in that we were all on the same page –
we didn‟t have to explain to each other and there weren‟t the same kinds of barriers
that bringing in ideas to do with, particularly peer review, might have had if we hadn‟t
done it all together. We did encounter things from other members of staff like „Oh,
peer review is just a way for the University to keep us under surveillance‟. But
because so many of us had done the course at the same time, we were able to
logically refute that kind of view.

What changes have you noticed in the student body with this new
approach to assessment?
The main reason why I‟m so excited by the ongoing research project way and above
any of the other things that we‟ve spoken about and way over any of the assessment
stuff that we‟ve done, is that it changes our students. I‟ve never seen classes as
vibrant and as talkative and as enthusiastic. I actually have students email me every
week if they can‟t make it and apologise. And I‟ve said to them “You don‟t have to
attend lectures”. Every week, I have a full lecture class and they apologise if they
can‟t make it.
We also have i-lecture, so they can easily watch the lecture online. One of the
classes I teach is in a room that doesn‟t have a console, so I go and get the tape-
recorder and get that loaded up onto i-lecture. Most of the time, I don‟t need to. I‟ve
never seen anything like it. It‟s so different to the way that my student groups used to
be.




Learning and Teaching Centre

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:16
posted:8/23/2011
language:English
pages:11