The Final Judgement A template for evaluating post-graduate dance

Document Sample
The Final Judgement A template for evaluating post-graduate dance Powered By Docstoc
					The Final Judgement:
A template for evaluating post-graduate dance research

Nicholas Rowe and Ralph Buck
National Institute of Creative Arts and Industries
University of Auckland


Creating a template for the evaluation of post-graduate research can be a highly contentious
endeavour. In determining which aspects of a research project should be valued and how
much they should be valued, such templates ultimately become an institution’s way of
defining the limits of useful knowledge on a subject. For research into creative practice,
these limits can easily feel arbitrary and stifling. The absence of such a template can,
however, render examination results (and thus research degrees) meaningless, subject to
fluctuations in the taste, expectations and values of different examiners over time. Moreover,
in a litigious world in which the awarding (or non-awarding) of degrees has career and
financial consequences, a capricious process of examination can be particularly hard to

It was with such concerns in mind that we began to develop a template for evaluating
creative components of post-graduate research degrees in dance (figure 1). To be used in
conjunction with our existing template for evaluating written exegeses, dissertations and
theses (figure 2), this evaluation template has been designed to clarify our expectations of a
creative research output. It was developed through successive drafts, subject to input from
first our own departmental postgraduate committee and others in our faculty and university
and subsequently from the external examiners that we engaged both nationally and

It should be noted that in this process we have not sought to establish a universal set of
criteria for the evaluation of creative components in post-graduate dance research. Whilst
acknowledging the multiplicity of approaches to such research, we do not attempt to
facilitate all of them through our PhD and Master’s degrees. We have instead identified our
strengths as an institution and considered how we might enhance the relevance of our
dance programme within our own particular community and region. This relevance is
overwhelmingly shaped by a sense that our institution is a research centre for the South
Pacific, and that our dance research must accommodate the exceptionally multi-cultural
context of these environs.
Meanings of Creative Research

Understanding our particular research focus has involved developing contrasts and
connections with contemporary theories on post-graduate research in creative practice. This
initially involved clarifying that our examination template is for research projects that present
dance as a research output rather than simply as part of the research process, or as a
means of reporting rather than just informing research. Conventions already exist within the
arts and humanities in academe for evaluating theses that simply engage in creative dance
practice as a phenomenological method of realizing ideas (which formed the basis of our
template for evaluating written exegesis, theses and dissertations- Figure 2). The complexity
arises in the evaluation of a danced research product that is presented for examination. In
this regard we consider any creative practice component that is presented for evaluation is
considered a research product. Any presentations of creative process are considered in the
same way that appendices are within a traditional thesis: for deeper illumination but not

Our template also does not seek to establishing criteria for what has been described as
practice-led research, which diverges from the traditional problem-led approach to academic
research by allowing research ideas to emerge spontaneously through engagement in
practical activity (Haseman 2005). Whilst we recognize the importance of free-association
and unbounded experimentation as a means of realizing valuable departure-points for
investigation, we remain convinced that research projects are more viable and rigorous (and
yield more complex insights) when constrained by a specific research query.

Our template has been very influenced by the recommendations of the recent report
Dancing Between Diversity and Consistency: Refining Assessment in Postgraduate Degrees
in Dance. We are particularly drawn to the idea of how more reflective practitioners might
emerge through a process that integrates a ‘spectating’ academic that “looks back to arrive
at knowledge” within a creative practitioner that “projects forward towards potential
‘transformative events’ in the production of knowledge” (Phillips, Stock & Vincs 2008, p. 24).
Whilst extending upon numerous aspects of this report’s guidelines for best practice in
Australian doctoral and masters by research examination, we do nonetheless find ourselves
challenged by several arguments presented in the report. These distinctions may in some
ways be attributed to the specificity of our creative practice research programme, which
contrasts with the more generic and all-encompassing “multi-modal” (Phillips, Stock & Vincs
2008, p. 15) approach covered by the report. Other distinctions perhaps indicate the cultural
specificity of the report itself.

Of the latter, perhaps the most overt example is the report’s assumption that creative
practice dance research means choreographic research, in which choreographers generally
perform their own work. More distinct approaches to creative practice research in dance
subsequently seem neglected. These include the interpretive-creative processes brought to
dance products by performers and rehearsal directors. Should we accord less creative
research value to an innovative and sophisticated interpretation of an existing choreography
by a dancer in, for example, a researched performance of the Sugar Plum Fairy solo, or the
creative guidance involved in the rehearsing of existing choreography by a rehearsal director
who undertakes an investigative restaging of an existing work? We would argue that to do
so is fundamentally unproductive in the fostering of a holistic and non-hierarchical creative
arts research environment for dance. Such an environment can be particularly important for
artists and cultural groups that seek deeper understanding of subtle yet innovative
interpretations of traditional choreography (along with the authorship of new dance
compositions). In order to encompass such possibilities, we avoid references to
‘choreography’ within the template.

We also do not include the term ‘performance’ and ‘performance-based research’ within the
template. This again might be seen in contrast to the Dancing Between Diversity and
Consistency report, which prioritizes the “liveness” of performance and its reflexive impact
within creative research (p.25). In conflating the idea of theatre dance documented by the
camera with the idea of dance actually created for the camera, the report does not
adequately recognize what is perhaps one of the most dynamic forums for contemporary
choreography. The demand for examiners and dancers to interface live in “three-
dimensionality environments” (p.26), whilst valued within certain approaches to creative
research, would seem too exclusive for the research that we seek to facilitate.

This leads the report to a recommendation that we consider to be the most problematic: the
appointment of industry professionals rather than academics as examiners. The concerns
we have over the use of industry professionals as academic evaluators stem from what we
consider to be a confusion over the function of research degrees in creative practice.
Ultimately, we do not consider research degrees to be the most effective location for
identifying artistic merit and promoting a quantitative evaluation of artistic quality. Research
degrees are, however, very effective at determining whether or not the creative researcher is
able to firstly engage in a rigorous research process and secondly clearly articulate that
research process. These two queries lead to many others, including an ability to identify
ideas worthy of investigation, an ability to contextualize such an investigation against
existing knowledge, an ability to strategically plan and responsively develop a research
project, an ability to critically analyze and reflect upon research outcomes, an ability to
synthesize these reflections so that they remain relevant to the research investigation and
an ability to present these synthesized ideas to others in ways that are accessible,
meaningful and relevant to the area of research. We value these particular abilities, and
therefore seek to evaluate them, as we consider the recognition of such skills more crucial in
the career development of a creative practice researcher than any perspective we might
have on the value of their artistic work.

Given these expectations, our research degrees require examiners who have a commanding
knowledge of the protocols of academic research. We recognize that industry professionals
can contribute vital analysis on the artistic value of a creative research product, but it seems
unfair to expect those who have not engaged in postgraduate research to be fluent in its
demands. Whilst we support the Dancing Between Diversity and Consistency report’s
intention to more effectively wed dance practitioners with dance academia, we do not feel
the examination process is the most functional location for such a union (any more than
situating a non-practicing dance academic as a choreographer in a professional dance

The Creative Practice Component Evaluation Template

This ultimately raises the question of why a creative practice component should be
presented for evaluation within a research degree. If not as an exhibition of artistic merit,
what is its function? We would contend that it presents an opportunity to articulate
researched ideas through a kinesthetic or embodied medium, allowing for both an
ambiguous artistic interpretation and a clear academic engagement with a research issue.
What then, is the creative practice component’s role in relationship to the written exegesis?

It could be suggested that in presenting a new idea in the form of an argument (which
distinguishes it from other ideas), a research project is obliged to present three components:
a definition of the argument, a rationale for the argument, and evidence for the argument.
Our template thus seeks to evaluate how the creative practice component has clarified the
boundaries of the research, engaged in critical decision-making and clearly illustrated the
outcomes of those decisions.

Within our postgraduate degrees, written and creative practice components of a research
project are assessed as an integrated whole. Within the evaluation process however, we
seek to deconstruct the examiner’s decisions so that grades for the creative practice
component are collated separately from the written thesis component. This is to clarify for

students, supervisors and examiners which aspects of the research project were valued and

This has led to the construction of two evaluation templates: one for the written component
and one for the creative practice component. The criteria for the written component reflect
traditional concerns for a written thesis: the contextual review, the methodology, the analysis
and the overall presentation. The creative practice component addresses the expectations
outlined previously in relation to the definition, rationale and illustration of the research
argument through dance.

Whilst the rationale for the ideas expressed in this template involved lengthy discussion and
debate, perhaps the same amount of time was dedicated to the actual vocabulary used to
express these ideas. Generic and specific terms wrestled each other, within both phrases
clarifying what is being evaluated and the adjectives clarifying the measured value. In
attempting to accommodate a diversity of cultural contexts for such research, the more
specific wrestled with the more generic terminology. This occurred within both the adjectives
determining value and the phrases clarifying what is being valued. Phrases such as
“professional standard” were omitted from the evaluation of presentation, in order to
accommodate artistic research that might seek to confront the cultural hegemony of
capitalism in arts practice. Ideas such as “contexts/ genres/ histories” were conflated, so that
the arts practitioner might more easily determine their own artistic boundaries for the


The template presented here remains for us a tentative work-in-progress. We present it to
stimulate discussion on how the evaluation of creative components in postgraduate research
might further be refined. We acknowledge that there is a limited pool of academic examiners
for creative components of postgraduate dance research, and that those overwhelmingly
come from an arts and academic culture defined by postmodernist artistic ideals. We have
thus sought to establish criteria through these templates that might guide examiners away
from the capri of artistic subjectivity and into the particular cultural/artistic ideals being
investigated by the dance researcher. In doing so we hope to broaden the artistic directions
and cultural contexts that might accommodate dance research in the multi-cultural
community that surrounds our institute.


Haseman, B. 2005. Monitoring Quality in the Practice-Led PhD. ATN Research on Research
Conference. University of South Australia.

Phillips, M., Stock, C. & Vincs K. 2008. Dancing Between Diversity and Consistency:
Refining Assessment in Postgraduate Degrees in Dance. Edith Cowan University/ ALTC.

                           Criteria and Standards for Written Components of Dissertations and Theses

Standard            A. WHAT: Engagement with specified               B. WHY: Aims, questions and critical            C. HOW: Standard of creative presentation
                       artistic parameters: contexts/ genres/        reflections                                        relevant to artistic/cultural context
Pass, No Hons       A1 Superficial and unclear engagement with       B1 Revealed some engagement with aims           C1   Inconsistent standard of presentation,
(50-64%)               specified parameters (contexts/ genres/          and questions, although inconsistent and          reflects uncertain understanding of
                       histories). Some consideration of historic       not clearly organised, little critical            relevant mode of presentation. Adequate
                       parameters, Technical inconsistencies/           reflection on alternative approaches.             organization of material, some attention to
                       weaknesses.                                                                                        relevant structural concerns.

2nd Class           A2 Considered but inconsistent engagement        B2 Displays consistent and organized            C2   Consistent and organized standard of
Div 2 Hons             with parameters (contexts/ genres/               engagement with aims and questions,               presentation, reflects an understanding of
(65-72%)               histories). Reflects knowledge of specified      limited scope of approaches, some critical        the demands of presentation at a high
                       physical vocabularies, techniques and            reflection on alternatives.                       level. Organisation of material and
                       aesthetic ideas.                                                                                   structure relevant to themes/form.

2nd Class           A3 Systematic and effective use of specified     B3 Clear and well organized engagement          C3   Refined presentation, reflecting a
Div 1 Hons             parameters (contexts/ genres/ histories).        with aims and questions, broad                    competent ability to address relevant
(73-79%)               Appropriate, well-considered                     understanding of approaches, analysed             context of artistic delivery. Concise,
                       interpretations of physical vocabularies,        problems and critically reflected well on         economical organization of material, with
                       techniques and aesthetic ideas.                  alternatives.                                     attention clearly given to relevant
                                                                                                                          structural concerns.

No 2s and more 4s   A4 Innovative and effective use of specified     B4 Clearly reflects a rigorous and defined      C4   Highly refined, cohesive presentation,
than 3s)                parameters (contexts/ genres/ histories),       investigation of aims and questions,              reflecting an ability to set new standards
1st Class Hons          insightful and creative interpretations,        problems identified, critically reflected         in the presentation of dance work. Clear
(80-100%)               strong understanding of physical                upon and resolved, illustrated use of             contribution to field.
                        vocabularies, techniques and aesthetic          innovative approaches to research.

                            Criteria and Standards for Written Components of Dissertations and Theses

Standard         A. Comprehension of topic and            B. Methods and techniques            C. Analysis and interpretation of       D. Written expression and
                    field                                                                         data                                    structure of dissertation
Pass, No Hons    A1 Basic comprehension of core           B1 Obvious limitations of the        C1 Generally ineffective analysis of    D1 Reasonable sentence
(50-64%)             concepts relating to topic and          methods selected that relate to      data and draws some                       construction but with
                     limited location of the topic           and illuminate the research          inappropriate conclusions,                occasional lapses, limited
                     within more general                     problem. Evidence of limited         some errors in referencing with           vocabulary, organisation of
                     knowledge of the field of               technical competence and             limited relevance to                      material does not consistently
                     study.                                  some technical errors in             interpretation.                           address the topic, regular
                                                             collection resulting in poor                                                   grammatical and spelling
                                                             data.                                                                          errors.
2nd Class        A2   Satisfactory comprehension of       B2 Generally sound selection of      C2 Correct but not always               D2 Reasonable sentence
Div 2 Hons            core concepts and information          methods that relate to and           systematic analysis of data,              construction and appropriate
(65-72%)              relating to topic though limited       illuminate the research              draws conclusions that shows              though limited vocabulary,
                      location of the topic within           problem. Evidence of sound           limited insight, correct                  some organisation of material
                      more general knowledge of              technical competence with            referencing that contributes              to address the topic, generally
                      the field of study.                    some errors in collection            generally to interpretation.              correct grammar and spelling.
                                                             affecting the quality of data.
2nd Class        A3   Good comprehension of core          B3 Clearly appropriate and           C3 Systematic and effective             D3   Good sentence construction,
Div 1 Hons            concepts and information               relevant methods selected that       analysis of data, appropriate             vocabulary and organisation
(73-79%)              relating to topic and location of      relate to and illuminate the         interpretations and defensible            of material that addresses the
                      the topic within more general          research problem. Evidence of        conclusions that have some                topic effectively, few minor
                      knowledge of the field of              sound technical competence           potential to contribute to                grammatical or spelling errors.
                      study.                                 resulting in the collection of       knowledge in the field, correct
                                                             good data.                           referencing that contributes to
                                                                                                  the quality of interpretation.
No 2s and more   A4   Excellent comprehension of          B4 Clearly appropriate and           C4 Systematic and effective             D4   Clear, concise and
4s than 3s)           core concepts and information          creative methods selected that       analysis of data, insightful and          economical writing, wide
1st Class Hons        relating to topic and creative         relate to and illuminate the         creative interpretations,                 vocabulary, excellent
(80-100%)             and/or insightful links to more        research problem. Evidence of        conclusions that have clear               organisation of material
                      general knowledge of the field         mastery of techniques resulting      potential to contribute to                illuminates topic directly,
                      of study.                              in the collection of excellent       knowledge in the field,                   consistently good grammar
                                                             data.                                referencing contributes clearly           and spelling.
                                                                                                  and specifically to the quality of


Shared By:
Description: The Final Judgement A template for evaluating post-graduate dance