Suspect science_

Document Sample
Suspect science_ Powered By Docstoc

[This is an incomplete first draft of a collaborative paper.]

Suspect science: Expert identification evidence derived from
photographic and video images

Katherine Biber†, Gary Edmond•, and Richard Kempø

Abstract: This paper provides a critical overview of the emergence of expert
identification evidence based on images in Australia. It reviews the
case law and endeavours to explain methodological and technical problems
with facial mapping evidence. It also suggests that extant admissibility
jurisprudence and traditional safeguards associated with expert opinion
evidence and the adversarial trial might not adequately protect those
accused of committing criminal acts.

Photography was present at the birth of the forensic sciences. Its use by Alphonse
Bertillon (1853-1914), as a system for recording anthropometric measurements,
places it among the earliest of the modern criminal investigation technologies.1
Photographic and video images continue to play important roles in the criminal justice
system, particularly in the investigation of crime. Yet, photographic images do not
speak for themselves. They require interpretation. Critical writing about photography
has cautioned us about using images to prove things.2 Photographic and video
evidence is fraught with dangers.
   In recent years, following a general prohibition on police identification evidence by the
High Court in Smith v The Queen (2001), police and prosecutors in Australia have
increasingly looked to experts to provide identification evidence based on the examination
of photographs and videos. Expert identification evidence based on such images is often,
though not invariably, described as facial mapping. Initially, the experts called by the
prosecution sought to provide positive identification evidence, where they opined that the
accused and the person appearing in security and surveillance images were one and the
same. Appellate courts, exhibiting some concerns about these developments, restricted the

  Faculty of Law, UTS.
  Faculty of Law, UNSW. Edmond has observed several cases where expert identification evidence from images
has been considered on the voir dire and admitted. This research forms part of Australian Research Council
Discovery Grant (DP0771770).
  Department of Psychology, UNSW. Kemp has acted as a rebuttal expert in several criminal cases where the
prosecution adduced and relied upon expert identification evidence. He has also been engaged by applicants whose
immigration status was challenged on the basis of facial mapping evidence. This research was partially supported
by Australian Research Council Discovery and Linkage International grants (DP0881623 and LX0883067).
  J Mnookin ‗The image of truth: Photographic evidence and the power of analogy‘ (1998) 10(1) Yale Journal of
Law & the Humanities 1; M Valverde, Representing Law and Order, 2006, A Young, Imagining Crime, 1996; A
Young, Art, Value, Law, 2005; K Biber, Captive Images, 2007; EF Loftus, ‗Unconscious transference in
eyewitness identification‘ (1976) 2 Law and Psychology Review 93.
  S Sontag, On Photography, 1971; R Barthes, Camera Lucida, 1981; A Sekula, Photography Against the Grain,
1984; A Solomon-Godeau, Photography at the Dock, 1991; J Tagg, The Burden of Representation, 1988; Tagg,
Grounds of Dispute, 1992; K Biber, Captive Images, 2007.

scope of this testimony; limiting expert identification evidence to opinions about
similarities between an unknown offender and the accused.
   This paper is critical of this expert identification evidence, the emerging jurisprudence
and some of the associated forensic practices. The validity and accuracy of virtually all of
the facial mapping, morphological and comparative techniques now routinely accepted in
Australian courts are uncertain. Consequently, it is our contention that prosecutors should
be reluctant to adduce such evidence, judges should not admit it, the defence should not be
obliged to defuse it, and jurors should not be required to hear or determine its value.

‘Expert’ identification evidence based on images: Face and body mapping
There is no standard definition of facial mapping. This is not particularly surprising as the
term seems to have been coined by the British media in the 1980s and there are no teaching
institutions, formal courses, regulatory bodies or publications based on research. Generally
the term is used to describe a variety of evolving techniques in which characteristics or
measurements of the face are used to assess the identity of a person of interest in
photographs or video. Kemp and Coulson explain:

         Two or more photographic, video or other images are compared to determine whether they depict the
         same person, or a photographic, video or other image is compared to a ‗live‘ (i.e. corporeal) target …
         to determine whether the image is of the target individual. The process of comparison might involve
         taking measurements, noting characteristics, or other techniques, and might concentrate on the face
         alone or might include other parts of the body. 3

This description excludes a range of forensic scientific activities, like facial reconstruction,
which are not predicated primarily on forms of comparison or identification from
photographs and video. It also recognises that facial mapping and comparative evidence is
not strictly limited to facial features.
   Broadly, there seem to be two ways to classify expert identification evidence: one
quantitative and the other qualitative. In practice, though, the two approaches are not
mutually exclusive. Generally, those using the more quantitative approach rely on the
measurement of key features of the face and body (often termed landmarks), as well as the
distances and angles between these features. Such methods are often described as photo-
anthropometry (see Appendix 1). The other, more qualitative approach, is concerned with
the presence or absence of characteristic features and the degree of similarity. This
approach is usually referred to as morphological analysis or morphological comparison.
The following definitions were an exhibit in R v Tang.

         Photo-anthropometry is a technique that attempts to metrically compare the proportional
         relationships of one photo to another rather than determine absolute visual similarities, as is
         done in morphological comparisons. It involves the analysis of anthropometric landmarks,
         dimensions and angles [simply] to quantify facial characteristics and proportions from a

         Morphology analysis
         The feature by feature approach to evaluating faces, heads and bodies. It involves the
         comparison of two images – one from the crime scene and one of the suspect. It involves
         subdividing the face, head and body into components to obtain a thorough qualitative analysis
         and to determine visual similarities or differences. 4

    Kemp & Coulson (in prep).
    R v Tang [2006] NSWCCA 167, [18].

   Even a cursory examination of the facial mapping evidence received in Australian
courts reveals a variety of techniques and tools relied upon by facial mappers and others
involved in analysing and comparing individuals captured by cameras. Australian courts
have encountered photo-anthropometry, morphological analysis, the comparisons of faces
and bodies (using video, photographs and enlarged photoboards), the overlaying of two
similar size images (photograph superimposition), the rapid transition between two images
(blink technique), as well as photo and video enlargement, enhancement and manipulation
(see Appendix 2).
   Here, it is important to emphasise, that in describing these techniques we do not mean to
suggest they are standardised or even consistently applied by individual facial mappers.
More particularly, we do not intend to suggest that these techniques are widely accepted,
scientific or reliable. One of the most conspicuous features of facial mapping evidence and
related forms of expert identification evidence is the absence of accepted procedures and
protocols. The unknown validity and accuracy of facial mapping techniques are among our
principle concerns.
   In this paper the reader will encounter references to body mapping as well as face and
body mapping. Body mapping involves applying similar techniques to the body shape,
features, and movement. As with facial mapping, those engaged in body mapping and
using movement for purposes of identification may attempt to apply quantitative measures
(e.g. anthropometric calculations around height) and/or describe similarities and
differences between body shapes and movements (e.g. right-handedness). Often the same
expert will be engaged in, and allowed to testify about, face and body comparisons. With
the necessary adaptations virtually all of the issues canvassed below would seem to be
applicable to so-called body mapping as well as the combination of face and body
   Not all of the individuals providing expert identification evidence based on images
would describe themselves as facial mappers or their opinion evidence as facial mapping.
Some would distinguish themselves or their activities from facial mapping on the basis of
the methods used (e.g. qualitative rather than quantitative) or underlying expertise (e.g.
comparative anatomy rather than computing or photography). To varying degrees such
alignments embody professional and disciplinary boundaries as well as diachronic
responses to judicial characterisations and criticism.5

Two caveats
One way to interpret this paper would be as an indictment of one or two ‗rogue‘ experts.
That, however, would be to trivialise the inability and/or unwillingness of criminal justice
systems—and a range of participants such as police, experts, prosecutors, defenders and
judges—to comprehend and address some of the persistent problems with images. Instead,
this paper endeavours to illustrate how adjectival law, formal commitment to a fair trial,
professional codes of conduct and expert guidelines do not adequately regulate
incriminating expert identification evidence.
   Second, this paper does not concern itself with the opinions of lay witnesses who are
very familiar with the person said to appear in images relevant to a crime. There are good
reasons to believe that lay people perform well when identifying individuals with whom
they are very familiar. There are dangers with lay identifications from images, but few
prima facie reasons to prevent lay individuals giving their opinions about the identity of
those with whom they are very familiar, especially where the images afford opportunities
to observe the face and/or movement.

    A Abbott, System of Professions (1988); T Gieryn. Cultural boundaries of science (1998).

Admissibility jurisprudence: Background
Most of the cases discussed in this paper were tried in NSW. The relevant admissibility
framework is the Evidence Act 1995 (NSW).6 According to the Act, to be admissible all
evidence must be relevant:

       55 Relevant evidence
       (1) The evidence that is relevant in a proceeding is evidence that, if it were accepted, could
       rationally affect (directly or indirectly) the assessment of the probability of the existence of a
       fact in issue in the proceeding.

       56 Relevant evidence to be admissible
       (1) Except as otherwise provided by this Act, evidence that is relevant in a proceeding
       is admissible in the proceeding.
       (2) Evidence that is not relevant in the proceeding is not admissible.

       Evidence Act Dictionary
       probative value of evidence means the extent to which the evidence could rationally affect
       the assessment of the probability of the existence of a fact in issue.

The intention behind the introduction of the Evidence Act, as section 56 makes clear, was
to encourage a more straightforward and inclusive approach to admissibility.
   Opinion evidence, even if relevant, is presumptively inadmissible. The opinion rule
(section 76) states that ‗evidence of an opinion‘ is not admissible ‗to prove the existence of
a fact about the existence of which the opinion was expressed‘. There are, however, several
exceptions to the exclusionary impact of the rule. Although it does not attempt to codify
the common law, section 79 provides the major exception for expert opinion evidence. It

       79 Exception: opinions based on specialised knowledge
       If a person has specialised knowledge based on the person’s training, study or experience,
       the opinion rule does not apply to evidence of an opinion of that person that is wholly or
       substantially based on the knowledge.

Provided an ‗opinion‘ is ‗wholly or substantially‘ based on ‗specialised knowledge‘ which
is based on ‗training, study or experience‘ it is not caught by the exclusionary opinion
rule.7 Where these conditions are satisfied, a witness can give relevant opinions about
facts in issue, subject only to the exclusionary discretions. These are sometimes referred to
as the Christie discretions.8
    In criminal proceedings trial judges have a responsibility to balance the probative value
of incriminating evidence against the danger of unfair prejudice to the accused.

       137 Exclusion of prejudicial evidence in criminal proceedings
       In a criminal proceeding, the court must refuse to admit evidence adduced by the prosecutor
       if its probative value is outweighed by the danger of unfair prejudice to the defendant.

Section 135 confers a discretion using similar, if somewhat less stringent, terminology.


  This is similar to the Evidence Act used in the ACT and the Federal Courts.
  See, for example: HG v The Queen (1999) 197 CLR 414, [39] (Gleeson CJ).
  R v Christie [1914] AC 545.

Before moving to consider the main Australian authorities it might be useful to succinctly
describe the position in England. English courts have been very receptive to identification
evidence derived from images.9 In a decision responding to a reference from the Attorney
General, the Court of Appeal (Criminal Division) reviewed the relevant English case law.

         In our judgment, on the authorities, there are, as it seems to us at least four circumstances in which,
         subject to the judicial discretion to exclude, evidence is admissible to show and, subject to appropriate
         directions in the summing-up, a jury can be invited to conclude that the defendant committed the
         offence on the basis of a photographic image from the scene of the crime:

             (i)   where the photographic image is sufficiently clear, the jury can compare it with the defendant
                   sitting in the dock (Dodson and Williams);
             (ii) where a witness knows the defendant sufficiently well to recognise him as the offender
                   depicted in the photographic image, he can give evidence of this (Fowden and White, Kajala
                   v Noble, Grimer, Caldwell and Dixon and Blenkinsop); and this may be so even if the
                   photographic image is no longer available for the jury (Taylor v Chief Constable of Chester);
             (iii) where a witness who does not know the defendant spends substantial time viewing and
                   analysing photographic images from the scene, thereby acquiring special knowledge which
                   the jury does not have, he can give evidence of identification based on a comparison between
                   those images and a reasonably contemporary photograph of the defendant, provided that the
                   images and the photograph are available to the jury (Clare and Peach);
             (iv) a suitably qualified expert with facial mapping skills can give opinion evidence of
                   identification based on a comparison between images from the scene, (whether expertly
                   enhanced or not) and a reasonably contemporary photograph of the defendant, provided the
                   images and the photograph are available for the jury (R. v Stockwell (1993) 97 Cr App R
                   260, R. v Clarke [1995] 2 Cr App R 425 and R. v Hookway [1999] Crim LR 750).10

The emergence of expert identification evidence—(iv) above—has followed quite a
different trajectory in Australia. Only in recent years, following an appeal to the High
Court, have Australian judges begun to reflect on the admissibility and value of expert
opinion derived from incriminating images.


A. The High Court creates a need: Smith v The Queen (2001)
In order to understand the emergence of expert identification evidence based on images in
Australia, we need momentarily to backtrack to consider the case of Smith v The Queen.
The different approaches in the Smith appeals are, in this context, significant. Appeals to
the NSW Court of Criminal Appeal (CCA) and the High Court focused on the
admissibility of positive identifications of Smith by police officers who examined
photographs of a robbery taken by a bank‘s security system (see Figure 1). Two police
officers, apparently independently, identified one of the robbers as Mundarra Smith and
were allowed, over objection, to give that evidence before the jury at trial. On appeal, the
admissibility of this testimony was challenged. Here, we might note, that the police
officers who purported to identify the offender had each had about half a dozen encounters,
including several ‗brief (under five minute) conversations‘, with Smith in the months
preceding the robbery. One of the officers had arrested Smith twice and estimated spending
two to three hours with him each time.

    There have been notable exceptions, such as R v Gray [2003] EWCA Crim 1001, [16]
     Attorney General's Reference No 2 of 2002 [2002] EWCA Crim 2373.

Figure 1        The person standing at the extreme right (with the dark hat) was identified as Smith

   On appeal before the CCA the primary issue was whether the police identification
evidence was opinion or fact.11 That Court concluded that the evidence of the police
officers was evidence of fact—that is, recognition evidence—and therefore not caught by
the exclusionary opinion rule. In the High Court, most of the judges took a different
approach. There, for the first time, the issue of relevance emerged.
   A majority comprised of Gleeson CJ, Gaudron, Gummon and Hayne JJ concluded that
the evidence of the police officers was irrelevant and therefore inadmissible. The joint
judgment explains their reasoning:

       The police witnesses were in no better position to make a comparison between the appellant and the
       person in the photographs than the jurors or, for that matter, some members of the public who had
       been sitting in court observing the proceedings. … Because the witness‘s assertion of identity was
       founded on material no different from the material available to the jury from its own observation, the
       witness‘s assertion that he recognised the appellant is not evidence that could rationally affect the
       assessment by the jury … The fact that someone else has reached a conclusion about the identity of
       the accused and the person in the picture does not provide any logical basis for affecting the jury‘s
       assessment of the probability of the existence of that fact when the conclusion is based only on
       material that is not different in any substantial way from what is available to the jury .

The majority, because they deemed the evidence of the police officers to have been
irrelevant, were disinclined to address the fact/opinion dichotomy.
    Although he agreed that the evidence of the police officers was inadmissible, Kirby J‘s
approach seems less controversial. For Kirby J, the evidence of the police officers was not
irrelevant. He accepted that their evidence might possess low probative value but all that
   R v Smith (1999) 47 NSWLR 419. There was argument about whether the evidence was identification—
classified as ‗opinion‘—or recognition evidence—and therefore ‗factual‘. This nomenclature has admissibility
implications because all witnesses can opine about facts but are often limited in terms of opinion.
   Smith v The Queen [2001] HCA 50, [11].

was required under the inclusive Evidence Act was an ability to ‗rationally affect … the
assessment of the probability of the existence of a fact in issue‘. The evidence of the police
officers could, if accepted, ‗rationally affect‘ such an assessment.
   Having deemed the evidence relevant, Kirby J concluded there was no applicable
exception to the opinion rule that would enable the police officers to testify about the
identity of the offender.

       Two possible bases for exception from the opinion rule were explored in argument. The exception for
       admitting evidence based on specialised knowledge provided by s 79 of the Act can be disregarded, as
       no suggestion of such expertise was made in relation to the police officers. Their prior contact with the
       appellant did not amount to ad hoc expertise based on familiarity, nor did they claim any expertise in,
       for example, anatomical or photographic comparisons. The exception for lay opinion evidence
       provided by s 78 of the Act states:
           The opinion rule does not apply to evidence of an opinion expressed by a person if:
                (a) the opinion is based on what the person saw, heard or otherwise perceived about a
                      matter or event, and
                (b) evidence of the opinion is necessary to obtain an adequate account or
                      understanding of the person's perception of the matter or event.
       Neither police officer was present at the ―matter or event‖ in question in the appellant's trial, namely
       the robbery. Although the security photographs record the robbery taking place, the opinion of the
       police officers is ―based on‖ the photographs and not, as such, ―based on‖ the robbery itself which
       they did not see, hear or otherwise perceive.

For Kirby J, the opinion evidence proffered by the police officers was relevant and might
legitimately be used in the investigation of the robbery. In the absence of an exception it
would not, however, be admissible in any trial. Here we might recall the limited contact
between the police officers and the accused. For Kirby J, this was insufficient familiarity to
establish an exception to the opinion rule based on specialised knowledge (or ‗ad hoc
   These decisions did not mean that police officers (and others) would be permanently
prevented from testifying about the identity of persons in images associated with criminal
acts. The majority described a range of circumstances where the evidence of police
officers—whether construed as evidence of fact or opinion—might yet prove relevant and

       In other cases, the evidence of identification will be relevant because it goes to an issue about the
       presence or absence of some identifying feature other than one apparent from observing the accused
       on trial and the photograph which is said to depict the accused. Thus, if it is suggested that the
       appearance of the accused, at trial, differs in some significant way from the accused's appearance at
       the time of the offence, evidence from someone who knew how the accused looked at the time of the
       offence, that the picture depicted the accused as he or she appeared at that time, would not be
       irrelevant. Or if it is suggested that there is some distinctive feature revealed by the photographs (as,
       for example, a manner of walking) which would not be apparent to the jury in court, evidence both of
       that fact and the witness's conclusion of identity would not be irrelevant.

On this point, to the extent that the identification evidence was ‗based on familiarity‘,
Kirby J concurred.
   Before Smith, police officers and other investigators routinely made identifications from
images based on prior interaction with the accused or the limited familiarity gained
through the course of a covert investigation. After Smith, investigators and prosecutors had
   Smith [59]-[60].
   For a more detailed discussion consider Edmond and San Roque, ‗Petit truths: ad hoc experts and legal
legitimacy‘, paper presented at the Law & Society Conference, University of Melbourne, 29 November 2007.
   Smith [15].

two options. First, they could endeavour to utilise the exceptions to irrelevance which the
majority had identified in the extract above. These exceptions enabled investigators and
police, in an ever-expanding range of circumstances, to proffer identification evidence
linking a known individual to a suspect in a photograph or video associated with a criminal
act.16 Second, police and prosecutors could seek the assistance of those with expertise—
that is, pre-existing expertise—in disciplines such as anatomy and physical anthropology.
These experts, it seemed, would be able to give opinion evidence about the identity of
persons in images based on their ‗specialised knowledge‘.
   We are primarily concerned with the second option.

B. Disciplining professional vision: From positive identification to similarities
Before 2003 expert identification evidence based on images seems to have received limited
jurisprudential consideration in Australian courts and tribunals. A more sceptical approach
to asylum seekers, especially in wake of the incident with the MV Tampa, led the state to
rely upon facial mapping evidence in immigration cases where questions about identity
were sometimes aggressively contested. In SHJB v Minister for immigration &
Multicultural & Indigenous Affairs, for example, Selway J considered an array of evidence
in his review of a decision by the Refugee Review Tribunal. This evidence included the
facial mapping evidence of a forensic anthropologist. Dr Watt‘s identification of the
applicant as a known Pakistani national rather than a refugee from Afghanistan was relied
upon by the Tribunal and accepted by Federal Court.17 In the criminal courts expert
identification evidence based on images became conspicuous shortly thereafter.
   In this genealogy, the District Court of NSW is interesting because in the first recorded
Australian criminal decision—an appeal over the admissibility of facial mapping
evidence—Justice Blanch adopted a critical stance toward this novel identification
evidence. In R v BLM facial and body mapping evidence was admitted in a trial at the
Bidura Children‘s Court and used to link M to an armed robbery.18 Dr Meiya Sutisno, ‗a
doctor of philosophy and medicine from the University of Sydney‘ relied upon photo
anthropometry, morphology and photo superimposition to make a positive identification—
that the accused and the offender ‗were the same person‘—based upon her examination of
images recorded by a CCTV system. 19 At this stage Dr Sutisno was apparently unwilling
to explain her techniques, particularly the points she relied upon for facial comparisons and
identification, because of concerns about her intellectual property.20 On appeal, Blanch
DCJ expressed his profound disconcert at the unwillingness of this expert to disclose or
explain the techniques which supposedly enabled her to make positive identifications.
Without an understanding of the basis of her approach, the tribunal of fact was in no
position to assess Dr Sutisno‘s opinion evidence. Even though Blanch DCJ thought that
some of the comparison evidence might be admissible the magistrate‘s decision to admit
Dr Sutisno‘s identification evidence was deemed inappropriate and the appeal was upheld.
   These ignominious beginnings were not the end for facial mapping evidence in

   See, for example, R v Leung and Wong [1999] NSWCCA 287 (Simpson J, Spigelman CJ and Sperling J
concurring); R v Li [2003] NSWCCA 290 (Ipp JA, Whealy and Howie JJ concurring).
   SHJB v Minister for Immigration and Multicultural and Indigenous Affairs [2003] FCA 502 (Selway J, Watts
providing a positive identification using facial mapping techniques and Dr Kemp criticising this evidence); NAFC
v Minister for Immigration and Multicultural and Indigenous Affairs [2003] FCAFC 22. See, more generally,
Anthony Good, Anthropology and expertise in the Asylum courts (2006).
   In the matter of the appeal of BLM (unreported) District Court of NSW, 14 September 2005.
   BLM, 3.
   BLM. 4, 10.

    Later in 2005 Bradley John Murdoch was convicted of the murder of Peter Falconio and
the abduction of Joanne Lees in the Supreme Court of the Northern Territory. During the
trial prosecutors adduced and relied upon Dr Sutisno‘s morphological evidence. The
circumstantial prosecution case included poor quality images of a person refuelling a
vehicle similar to one owned by Murdoch, at a truck stop in the relevant vicinity at a time
proximate to the murder and abduction. The images were of such poor quality that the
vehicle‘s number plate, adjacent to where the person of interest was standing, could not be
    Nevertheless, at trial Dr Sutisno was allowed to testify, based on a comparison of the
video images taken at the truck stop and reference images of Murdoch, that in her opinion
it was Murdoch at the truck stop. Chief Justice Martin made the following comments in his
admissibility determination:

       Dr Sutisno was an impressive witness. She is highly qualified and experienced in her field. …
       Applying both morphological analysis and photographic superimposition, Dr Sutisno concluded that
       the person depicted in the image taken from the security film was the accused. She was unable to
       identify any differences of significance.

BLM was distinguished because Martin CJ was satisfied that ‗the principles underlying the
work of Dr Sutisno can be readily understood.‘22
   At about the same time the prosecution adduced and relied upon Dr Sutisno‘s facial
mapping evidence in the Supreme Court of NSW. In R v Tang Dr Sutisno‘s positive
identification evidence was admitted in a prosecution for armed robbery. Dr Sutisno
testified, on the basis of her morphological analysis, that Hien Puoc Tang and the offender
were ‗one and the same‘.23 This was based on her morphological comparisons and the
presence of what were described as ‗unique identifiers‘.24 Dr Sutisno concluded that:

       … it is my opinion that the multiplicity of common morphological features, unique identifiers and
       Asian characteristics to (sic) lend support that the offender and alleged suspect Hien TANG are one
       and the same.

During cross-examination Dr Sutisno rejected the suggestion ‗that there was a degree of
subjectivity in her assessment‘.26
   The evidence admitted in Tang and Murdoch represents the apotheosis of expert
identification from images in Australia.
   The appeal in Tang provided an appellate court with the first opportunity to consider
facial mapping evidence. Tang is an important, if unsatisfactory, authority. Writing for the
Court, Chief Justice Spigelman expressed doubts about the facial and body mapping
evidence presented by Dr Sutisno. In his opinion facial mapping—‗let alone body
mapping‘—evidence did not constitute ‗specialised knowledge‘ and was incapable of
supporting expert opinion regarding identity.

       The evidence in this trial did not disclose, and did not permit a finding, that Dr Sutisno‘s evidence was
       based on a study of anatomy. That evidence barely, if at all, rose above a subjective belief and it did

   R v Murdoch [2005] NTSC 78, [95].
   Murdoch, [112]. In Murdoch, Dr Sutisno did not rely upon anthropometrical techniques and evidence.
   R v Tang [2006] NSWCCA 167, [23]-[28].
   Tang [27], [18], [26]. Interestingly, in practice, ‗unique identifiers‘ were not as unique as might be imagined.
Rather than scars and tattoos, features such as ‗wide and thick upper and lower lips, wide square chin with dimple‘
and ‗relatively upright posture‘ became ‗unique identifiers‘ that enabled Dr Sutisno to testify in the misleading
oxymoronic terminology of ‗definitive resemblances‘.
   Tang, [28].
   Tang, [78]-[81]. That is, her morphological comparisons.

       not, in my opinion, manifest anything of a ―specialised‖ character. It was not, in my opinion, shown
       to be ―specialised knowledge‖ within the meaning of s79.27

       Facial mapping, let alone body mapping, was not shown, on the evidence in the trial, to constitute
       ―specialised knowledge‖ of a character which can support an opinion of identity. 28

Nevertheless, rather than exclude the opinion, the Chief Justice accepted that Dr Sutisno‘s
anatomical qualifications and experience enabled her to observe and describe similarities
and differences between the person in the security images and the reference photographs of
the accused. Her opinion evidence would be admitted as ‗ad hoc expertise‘.

       The identification of points of similarity by Dr Sutisno was based on her skill and training,
       particularly with respect to facial anatomy. It was also based on her experience with conducting such
       comparisons on a number of other occasions. Indeed, it could be supported by the experience gained
       with respect to the videotape itself through the course of multiple viewing, detailed selection,
       identification and magnification of images. By this process she had become what is sometimes
       referred to as an ―ad hoc expert‖.29

For the CCA, Dr Sutisno‘s positive identification evidence was improperly admitted. Tang
was entitled to a re-trial where Dr Sutisno would be restricted to commenting on
similarities (and, in theory, differences).30
       The evidence of Dr Sutisno of similarity, at least with respect to the facial features, is capable of
       adding strength to the Crown‘s circumstantial case. Even if she is not able to express the conclusory
       opinions of the character she did express, she can give evidence which supports of (sic) the Crown

   Of significance, in its review of the admissibility of opinion evidence based on
‗specialised knowledge‘ under section 79 of the Evidence Act, the CCA explicitly
dismissed the need for reliability:

       The focus of attention must be on the words ―specialised knowledge‖, not on the introduction of an
       extraneous idea such as ―reliability‖. 32

Here, in an appeal concerning the admissibility of incriminating opinion evidence, the
validity and reliability of the expert‘s techniques and interpretations appear to have little, if
any, significance. Notwithstanding the absence of a body of ‗specialised knowledge‘
(something akin to a ‗field‘ at common law), experts would not be prevented from
proffering their incriminating opinions in the absence of ‗specialised knowledge‘, a ‗field‘,
validated techniques or demonstrated proficiency. The fact that the expert opinion evidence
was not demonstrably reliable was not enough to warrant its exclusion under the opinion
rule and its exceptions (sections 76 and 79) or the exclusionary discretions (sections 137
and 137).
   Tang did not have a salutary or particularly exclusionary impact on expert identification
evidence. In the Supreme Court of NSW, just months after the appeal, Justice Hall
admitted Dr Sutisno‘s facial mapping evidence about a person in an ATM photograph. He

   Tang, [140]. Though compare [135].
   Tang, [146].
   Tang, [120].
   At the re-trial facial mapping evidence was not relied upon and Tang was again convicted.
   Tang, [157].
   Tang, [137].

       The evidence does establish that Dr. Sutisno does have specialised knowledge based on study and
       experience in relation to facial characteristics in the context of issues concerned with establishing
       identification both of deceased persons and otherwise. Such has become a recognised field for expert
       analysis, albeit of fairly recent origin. The opinion she has expressed as to similarities is an opinion
       within that field of specialised knowledge.33

In R v Jung (2006) Dr Sutisno was allowed to give facial mapping evidence but, following
Tang, was prevented from making positive assertions about identity. Her written reports
gave ‗conclusory opinions‘ but her testimony was restricted to the terms preferred by
Spigelman CJ in Tang.34 Perhaps unremarkably, Justice Hislop came to a similar position
in R v Kaliyanda.35 Here, again, most of the emphasis seems to have been placed on Dr
Sutisno‘s qualifications, training and experience as an anatomist specialising in facial
reconstruction rather than the more fundamental question of the existence of ‗specialised
knowledge‘ and whether her techniques and opinions were valid and reliable. None of the
judges involved in Tang, Jung and Kaliyanda inquired whether Dr Sutisno could actually
do what she claimed.
   In the course of just a few months and notwithstanding Spigelman CJ‘s apparent
incredulity facial mapping had become a legally recognised form of ‗specialised
   This trend was confirmed at the end of 2006 when Murdoch appealed his conviction to
the Northern Territory Court of Criminal Appeal. Among the grounds of appeal was the
admissibility of Dr Sutisno‘s incriminating facial mapping evidence. Employing common
law criteria, that Court basically embraced the approach to facial mapping evidence
proposed by the NSW CCA in Tang.

       This Court has found that the technique employed by Dr Sutisno did not have a sufficient basis to
       render the results arrived at by that means part of a field of knowledge which is a proper subject of
       expert evidence. However the evidence given by Dr Sutisno was capable of assisting the jury in terms
       of similarities between the person depicted in the truck stop footage and the appellant. It was evidence
       that related to, and was admissible as, demonstrating similarities but was not admissible as to positive
       identity. Dr Sutisno was not qualified to give evidence, as she did, based on ―face and body mapping‖
       as to whether the two men were, indeed, the same man. Her evidence in this regard should not have
       been received.

The Northern Territory CCA concluded that Dr Sutisno should not have positively
identified the person in the images taken at the truck stop. Instead, she should have been
restricted to describing similarities between Murdoch and the person of interest. In this
appeal, an expert‘s positive identification before a jury was not enough, however, to
warrant a re-trial.
   Smith effectively created a need for experts to assist the prosecution and the court with
the interpretation of images. Since Tang (and Murdoch) experts have been prevented from
making positive identifications (in court). Now, as Jung, Kaliyanda, and a number of
subsequent cases—such as R v Alrekabi and R v Pera in the District Court of NSW—attest,
facial mapping testimony has become a legally recognised form of ‗specialised knowledge‘
admissible in criminal proceedings.37 Police and prosecutors in NSW now routinely retain
the services of experts, like anatomists, to examine images and to testify about apparent
similarities between the accused and images of a person of interest.
   R v Jung (2006) NSWSC 658, [55].
   Jung, [9]. During cross-examination, on the voir dire, Dr Sutisno ‗was not prepared to admit that she could make
a mistake‘ using her techniques.
   R v Kaliyanda (unreported interlocutory judgment) Supreme Court of NSW (17 October 2006).
   R v Murdoch [2007] NTCCA 1, [300].
   R v Alrekabi [2007] NSWDC 110.

In this section we endeavour to explain why expert identification evidence based on
images may be unreliable and unfairly prejudicial. Our argument has three major
components. First, we explain that facial mapping has never been subjected to any form of
evaluation, and as a result we have no way of knowing whether the technique is accurate.
We then consider a range of factors which we believe are likely to undermine the accuracy
of the technique and increase the chance of erroneous conclusions. Finally, we provide an
outline of experiential evidence which indicates that jurors are likely to be strongly
influenced by facial mapping evidence, regardless of its accuracy.

A. Validation and proficiency testing
Despite the widespread use of facial mapping evidence in courts in Australia and the UK,
there is no empirical evidence regarding the validity or reliability of the technique(s). The
validity of a technique is the extent to which it does what it is claimed to do, while
reliability is a measure of consistency or the extent to which the technique will produce the
same results when repeatedly applied under varying conditions or by different operators.
Reliability and validity are characteristics of all measurement systems, and must be known
to the operator if he or she wishes to reach appropriate conclusions based on the use of the
system. Because the reliability or validity of facial mapping has never been assessed we do
not know whether the technique can determine whether two images are of the same person,
or the extent to which the conclusions reached using this technique will depend on factors
such as the training of the operator, or the precise details of the case in question.
    The reliability of a measurement technique is one of the factors which determines its
accuracy and the probability that its application will result in an error (the error rate). All
measurement systems have some level of error associated with their use, and this is
acceptable provided the error rate and accuracy of the system are known and taken into
account. Different levels of accuracy of measurement are required depending on the use to
which the measurement is being put. For example, the accuracy with which we need to
determine the dimensions of a room will depend upon whether we intend to cut carpet for
the floor or estimate the amount of paint required to decorate it. The former use requires a
higher degree of accuracy than the latter.
    When, as in the case of facial mapping, we use a measurement technique to make a
binary Yes/ No decision, the situation is more complicated. In these circumstances, there
are two measures of accuracy and two measures of error rate to consider. Assuming that
the objective of facial mapping is to decide whether two images are of the same person,
then there are four possible outcomes to this binary decision process. First of all, if the
images are of the same person, we can either correctly decide that they are of the same
person (a hit) or incorrectly decide they are of different people (a miss or false negative
error). If, on the other hand, the images are of different people we might either wrongly
conclude that they are of the same person (a false alarm or false positive error) or correctly
decide they are of different people (a correct rejection). These four outcomes are not
independent of one another. If, for example, an operator always concludes that two images
are of the same person, regardless of the evidence presented to them, the operator will have
a perfect hit rate (i.e. 100 per cent), but will make a false positive error whenever presented
with images of different people. In order to measure the true ability of the operator to
determine whether or not the two images are of the same person we need to measure both

the hit rate and the false positive rate and combine these using a statistical technique
known as signal detection theory.38
   The reliability, hit rate, false alarm rate and hence the validity of a system can quite
easily be established by a validation study. In the case of facial mapping, in a typical
validation study an experimenter would ask operators to evaluate pairs of images which the
experimenter knew to be of either the same or different people. In this way we can
compare the operators‘ decisions to the ground truth. No such evaluation of facial mapping
has ever been attempted, and as such we have no way of knowing how well the operators
would perform in this situation, how accurate their decisions would be, whether different
operators would reach the same conclusions or even whether the same operator would
reach the same conclusion when evaluating a pair of images on two separate occasions.
   Defenders of facial mapping might argue that it is setting the bar too high to require a
technique to be subject to a validation study before we allow it to be presented in court. In
support of this position they could point out that other widely accepted forensic techniques,
including fingerprinting, have never been formally evaluated in this way.39 There is some
merit to this argument, but facial mapping and fingerprint analysis vary in one very
important way. Although not subject to validation studies, the work of fingerprint analysts,
unlike facial mappers, is subject to proficiency testing designed to establish the analyst‘s
knowledge of the techniques and procedures they should apply and their ability to apply
these techniques. Although proficiency testing doesn‘t tell us whether the technique is
valid, it does at least tell us whether the individuals involved understand what they are
required to do and undertake the task competently. In the case of latent fingerprint
examiners we know how well a particular analyst can apply the technique in which they
have been trained, but we don‘t know how valid that technique is. In the case of facial
mapping, not only do we not know if the procedure is valid, we don‘t even know if the
operators are doing the same thing. We will return to this point later.

B. Some factors likely to undermine the validity of facial mapping
Having established that facial mapping has never been subjected to a validation study, and
that we therefore do not know whether it is a valid technique, it is our intention to outline a
number of factors which are likely to undermine the validity of expert identifications
derived from images.

Characteristics of the image
Facial mapping often involves the comparison of a reference image of the suspect to an
image recorded using a surveillance system such as a CCTV camera. In these cases the
quality of the surveillance images may significantly limit the accuracy with which
comparisons can be made. Surveillance cameras, such as those used in shops or public
places, are usually set to monitor a wide area, so the person of interest will typically
occupy only a small percentage of the image area. In addition, the cameras are often
positioned some distance off the ground to protect them from interference, and as a result
often shown an unusual angle of view. These cameras may also have to cope with large
variations in lighting, and may not be subject to regular maintenance. In older systems the
images are often recorded to analogue devices, such as VHS tape and the tape is often
reused many times, giving rise to a very significant degradation of image quality.
Furthermore, to reduce storage requirements, many systems record at a rate far below the
standard 24 frames per second, with some recording only one frame per second.

     Simon Cole (2006).

Consequently, most of the information, and detail of characteristics such as movement
patterns, may not be recorded.
    In addition, the characteristics of an image impose an absolute limit on the amount of
information that can be extracted from the image. Forensic science based television
programs such as CSI often feature image enhancement systems which allow the operator
to zoom into images to examine minute details of the scene. Unfortunately such systems
are fictitious. No amount of image enhancement or clever technology can restore
information which was not recorded because it was beyond the resolution of the system.
Recall the number plate in R v Murdoch. The location of an image component can only be
resolved to an accuracy equivalent to the size of two pixels, and details which are below
the size of a single pixel are not recorded. The quality of the camera‘s optics and recording
system may further limit the amount of information contained in the image, and the optics
of the lens may introduce considerable distortion. It is important to recognise that the
image used in the facial mapping process is a limited, degraded and distorted
representation of reality, and nothing can be done to overcome these limitations to image
    The angle of view at which an image is recorded will also have a significant impact on
its appearance. A face is a complex three dimensional structure which in a photograph is
rendered into a two dimensional image. As a result small changes to the angle of view will
result in marked changes to the appearance of the image. For example, if we consider a
portrait style photograph of the face, the apparent size of the nose will change with even
very small changes in the tilt of the head. Raising the chin toward the camera will make the
nose appear smaller while dropping the chin away from the camera will make it appear
longer. Although the human visual system is capable of using high level information such
as the shape of shadows to estimate three dimensional structure, the accuracy with which
this can be achieved is likely to be limited and will be affected by lighting and other
factors. Any measurements taken directly from an image are likely to change with angle of

Statistical issues
To be reliable and accurate facial mapping analysis would have to be based on very a
detailed knowledge of the frequency of occurrence of various facial features within the
relevant population. Without such data no conclusions can be drawn from the observation
that two images share one or more characteristics. The observation that two images both
display a particular facial characteristic, say a particular nose shape, only becomes
probative if we know how frequently that characteristic occurs within the relevant
population. Over the course of the last century anthropologists and others have measured
relatively small groups of individuals, but most experts recognise that we would need far
more detailed information to reliably estimate the frequency of the occurrence of a set of
features.41 Mardia et al explain that:

          The science behind this method, termed ‗facial mapping‘ by the UK press, is still emerging and the
          establishment of a reliable database of population statistics is now viewed as essential if we are to be
          able to lay down standards for facial comparisons which will ensure that any positive identification
          made is regarded as safe.

   This creates serious problems for those using anthropometry, especially if they do not use the original equipment
in its original configuration.
   e.g. Bertillion.
   Mardia, Coombes, Kirkbride, Linley & Bowie (1996) p.658. Compare DNA population statistics.

To develop the required database would be a considerable undertaking. The database
would need to include accurate measurements of a large number of characteristics across
many thousands of individuals. Furthermore, it is likely that we would require different
databases for different ethnic groups.43 We know that the occurrence of blue eye colour is
fairly common in some populations but very rare in others. If we want to draw a
conclusion based on the observation that two images both show individuals with the same
eye colour, we need information about the frequency of different eye colours in the
appropriate population. It is not clear how narrowly we would need to define a population
in order to support valid reasoning. For example, would it be sufficient to have different
databases for broad groups such as ‗Caucasian‘ and ‗Asian‘ populations, or would it be
necessary to subdivide these populations.
    It is not enough just to know how frequently each facial characteristic occurs within the
relevant population, we also require information regarding the statistical independence of
the features from each other.44 If two or more features are correlated with each other (that
is, if the occurrence of one feature predicts the occurrence of another feature) then the
probability that two images share both these features is higher than is the case if the
features are statistically independent of one another. In the extreme case where the
correlation coefficient is 1.0 and the occurrence of the first feature perfectly predicts the
occurrence of second feature, then the observation that the two images share both features
is no more informative that the observation that they share just one of these features.
Imagine that two images share three characteristics which we know occur with a frequency
of 0.2, 0.4 and 0.6 in the relevant population. If these features are totally independent of
each other, then the probability that two images sampled at random from the relevant
population will share all three characteristics is 0.048 (i.e. 0.2 x 0.4 x 0.6 = 0.048).
However, if these characteristics are not independent of one another then the calculation
becomes much more complex and the likelihood that two randomly selected images will
share all three of these characteristics rises dramatically. In their analysis of the features of
358 young white males Mardia et al (1996) observed correlations as high as 0.94.
    Some courts have ruled that expert witnesses should not present their conclusions in the
form of probabilities (e.g. ‗It is 98% certain that these two images are of the same person‘)
requiring instead that the witness simply notes the similarities they have observed in the
two images. It is important to note that avoiding the use of probabilistic language in this
way does not overcome the statistical problems detailed above. This change represents a
switch from an explicit to an implicit probability. The change in the language use to
express conclusions may disguise faulty reasoning behind the expert‘s opinion evidence.

The absence of a standardised procedure
Kemp et al undertook an international survey of all individuals involved in providing
identification evidence from the examination of images.45 A total of 26 individuals from
several countries and with an average of 17 years of experience responded to a series of
questions about their background, training, the techniques they employ and factors which
affected the accuracy of their conclusions. The most striking finding was the diversity of
the responses. For example, while some respondents reported that they always obtained
comparison images taken at the same location as the original surveillance image, others
reported that they never did this, and while some reported that, if it was available, they
   This has raised problems for population statistics associated with DNA profiling. Consider Bropho v The State of
Western Australia [2007] WADC 77.
   Interestingly, most of the available facial mapping decisions in Australia involve trials of people of non-
Caucasians descent: Smith and Kaliyanda (Aborigines), Jung and Tang (Asian), Alrekabi (Middle Eastern) and
Pera (South Pacific Islander).
   (2008) ref to EAPL conference.

always used the same camera to capture the comparison images, others never did this.
Although about one third of the respondents claimed to use published data to compute the
probability that the two images were of the same person, several others indicated that data
did not exist to support such an approach. While some respondents reported that the
technique they used was largely unaffected by differences in the angle of view of the two
images, limitations in image quality or the elapsed time between two images, others
reported that each of these factors would affect the accuracy of their results.
   We noted earlier that, unlike fingerprint examiners, individuals who engage in facial
mapping are not subject to proficiency testing. Kemp‘s survey sheds light on this area.
Proficiency tests will normally only occur within a field if there is a regulatory body in
place and an agreed set of operating procedures which practitioners recognise and against
which performance can be assessed. In the case of facial mapping there are no accepted
operating standards, and as a result operators use different techniques and have diverging
opinions about best practice and the reliability and accuracy of their techniques.

Unconscious influence
The results of several decades of experimental psychology research demonstrate that an
individual‘s expectations and prior beliefs can influence their judgement. There have been
many demonstrations of this phenomenon, but one recent study is of particular relevance to
facial mapping because it suggests that even trained experts are subject to influence and
bias. Dror et al (2006) demonstrated that fingerprint experts‘ professional judgments could
be influenced by extraneous information. In this study five very experienced fingerprint
examiners were each re-presented with a pair of prints which, five years earlier, they had
declared a match. The experts were unaware that they had previously matched the prints
and on the second occasion the prints were accompanied by information which suggested
that they were from two different people. Of the five experts tested only one did not
change his opinion regarding the prints. Four of the examiners now concluded there was no
match and the other that it was not possible to make a definite determination. This is a
clear demonstration that even highly trained individuals using standardised techniques can
be influenced by expectations induced through contextual information.
   In the case of facial mapping there are probably several significant biasing factors
which might influence the operator. For example, if the operator examines the surveillance
image and the reference image at the same time, there is a danger that their interpretation
of ambiguous characteristics of the surveillance image will be influenced by their
knowledge of the appearance of the reference image. There is also a danger that any
suggestion that other people believe the two images are of the same person might bias the
operator towards a similar decision. Such an influence is likely to occur during the
presentation of the case materials to the operator. In cases we have observed, the letter of
instruction from the investigating police officers makes it clear that they believe that the
images are of the same person and the reference photographs and the security images are
typically included in the same package.
   Cambell-Tiech describes a murder trial which provides another example of the
operation of such influence in facial mapping cases.46 In this case four independent facial
mapping analysts were asked to provide reports comparing surveillance images to images
of the suspect. Initially all the analysts agreed that there was some support for the
proposition that the surveillance images were of the suspect. However, some time later the
police decided they had arrested the wrong man and identified a new suspect. The same
analysts were asked to compare the CCTV images to photographs of the new suspect. This


time two of the experts stated that there was ‗no support‘ for the proposition that the
suspect was the person in the video, the third concluded that there was support for the
position that the images were not of the same person and the fourth that there was
‗powerful support‘ that they were the same person. The police then asked the third expert
to reconsider his analysis and made it clear to him that they felt his previous conclusions
were wrong. After reviewing his findings and without access to additional material this
expert concluded that he could not exclude the possibility that the suspect was the person
shown in the CCTV images.

C. The influence of facial mapping evidence
In the previous sub-sections we explained that facial mapping techniques have not been
formally evaluated and proceeded to identify a number of factors which are likely to
undermine the validity of the technique. In this final section we review evidence which
suggests that facial mapping evidence is likely to exert considerable influence on the
tribunal of fact.

The psychology of face perception
Experimental research has demonstrated that, although we are able to recognise familiar
faces from very poor quality images, we struggle to identify unfamiliar faces even when
image quality is high. Kemp et al investigated the ability of shop assistants to determine
whether a photograph on a credit card was of the unfamiliar person attempting to use the
credit card to purchase goods.47 Although this task did not require the participants to
remember the faces, overall performance was less than fifty per cent. Burton et al showed
students images from a CCTV system which monitored the entrance of a University
building.48 Despite very poor image quality students were able to recognise academics who
taught them, but were unable to match high quality images of unfamiliar academics.
Overall these and other studies show that people make many errors when trying to
determine whether two different images are of the same unfamiliar person. This, of course,
is the task facing jurors when presented with surveillance images and a suspect standing in
    In a recent experimental study of the influence of facial mapping evidence, Kemp et al
(2008) provided participant-jurors with video sequences of a defendant in court. The jurors
were then shown CCTV sequences featuring either the defendant or a different individual
fleeing a crime scene. Participant jurors were required to decide whether the person in the
CCTV material was the defendant and were then presented with expert opinion evidence
led by either the prosecution or the defence. The results showed that the participants‘ initial
judgements were often inaccurate. About half of the jurors who saw someone other than
the defendant were prepared to convict the defendant. However, once the juror had read the
expert report, his or her verdict was predicted only by its content, and not by the ground
truth. That is, most jurors were persuaded to adopt the position advocated by the facial
mapping witness, even when that assessment was wrong.
    In summary, the validity of facial mapping has not been tested, and there are a number
of reasons to doubt the accuracy of the technique. All this would be unimportant if jurors
were able to ignore facial mapping evidence and could reliably decide for themselves
whether the surveillance images were of the accused. Unfortunately research tells us that
jurors are prone to make identification errors, and are very likely to be influenced by expert
identification evidence, regardless of its accuracy.

  Kemp at et (1997).
  Mike Burton et al, ‗Face recognition in poor-quality video: Evidence from security surveillance‘ (1999) 10
Psychological Science 243.

Conventionally, judges tend to rely upon the strengths of the adversarial criminal trial
when rationalising the admission and use of incriminating expert evidence adduced by the
prosecution. This section builds upon the previous discussion by suggesting that many of
the traditional adversarial safeguards are not particularly effective, not applied in ways that
actually protect the accused and that many of the risks associated with expert identification
evidence actually fall upon the accused.
   The first issue to consider is why our admissibility jurisprudence has not prevented such
suspect expert opinion evidence from contaminating criminal trials.

Admissibility standards and discretionary exclusions
Our existing admissibility standards and exclusionary discretions have done little to
alleviate problems with expert identification evidence.49 The common law, with its
emphasis on the existence of a ‗field‘ and whether a person is an expert (in that field), and
the emphasis on ‗specialised knowledge‘ and the ‗basis‘ of opinions under section 79 of
the Uniform Evidence Law, have not encouraged judges to attend to the validity or
reliability of expert opinion evidence. Instead, admissibility criteria have been
operationalised in ways that place greater emphasis on an expert‘s formal qualifications
and experience in court than whether their techniques have been successfully tested.
Earlier we observed how the NSW CCA, in Tang, interpreted ‗specialised knowledge‘ in a
manner that explicitly repudiated the need for ‗reliability‘.
   Whether judges are concerned about ‗specialised knowledge‘ or whether the witness is
an ‗expert‘ offering an opinion based on a recognisable ‗field‘, facial mapping evidence
and the comparison of images should induce concern. There is no facial mapping ‗field‘ or
discipline, no specialised facial mapping knowledge, no university courses, no dedicated
journals, no attempts to standardise or regulate techniques, and no serious attempt to test or
validate the range of methods used by its disparate practitioners. Almost all of the extant
studies with relevance to identification from images are critical of facial mapping. Most
suggest that things are far more complicated than those proffering testimony would seem—
on the basis of their testimony and reports—to appreciate (or publicly concede).
   To the extent that judges allow anatomists (or others) to testify we can only wonder
what the ‗basis‘ in their ‗training, study and experience‘ actually is. What in a person‘s
anatomical training makes them skilled or proficient at making comparisons between two-
dimensional images, especially where the images are of very poor quality. What relevance
does facial reconstruction or anatomical terminology have to photographic comparisons
and identifications? Abilities should be demonstrated rather than asserted, particularly
where techniques and methods have obvious limitations and have been subjected to serious
and unanswered criticism. Existing approaches to admissibility have allowed considerable
slippage between an expert‘s qualifications, training, study and experience and the kinds of
subjects on which they are allowed to pronounce.
   In the wake of Makita (Australia) Pty Ltd v Sprowles (2001) judges and lawyers were
encouraged to take a renewed interest in the basis of opinions derived using ‗specialised
knowledge‘. This, however, has tended to encourage a rather perfunctory interest in the
techniques and procedures relied upon by experts. In the case of expert identification
evidence, most judges have been satisfied with a vague description of the process used by
the witness. The trouble with indifference to validity and reliability is that individuals with
  For a much more expansive discussion of these issues see Edmond, ‗Specialised knowledge, the exclusionary
discretions and reliability: Reassessing incriminating expert opinion evidence‘ (2008) UNSWLJ (in press).

formal expertise are able to give opinion evidence as long as they have engaged in some
comprehensible procedure, even if the procedure has not been shown to reliably produce
the kinds of results claimed.50
    The exclusionary discretions, based on Christie at common law and sections 135 and
137 under the Uniform Evidence Law, have exerted almost no discernible impact on the
reception of expert opinion evidence in Australia. Designed to balance the probative value
of evidence against the danger of unfair prejudice to the accused, they are rarely used to
exclude incriminating expert opinion evidence adduced by the prosecution. Given the
many weaknesses with facial mapping evidence and the status of the witnesses presenting
identification evidence it might be thought that the unknown probative value of the
evidence would be outweighed by the danger of unfair prejudice to the accused. Here, one
prominent danger is that a highly-credentialed expert might be mistaken in circumstances
where it is very difficult to demonstrate that they are actually wrong.
    Instead, judges have exhibited an incomprehensible reluctance to make any assessment
of the probative value of the prosecution‘s expert opinion evidence. Unwilling to trespass
on the prerogatives of the jury, when applying the exclusionary discretions judges assume
that the evidence might be accepted by the jury. In consequence, when balancing the
probative value of the evidence against the danger of unfair prejudice to the accused, they
take the evidence at its highest.51 Taking expert opinion evidence ‗at its highest‘ tends to
mean accepting that the expert can reliably do what he or she claims. Where the evidence
is assumed to be reliable, because a jury might come to that conclusion (i.e. ‗if accepted‘),
the risks of unfair prejudice to the accused, in theory, evaporates. For, if we assume that
the expert evidence is reliable then any dangers, unfair prejudice and forensic disadvantage
caused by unreliability, simply disappear.

Defence, prosecution and the adversarial criminal trial
Judges tend to admit incriminating expert opinion evidence because they believe that
problems with validity and reliability, like the credibility of an expert, can be productively
canvassed during the adversarial trial. There is a tendency to trivialise the very different
roles that prosecution and defence experts fulfil in criminal trials. Ordinarily, expert
identification evidence forms part of a circumstantial case against the accused. This
evidence is usually integrated into a coherent and interrelated assemblage. In contrast, and
consistent with the distribution of burdens of proof and the accused‘s interest in doubt, the
evidence of defence (i.e. rebuttal) experts is usually critical. Rebuttal experts usually focus
on limitations, uncertainties and problems with the techniques employed by what are often
very experienced and sometimes extremely well-credentialed forensic scientists. It is not
uncommon for rebuttal experts to report on difficulties and problems without undertaking
their own assessment or investigation.52
   Adopting what might be considered to be an even-handed approach to expert opinion
evidence adduced by the prosecution and defence can be quite misleading and, in the
context of a criminal proceeding, unfair. The decision to admit unreliable expert opinion
evidence or expert opinion evidence of unknown reliability creates forensic disadvantage
for the defence and prejudice to the accused. The defence is almost always required to
respond. The defence must consider if and how it might challenge incriminating expert
opinion evidence and whether they can afford or, more practically, obtain the services of a
rebuttal expert. Preparation for cross-examination can require a considerable investment in

   Here, a ‗procedure‘ might include the comparison of two images on a computer screen (or across two computer
screens), or examining images of the unknown offender prior to looking at police reference photos of the accused.
   R v Shamouil [2006] NSWCCA 112.
   See R v Madigan [2005] NSWCCA 170 at [107] (Wood CJ at CL, Grove and Hoeben JJ concurring).

time and often requires the assistance of an expert. Rather than placing responsibility for
demonstrating validity and reliability squarely on the prosecution and investigators—that
is, the state—admitting potentially unreliable expert identification evidence transfers
responsibility for demonstrating unreliability onto the defence. This represents
inappropriate cost and burden shifting.
    Where experts called by the prosecution are allowed to opine on issues at the
peripheries of their expertise (or beyond), or loosely connected to their ‗training, study and
experience‘, it can be practically difficult to contest their evidence or persuasively convey
the seriousness of limitations. This danger is accentuated by the overall strength and depth
of the prosecution case mitigating weaknesses in their expert‘s techniques or incriminating
opinions. Where testimony is based merely on an expert‘s interpretations and assertions,
what is sometimes referred to as ipse dixit, in the absence of established methods,
techniques and means of evaluating the interpretation, expert disagreement is reduced to a
credibility contest. The defence are obliged to challenge the expert‘s competence and
credibility in a context where the state, the court and the prosecutor each implicitly confers
its imprimatur.
    The conventional adversarial trial is over-rated as a means of consistently exposing
limitations with incriminating expert opinion evidence.

       Just because cross-examination might be able to expose problems with a technique or an expert might
       be found who is willing to testify about weaknesses in the forensic scientific evidence, does not mean
       that these should provide the primary bulwark against unreliable and prejudicial expert opinion
       evidence adduced by the prosecution. Facilitating cross-examination or allowing the defence to call
       rebuttal expertise does not make a trial fair. Structural symmetry is not the same as substantial

Expert disagreement and the jury
As things stand there is a tendency to allow incriminating expert opinion evidence based
on images and related expert disagreement to go before lay jurors. Often this is linked to
the impression that the jury ‗would be assisted by‘ expert opinion evidence.54 In many
circumstances, however, granting primacy or deferring to the jury is inappropriate. Where
expert opinion evidence is not demonstrably reliable we must wonder about the propriety
and fairness of leaving that evidence or related disagreement to the jury. The lay jury
should not be required to judge unreliable evidence or speculative opinions.
   Similarly, the fact that two or more experts disagree does not mean that there is a
genuine scientific dispute or that both ‗sides‘ should be entitled to go before the jury.
Where the proponent of opinion evidence has not undertaken validation and reliability
studies, the existence of criticism does not transform their claims into a credible scientific
position that can or should be heard and resolved by a jury. Where the evidence is of a kind
that is likely to proliferate the need for evidence of validity and reliability is acute.

Directions and warnings
Frequently, facial mapping (and other kinds of questionable expert opinion evidence) is
admitted with the proviso that problems can be explored through cross-examination and
rebuttal evidence, and risks reduced by judicial instruction—that is, directions and

   ‗Specialised knowledge‘. In Murdoch, Jung, R v Alrekabi (2007) and R v Pera (2007 and 2008) trial judges and
the NT Court of Criminal Appeal were comforted by the participation of rebuttal experts—called by the defence.
   For example, Murdoch, [107].

warnings. Unfortunately, almost all of the empirical research suggests that judicial
instructions, directions and warnings are, at best, ineffective.55
   We should be reluctant to allow unreliable expert opinion evidence to go before juries
on the basis that limitations and dangers will be identified and explained through judicial
directions and warnings. (We should not forget that the judges giving these instructions are
the same judges who admit the incriminating expert identification evidence.) Individually
and/or in combination, cross-examination, defence witnesses, and judicial instructions do
not guarantee that serious limitations with facial mapping evidence and expert
identification evidence will be conveyed or appreciated. Individually and/or in
combination they do not provide adequate safeguards against unreliable expert opinion

Similarity/difference evidence
To be fair, senior judges have imposed some limitations on the provision of expert
identification evidence derived from images. In NSW, for example, the CCA has
endeavoured to prevent experts from making positive identifications (in court, at least).
These limitations have been imposed as a way of managing facial mapping evidence and
limiting potential prejudice. Restricting the expert opinion evidence to similarities might
not provide much protection to the accused. In most trials the jury is left with little doubt
that it is the expert‘s considered opinion that there are no meaningful differences between
the accused and the unknown person in the images.
   Interestingly, legal conventions rather than scientific norms prevent experts from
proffering their opinion and the strength of their conviction.56 Ironically, these legal
constraints may make experts retained by the prosecution appear more cautious and more
credible then they actually are. (It will not usually be in the accused‘s interest to attempt to
expose an expert‘s personal view about identity as a way of challenging their credibility or
methodological apathy.)
   Allowing experts to provide similarity evidence simply elides questions about whether
anatomists and physical anthropologists, notwithstanding a formidable technical
vocabulary, are actually able to see similarities and differences better than lay people and
whether the similarities (and differences) in the images are real or artefactual. Furthermore,
the ability to identify a few similarities from a large number of images raises questions
about selection biases.

        Why the many images which do not provide similarities should be ignored or trivialised is never
        adequately explained. Why should images that appear to present similarities be preferred over those
        that are unclear or might suggest dissimilarity? There are no analytical or statistical techniques to
        explain these prejudicial preferences.

Circumstantial evidence, independence and necessity
Usually, facial mapping evidence is presented as part of a more complex and integrated
case against the accused. One of the dangers with unreliable expert opinion evidence is that
the limitations with the techniques or approach may be overlooked or discounted because
of corroborative evidence. This is regrettable. For admissibility, the validity and reliability

   Joel Lieberman and Bruce Sales, ‗The effectiveness of jury instructions‘ in W. Abbott and J. Batt (eds), A
Handbook of jury research (1999) 18.1; James Ogloff and Gordon Rose, ‗The comprehension of judicial
instructions‘ in N. Brewer and K. Williams (eds), Psychology and law: An empirical perspective (2005) 407.
   I Mitroff, The subjective side of science (1974); M Mulkay, ‗Interpretation and the use of rules: the case of the
norms of science‘ in T. Gieryn (ed) Science and Social structure (1980).
   ‗Specialised knowledge‘. See also Geoffrey Bowker and Susan Star, Sorting things out: Classification and its
consequences (1999).

of expert opinion evidence should be considered independently from the other evidence.
The overall strength of a case or the existence of other evidence should not be used to
supplement techniques and opinions that are not demonstrably reliable.
   When considering the admissibility, probative value, prejudicial effect and reliability of
expert opinion evidence, it is useful to know something about the provenance of an
expert‘s opinion. If experts are in conversation with other investigators and are aware of
other features of the case against the accused then this may—consciously or
unconsciously—influence their interpretations. To the extent that experts are engaged in
interpretive tasks, prejudicial information compromises independence and tends to
undermine reliability. (Remember the study of the fingerprint examiners.) Where facial
mappers know about other aspects of the case it is a mistake to treat their opinions as
independent corroboration. (This operates in addition to the range of methodological
   Necessity, or the apparent necessity, of identification evidence does not provide an
adequate basis for the admission of expert opinion. If expert witnesses cannot satisfy a
court that their techniques are valid and reliable then any opinions, however interesting or
incriminating, are neither necessary nor desirable.

Changing techniques to evade criticism (i.e. law-science co-production)58
One of the interesting developments in expert identification evidence based on images is
that those testifying in Australian courts have exhibited a conspicuous preference for
qualitative methods. Under intense scrutiny (and perhaps pressure from prosecutors) expert
witnesses in Australia seem to have abandoned their more quantitative approaches—
anthropometry and concern about the distribution of features among populations—in
preference for more qualitative and subjectively-based interpretations. Morphological
approaches enable experts to effectively identify an offender or accumulate similarities
without having to address questions about the distribution of features (i.e. are particular
features or sets of features common within certain populations), whether features are
independent of one another, or whether features are real or merely artefacts of the CCTV
and surveillance images. The prosecution witness identifies similarities and then the
defence are required to overcome the prejudicial effect of this expert opinion evidence.
   Quantitative methods bring some of the technical problems with the use of images to the

Improving expert identification evidence based on images: Prophylaxis
Our contention, which should now be clear, is that to gain admissibility, there should be
evidence supporting the validity and reliability of incriminating expert opinion evidence.59
Prosecutors, defence lawyers and judges should all be expecting to see validation studies
and error rates. Judicial concerns with admissibility should be focused on whether the
techniques and opinions are sufficiently reliable for admissibility and whether the
techniques and theories used are sufficiently similar to those tested and published in
mainstream scientific, biomedical and forensic science journals, as well as evidence of the
particular expert‘s credentials and proficiency.
   Even in the absence of validation studies courts (and investigative agencies) could
dramatically improve the standard of expert identification evidence based on images. They
could, for example, require two different experts to each examine a single set of
photographs—one would examine a set of images of the accused and the other a set of

  Sheila Jasanoff (ed), States of knowledge: The co-production of science and social order (2006).
  We would argue that the prosecution must satisfy the trial judge, using evidence, that on the balance of
probabilities the techniques are reliable. See also E Beecher-Monas, Evaluating scientific evidence (2006).

images of the unknown person associated with a crime—using a pre-determined set of
anatomical parameters. Attention could even be focused on particular regions of the body
or features, such as ears or nose or head shape, based on a preliminary examination of the
images of the unknown person. The two sets of descriptions prepared by experts could then
be compared. Differences in description would presumably lead to the exclusion of the
evidence. Low level consistency might not be sufficiently probative to overcome the
dangers (section 137) or the expenditure of time and resources (section 135). Such an
approach would not guarantee reliability but it would prevent the most egregious expert
identification evidence going before a jury.
   Obviously, the preferable approach is to actually test the techniques and the proficiency
of individuals.

There are good reasons to doubt the value of expert identification evidence based on
images as it currently appears in Australian and English courts. There are also good
reasons to question many of the safeguards routinely invoked to reassure those concerned
about the reliability of expert evidence, the fairness of trials and the safety of criminal

[To be developed.]

APPENDIX 1             Photo-anthropometry

            Offender                              Mr Moamanu

      An illustration of the photogrammetric points.

APPENDIX 2             Image Superimposition

                  Offender                         Mr Moamanu


Source: Rod McCourt, The requirement for expert imagery analysis available at

Shared By: