The effects of relevance of on-screen information on gaze

							The effects of relevance of on-screen
information on gaze behaviour and
 communication in 3-party groups


                  Emma L Clayes
              University of Glasgow
          Supervisor: Prof. Anne Anderson
             Co-Supervisor: Jim Mullin
          BT Supervisor: Dr. David Hands
          Sponsored by UK ESRC and BT
Outline



  Background & Hypotheses
  Design and Method
  Results
  Summary
  Future work
  Conclusion
Representations: form and location

    Many studies concerning mediated
     communication (Finn et al, 1997) with
     different technologies, measures and
     tasks

    Little research on the impact of
     different representations (e.g Sellen,
     1995; Parise et al, 1996)
Representations and relevance

    How does the form and location of
     representations impact on gaze and
     communication?

    How does this interact with the relevance
     of on screen information ( e.g. task related
     information and representations in terms
     of the task role)
Evaluation techniques in CMC

    Many different measures used to examine
     computer mediated communication (e.g.
     dialogue analysis, task performance,
     questionnaires)

    Additional factors (e.g. task, no of
     participants, technology) and different
     methods have led to conflicting results
Methodological issues

  Three-party groups
  Problem solving and social tasks
  Communication analysis,
   questionnaire responses and task
   performance
  Eye-tracking as an evaluation
   technique
Eye-tracking in Psychology and HCI

    Well established measure of human
     information processing (Rayner, 1998)

    Eye-gaze computer interfaces (Jacob,
     1991)

    Recent studies examining gaze and CMC-
     Velichkovsky et al (1997), Mullin et al
     (2001)
Communication and Attention

   Eye-tracking as an evaluation technique:
       What do users attend to when using
        remote communication systems?

   Does the relevance of on screen
    information impact on gaze behaviour?

   Are patterns of gaze related to patterns of
    interaction within the group?
Eye-tracking in CMC

   Exploratory questions:

 Is it possible to obtain significant amounts
  of eye data during a non-restricted
  interaction?
 Are participants consistent in their
  patterns of gaze across a screen?
 Are these patterns meaningful?
Communication and Relevance


   Relevance of on screen information (e.g.
    video data and video links)

   Shared visual data more useful than video
    conference links (Daly-Jones et al 1998)
Communication and Relevance


    Relevance of other collaborators (e.g. task
     role and status)

    Status effects in mediated communication
     (Dubrovsky et al 1991, France et al, 2000)
Study 1

 What on-screen features attract most
  gaze?
 Does the position of representations on
  screen impact on mediated
  communication?
 Does this interact with the type of task and
  relevance of information presented on
  screen?
Task 1-Problem-solving task

 Map task-collaborative problem-solving
  task
 Two Instruction Givers (IG1 and IG2) have
  to instruct one other person (Instruction
  follower-IF) on how to draw a route on their
  map
 Therefore, relationship between IG and IF
  more relevant to task success than IG1
  and IG2
Task 1-Display Screen
Task 1-Hypothesis I


   Hypothesis I- Users will gaze more
    often at shared visual data ( i.e. the
    map) than video links of remote
    collaborators
Task 1-Hypothesis II


    Do participants look equally often at
     the IF and IG video images?

    Does the location of the video image
     affect gaze behaviour?
Task 1-Hypothesis III


    Do patterns of gaze reflect patterns of
     interaction within the group?

    I.e. Do participants talk more often to
     the person they look most often at?
Design

     2 factor mixed design

    Relevance (map, video-IF, video-IG)
     within subjects

    Location (video-IF top, video-IF
     bottom) between subjects
Method

 Subjects- 10 groups of 3, 20 eye-
  tracked
 2 maps, order of maps and location
  of IF video balanced
 3 participants in different rooms-high
  quality audio and video links-
 Eye tracked participant always an IG
Results


   Screen divided into areas of interest for
    eye gaze analysis

   Percentage analysis on different areas of
    interest and different fixations
Eye Data-Pictorial analysis
Results-Hypothesis I


   Participants spent significantly more time
    looking at the map (72.4%) than video
    image of IF (12%) and IG (10%)

   Relevance of on-screen information
    impacts on gaze behaviour
         F(2,36)=258.15, p<.001
Results-Hypothesis II


 Do participants look more often at IF
  compared to IG and is this affected
  by location?
 Main effect of relevance (IF 12%, IG
  10%) p<.05
 No effect of location p>.05.
Results-Hypothesis II

   Significant interaction between relevance
    of video image and location of video
    image F(1,18)=5.73,p<.05

   Participants looked more often at the
    instruction follower - only significant when
    the IF video was located in the top left of
    the screen (F=10.48, p<.05)
Results-Hypothesis III

 Do patterns of gaze reflect patterns of
  interaction within the group?
 Does the eye tracked participant direct
  more turns of speech to the IF or IG?
 Turn combination analysis: IG1-IG2, IG1-IF
 No effects of relevance (task role-IG2/IF),
  location of IF or an interaction between
  the two.
Results-Hypothesis III

     Turn Combination results
                 IG1-IF     IG1-IG2
     IF top left       20.7       22.6
     IF bottom left    19.5       23.2
                       20.1       22.9

     Average no of turns by IG1-43
Task 1-Summary


    Users look more often at shared visual
     data (map 72.4%) than video images (22%)

    Users gaze more often at IF than IG-only
     significant when IF video is located in top
     left of screen
Task 1-Summary


   Communication patterns reflect that
    IG directs almost equal number of
    turns to IG and IF- does not reflect
    patterns of gaze

   Positional effect for salient video
    images?
Task 2-Relevance and Status

 Realistic task-mediated business
  meeting
 Two confederates- one high status
  (boss) and one low status (marketing
  assistant)
 Video data presented not essential to
  complete task
Task 2-Relevance and Status


    Hypothesis I- Do participants gaze more
     often at shared data compared to video
     images when the information is relevant,
     but not essential to complete the task.
Task 2-Relevance and Status


    Hypothesis II- Do participants look more
     often at a high-status collaborator?

    Does the location of the high status video
     image impact on gaze behaviour?
Task 2-Relevance and Status


    Hypothesis III- do patterns of gaze reflect
     patterns of interaction within the group?

    I.e do participants talk more often to the
     person they look most often at?
Task 2-Design

2 factor mixed design

   Relevance (visual graphic, video-high
    status, video-low status) within subjects

   Location (video-high status top, video-
    high status bottom) between subjects
Task 2-Method


   2 confederates, 20 eye-tracked
    participants

   3 participants in different rooms-high
    quality audio and video links
Task 2-Display Screen
Results-Hypothesis I

   Participants spent significantly less time
    looking at the low status video image
    (18%) than both the task feature (24.5%)
    and the high status video image (35%)

   Relevance of on-screen information
    impacts on gaze behaviour (F(2, 36)=8.19,
    p<.05), no effect of location or an
    interaction.
Results-Hypothesis II
   Do participants gaze more often at the high
    status confederate compared to the low
    status confederate and does this interact
    with the location of video images?

 Only sig main effect of status
  F(1,18)=22.19,p<.05, no effect of location or
  an interaction
 Participants gazed more often at the high
  status confederate (35%) compared to the
  low status confederate (24.5%)
Results-Hypothesis III

   Do patterns of gaze reflect patterns
    of interaction within the group?

   Do participants direct more turns of
    speech to the high status
    confederate compared to the low
    status confederate?
Results-Hypothesis III

   Turn combination analysis: sig effect
    of status, no effect of location or an
    interaction

   Participant directs more turns of
    speech to high status confederate
    and this reflects patterns of gaze
Results-Hypothesis III

Turn Combination results
            Participant-LS Participant-HS
HS top left             2.9           10.5
HS bottom left          2.7            9.3
                        2.8            9.9

Average no of turns by Participant-13
Task 2-Summary

 Participants gaze less often at low status
  video compared to high status video and
  task feature
 Participants gaze more often at high
  status video regardless of location
 Interaction in Task 1 not replicated- video
  images attract more gaze and main effect
  of status found
 Patterns of gaze reflect patterns of
  communication
Study 1-Summary

 Participants gaze more often at shared
  data in task 1, not in task 2
 Significant interaction between location
  and relevance of video images in task 1,
  not in task 2
 Overwhelming impact of status on gaze
  and communication in task 2
 Communication patterns reflect patterns
  of gaze in Task 2, not Task 1
Study 1-Summary


 Eye tracking provides valuable
  information about mediated
  interaction
 Distribution of attention related to
  experimental task manipulations
 Positional effect for salient video
  images?
Study 1-Eye-tracking in CMC

 Exploratory questions about eye tracking
  answered
 Data capture rate high –managed to track
  on average 70% of participants who took
  part
 Average of 80% of fixations directed on
  screen during task
 Patterns of gaze consistent and
  meaningful
Future Work

   Same tasks, different design
   Video images placed in 4 corners of the screen,
    therefore 4 conditions for each task

   Task 1:Provisional results suggest interaction not
    replicated- always gaze more often at the map
   Task 2:Difference in gaze distribution to videos
    smaller when video images are placed on same
    side of the screen than when they are placed on
    opposite sides of the screen
Conclusion

  Eye tracking can be used successfully
   as an evaluation technique
  Task differences and status effects
   robust- reflected in both patterns of
   gaze and communication
  May be positional effect for salient
   video images?
Conclusion

 Implications for the design of remote
   communication systems

  Context in which system is to be
   applied very important e.g. social or
   problem solving
  Further research required on positional
   effects-may be used to enhance or
   reduce amount of gaze directed to
   representations