Qualitative Analysis - PowerPoint by 0bzcVx



 Ethics
 Schedule Adjustments
 Mid-term practice exercise

The research problem

 The Faculty of CS will be installing work
  pods on the 4th floor whereby a small
  group of students can gather around a
  large wall display positioned at the end of
  a work table to perform ad hoc group
 BEFORE the installation happens, you
  would like to investigate how students will
  use the space.
 You would like to use the findings to inform the
  design of software that will allow students to
  seamlessly interact with their various shared
  resources and display them as appropriate on
  the large display (something other than just
  plugging in the video connector).
 You have been given a team of 2 people to
  conduct this research over the course of one
  semester and have funding to purchase any
  necessary equipment/software and to pay up to
  $2000 in participant honorariums.

  Primary Research Questions
 1. For what kinds of collaborative tasks would
  students like to use the work pods?

  2. What kinds of documents and other work
  products would they like to share?

  3. What kinds of interaction abilities are needed
  (e.g., sequential or multi-user interactions)?

  4. What privacy issues will arise if displays/work
  products are shared?                             4
What approach would you take
to answer these research
 Describe the study methodology you would
 Briefly discuss the benefits and limitations of
  this approach and why it is appropriate given
  the resources and the research questions.
 Make sure to touch upon the tradeoffs of
  your approach in terms of generalizability,
  precision, and realism.
What would your study look
 Briefly describe your intended participants
  and any inclusion/exclusion criteria.
 Briefly describe your proposed study protocol
  (what would participants do).
 Briefly describe your data collection
  techniques. If appropriate, briefly describe
  the types of questions you would ask

How good is this approach?

 Make sure that it is clear how the protocol
  and data collection will support your ability to
  answer the research questions.
 Make sure both the limitations of your
  methodological decisions as well as their
  benefits are clearly articulated throughout.
 Sell it!


1. Analysis Considerations
1 Words
2 Context (tone and inflection)
3 Internal consistency (opinion shifts during
4   Frequency and intensity of comments (counting,
    content analysis)
5   Specificity
6   Trends/themes
7   Iteration (data collection and analysis is an
    iterative process moving back and forth)
2. The Procedures
1 Coding/indexing
2 Categorisation
3 Abstraction
4 Comparison
5 Dimensionalisation
6 Integration
7 Iteration
8 Refutation (subjecting inferences to scrutiny)
9 Interpretation (grasp of meaning - difficult to
  describe procedurally)
Exercise 1

 Coding with pre-defined categories
 Deductive analysis
 Theory Testing

Exercise 2

 Open coding
 Inductive analysis
 Exploratory research
 Theory building research


 Qualitative Inquiry - Purpose

The purpose of qualitative inquiry is to produce
findings. The Data Collection process is not an
end in itself. The culminating activities of
qualitative inquiry are analysis, interpretation,
and presentation of findings.
 Qualitative Inquiry - Challenge

To make sense of massive amounts of data,
reduce the volume of information, identify
significant patterns and construct a framework
for communicating the essence of what the
data reveal
 Qualitative Inquiry - Problem

‘…have few agreed-on canons for qualitative
data analysis, in the sense of shared ground
rules for drawing conclusions and verifying
sturdiness’ (Miles and Huberman, 1984)
The Creativity of Qualitative
 ‘..the human element of qualitative inquiry is both
     is strength and weakness - its strength is fully
  using human insight and experience, its weakness
   is being so heavily dependent on the researcher’s
   skill, training, intellect, discipline, and creativity.
     The researcher is the instrument of qualitative
     inquiry, so the quality of the research depends
      heavily on the qualities of that human being’
                       (Patton, 1988)
The Science and Art of Qualitative
Inquiry (Patton, 1988)
                    The Science
      The scientific part is systematic, analytical,
    rigorous, disciplined, and critical in perspective
                       The Art
         The artistic part is exploring, playful,
         metaphorical, insightful, and creative
         Critical Thinking
‘Critical Thinking calls for a persistent effort to
   examine any belief or supposed form of
  knowledge in the light of the evidence that
   supports it and the further conclusions to
          which it tends’ (Glaser, 1941)
                or more simply!
   Critical Thinking means weighting up the
  arguments and evidence for and against.
             Critical Thinking
• Key points (Glaser, 1941):
  – Persistence: Considering an issue carefully
    and more than once
  – Evidence: Evaluating the evidence put
    forward in support of the belief or viewpoint
  – Implications: Considering where the belief
    or viewpoint leads; what conclusions would
    follow; are these suitable and rational; and
    if not, should the belief or viewpoint be
    Guidance for Creative Thinking

1   Be open
2   Generate options
3   Divergence before convergence
4   Use multiple stimuli - triangulate
5   Side track, zig-zag, and circumnavigate
6   Change patterns
7   Make linkages
8   Trust yourself
9   Work and play at it
   The Credibility of Qualitative
1 Rigorous techniques and methods for gathering high-
  quality data that is carefully analysed, with attention
  to issues of validity, reliability, and triangulation
2 The credibility of the researcher, which is dependent
  on training, experience, track record, status, and
  presentation of self
3 Philosophical belief in the phenomenological
  paradigm, that is, a fundamental appreciation of
  naturalistic inquiry, qualitative methods, inductive
  analysis and holistic thinking
 A Credible Qualitative Study
A credible qualitative study needs to address
  the following issues:
1 What techniques and methods were used to
  ensure the integrity, validity, and accuracy of
  the findings
2 What does the researcher bring to study in
  terms of qualifications, experience, and
3 What paradigm orientation and assumptions
  ground the study
Principles of Analysing
Qualitative Data
1 Proceed systematically and rigorously (minimise human
2   Record process, memos, journals, etc.
3   Focus on responding to research questions
4   Appropriate level of interpretation appropriate for
5   Time (process of inquiry and analysis are often
6   Seek to explain or enlighten
7   Evolutionary/emerging
Qualitative Research: Common Features of
Analytic Methods (Miles & Huberman,1994)
1 Affixing codes to a set of field notes drawn
  from data collection
2 Noting reflections or other remarks in margin
3 Sorting or shifting through the materials to
  identify similar phrases, relationships
  between themes, distinct differences
  between subgroups and common sequences
Qualitative Research: Common Features of
Analytic Methods (Miles & Huberman,1994)
4 Isolating patterns and processes,
  commonalties and differences, and taking
  them out to the filed in the next wave of data
5 Gradually elaborating a small set of
  generalisations that cover the consistencies
  discerned in the data base
6 Confronting those generalisations with a
  formalised body of knowledge in the from of
  constructs or theories
                            Interface Design and Usability Engineering
              Articulate:                         Brainstorm                 Refined                 Completed
              •who users are                      designs                    designs                 designs
Goals:        •their key tasks

                                       Psychology of                    Graphical
            centered                                    Participatory
                                       everyday                         screen
            system                                      interaction
                            Evaluate   things                           design          Usability          Field
                                       User             Task /          Interface       testing            testing
Methods:    Participatory                               Cognitive
                                       involvement                      guidelines
            design                                      scenario
                                       Representation   walk-through    Style           Heuristic
                                       & metaphors                      guides          evaluation

                                         low fidelity                   high fidelity
                                         prototyping                    prototyping
                                         methods                        methods

Products:        User and                        Throw-away                Testable                  Alpha/beta
                 task                            paper                     prototypes                systems or
                 descriptions                    prototypes                                          complete
 the point is:
    to generate MANY, WIDE-RANGING ideas
     nutty and absurd are GOOD. go for the extremes
      (to get out of the rut)
     riff off other’s ideas.

 the point is NOT:
    to generate excellent, complete, feasible ideas
      … pressure stifles
    to develop or critique ideas
      … go wide. deep is for later.

1. prepare a list of topics / questions
   ahead of time; or in a preliminary brainstorm

2. facilitator takes team through list of topics
   switch topic when energy ramps down

3. notetaker takes notes (very important)
4. switch roles so everyone can play
5. ground rules
6. followup
brainstorming is like
ground rules
   Postpone and withhold your judgment of ideas:
    never criticize
   Encourage wild and exaggerated ideas
   Quantity counts at this stage, not quality
   Switch topics when the popcorn slows down
   Build on the ideas put forward by others
   Every person and every idea has equal worth
   Elect a facilitator (calls switches) and a note-
Form groups of 8-10
assign a facilitator, note taker

          User interface for a car proximity
                  detection system

            Brainstorm 3 aspects of the
           problem: (e.g., current problems,
             physical form factor, activity
           metaphor, input techniques, etc.)
                  go: 5 minutes
follow up
  collect the notes

  go through carefully, with judgment turned on
  look for
    interesting, surprising ideas that might work
    ideas that will combine well
    promising directions on which you should
     brainstorm more

  keep your notes. at a later design stage, come
   back to them and see if anything else has
   become useful in the meantime.
 work consolidation:
 abstracting specific insights

 one tool: the affinity diagram

 can use to “consolidate” insights from collected or
generated data. for example:
     brainstorming about design problems
       categories of problems
     brainstorming about design ideas
       categories of ideas
     comments from users
       categories of desirable / successful features
 how do you make an affinity diagram?
1.    team writes down all data & insights on post-it notes;
      be sure you can link the post-it detail back to its source!
2.    stick one post-it on the wall
      a whiteboard or big sheet of butcher paper is best
3.    arrange the other post-its around it, grouping by affinity
      to each other. iteration will be required.
4.    look at each group and see what it has in common;
      name and describe each group.
5.    “snapshot” the result for documentation
     • digital photo  your design website or notebook
     • transfer post-its onto xerox paper, 1 sheet / notes-cluster
        scan  website
    why does an affinity diagram work?

•    use physical arrangement/proximity to
     understand connections
•    openness to serendipity
•    low cost to rearrange ideas
•    many variants:
     arrange along axes rather than by affinity
     tie causes to effects
     group evidence under assertions
affinity diagram exercise

   Now take your notes from the earlier
brainstorming and create an affinity diagram

              go: 8 minutes

To top