Get documents about "Qualitative Analysis - PowerPoint
Document Sample
Housekeeping
Ethics
Schedule Adjustments
Mid-term practice exercise
1
The research problem
The Faculty of CS will be installing work
pods on the 4th floor whereby a small
group of students can gather around a
large wall display positioned at the end of
a work table to perform ad hoc group
work.
BEFORE the installation happens, you
would like to investigate how students will
use the space.
2
You would like to use the findings to inform the
design of software that will allow students to
seamlessly interact with their various shared
resources and display them as appropriate on
the large display (something other than just
plugging in the video connector).
You have been given a team of 2 people to
conduct this research over the course of one
semester and have funding to purchase any
necessary equipment/software and to pay up to
$2000 in participant honorariums.
3
Primary Research Questions
1. For what kinds of collaborative tasks would
students like to use the work pods?
2. What kinds of documents and other work
products would they like to share?
3. What kinds of interaction abilities are needed
(e.g., sequential or multi-user interactions)?
4. What privacy issues will arise if displays/work
products are shared? 4
What approach would you take
to answer these research
questions?
Describe the study methodology you would
use.
Briefly discuss the benefits and limitations of
this approach and why it is appropriate given
the resources and the research questions.
Make sure to touch upon the tradeoffs of
your approach in terms of generalizability,
precision, and realism.
5
What would your study look
like?
Briefly describe your intended participants
and any inclusion/exclusion criteria.
Briefly describe your proposed study protocol
(what would participants do).
Briefly describe your data collection
techniques. If appropriate, briefly describe
the types of questions you would ask
participants.
6
How good is this approach?
Make sure that it is clear how the protocol
and data collection will support your ability to
answer the research questions.
Make sure both the limitations of your
methodological decisions as well as their
benefits are clearly articulated throughout.
Sell it!
7
QUALITATIVE ANALYSIS
CODING (INTRO EXERCISES)
QUALIT
AFFINITY DIAGRAMS
THE QUALITATIVE ANALYTICAL
PROCESS
1. Analysis Considerations
1 Words
2 Context (tone and inflection)
3 Internal consistency (opinion shifts during
groups)
4 Frequency and intensity of comments (counting,
content analysis)
5 Specificity
6 Trends/themes
7 Iteration (data collection and analysis is an
iterative process moving back and forth)
2. The Procedures
1 Coding/indexing
2 Categorisation
3 Abstraction
4 Comparison
5 Dimensionalisation
6 Integration
7 Iteration
8 Refutation (subjecting inferences to scrutiny)
9 Interpretation (grasp of meaning - difficult to
describe procedurally)
Exercise 1
Coding with pre-defined categories
Deductive analysis
Theory Testing
12
Exercise 2
Open coding
Inductive analysis
Exploratory research
Theory building research
13
CONCEPT GENERATION &
AFFINITY DIAGRAMS
14
Qualitative Inquiry - Purpose
The purpose of qualitative inquiry is to produce
findings. The Data Collection process is not an
end in itself. The culminating activities of
qualitative inquiry are analysis, interpretation,
and presentation of findings.
Qualitative Inquiry - Challenge
To make sense of massive amounts of data,
reduce the volume of information, identify
significant patterns and construct a framework
for communicating the essence of what the
data reveal
Qualitative Inquiry - Problem
‘…have few agreed-on canons for qualitative
data analysis, in the sense of shared ground
rules for drawing conclusions and verifying
sturdiness’ (Miles and Huberman, 1984)
The Creativity of Qualitative
Inquiry
‘..the human element of qualitative inquiry is both
is strength and weakness - its strength is fully
using human insight and experience, its weakness
is being so heavily dependent on the researcher’s
skill, training, intellect, discipline, and creativity.
The researcher is the instrument of qualitative
inquiry, so the quality of the research depends
heavily on the qualities of that human being’
(Patton, 1988)
The Science and Art of Qualitative
Inquiry (Patton, 1988)
The Science
The scientific part is systematic, analytical,
rigorous, disciplined, and critical in perspective
The Art
The artistic part is exploring, playful,
metaphorical, insightful, and creative
Critical Thinking
‘Critical Thinking calls for a persistent effort to
examine any belief or supposed form of
knowledge in the light of the evidence that
supports it and the further conclusions to
which it tends’ (Glaser, 1941)
or more simply!
Critical Thinking means weighting up the
arguments and evidence for and against.
Critical Thinking
• Key points (Glaser, 1941):
– Persistence: Considering an issue carefully
and more than once
– Evidence: Evaluating the evidence put
forward in support of the belief or viewpoint
– Implications: Considering where the belief
or viewpoint leads; what conclusions would
follow; are these suitable and rational; and
if not, should the belief or viewpoint be
reconsidered
Guidance for Creative Thinking
1 Be open
2 Generate options
3 Divergence before convergence
4 Use multiple stimuli - triangulate
5 Side track, zig-zag, and circumnavigate
6 Change patterns
7 Make linkages
8 Trust yourself
9 Work and play at it
The Credibility of Qualitative
Analysis
1 Rigorous techniques and methods for gathering high-
quality data that is carefully analysed, with attention
to issues of validity, reliability, and triangulation
2 The credibility of the researcher, which is dependent
on training, experience, track record, status, and
presentation of self
3 Philosophical belief in the phenomenological
paradigm, that is, a fundamental appreciation of
naturalistic inquiry, qualitative methods, inductive
analysis and holistic thinking
A Credible Qualitative Study
A credible qualitative study needs to address
the following issues:
1 What techniques and methods were used to
ensure the integrity, validity, and accuracy of
the findings
2 What does the researcher bring to study in
terms of qualifications, experience, and
perspective
3 What paradigm orientation and assumptions
ground the study
Principles of Analysing
Qualitative Data
1 Proceed systematically and rigorously (minimise human
error)
2 Record process, memos, journals, etc.
3 Focus on responding to research questions
4 Appropriate level of interpretation appropriate for
situation
5 Time (process of inquiry and analysis are often
simultaneous)
6 Seek to explain or enlighten
7 Evolutionary/emerging
Qualitative Research: Common Features of
Analytic Methods (Miles & Huberman,1994)
1 Affixing codes to a set of field notes drawn
from data collection
2 Noting reflections or other remarks in margin
3 Sorting or shifting through the materials to
identify similar phrases, relationships
between themes, distinct differences
between subgroups and common sequences
Qualitative Research: Common Features of
Analytic Methods (Miles & Huberman,1994)
4 Isolating patterns and processes,
commonalties and differences, and taking
them out to the filed in the next wave of data
collection
5 Gradually elaborating a small set of
generalisations that cover the consistencies
discerned in the data base
6 Confronting those generalisations with a
formalised body of knowledge in the from of
constructs or theories
Interface Design and Usability Engineering
Articulate: Brainstorm Refined Completed
•who users are designs designs designs
Goals: •their key tasks
Task
Psychology of Graphical
centered Participatory
everyday screen
system interaction
Evaluate things design Usability Field
design
User Task / Interface testing testing
Methods: Participatory Cognitive
involvement guidelines
design scenario
Representation walk-through Style Heuristic
User-
& metaphors guides evaluation
centered
design
low fidelity high fidelity
prototyping prototyping
methods methods
Products: User and Throw-away Testable Alpha/beta
task paper prototypes systems or
descriptions prototypes complete
specification
28
brainstorming
the point is:
to generate MANY, WIDE-RANGING ideas
nutty and absurd are GOOD. go for the extremes
(to get out of the rut)
riff off other’s ideas.
the point is NOT:
to generate excellent, complete, feasible ideas
… pressure stifles
to develop or critique ideas
… go wide. deep is for later.
process
1. prepare a list of topics / questions
ahead of time; or in a preliminary brainstorm
2. facilitator takes team through list of topics
switch topic when energy ramps down
3. notetaker takes notes (very important)
4. switch roles so everyone can play
5. ground rules
6. followup
brainstorming is like
popcorn
ground rules
Postpone and withhold your judgment of ideas:
never criticize
Encourage wild and exaggerated ideas
Quantity counts at this stage, not quality
Switch topics when the popcorn slows down
Build on the ideas put forward by others
Every person and every idea has equal worth
Elect a facilitator (calls switches) and a note-
taker
Form groups of 8-10
assign a facilitator, note taker
Problem:
User interface for a car proximity
detection system
Brainstorm 3 aspects of the
problem: (e.g., current problems,
physical form factor, activity
metaphor, input techniques, etc.)
go: 5 minutes
follow up
collect the notes
go through carefully, with judgment turned on
look for
interesting, surprising ideas that might work
ideas that will combine well
promising directions on which you should
brainstorm more
keep your notes. at a later design stage, come
back to them and see if anything else has
become useful in the meantime.
work consolidation:
abstracting specific insights
one tool: the affinity diagram
can use to “consolidate” insights from collected or
generated data. for example:
brainstorming about design problems
categories of problems
brainstorming about design ideas
categories of ideas
comments from users
categories of desirable / successful features
how do you make an affinity diagram?
1. team writes down all data & insights on post-it notes;
be sure you can link the post-it detail back to its source!
2. stick one post-it on the wall
a whiteboard or big sheet of butcher paper is best
3. arrange the other post-its around it, grouping by affinity
to each other. iteration will be required.
4. look at each group and see what it has in common;
name and describe each group.
5. “snapshot” the result for documentation
• digital photo your design website or notebook
• transfer post-its onto xerox paper, 1 sheet / notes-cluster
scan website
why does an affinity diagram work?
• use physical arrangement/proximity to
understand connections
• openness to serendipity
• low cost to rearrange ideas
• many variants:
arrange along axes rather than by affinity
tie causes to effects
group evidence under assertions
affinity diagram exercise
Now take your notes from the earlier
brainstorming and create an affinity diagram
go: 8 minutes
debrief
