qualitative social science - Research-Practice Interaction - Wikispaces by linxiaoqin



Research and Practice
                  IUE 2010 Panel

                    Keith Instone
 Danielle Cooley, Mark Newman, Susan Weinschenk
                  Introduction                              2

• As a user experience practitioner, do you wish you really
  understood "scientific research" (like CHI papers) and
  were able to apply it to help you do a better job?

• Have you tried to read "the literature" but been bored,
  confused and disappointed?

• Do you have a list of burning questions you wish "those
  academics" would get answers to?

• Would you like to talk with HCI researchers to get to
  know them better so that one day you might able to
  collaborate with them, but you have no idea how to get
          First, let’s hear from YOU                       3

• Are you a “researcher” or a “practitioner” or both or

• What do you think of when we say “research”?

• Why are you here (vs. the other panel, vs. going home,
  vs. starting early on the pub crawl)? What caught your

• What do you want to get out of this session?

                    Agenda                                5

• We have more background information on the topic
  (keep to a minumum)
   – Background: on demand/in response to questions,

• Each panelist has some bullet points and details (but not
  have them talk the whole time)

• DISCUSSION with you: that is the goal of the session
   – Can become “lectures” or “panelists debate” if you
     want, however
                   About the panel                                     6

• Keith Instone, “organizer”, IBM practitioner, ex-BGSU CS
  researcher, been learning a lot about this topic for the past year

• Danielle Cooley, consultant, 12 years experience as usability
  practitioner, MSHFID from Bentley

• Mark Newman, assistant professor, School of Information, University
  of Michigan, research interests: Ubiquitous computing, end-user
  programming, Ex-PARC

• Susan Weinschenk, author of books and blogs that "translate"
  research into practice, Ph.D. in psychology, reads hundreds of
  research papers every year for the last ?? years, Chief of User
  Experience Strategy, Americas for HFI
 If Don Norman is talking about it, then it
            must be relevant
• “The research-practice gap: The Need for Translational
   – http://jnd.org/dn.mss/the_research-practice_gap_1.html
• Interactions, July/August 2010
   – http://portal.acm.org/citation.cfm?doid=1806491.1806494
      See http://en.wikipedia.org/wiki/Pasteur%27s_Quadrant for a different visualization

                                                                                Inventors such as Thomas
  Aimed at                     Pasteur: fundamental                             Edison fit the quadrant of
  some                         research aimed at solving                        searching for relevant
  practical                    important applied                                knowledge to solve an applied
  problem                      problems                                         problem, but without any
                                                                                attempt to expand our general
                                                                                understanding of phenomena

                               Researchers most often play                      A third quadrant is filled with
                               in the fun quadrant, finding                     tinkerers who produce
                               lovely problems to work on                       inventions that neither add
Pure science.                  without regard for whether                       to fundamental
No application                 anyone cares outside of                          understanding nor have any
in mind                        their fellow research in-                        use

                           Search for new understanding                              Apply existing knowledge
          Keith (in 5 minutes or less)                  9

• CHI workshop
   – There are “immovable objects” of culture:
     HCI/research, UX practice/corporate
   – Focus on bridges: Education, Knowledge,
   – Possible to make a difference, people want to change
     (e.g., CHI 2011)

• Information Architecture Summit
   – IA still in early stages (cf. HCI, usability)

• The topic is getting some attention
   – Don Norman interactions article
        Danielle (in 5 minutes or less)                                     10

• “Back off, man. I’m a scientist.”

• UX Practice subspecialty makes a difference
   – Tim Brown’s “T-shaped People”
                                                 IA visual copy ID
• Practice changes too quickly for
  academia to keep up with specifics

                                                        Usability testing
• Issues with the Norman article
   – What about rigor?
   – Qualities like “intensity,” “attention to
     detail” and patience” not unique to
     one or the other.
   – How do design patterns fit in?
        Susan (in 5 minutes or less)                     11

• Research is critical in UX practice
• Research papers are hard to read and interpret
• There is a lot of misunderstanding by practitioners about
• It takes a special (weird?) person
• Research is power
• The best papers may be outside of your field
• Research may not give you answers
• There are some great blogs and websites that are
  "translators" of research
           Mark (in 5 minutes or less)                             12

• Types of “research”
   – “Scientific" model is NOT only type: design, engineering,
     qualitative, exploratory
   – Hard to digest different types, evaluate “goodness”
• Pasteur's Quadrant
   – Useful tool for thinking about problem selection
   – Not useful: helping researchers communicate results
• Translational developers (vs. researchers)
• Translational skills
   – What are the translation skills, who has them, how to motivate
     them? Lucrative publishing/consulting career a la Nielsen?
• Translational research (at NIH)
   – Big pots of money DO motivate researchers to shift their focus.
     Would it work for HCI?
                 DISCUSSION!                               13

• Goal is for this to be the lion share of the session. Take
  notes here….
   A few “problem statement” quotes from Don             14

• There is an immense gap between research and practice

• The gap is fundamental. The knowledge and skill sets
  required of each group differ

• In science, there are clear links among hypotheses,
  conclusions, and evidence

• In the practices of most professions, the links are
  tenuous at best…there is much reliance upon “best

  Aimed at
  problem               Applied scientist                   Practitioner

Pure science.           Pure scientist                       Tinkerer
No application
in mind

                  Search for new understanding          Apply existing knowledge
                  Don’s conclusion                                    16

• Third discipline:
   – Translate between the abstractions of research and the
      practicalities of practice
   – Translating research findings into the language of practical
      development and business
   – Translating the needs of business into issues that researchers
      can address

• A new kind of practitioner: the translational developer
    Snippets from CHI position papers (Keith)
   http://research-practice-interaction.wikispaces.com/            17

• Arnie Lund (Microsoft)
   – Biz sharing with research: …“This is too secret for MSR to see!”
   – Research collaborating with biz: who would get credit for idea?
• John Karat (IBM)
   – Research papers are for the research community to evaluate
• Kath Straub
   – HFI’s Research Update Newsletter: practitioners understand that
     recent research can speak to relevant issues
• Nigel Bevan
   – UPA Usability Body of Knowledge
   – Research-Based Web Design & Usability Guidelines (produced
     with US Government funding)
                  Immovable objects (Keith)                                18

                                                UX practice culture
      HCI research culture
                                               No time for “research”:
     Publish for researchers,
                                                    good enough
       not for practitioners
                                              Rapidly evolving practice
         Expanding field
                                              Status within corporations
     Status within academia

                                                     Corporate culture
   Research culture
                                                     “Produce or perish”
   “Publish or perish”
                                                    Wants broad answers
Answers narrow questions
                                                    Strategic advantage
     Open sharing
    Experimentation                                     Fear of failure
                               Bridges (Keith)                                                        19

                                        Little shared language
                                     Speed-of-operation differences
                                          Finding each other
                                 Fragmented professional organizations
                                      Mapping “research answers”
                                        to “practical questions”
                                                                           UX practice culture
     HCI research culture
                                                                          No time for “research”:
    Publish for researchers,                 Knowledge
                                                                               good enough
      not for practitioners          Need shared knowledge base
                                                                         Rapidly evolving practice
        Expanding field               Hard to organize research
                                                                         Status within corporations
    Status within academia                 for practical use

                                      HCI education vs. practice
                                        Amateur practitioners                  Corporate culture
   Research culture
                                  HCI education for CS (etc.) degrees          “Produce or perish”
   “Publish or perish”
                                       Training for practitioners             Wants broad answers
Answers narrow questions
                                                                              Strategic advantage
     Open sharing
    Experimentation                                                               Fear of failure
                                     CHI 2011 Action items
                          Editing “benefits” statements for the program
                        Awards: best case study, impact for research paper
                                      Community leadership                                          20
                          Add “practitioner take-aways” to the program

                             Brokerage system to connect R & P with
                                       common interests
                                   Translators between R & P
                             Face-to-face meetings (to start building
                                    long-term relationships)
                                          Social media

                                                                             UX practice culture
 HCI research culture
  Grand challenges                       Knowledge
                                 SIGCHI collaborate on UPA BoK

                                      Training for managers
                                                                                Corporate culture
Research culture
                                                                                Improve UX status

                          Ideas / Solutions
                                     (very incomplete)
       IA Summit discussions (Keith)                     21

• Research and practice in IA are fractured
   – Not unique: many disciplines face this challenge)
• Need an IA research agenda…
   – Or, there is no such thing as IA research by itself:
      there is HCI, IS, or LIS research that IAs care about
• A huge percentage of time, people, and resources in the
  field are devoted to IA practice only
   – Few resources are committed to scientific IA research
• IA research: aggregation that looks across disciplines
  and present boundaries
       IA Summit discussions (Keith)                         22

• Many people still confuse “project research” with
  “scientific research”
   – Specific knowledge vs. general knowledge

• Easy to find existing research, hard to make sense of it
   – Need for “Carl Sagan like figure” to translate

• Example of translation: BJ Fogg, researcher who spent a
  lot of time explaining his research to a company

From http://www.flickr.com/photos/resmini/sets/72157623664715269/
See also:
Maturing a Practice, Journal of IA, http://journalofia.org/volume2/issue1/04-
ASIS&T Bulletin
          Why this is important (Susan)                                   25

• Research is critical in UX practice
   – There are far too many "urban legends" that people base their
     designs on that just don't hold true when you look at the

• There is a lot of misunderstanding by practitioners about research
   – "I need research on XYZ. Why aren't you talking about that?"
     (There isn't any!), "That research is from 2003. It's too old". (If it's
     valid research it doesn't necessarily lose truth with age).

• Research is power
   – Rather than say, "I think that" about a design question you can
     say, "A study by xxx showed that..., and therefore we should do
             Why this is hard (Susan)                                 26

• Research papers are hard to read and interpret
    – They are often poorly written, and may not even be interesting or
       have results that can be practical. Often the results cannot be or
       should not be generalized. Sometimes the data doesn't match
       the conclusions of the authors. You have to read carefully.
• It takes a special (weird?) person
    – To be able to read and understand the papers and even more
       special to LIKE doing that. I agree with Norman that we really do
       need "translators" who know what they are doing, and are willing
       to do it.
• The best papers may be outside of your field
    – There is great research that is applicable to UX that may be in
       journals that have to do with social psychology, or marketing or
       advertising. If you are trolling for research you should look
       outside your field.
• Research may not give you answers
    – The best research results in more questions than answers. It's a
       different mindset that you have to accept if you are going to try
       and get anything out of research.
    Good places to go for “the research” (Susan)                                                        27

•   http://bps-research-digest.blogspot.com
     – The British Psychological Society. More research articles in a week than you can possibly
          keep up with. Everything psychology, including some research you are probably not
          interested in, but it’s comprehensive!

•   http://www.spring.org.uk
     – This blog is PsyBlog, written by Jeremy Dean, a researcher at University College London,
          More great psychology research.

•   http://scienceblogs.com
     – The Weizmann Institute of Science is a basic research institute in Rehovot, Israel and they
          have a blog that covers research in many fields, including politics, education, etc. I follow the
          Brain and Behavior blog.

•   http://thesituationist.wordpress.com
     – From the Project on Law and Mind Sciences at Harvard Law School, this blog is “devoted to
          identifying, inventorying, archiving, blogging, and otherwise promoting research, writing,
          conferences, colloquia, and presentations directed toward understanding the implications of
          social psychology, social cognition, and other related mind sciences for law, policymaking,
          and legal theory. The Project’s director is Jon Hanson, the Alfred Smart Professor of Law at
          Harvard Law School.” In other words, they report on some good research in social

•   http://www.dericbownds.net
     – I love Deric Bownds blog. He has little snippets of research about the intersection between
          the brain and biology.
           Types of “research” (Mark)                                28

• There are a number of different research styles/paradigms at work in
• The "scientific" model is only one--and I would argue it is not
  necessarily the most dominant.
• Other paradigms that are strongly at work are design, engineering,
  and qualitative social science (which operates somewhat
  differently from quantitative social science which more closely
  mirrors the more traditional scientific paradigm with hypotheses-
• This actually makes it EVEN HARDER to understand and digest the
  results of research, which is something I'm learning more and more
  as I try to teach Masters' and doctoral students how to read and
  write research.
• It's even harder to figure out (and teach people how to figure out)
  how to separate good, important, relevant, impactful research from
  not-so-good stuff. (And it's even harder to figure out how to actually
  DO the good, important, relevant, impactful research.)
• Summary: Even researchers have a hard time with this stuff!!
         Pasteur's Quadrant (Mark)                       29

• Pasteur's Quadrant comes up A LOT in academic
  circles--especially those like HCI that see themselves as
  working in "applied" domains.
• As far as I can tell, everyone in HCI and related fields
  (e.g., everyone in an iSchool) thinks they are working in
  Pasteur's Quadrant or at least is aware that they should
• Essentially, PQ is a useful tool for thinking about
  problem selection, but not very useful for informing
  researchers about how they should communicate their
• Being unaware of Pasteur's Quadrant is NOT the reason
  that research results are irrelevant to practitioners,
      Translational developers (Mark)                       30

• Norman talks about "translational developers" but not
  about "translational researchers." Odd, because that's
  exactly how I would characterize Norman himself (as
  well as his current business partner Jakob Nielsen).
• I agree that translation requires skills and motivations
  that are different from those that are required of either
  researchers or practitioners.
• I also agree that it is critically important for research to
  have impact on practice.
• Not quite sure what to do about this except to figure out
  what those skills are, who has 'em, and how to motivate
• I'm sure the prospect of a lucrative publishing/consulting
  career a la Nielsen would appeal to many, but what are
  the steps along the way?
       Translational research (Mark)                       31

• Interesting note: "translational research" is apparently a
  significant buzzword at NIH these days--and it denotes
  exactly what is being called for here: research that seeks
  to "translate" findings/practices/etc. from "bench science"
  to "clinical practice."
• Most notably, they have specific funding programs in
  place to incentivize such projects.
• Big pots of money DO motivate researchers to shift their
  focus, by the way. Could anyone imagine such programs
  in the field of HCI, UX, IxD?
• Who would fund it? Would it help?

To top