1 Research and Practice Interaction IUE 2010 Panel http://www.iue2010.com/P33.html Keith Instone Danielle Cooley, Mark Newman, Susan Weinschenk Introduction 2 • As a user experience practitioner, do you wish you really understood "scientific research" (like CHI papers) and were able to apply it to help you do a better job? • Have you tried to read "the literature" but been bored, confused and disappointed? • Do you have a list of burning questions you wish "those academics" would get answers to? • Would you like to talk with HCI researchers to get to know them better so that one day you might able to collaborate with them, but you have no idea how to get started? First, let’s hear from YOU 3 • Are you a “researcher” or a “practitioner” or both or neither? • What do you think of when we say “research”? “Practice”? • Why are you here (vs. the other panel, vs. going home, vs. starting early on the pub crawl)? What caught your attention? • What do you want to get out of this session? • DRAW IT, DON’T SAY IT 4 Agenda 5 • We have more background information on the topic (keep to a minumum) – Background: on demand/in response to questions, ideally • Each panelist has some bullet points and details (but not have them talk the whole time) • DISCUSSION with you: that is the goal of the session – Can become “lectures” or “panelists debate” if you want, however About the panel 6 • Keith Instone, “organizer”, IBM practitioner, ex-BGSU CS researcher, been learning a lot about this topic for the past year • Danielle Cooley, consultant, 12 years experience as usability practitioner, MSHFID from Bentley • Mark Newman, assistant professor, School of Information, University of Michigan, research interests: Ubiquitous computing, end-user programming, Ex-PARC • Susan Weinschenk, author of books and blogs that "translate" research into practice, Ph.D. in psychology, reads hundreds of research papers every year for the last ?? years, Chief of User Experience Strategy, Americas for HFI If Don Norman is talking about it, then it 7 must be relevant • “The research-practice gap: The Need for Translational Developers” – http://jnd.org/dn.mss/the_research-practice_gap_1.html • Interactions, July/August 2010 – http://portal.acm.org/citation.cfm?doid=1806491.1806494 See http://en.wikipedia.org/wiki/Pasteur%27s_Quadrant for a different visualization 8 Inventors such as Thomas Aimed at Pasteur: fundamental Edison fit the quadrant of some research aimed at solving searching for relevant practical important applied knowledge to solve an applied problem problems problem, but without any attempt to expand our general understanding of phenomena Applicability Researchers most often play A third quadrant is filled with in the fun quadrant, finding tinkerers who produce lovely problems to work on inventions that neither add Pure science. without regard for whether to fundamental No application anyone cares outside of understanding nor have any in mind their fellow research in- use group Search for new understanding Apply existing knowledge Knowledge Keith (in 5 minutes or less) 9 • CHI workshop – There are “immovable objects” of culture: HCI/research, UX practice/corporate – Focus on bridges: Education, Knowledge, Communication – Possible to make a difference, people want to change (e.g., CHI 2011) • Information Architecture Summit – IA still in early stages (cf. HCI, usability) • The topic is getting some attention – Don Norman interactions article Danielle (in 5 minutes or less) 10 • “Back off, man. I’m a scientist.” • UX Practice subspecialty makes a difference – Tim Brown’s “T-shaped People” IA visual copy ID • Practice changes too quickly for academia to keep up with specifics Usability testing • Issues with the Norman article – What about rigor? – Qualities like “intensity,” “attention to detail” and patience” not unique to one or the other. – How do design patterns fit in? Susan (in 5 minutes or less) 11 • Research is critical in UX practice • Research papers are hard to read and interpret • There is a lot of misunderstanding by practitioners about research • It takes a special (weird?) person • Research is power • The best papers may be outside of your field • Research may not give you answers • There are some great blogs and websites that are "translators" of research Mark (in 5 minutes or less) 12 • Types of “research” – “Scientific" model is NOT only type: design, engineering, qualitative, exploratory – Hard to digest different types, evaluate “goodness” • Pasteur's Quadrant – Useful tool for thinking about problem selection – Not useful: helping researchers communicate results • Translational developers (vs. researchers) • Translational skills – What are the translation skills, who has them, how to motivate them? Lucrative publishing/consulting career a la Nielsen? • Translational research (at NIH) – Big pots of money DO motivate researchers to shift their focus. Would it work for HCI? DISCUSSION! 13 • Goal is for this to be the lion share of the session. Take notes here…. A few “problem statement” quotes from Don 14 • There is an immense gap between research and practice • The gap is fundamental. The knowledge and skill sets required of each group differ • In science, there are clear links among hypotheses, conclusions, and evidence • In the practices of most professions, the links are tenuous at best…there is much reliance upon “best practice” 15 Aimed at some practical problem Applied scientist Practitioner Applicability Pure science. Pure scientist Tinkerer No application in mind Search for new understanding Apply existing knowledge Knowledge Don’s conclusion 16 • Third discipline: – Translate between the abstractions of research and the practicalities of practice – Translating research findings into the language of practical development and business – Translating the needs of business into issues that researchers can address • A new kind of practitioner: the translational developer Snippets from CHI position papers (Keith) http://research-practice-interaction.wikispaces.com/ 17 • Arnie Lund (Microsoft) – Biz sharing with research: …“This is too secret for MSR to see!” – Research collaborating with biz: who would get credit for idea? • John Karat (IBM) – Research papers are for the research community to evaluate • Kath Straub – HFI’s Research Update Newsletter: practitioners understand that recent research can speak to relevant issues • Nigel Bevan – UPA Usability Body of Knowledge – Research-Based Web Design & Usability Guidelines (produced with US Government funding) Immovable objects (Keith) 18 UX practice culture HCI research culture No time for “research”: Publish for researchers, good enough not for practitioners Rapidly evolving practice Expanding field Status within corporations Status within academia Corporate culture Research culture “Produce or perish” “Publish or perish” Wants broad answers Answers narrow questions Strategic advantage Open sharing Experimentation Fear of failure Bridges (Keith) 19 Communication Little shared language Speed-of-operation differences Finding each other Fragmented professional organizations Mapping “research answers” to “practical questions” UX practice culture HCI research culture No time for “research”: Publish for researchers, Knowledge good enough not for practitioners Need shared knowledge base Rapidly evolving practice Expanding field Hard to organize research Status within corporations Status within academia for practical use Multi/inter-disciplinary Education HCI education vs. practice Amateur practitioners Corporate culture Research culture HCI education for CS (etc.) degrees “Produce or perish” “Publish or perish” Training for practitioners Wants broad answers Answers narrow questions Strategic advantage Open sharing Experimentation Fear of failure CHI 2011 Action items Editing “benefits” statements for the program Awards: best case study, impact for research paper Community leadership 20 Add “practitioner take-aways” to the program … Communication Brokerage system to connect R & P with common interests Translators between R & P Face-to-face meetings (to start building long-term relationships) Social media UX practice culture HCI research culture ? Grand challenges Knowledge SIGCHI collaborate on UPA BoK Education Training for managers Corporate culture Research culture Improve UX status ? Ideas / Solutions (very incomplete) IA Summit discussions (Keith) 21 • Research and practice in IA are fractured – Not unique: many disciplines face this challenge) • Need an IA research agenda… – Or, there is no such thing as IA research by itself: there is HCI, IS, or LIS research that IAs care about • A huge percentage of time, people, and resources in the field are devoted to IA practice only – Few resources are committed to scientific IA research • IA research: aggregation that looks across disciplines and present boundaries IA Summit discussions (Keith) 22 • Many people still confuse “project research” with “scientific research” – Specific knowledge vs. general knowledge • Easy to find existing research, hard to make sense of it – Need for “Carl Sagan like figure” to translate • Example of translation: BJ Fogg, researcher who spent a lot of time explaining his research to a company 23 From http://www.flickr.com/photos/resmini/sets/72157623664715269/ See also: Maturing a Practice, Journal of IA, http://journalofia.org/volume2/issue1/04- hobbs/ ASIS&T Bulletin 24 Why this is important (Susan) 25 • Research is critical in UX practice – There are far too many "urban legends" that people base their designs on that just don't hold true when you look at the research. • There is a lot of misunderstanding by practitioners about research – "I need research on XYZ. Why aren't you talking about that?" (There isn't any!), "That research is from 2003. It's too old". (If it's valid research it doesn't necessarily lose truth with age). • Research is power – Rather than say, "I think that" about a design question you can say, "A study by xxx showed that..., and therefore we should do ....." Why this is hard (Susan) 26 • Research papers are hard to read and interpret – They are often poorly written, and may not even be interesting or have results that can be practical. Often the results cannot be or should not be generalized. Sometimes the data doesn't match the conclusions of the authors. You have to read carefully. • It takes a special (weird?) person – To be able to read and understand the papers and even more special to LIKE doing that. I agree with Norman that we really do need "translators" who know what they are doing, and are willing to do it. • The best papers may be outside of your field – There is great research that is applicable to UX that may be in journals that have to do with social psychology, or marketing or advertising. If you are trolling for research you should look outside your field. • Research may not give you answers – The best research results in more questions than answers. It's a different mindset that you have to accept if you are going to try and get anything out of research. Good places to go for “the research” (Susan) 27 • http://bps-research-digest.blogspot.com – The British Psychological Society. More research articles in a week than you can possibly keep up with. Everything psychology, including some research you are probably not interested in, but it’s comprehensive! • http://www.spring.org.uk – This blog is PsyBlog, written by Jeremy Dean, a researcher at University College London, More great psychology research. • http://scienceblogs.com – The Weizmann Institute of Science is a basic research institute in Rehovot, Israel and they have a blog that covers research in many fields, including politics, education, etc. I follow the Brain and Behavior blog. • http://thesituationist.wordpress.com – From the Project on Law and Mind Sciences at Harvard Law School, this blog is “devoted to identifying, inventorying, archiving, blogging, and otherwise promoting research, writing, conferences, colloquia, and presentations directed toward understanding the implications of social psychology, social cognition, and other related mind sciences for law, policymaking, and legal theory. The Project’s director is Jon Hanson, the Alfred Smart Professor of Law at Harvard Law School.” In other words, they report on some good research in social psychology. • http://www.dericbownds.net – I love Deric Bownds blog. He has little snippets of research about the intersection between the brain and biology. Types of “research” (Mark) 28 • There are a number of different research styles/paradigms at work in HCI. • The "scientific" model is only one--and I would argue it is not necessarily the most dominant. • Other paradigms that are strongly at work are design, engineering, and qualitative social science (which operates somewhat differently from quantitative social science which more closely mirrors the more traditional scientific paradigm with hypotheses- experiments-results). • This actually makes it EVEN HARDER to understand and digest the results of research, which is something I'm learning more and more as I try to teach Masters' and doctoral students how to read and write research. • It's even harder to figure out (and teach people how to figure out) how to separate good, important, relevant, impactful research from not-so-good stuff. (And it's even harder to figure out how to actually DO the good, important, relevant, impactful research.) • Summary: Even researchers have a hard time with this stuff!! Pasteur's Quadrant (Mark) 29 • Pasteur's Quadrant comes up A LOT in academic circles--especially those like HCI that see themselves as working in "applied" domains. • As far as I can tell, everyone in HCI and related fields (e.g., everyone in an iSchool) thinks they are working in Pasteur's Quadrant or at least is aware that they should be. • Essentially, PQ is a useful tool for thinking about problem selection, but not very useful for informing researchers about how they should communicate their results. • Being unaware of Pasteur's Quadrant is NOT the reason that research results are irrelevant to practitioners, IMHO. Translational developers (Mark) 30 • Norman talks about "translational developers" but not about "translational researchers." Odd, because that's exactly how I would characterize Norman himself (as well as his current business partner Jakob Nielsen). • I agree that translation requires skills and motivations that are different from those that are required of either researchers or practitioners. • I also agree that it is critically important for research to have impact on practice. • Not quite sure what to do about this except to figure out what those skills are, who has 'em, and how to motivate them. • I'm sure the prospect of a lucrative publishing/consulting career a la Nielsen would appeal to many, but what are the steps along the way? Translational research (Mark) 31 • Interesting note: "translational research" is apparently a significant buzzword at NIH these days--and it denotes exactly what is being called for here: research that seeks to "translate" findings/practices/etc. from "bench science" to "clinical practice." • Most notably, they have specific funding programs in place to incentivize such projects. • Big pots of money DO motivate researchers to shift their focus, by the way. Could anyone imagine such programs in the field of HCI, UX, IxD? • Who would fund it? Would it help?
Pages to are hidden for
"qualitative social science - Research-Practice Interaction - Wikispaces"Please download to view full document