by KATHY BAXTER, Salesforce; CATHERINE COURAGE, DocuSign; & KELLY CAINE, Clemson University
Ten years is an eternity in the tech world. But the speed of change makes the classic “decade of reflection” even more valuable for assessing which changes really count and why. We had a chance to reflect systematically on the last decade of user experience research for the second edition of our book “Understanding Your Users,” which was first published in 2005. Since then, user research has become more widespread and more sophisticated. It has also responded to challenges of faster development cycles while simultaneously contributing more, not just to product development, but to the bigger picture of strategic innovation.
So among the noise of trendy terms and fashionable phrases we found five key trends that really do matter:
1. Usability to User Experience
One of the biggest changes is the shift from a focus on “usability” to “user experience.” “Usability” is an attribute of a system or user interface (UI). It refers to the effectiveness, efficiency, and satisfaction with which users can achieve tasks when using a product. “User experience,” on the other hand, is human-centered and is the co-creation of an interaction between a person or persons and an artifact. User experience includes usability and all aspects a user encounters when dealing with a product or service from the branding to the motion and audio design to customer support. It is about how the product or service makes the user feel and encompasses both user behavior and the social context of the user-artifact relationship. You can see this change reflected in job titles (e.g., from “Usability Engineer” in the 1990’s and early 2000’s to “User Experience Researcher” today) and professional organizations (e.g., the “Usability Professionals Association” changed their name to “User Experience Professionals Association” in 2012). The result of the change in focus means that a User Experience Researcher (UER) must conduct research that goes beyond the UI and think about all of the elements influencing a user’s experience.
2. Lightweight, iterative, or agile approaches
A second change in the profession is the popularity of lightweight, iterative, project-based approaches development processes such as agile. Agile is a rapid development process used by a cross-functional team that ideally works together closely and meets daily. While agile development does include continuous customer or stakeholder involvement, the competing requirement to iterate quickly in small operable chunks means it can be difficult to incorporate user research into the process. This makes conducting early user research to understand user requirements critical because there may be less time in later cycles to conduct longer-term research. Regardless of the development process a company uses, it is imperative to take a cumulative approach to user research, constantly building upon previous work and triangulating across methodologies to create a holistic understanding your users, domain, industry, etc.
3. Innovation at speed
A third development is the increasing variety of technology and contexts where research is needed. For example, products that were considered science fiction ten years ago such as wearables (e.g., smart watches, Google Glass), self-driving cars, and increasingly complex consumer medical devices (e.g., contact lens that measure blood glucose in tears) are becoming reality. In addition, with the explosion of mobile apps and devices means that work is not just being done at a desk or even at one particular place. Our research is less often conducted inside a usability lab and more often done in context. This means we must understand not just whether or not users can find the feature they want and whether it meets a real user need but also how people and things around those the device(s) feel and respond.
4. Vulnerable user types and research ethics
The importance of research ethics isn’t a new development; however, our ability to do research with individuals we have never considered or experienced before has become easier than ever. When you conduct research with vulnerable populations (e.g., children, illiterate, homeless, sick, aging), you must be aware of other factors that could mean your typical research protocol is coercive or not transparent enough about the risks to the participant and obligations of the researcher. For example, if a participant has low or no literacy, s/he cannot read the non-disclosure agreement (NDA) or informed consent form and must take your word when you provide a verbal protocol. The incentive amount you might offer participants for studies in your lab in Silicon Valley can be greater than the monthly income of a participant in a field study in another country and could be coercive if they are uncomfortable inviting you into their home but they desperately need that money. When you provide a prototype of your product for a child to use in his/her home, all of their family members will likely interact with it and therefore, become unknowing participants in your study, even if they did not sign an NDA or provide informed consent. These are issues that increasingly researchers must think about, and take personal responsibility for. Especially since most companies do not have formal Institutional Review Boards (IRBs) to review and approve research, it is all too rare for these issues to be discussed in an open and neutral environment. The push to collect user data faster and faster often negatively impacts this type of deep review.
5. Remote Research
A fifth development is the dramatically increased availability of tools that enable remote user research. These range from tools created specifically to make the job of a user researcher easier (e.g., Mechanical Turk for crowdsourcing data analysis, Paco for experience sampling methodology, UserTesting.com for fast, large scale usability evaluations), to general purpose communication tools like Skype, Citrix GoToMeeting, or Google Hangouts that now make conducting interviews across the country as convenient as conducting them in the user researcher’s home city. Tools that analyze user sentiment on social networks allow you to hear from users in the public sphere and understand your perceived brand.
We’d love to hear from you the changes you have observed in our field in the last 10 years, for better or worse. What changes have you noticed? Which do you feel have made the most impact? What has made doing UX research easier? Harder?
Get 30% off and free shipping worldwide on the second edition of “Understanding Your Users”—use promo code COMP315 here. We’d love to have your feedback!
Kathy is a Principal User Researcher at Salesforce in San Francisco. Previously, she worked at Google for over 10 years as a Staff User Experience Researcher & UX Infrastructure Manager. Prior to 2005, she worked as a Sr. UER at eBay and Oracle. Kathy is active in the UX community, volunteering on the EPIC and CHI conference committees, as well as teaching courses and mentoring young girls in STEAM careers. She received her MS in Engineering Psychology and a BS degree in Applied Psychology from the Georgia Institute of Technology. The second edition of the book she coauthored, Understanding your Users, was published in May 2015 & was the #1 New Release in HCI & Software & Product Design on Amazon the first several weeks it was on sale. https://www.linkedin.com/in/kathykbaxter
Catherine’s passion is transforming corporate culture by making customer-focus a driver of innovation and change. She leads the DocuSign Customer Experience team where her group’s mission is to create world-class products and services for customers, partners and employees. Catherine co-authored Understanding Your Users and is an active writer and speaker on creativity, innovation and design. She has been featured in Harvard Business Review, The Wall Street Journal, Fast Company, Huffington Post, and TEDx. She has twice been selected by the Silicon Valley Business Journal – in 2011 as one of Silicon Valley’s tech leaders, and in 2013 as one of Silicon Valley’s 100 Most Influential Women. Also in 2013, Catherine made Forbes list of “Top 10 Rising Stars at The World’s Most Innovative Companies.” In 2014, the National Diversity Council named her one of the Top 50 Most Powerful Women in Technology. https://www.linkedin.com/in/catherinecourage, @ccourage
Kelly is a researcher and professor working at the intersection of people and technology. She directs the Humans and Technology Lab at Clemson University where she and her students advocate for users and create easy to use, useful technology that meets people’s needs. She co-authored a Understanding your Users, has published dozens of peer-reviewed papers and is regularly cited by media such as the AP, Washington Post, NPR, and New York Times, making her a sought-after speaker, thinker and writer on understanding people and their relationship to technology. @kellycaine
Thanks for the great article. I’m glad to see that research professionals have move on from the position of treating the customer as a component that can be quantified and measured. It was always a version of Taylorism, which was a flawed or incomplete approach to design.
I believe that the most important role researchers can play is to help build empathy by arranging for business and design teams to meet and work with the users. Meeting people face to face has a positive impact that can’t be compared to reading spreadsheets and disembodied quotes.
As we heard at the conference, organisations are structurally designed to resist change, however I’ve witnessed first hand managers that have overturned poor decisions after they have spent time with the customers and made a human connection.
Leave the iterative optimisation up to the AB testing software and automated processes and let’s spend our time instead bringing users and designers together.
End of sermon!
Hi Simon! I think one of the biggest challenges is finding *representative* customers to observe. Living in the Silicon Valley (as I do) is like working in an echo chamber. If we want to interact with non-tech savvy users with a diversity of experiences, backgrounds, and needs, travel is required. Unfortunately, on increasingly short development cycles and tiny travel budgets, getting even some of your team to travel around the US, much less internationally, can be a real obstacle. This is when we must create deliverables that help the team experience these customers from their offices. You are right that quotes don’t cut it. A doc or even poster of various personas is limited in its effectiveness. It’s takes time and money and a culture to demand it but you need videos, artifacts, a day-in-the-life exercises, etc. to engage those that couldn’t visit the customers with you. I’d love to hear how you and others engage those that can’t go on-site with you.
I’m so glad you raised the issue of research ethics with vulnerable populations. It is a critical part of strong work, and one that can be easily forgotten as we innovate at speed and move toward agile methods. Whether working with children or on medical devices for those in need, ethical research must protect the individuals who participate.
Hi Katie! Research ethics is an area of particular interest and concern for me. Many individuals conducting user research today do not have a formal background in psychology, anthropology, ethnography, etc. so they didn’t have a class in research ethics or experience working with an IRB. I believe research ethics is a mindset, not just a set of rules to remember and so if you didn’t begin your research experience with these kinds of ethical questions at the start of every study you design, it is easy to unknowingly violate ethical principles. I’d love to see a volunteer organization that can review studies for researchers (under NDA, of course) that want a neutral third party review of their study design! Perhaps this is something that EPIC can create? 🙂
Thank you for a great article. Are there documented methodologíes for UX research beyond the traditional ethnography tolos?
tools
Hi Jose! The tools we describe are broadly used across user experience and not limited to ethnographic or anthropological research. Is there a specific tool or methodology you are wondering about?
Whole heartedly agree with the observations listed! Additional observations I would throw in…
6. Higher Education programs for UX / HCI evolved from graduating “academics” to “practitioners”.
– The Master’s programs at schools like Berkeley, U Washington, CMU, etc. are delivering graduates that can be instantly plugged into “real-world” teams.
7. Similar to the shift of “usability” to “User Experience”, I would say the explosion of the Interactive Designer role has been a key change.
– Being a usability engineer 10 years ago was a lot easier, when evaluating designs made by a Product Manager in PowerPoint.
– Strong interaction designers rarely leave low-hanging fruit on the floor, though they can be blinded by over-confidence which is just another reason that researchers have had to improve their deliverables and processes in the past 10 years!
8. Customers are talking to other customers.
– Reviews in the mobile app store, amazon, etc, make it a lot harder for a bad experience to hide behind a strong brand name name.
That said, I do wonder what percentage of these observations are true beyond the halo of silicon valley, seattle, and other technology hubs? Not saying they don’t exist, but my prediction (and hope) in the next 10 years is having more UX roles embedded into the DNA of companies throughout the US!
Thanks Kathy and Catherine for putting this list together!
Hi Scott!
These are excellent additions! I agree that, in the rare occasion I am in the lab, there is far less low-hanging fruit. The research questions have gotten much more complex to answer and can rarely be answered with a single method. Triangulating with multiple methods is required.
The use of social media can be a great thing from a research standpoint but it can be difficult to know how representative those voices are of your broader user base.
I would love to hear perspectives from practitioners in other areas of the country but most particularly OUTSIDE the US. As US companies increasingly try to get a foothold in emerging markets, how has UX research in those countries changed? In our book, Apala Chavan wrote a great case study about how she had alter her research methodologies for each of the countries and user types she studied. I would love to hear more of those stories!