Uncovering Unknown Unknowns: 3 Times Design Research Revealed Critical Information

3 examples of how design research unlocks the “why” behind user behavior to provide essential insights.

By Mark Power-Freeman, UX Lead in Design at Praxent.

We know research is important, but just how important is often unrealized. Design research is absolutely essential, as it answers important questions and allows us to get at the “why” behind why users do what they do. It’s only when we understand user behavior – and the driving motivations behind it – that we can effectively design a product tailored to their needs and wants.

What Is Design Research?

Design research is a type of user research that examines how users interact with a product, specially as it relates to their user experience. Design research can be highly in-depth, or it can be done more quickly without breaking the bank. Either way, it’s essential and can be conducted in several ways:

• User surveys
• User interviews
• Contextual inquiries

At Praxent, we prioritize user research, as we know how essential it is in unlocking the key insights that drive product design — and innovation. Design research can be particularly helpful in scenarios where we know the user isn’t behaving as expected, yet we don’t know why. When it comes to new product design, then, design research can provide clues on how a user will act.

Below, we look at three design research examples that illustrate just how revealing research can be – and the far-reaching implications it can have for a product.

>>> Want to learn more about user research? Check out our article: Making Sense of User Research: 5 Tools for Finding & Refining Winning Product Ideas

Design Research Case Study #1: Streaming Media Company

Data showed that users weren’t engaging with additional content after watching an initial video clip. While the team thought either the playlists weren’t prominent enough or they were too long, the design research revealed something else entirely.

The UX Issue: Low Engagement

A company specializing in streaming live events, replays, and documentaries from sports that fall outside of the main areas of coverage for ESPN and other major players in the industry noticed something curious on their platform. Live events and documentaries enjoyed robust viewership numbers, but Google Analytics showed that subscribers weren’t staying engaged with automated or curated playlists after watching video clips. They would exit the playlist or article that had drawn their attention after watching only the single clip.

This hurt a number of engagement metrics. It also suggested the content creation and product design teams were wasting resources lining up content that was going unnoticed by subscribers.

Theories and Attempted Solutions: Content Placement and Length

Teams and department heads talked through this issue, expressing myriad opinions about the root of the problem and what we could do to fix it. There were two theories that won a Darwinian competition to determine the cause and the solution for this problem:

Theory 1: The playlists weren’t prominent enough, so people weren’t clicking on them.

Theory 2: The clips were too long, and people weren’t sticking around. 

We addressed the first theory by moving the playlists around and giving them different visual design treatments. We attacked theory number two by experimenting with different lengths and formats. Neither solution moved the needle enough to make everyone happy. We looked to be locked into an endless cycle of debate followed by fruitless action.

That’s when the product team pointed out that design research could put us on the right track by connecting with users and getting their thoughts on the subject.

Design Research: 1-Hour Conversations

We wanted to move fast, so we recruited from a local user pool. We conducted a series of one-hour research conversations with subscribers known personally to some of the high-level stakeholders. Despite the personal connection the participants had to the company, as professional and semi-professional athletes they were not shy at all about sharing their opinions. One quote from these sessions was especially memorable: In response to a question about infinite scrolling on the home page, the participant snorted and said: “Don’t do this. Infinite scrolling is death.” Noted!

Our goal was a usability assessment of the sites, but we focused a significant portion of our sessions on video consumption patterns. It didn’t take long for an observable user behavior pattern to emerge.

The Outcome: Change How Clips Were Assembled

We spoke with 12 users in all, but developed a working theory by the end of session 3. Our users simply weren’t interested in the way clips were chosen for the pre-assembled playlists. Instead of themed playlists (for example, a separate video for each preliminary heat in a race, or each match in a weight class for wrestling), they wanted to watch clips about a particular athlete. This was almost always because of a personal connection: the athlete they were looking for was a daughter or son, a sibling, a training partner, or a close friend.

Analytics had told us What, but it took design research with actual users to tell us Why. Armed with this new information, the company changed not only the user experience but also the customer experience. Marketing had a new angle to pursue (watch your child compete), and product design and content creation were able to end wasted work, and instead focus on bringing other aspects of the celebration and promotion of the sports to fans’ attention.

Design Research Case Study #2: Energy Sector Company

Users were only using a small amount of the application’s functionality, and session time was short. Without a unified theory on what was happening, design research proved instrumental in revealing some of the app’s shortcomings from the customer’s point of view.

The UX Issue: Low Engagement With Custom Application

We worked with a company in the energy sector that had a platform built to track energy usage data, enabling traders and analysts to make effective business decisions regarding energy purchases. It came to light that the company’s customers were tapping into only a tiny amount of the app’s power, barely engaging with it for any significant length of time in a given session. They often preferred to have their account representatives do all of the heavy lifting instead of putting in the work themselves.

Theories and Attempted Solutions: Ideas But No Unified Theory

The stakeholders at the company had ideas about some of the stumbling blocks placed in front of their end users, but there was no “grand unified theory” that comprehensively explained the lack of engagement. It was concerning, though, as a lack of customer engagement is the stumble that often precedes a fall. Net Promoter Scores — seen as “late and not too lamented” by some — identified one aspect of customer behavior that marketing and sales teams should find frightening: disengaged customers are extremely vulnerable to being wooed away by competitors and letting subscriptions lapse.

There was also a tremendous opportunity cost for internal teams associated with lackluster user engagement, as hours spent developing unused features could have gone toward more impactful functionality.

Design Research: Remote Research Conversations

We conducted seven remote research conversations with participants who fit one or more of the three personas we had identified in collaboration with stakeholders and project sponsors. Each conversation lasted about an hour.

The Outcome: Core Functionality Was Invisible, Disappointing, Discouraging Further Exploration

We saw trends emerging after just the first few interviews. The features the users did know about weren’t meeting their needs, and a halo effect of this inadequacy was they simply weren’t bothering to explore the system more. They didn’t know all the system had to offer, nor was it surfacing the core functionality that mattered most to users. For example, during generative research interviews, we saw users were doing the same searches every time they accessed the system, failing to take advantage of the built in Saved Search functionality.

After our design research revealed that the primary experience issue centered on the invisibility of core functionality, we designed a system to bring the most useful features to the forefront via a dashboard and persistent navigation. When we put the new design in front of users for testing, one person said: “I avoided the application for a long time because it was really hard to navigate. This is so much more friendly than the other version.” 

Two words jump out from that quote: avoid and friendly. Avoiding a tool is even stronger than not using it. The redesign helped this user overcome a deep-seated aversion, and it struck the user as friendly and inviting in a way that its predecessor hadn’t.

 Design Research Case Study #3: Benefits Company

Rather than feeling positive about enrolling for valuable benefits, employees seemed confused and even acted carelessly in the process. Design research revealed unexpected interactions that began well before they clicked “enroll.”

The UX Issue: Confusion and Carelessness

In an ideal world, open enrollment for employer benefits would be an occasion for joy. We’d see people jumping up at their desks when they finished, and there’d be cigars, champagne, a DJ, and a laser light show. We’re not there yet, but there ought to be a little bit more of a celebratory air surrounding enrollment. After all benefits are a positive; they help you protect both your health and wealth.

Instead, when the email from HR announcing enrollment hits people’s inboxes, the reaction is one of confusion and carelessness. Yet we’re not quite sure why. That was the state of things when we began working with a company that builds and sells benefits administration platforms. They sought us out to help build a better enrollment process, but before we could do that we had to figure out the source of the trouble and confusion.

Design Research: Interviewing Specific User Personas

This was a mission that called for cordoning off personal feelings about the subject matter. We’re not energy traders, real estate impresarios, or managers of contractors; but every one of us has enrolled in benefits. So the trick here was to drill down on this subject without biasing the research and questions with our own theories.

To steer clear of our own thoughts, we spoke to 18 people representing six personas in the benefits narrative (carriers, employers, and insurance brokers). Fun fact: Gathering large amounts of data plays a critical role in pulling a researcher away from their biases when it comes time to examine session notes for patterns.

The Outcome: Behind-The-Scenes Ecosystem and Desire for Information

Our design research provided several major insights from the angle of each persona:

• Employees felt lost before they even began the process and needed help with the rationale for a benefits election
• Employers felt the tech did not effectively support them
• Carriers felt their products were not properly represented in the selection process

Many of these point to significant big picture changes, especially in the relationships among brokers, carriers, employers, and the insured. Our research revealed a previously unknown ecosystem and interdependence behind the information and options that appear when we click on a link from HR and start signing up.

We also learned how to parcel out information to employees during the pre-enrollment and enrollment phases. Essentially, insights revealed that benefit recipients want a great deal of supporting material when making choices, but they also don’t want to read. As more than one person put it, everybody just wants to zoom through the process, but they also want to feel like this: “Who just nailed their benefits selection? I did.” It’s our hope that as the new approach is deployed, we’ll be able to share plenty of success stories.

>>>3 Keys to Practical User Research for New Products [“Define Your Product” Series]

As these design research examples illustrate, design research is absolutely instrumental in ensuring your product meets client needs and wants. At Praxent, we prioritize design research and emphasize its importance in every project we do – whether that’s building custom software from scratch or improving upon existing systems. Research is one of the major tools that helps us unlock the insights that lead to disruption. Ready to get started? Speak with one of our experts.