Kumar Successfully Leverages Experience with UserTesting to Execute Her Vision for Data-Driven Design

When professor Ranjitha Kumar joined Illinois Computer Science in 2014, she knew exactly what she wanted to accomplish in the next phase of her career. As a first-time faculty member with a research focus on applying data mining and machine learning to user experience design, Kumar’s mission was to bring about a new era of data-driven design: tying design decisions to desired outcomes.

The only thing she wasn’t certain of was the precise route she would take to bring about her vision. Kumar came out of the Ph.D. program at Stanford University with a venture-backed startup called Apropose, which she built to construct data-driven design software for the Web. She then worked with Google and Adobe on open-source collaborative projects with her students at Illinois.

While both opportunities provided their own insights, the last three years she has spent as the Chief Scientist at UserTesting ended up being the most productive – with more than eight product releases over that span. UserTesting IPO’d two years into Kumar’s tenure there, and since that time, the company was taken private by private equity firm Thoma Bravo, and in early April announced a merger with UserZoom, another market leader in UX insights.

“UserTesting represented the best of both worlds that I’d experienced in industry,” Kumar said. “Launching a startup offers great rewards, but, ultimately, you don’t know if you’ll get to answer your questions. At a giant company, you have immense resources, but it’s much harder to move the needle on the business.”

“At UserTesting, I was able to run the right experiments to find out which parts of my research really mattered to the industry. Every time we validated a direction, we could deploy it to thousands of customers and hundreds of thousands of users. When you find alignment like that between academia and industry, the whole experience turns into a rocket ship.”

Kumar’s interest in UserTesting grew from its dedication to human insight, which the company describes as “offering a valuable understanding of customers that result from listening and observing with empathy and can be used to connect the dots between what they think, feel, say, and do. It provides the ‘why’ context that gives organizations the ability to understand customer needs and rethink ways to better serve them.”

Immediately preceding her time with UserTesting, Kumar built an interaction mining system that she unveiled through a paper entitled, “ZIPT: Zero-Integration Performance Testing of Mobile App Designs.”

The key idea, she said, was to build a remote usability testing platform for tasks on Android apps that required zero code integration, allowing users to run tests on any of the apps on the Google Play store. During the tests, the platform recorded user interaction and design data and combined these different data streams into structured representations that returned analytics on usability. It computed quantitative visualizations and metrics to aid in the qualitative analysis of usability tests.

“It was at this point, I was considering starting another company when UserTesting reached out,” Kumar said. “The questions I wanted to answer and the problems the company wanted to solve aligned, which was important to me. By bridging the quantitative and qualitative approaches to experience design we can provide a single platform for optimizing digital experiences holistically: quantitative methods to uncover what and how much and qualitative ones to understand why.”

Among the product features Kumar and a team of UserTesting employees released, here are a few examples:

• Interactive Path Flows, which represented the company’s “first step in bridging quantitative and qualitative experience testing.” Kumar and UserTesting was awarded a patent last November on the Interactive Path Flows technology.
• Machine-learning powered data visualizations, which allowed users to “see data from multiple participant sessions side-by-side, making it faster than ever to identify the most interesting moments and uncover actionable insights.”
• Instant insight, which helps users “save time during post-test analysis and generate actionable findings by automatically surfacing interesting patterns, anomalies, and correlations for task-based questions.”
• Friction Detection for enhanced behavioral analytics, which advances the roadmap that she’s built around behavioral analysis and machine learning.

“At the end of the day, all the emotion and frustration and delight that gets captured in UserTesting videos can really help inform decision makers. But people don’t want to wade through hours of video or reams of data by themselves,” Kumar said. “We’re creating a system that can automatically find and highlight the 20-second clip that you can show to your manager or CEO to help them understand whether something worked or didn’t.”

Kumar continued, “This is exciting because we’re leveraging the technology to bring basically everyone in an organization closer to understanding their customer.”

Kumar will get a chance to paint this vision at an upcoming keynote presentation for the 2023 ACM SIGIR Conference on Research and Development in Information Retrieval.

She also expressed optimism about accomplishing more in this realm while remarking on the gratitude she felt for the opportunity. And Kumar's optimism for the future comes from the comfort of knowing she has found her niche, something she strove for when she came to Illinois CS motivated by the possibilities included in technology transfer – or the process of creating something new in a research lab and turning that into commercial products.

“One of the things that originally drew me to Illinois CS was the way this department measures impact in all forms,” Kumar said. “I care deeply that the work my students and I do makes its way into the world above and beyond publishing papers, so I’m glad to have a department that recognizes that value and helps me achieve it.”