Concept Testing Methods

Concept testing is the process of evaluating likely customer response to a product idea prior to its introduction into the market. Seen through a jobs-to-be-done lens, the goal of the concept testing process is to validate that a product concept is better than competing solutions at helping customers get a job done. To make this determination, we must know what metrics the customers use to measure the successful execution of the job-to-be-done. Our methods work because they are built around these customer metrics. It is the only concept testing method to use this approach and is an integral step in our innovation process, Outcome-Driven Innovation (ODI).

Conjoint analysis, choice modeling, and other concept testing techniques ask customers to make trade-offs between product features or attributes, often in relation to price. They assume that the goal is to reveal the most desirable feature set. This thinking is the cause of many problems and is the myth that misleads. Concept testing is the process of validating that a concept (that has already been conceived) is significantly better at helping customers get a job done. It is not the process of creating the concept: that is ideation. Conjoint analysis and other methods attempt to do both, but they are not effective at either.

To validate that a product concept is better than competing solutions, companies must know what metrics its customers use to measure the successful execution of the job-to-be-done. In addition, they must find out how customers evaluate the new concept, using these metrics. Most concept testing methods are not designed to make these determinations. They don’t even include these metrics in the evaluation. Instead, the test involves asking customers to make a series of trade-offs between a small number of features or other attributes to reveal the relative importance of the concept’s component attributes.

This thinking is fraught with risk. One risk is that the features being ranked do not address the customers’ top unmet needs. If that’s the case, the test will reveal the winning feature set, but those winning features will still fail to help customers get the job done better, and customers will not value the product or service. Another risk is that the customer will fail to make the connection between their unmet needs and the features being tested, leading to highly inaccurate results. In either case, bad investment decisions will result.

We have discovered that customers use between 50 and 150 metrics (desired outcomes) to evaluate how well a product helps them get a job done. These metrics are the customers’ needs. In addition, they use numerous other metrics to evaluate the execution of the related consumption chain jobs, which include acquiring, setting up, learning to use, interfacing with, maintaining, repairing, and replacing the product. Traditional concept testing methods do not provide a qualitative means for capturing these metrics or a quantitative means for ascertaining their relative importance. Our approach does both. Without these insights, accurate concept testing results cannot be achieved.

Well before concept testing, we use these same metrics to determine which customer needs are unmet and construct the solution that best addresses them. As a result, this becomes a much simpler task because it requires only that we validate with customers that the proposed solution addresses the unmet needs (customer metrics) as well as we had envisioned. This is a test we are almost certain to pass because our concept has already been designed to be the product or service the customers want. Learn more about our growth strategy consulting services.

Turn Customer Input into Innovation
In this timeless 2002 Harvard Business Review article, Tony Ulwick first introduces the concept of Outcome-Driven Innovation and the opportunity algorithm to HBR readers. He explains how Cordis Corporation (now a division of Johnson & Johnson) used ODI to create and concept-test ideas that ultimately increased its angioplasty balloon market share from 1 percent to 20 percent.
Learn More

The Customer-Centered Innovation Map
In this groundbreaking 2008 Harvard Business Review article, Tony Ulwick and Lance Bettencourt reveal an important discovery they made while turning the jobs-to-be-done innovation theory into practice: job mapping. A job map breaks down the job the customer wants to get done into specific process steps, making it easier to discover the metrics customers use to measure success and bringing superior concept testing within reach.
Learn More

Giving Customers a Fair Hearing
In this provocative article in MIT Sloan Management Review, Tony Ulwick reveals how a customer need must be defined within the jobs-to-be-done framework to become a useful input into your innovation process. This article introduces timeless standards for understanding the types of customer needs that exist, how we use them to generate a big idea that creates value for customers, and how to validate the concept with new concept-testing methods.
Download the article

Ideas-First or Needs-First: What Would Edison Say?
In just over 30 years, Thomas Edison, one of the world’s greatest innovators, pioneered six industries that today have a cumulative market value of more than $1 trillion. How did he come up with these big ideas? Learn the answer, and how outcome-driven thinking can help your company grow, from Sarah Miller Caldicott.
Download the white paper

Silence the Voice of the Customer
This white paper by Strategyn founder Tony Ulwick highlights the business advantages of using our ODI methods to capture and prioritize customer needs. It explains why traditional voice-of-the-customer (VOC) practices continue to fail and makes a solid case for retiring VOC both for understanding customer needs and for driving the concept-testing process.
Download the white paper

What is Outcome-Driven Innovation?
Outcome-Driven Innovation (ODI) is the most effective innovation process in existence today. This white paper by Strategyn founder and ODI inventor Tony Ulwick explains why innovation has historically been an ineffective process, the discoveries he made that led to ODI, and how it enables us to come up with the big ideas and validate them through new concept-testing methods.
Download the white paper

A New Perspective on Strategy
The goal of business strategy formulation is to create a unique and valued competitive position. This white paper by Strategyn founder Tony Ulwick offers a different perspective on strategy, explaining why customer needs (not company activities) are the basic unit of competitive analysis. Ulwick also introduces innovation strategies we use to drive company growth and beat competition.
Download the white paper

What Customers Want
What Customers Want, the best seller by innovation thought leader Tony Ulwick, explains what Outcome-Driven Innovation (ODI) is and why it works. Ulwick, who pioneered jobs-to-be-done thinking and invented ODI, details how ODI transforms jobs-to-be-done theory into a practical method for ideation and concept testing that drives big ideas and company growth.
Learn More

Choose from a dozen case studies of companies such as Microsoft, Ingersoll Rand, Bosch, and others, and learn how we apply our ODI methodology to generate big ideas, validate them through new concept testing methods, and help companies grow.
Learn More