To validate that a product concept is better than competing solutions, companies must know what metrics its customers use to measure the successful execution of the job-to-be-done. In addition, they must find out how customers evaluate the new concept, using these metrics. Most concept testing methods are not designed to make these determinations. They don’t even include these metrics in the evaluation. Instead, the concept test involves asking customers to make a series of trade-offs between a small number of features or other attributes to reveal the relative importance of the concept’s component attributes.
This thinking is fraught with risk. One risk is that the features being ranked do not address the customers’ top unmet needs. If that’s the case, the test will reveal the winning feature set, but those winning features will still fail to to help customers get the job done better, and customers will not value the product or service. Another risk is that the customer will fail to make the connection between their unmet needs and the features being tested, leading to highly inaccurate results. In either case, bad investment decisions will result.