The Law of Market Failure Motivated Us to Pose the Question: Should we Invest in a new Voucher Feature on our Webshop – or Not?

The Law of Market Failure: “Most new ideas (80-90%) will fail in the market – even if competently executed” – Alberto Savoia

Note: This blog post was originally made available as an article on LinkedIn.

“The Law of Market Failure” is defined and explained in depth – as well as how to systematically overcome this innovation challenge – by Alberto Savoia (Google’s Innovation Agitator Emeritus) in his latest book “The Right It – Why So Many Ideas Fail and How to Make Sure Yours Succeed”.

The purpose of this article is to provide a real life example of how the innovation tools (thinking tools, testing tools using pretotyping experiments, analysis tools) were applied to test an idea (new voucher feature on a webshop – or not) against the market before investing major amounts of time and money (which could have resulted in a slow and expensive market failure – or a market success). Hopefully this can inspire you to apply these approaches – so you can avoid experiencing as slow and expensive failures as I have myself.

I have chosen to use the terminology from Alberto’s newest book (“The Right It…”), while the provided example predates this particular book’s release. The thinking, testing and analysis for making investment decisions is based on both Alberto’s previous and latest books and videos about “pretotyping” (sources provided at the end of this blog post).

Voucher Feature on Webshop or Not? – The Idea(s) to Test

Anyone can come up with an idea – and we all do – so there is no shortage of ideas. The question is whether or not a particular idea will satisfy customer needs to such a degree, that they will engage with it to such an extend – that it is also a viable and sustainable business idea (a market success).

A while ago I was in a context where multiple ideas related to introducing a new voucher feature on an existing webshop was in question. Some of the ideas are presented in short below.

Idea: “Our competitors have a voucher feature on their webshops, so we should have one too”.

Question: “Because competitors have a voucher feature on their webhop, do we then know that their customers engage with this feature and that this has been and is still worth their investment – and that it will be worth investing in for us”?

Answer: “No. We do not know. Concluding anything based on no valid data is not meaningful – and what Alberto Savoia labels Thoughtland”.

Idea: “If we introduce a voucher feature on our webshop, many customers will buy more of our products”.

Question: “Do we know how many customers – if any – will buy more of our products – and that it will be worth investing in for us”?

Answer: “No. We do not know. Concluding anything based on no valid data is not meaningful – and what Alberto Savoia labels as Thoughtland”.

Idea: “If customers buy more with our new voucher feature, we will make a bigger profit”.

Question:“Do we know that we will make a bigger profit if customers buy more of our products with the new voucher feature”?

Answer: “No. We do not know. Concluding anything based on no valid data is not meaningful – and what Alberto Savoia labels as Thoughtland”.

The challenge with the above ideas is, that they are abstract and fluffy – and they will not – in that form (at that point in time) – help overcome the paradox between inventors (those with the ideas), the investors (those deciding related to the bigger investments) and the customers (those engaging with the idea and ultimately paying for it with their time and money). So the ideas should be described in a hypothesis format as below.

Market Engagement Hypothesis

Market Engagement Hypothesis: “Many customers exposed to our new voucher feature on our webshop will use it while buying a product resulting in a bigger profit”.

This format is slightly better – but still so abstract and fluffy – that it cannot be tested. To refine this a bit further, it can be written in the XYZ Hypothesis format, where X is the percentage of customers engaging with the idea, Y is a description of the customer segment and Z is how the customers engage with the idea.

XYZ Hypothesis

XYZ Hypothesis: “At least 10% of returning customers exposed to the new vouchers feature on our webshop will use it while buying basic food products with a minimum 20% purchase increase“.

X = At least 10%

Y = Returning customers

Z = Buying basic food products with minimum 20% purchase increase

Using the XYZ Hypothesis the idea can now be tested against the market. Since it would be both time consuming and expensive to test against all of the market (and/or current customer base) – the overall XYZ Hypothesis can be split / refined (so-called “hypozooming”) into one smaller hypothesis or more labelled xyz1 Hypothesis, xyz2 Hypothesis and so on.

xyz1 Hypothesis

xyz1 Hypothesis: “At least 10% out of a sample of 1,000 returning customers will use our new voucher feature while buying basic food products with a minimum 20% purchase increase“.

Since the webshop was already up and running with thousands of returning customers, the sample of 1,000 returning customers could be reached fast (e.g. within a few days).

We did not want to invest a big amount of time and money before having some of our own data supporting such a decisions. So some pretotyping experiments were designed to test the different xyz hypotheses.

Pretotyping Experiments – Testing the xyz1 Hypothesis

To test the xyz1 Hypothesis two pretotyping types where utilized.

The Facade Pretotype

The Facade Pretotype was chosen to ensure that the sample of 1,000 returning customers, which were exposed to the pretotyping experiment got what they were looking for: They were able to buy the basic food products using the voucher feature hopefully having a satisfying customer experience. A small investment was made to include the voucher feature in the user interface of the webshop using A/B split testing techniques, so only 1,000 customers had the opportunity to engage with the new functionality while executing the pretotyping experiment. Basically the only difference from the existing webshop was text saying “Enter voucher code” and a form field to enter the voucher code into and a discount calculation in the shopping basket (this is very little invested effort with current web development technology).

The Mechanical Turk Pretotype

The Facade Pretotype was combined with a Mechanical Turk Pretotype where human beings manually handled all needed back office processes (not visible to customers), while running the experiment (so no investment was made upfront related to changing the IT systems). As the maximum of orders where manual work was needed was limited to 1,000 – it was much less expensive to collect data to learn whether or not to invest – than either buying/integrating or developing the voucher functionality in full.

Analyzing the Data from the Pretotyping Experiment (xyz1 Hypothesis)

“When it comes to gauging an idea’s market potential and likelihood of success, our deliberation must be based on hard data, and the data must come with some skin in the game” – Alberto Savoia

Alberto Savoia uses the term “Skin-in-the-Game Caliper” (based on “skin in the game” coined by Nassim Taleb) to evaluate to which degree data is meaningful related to pretotyping experiments. Examples worth 0 (zero) Skin-in-the-Game Points are opinions, comments, surveys, polls, focus groups, interviews etc. Common for all of these is that those answering have nothing to loose (e.g. “skin in the game”) while providing e.g. opinions or answers.

The data collected from the above pretotyping experiment was behavioral data, where those (unknowingly) providing the data had a lot of skin in the game – since they spent both their time, provided personal information (e.g. name, address, e-mail address, credit card information) and paid with real money for the bought products.

The following is not the actual data from the pretotyping experiment related to testing the xyz1 Hypothesis – but it conceptually shows the point at which data supported investing more time and money in developing the new voucher feature on the webshop. After a couple of pretotyping experiment iterations the data showed something like the below:

Data from Pretotyping Experiment: 16% out of a sample of 1,000 returning customers used the voucher feature while buying basic food products with a minimum of 23% purchase increase“.

The result compared to the previously defined xyz1 Hypothesis:

xyz1 Hypothesis:“At least 10% out of a sample of 1,000 returning customers will use our new voucher feature while buying basic food products with a minimum 20% purchase increase“.

From the above (slightly crude example) it should be clear, that this can be viewed as an idea, which might lead to market success – since 16% engaged with the idea (versus minimum 10%) and the purchase increase was 23% (versus minimum 20%) – so the results of the experiment (after a couple of iterations) exceeded the expectations.

Market Success: Results >= Expectations

I hope this served as a quick introduction to thinking tools, testing tools (pretotyping experiments) and analysis tools. If you want to learn more check out the “Sources” section below – and feel free to reach out to me.

Credits / Thank You

To Alberto Savoia and countless others for researching, developing and sharing approaches to overcome the challenges related to The Law of Market Failure (I wish I had known about pretotyping when embarking on earlier business ventures).

To Tim Vang for introducing me to and collaborating related to pretotyping.

To all of my business partners and colleagues who have designed and executed pretotyping experiments with me.

Sources

Leave a reply:

Your email address will not be published.

Site Footer