Validating Hypotheses

1. Purpose

In the Design Workshop (or optionally the Validate Workshop) you defined and prioritised your testable hypotheses, and figured out experiments to validate them. Again, be careful with the word "validate". In this context, validation means we already have a clear idea of what a new product might look like, a specific way of selling to customers, or a specific marketing slogan we want to test. If your level of knowledge about the idea is quite high, you can think in terms of hypotheses, and design experiments to validate — or more accurately, falsify — them. If your idea is still vague, or you're still trying to understand your innovation context, don't use the word "validate". This is not what you need yet. Turn to Exploring Questions and formulate open questions instead of hypotheses — similar to Step 2: Discover. This is common in the first iteration of a Business Design project.

Let's assume you have your hypotheses and want to validate them: now it's time to design, prepare and run experiments that gather the facts you need. There are many ways to find out if your business model is feasible, profitable, desired by customers and fits your organisation. No two experiments look the same — plan yours wisely.

While planning your experiments, consider their level of reality (or evidence). Some experiments — interviews, for example — only offer a low level of reality, even with a prototype involved. It's always an experimental setting (see a funny example in this video). The highest level of reality comes when you charge your customer and money changes hands. Look at the examples in the picture for inspiration.

Another example of the downside of market research in experimental settings is the 1985 launch of New Coke (see here). It's highly risky to focus on just one dimension — taste, in that case — and ignore all the others. If you want to test the desirability of your offering with customers and users, think about the "investment" you expect from them as the threshold of your experiment. Compliments aren't enough — they cost nothing! Investments can be financial, time-based or social, and show how committed customers and users are to your offerings. Examples:

Financial investments

• Pre-ordering products

• Deposit

Time investments

• Invitation to event / next meeting

• Co-creation (Contribution to development)

• Test user for trial version

• Active research on the web

• Dwell time on webpage

• Registration for newsletter / event / training

Social investments

• Introduction to colleagues or friends

• Recommendation

• Testimonial

• Exceptional emotions

Another option: release the new product or service (as a prototype) to customers and users under conditions as realistic as possible, so you can observe their behaviour. We look for reality when we generate facts — we don't rely on the individual opinions of managers, employees or customers. Here too, we're talking about a time investment.

The investment you ask for depends on you, your hypothesis and experiment, your prototype or Lean Offerings, and your customers and users. The higher the level of investment you're looking for, the more reality you need. Some examples:

F = Financial investment | T = Time investment | S = Social investment

2. Duration

5 weeks (see Step 4: Validate)

3. Key Activities

The following activities represent the core of validating hypotheses:

• Design experiment: The design of your experiment depends heavily on your topic, the stage you're at and your team. We can't give you a blueprint for the RIGHT experiment for your question or hypothesis. We always aim for experiments that are repeatable — meaning they always produce the same results when we run them. Experiments often combine several Tools & Methods for Phase IV (e.g. a landing page with a tracking mechanism, plus a survey). Create a structured plan for your experiment, including

  • Method(s) & tool(s)

  • Procedure & period

  • Participants

  • Prototype(s) or Lean Offerings

  • Timeframe

  • Threshold (and how you measure it)

  ...and do a dry run if you can. Discuss your experiment's level of reality with your team (see below).

• Prepare experiment: Once you have a clear picture of what the experiment will look like, the preparation work usually follows: developing guidelines or storylines, building a prototype or lean offering, identifying and inviting participants, running marketing activities, setting up tracking, briefing your team, and collecting the data you need for your calculations. Again, there's no blueprint — this depends entirely on your setup.

• Run experiment and analyse results: When you run the experiment, tracking results and measuring the output against your threshold is crucial. A dashboard helps you stay on top of it. Stay flexible, and adapt your research design immediately if you learn something new along the way — for example, how to improve your pitch's storyline or the headline on your landing page.

4. Participants

• Team Manager

• Team Members

• Team Coach

• Optional: Prototyping Expert

• Optional: Research Expert

• (Potential) Customers

5. Tools & Materials

• See Tools & Methods for Phase IV

• Hypotheses & Experiments

6. Instructions for Coaches

1. Make sure your team is testing the right things. Teams tend to focus on ease-of-use issues and forget the bigger question: whether there's a real problem/solution fit, or whether customers see value in the offering. If you sense your team is struggling to focus on the most important hypotheses, ask them whether they'd invest their own money based on the results of their experiments. If not, they should redesign experiments they can actually rely on.

2. Teams also tend to gather and interpret information in a biased way, to support their existing beliefs (see box below). Make sure they act like "real" researchers, and design, conduct and analyse their experiments neutrally.

3. Make sure the whole team is involved in analysing the results of their experiments, combined with your neutral view as Team Coach, to help mitigate biased interpretations.

4. Whenever your team wants to test the desirability of their offering, they should think about customer investments as a threshold (see below). Make sure your teams define appropriate investments before running experiments — the investment should fit the phase you're in. Low levels of investment are typical at the beginning of Step 4: Validate. As you allow for more reality, the level of investment should increase too.