Self-Understanding for Opportunity Creation
Chapter 7: Testing and Experimenting in Markets
7.1
Define experiments and describe the steps of experimentation.
7.2
Identify and describe the six steps of scientific experimentation.
Neck, Entrepreneurship. © SAGE Publications, 2018.
3
Learning Objectives
7.3
Demonstrate how to test hypotheses and identify customers.
7.4
Explore different ways of generating data and describe the rules of experimentation.
Neck, Entrepreneurship. © SAGE Publications, 2018.
4
Learning Objectives
7.5
Identify three types of experiments most commonly used.
7.6
Illustrate the power of storyboarding as a form of prototyping and a basis for experiments.
Neck, Entrepreneurship. © SAGE Publications, 2018.
5
Learning Objectives
Notable Quote
“Experiments are key to innovation because they rarely turn out as you expect, and you learn so much.”
Jeff Bezos, Amazon.com founder and CEO
Neck, Entrepreneurship. © SAGE Publications, 2018.
6
7.1 What Are Experiments?
7
Neck, Entrepreneurship. © SAGE Publications, 2018.
8
Experimentation
Experimentation is a method used to prove or disprove the validity of an idea or hypothesis.
A hypothesis is an assumption that is tested through study and experimentation.
What Are Experiments?
Neck, Entrepreneurship. © SAGE Publications, 2018.
9
Experimentation Includes
Asking questions
Taking action
Trying something new
Collecting real-world information
Testing new concepts
What Are Experiments?
7.2 The Six Steps of Scientific Experimentation
10
Neck, Entrepreneurship. © SAGE Publications, 2018.
11
The Six Steps of Scientific Experimentation
Asking lots of questions
Carrying out background research
Developing hypotheses
The Six Steps of Scientific Experimentation
“Steps of the scientific method” http://www.sciencebuddies.org/science-fair-projects/project_scientific_method.shtml accessed December 27, 2014.
11
Neck, Entrepreneurship. © SAGE Publications, 2018.
12
The Six Steps of Scientific Experimentation
Testing the hypotheses by running experiments
Analyzing the data
Assessing results
The Six Steps of Scientific Experimentation
“Steps of the scientific method” http://www.sciencebuddies.org/science-fair-projects/project_scientific_method.shtml accessed December 27, 2014.
12
Figure 7.1: The Scientific Method
Neck, Entrepreneurship. © SAGE Publications, 2018.
13
Source: Retrieved from http://generalchemistryfordson2013.weebly.com/scientific-method-flow-chart.html
Suppose you started a business and your original assumptions about your target customer base turned out to be incorrect? How would you deal with this setback?
Neck, Entrepreneurship. © SAGE Publications, 2018.
14
7.3 Hypotheses and Customer Identification
Neck, Entrepreneurship. © SAGE Publications, 2018.
16
Hypothesis and Customer Identification
In 2008, Subway hypothesized that selling select foot long sandwiches for $5 would increase overall sales.
Hypotheses and Customer Identification
In 2008, as the U.S. and other countries were hit with a recession, some marketers at Subway suggested a hypothesis that selling foot-long subs at the reduced price of $5 would increase sales. Yet others were concerned that the promotion would distract customers from purchasing the more expensive items on the menu.
So, Subway carried out an experiment: it tested the promotion in some Subway sites, but not in others, and for limited time periods, such as weekends. The results showed that the $5 subs did not detract from the overall sales. However, even if the experiment had failed to support the hypothesis—if customers had stopped buying the more expensive items on the menu—it still would have raised many interesting questions: Why were customers not buying the more expensive items? What could Subway do to attract them to the other items on the menu, while offering the promotion? The Subway experiment turned out to be a useful way for them to define customer tastes and preferences.
Thomas H. Davenport, “How to design smart business experiments,” Harvard Business Review, (February, 2009, pp. 68- 76).
16
Neck, Entrepreneurship. © SAGE Publications, 2018.
17
Limited, Low-Cost Experimentation
One example: Students who considered making an app to prevent classroom disengagement tested their idea using yellow flags that were thrown in the classroom.
Hypotheses and Customer Identification
Neck, Entrepreneurship. © SAGE Publications, 2018.
18
Involving Potential Customers
Helps in getting immediate feedback
Assists with making connections when the product is launched.
First, determine the types of customers to involve.
Hypotheses and Customer Identification
Neck, Entrepreneurship. © SAGE Publications, 2018.
19
Six Types of Customers
End users: customers who actually buy and use the product
Influencers: celebrities or other trendsetters
Hypotheses and Customer Identification
Steve Blank and Bob Dorf, The Startup Owner’s Manual. (California: K&S Ranch, 2012)Pp. 87-88.
19
Neck, Entrepreneurship. © SAGE Publications, 2018.
20
Six Types of Customers
Recommenders: industry experts, bloggers, those who offer opinions to the public
Economic buyers: customers who approve large purchases such as retail store buyers, office managers, etc
Hypotheses and Customer Identification
Neck, Entrepreneurship. © SAGE Publications, 2018.
21
Six Types of Customers
Decision-makers: ultimate decision-makers such as CEOs, CFOs, or mom and dad
Saboteurs: those who can veto purchases or slow down the purchasing process
Hypotheses and Customer Identification
Figure 7.2: Six Types of Customers
Neck, Entrepreneurship. © SAGE Publications, 2018.
22
Imagine starting a particular kind of business of your choice. How would you use social media to test a reaction of your target audience?
Neck, Entrepreneurship. © SAGE Publications, 2018.
23
The Rights of Research Participants
Ethical concerns
Legal requirements
Regulations on use of animals in testing
Neck, Entrepreneurship. © SAGE Publications, 2018.
24
ENTREPRENEURSHIP MEETS ETHICS
Before beginning testing and experimentation, the entrepreneur must consider some ethical concerns related to market research and the rights of research participants. Most notably, people participating in experiments have the right to informed consent; the right to be treated with dignity regardless of racial or ethnic background, sexual preference, or socioeconomic status; the right to privacy and confidentiality; and the right not to be deceived or harmed as a consequence of research participation. In addition, there are legal requirements for testing of food items and personal care products that come into contact with the human body, as well as regulations on the use of animals in product testing.
As long as the researcher is able to conduct the market research ethically and laws are followed, research doesn’t have to be expensive. For example, entrepreneurs may enlist a group of people to try out free samples of a product in exchange for submitting an evaluation or attending a focus group afterwards. There are also many laboratories that perform testing for regulatory compliance, with various price structures to suit different budgets.
24
Rights of Research Participants
The right to informed consent
The right to be treated with dignity regardless of ethnic background, sexual preference, or socioeconomic status
Neck, Entrepreneurship. © SAGE Publications, 2018.
25
ENTREPRENEURSHIP MEETS ETHICS
Rights of Research Participants
The right to privacy and confidentiality
The right not to be deceived or harmed as a consequence of research participation.
Neck, Entrepreneurship. © SAGE Publications, 2018.
26
ENTREPRENEURSHIP MEETS ETHICS
How would you ensure that the participants in your experiments are treated ethically and their rights are protected?
27
7.4 Generating Data and the Rules of Experimentation
28
Neck, Entrepreneurship. © SAGE Publications, 2018.
29
Rules of Experimentation
Focus on all stakeholders
Ask lots of questions
Think like a scientist, but do not act like a scientist
Build your learning into the next iteration
Generating Data and the Rules of Experimentation
1. Focus on all stakeholders. Entrepreneurs need to focus on all the stakeholders potentially involved in the startup – these include customers, partners, suppliers, distributors, even real estate agents.
2. Ask lots of questions. Remember, every question you have about your idea is fertile ground for an experiment.
3. Think like a scientist but don’t act like a scientist. In other words, it’s important to think through your hypothesis, what you want to test and how you are going to test, but don’t get bogged down in the rigor.
4. Build your learning into the next iteration. Don’t ignore negative information just as you don’t want to ignore positive information. A general rule of thumb is that six pieces of information saying the same thing can be a fact!
5. Keep track of your data. You may think a piece of information is not important but it is essential to keep track of everything.
6. Keep your experiments low cost and quick; and use them to shape and improve ideas.
7. Don’t just talk with stakeholders – interact with them.
8. Don’t ignore data just because you don’t like what it’s telling you.
29
Neck, Entrepreneurship. © SAGE Publications, 2018.
30
Rules of Experimentation
Keep track of your data
Keep experiments quick and low cost
Interact with stakeholders
Pay attention to the data
Generating Data and the Rules of Experimentation
1. Focus on all stakeholders. Entrepreneurs need to focus on all the stakeholders potentially involved in the startup – these include customers, partners, suppliers, distributors, even real estate agents.
2. Ask lots of questions. Remember, every question you have about your idea is fertile ground for an experiment.
3. Think like a scientist but don’t act like a scientist. In other words, it’s important to think through your hypothesis, what you want to test and how you are going to test, but don’t get bogged down in the rigor.
4. Build your learning into the next iteration. Don’t ignore negative information just as you don’t want to ignore positive information. A general rule of thumb is that six pieces of information saying the same thing can be a fact!
5. Keep track of your data. You may think a piece of information is not important but it is essential to keep track of everything.
6. Keep your experiments low cost and quick; and use them to shape and improve ideas.
7. Don’t just talk with stakeholders – interact with them.
8. Don’t ignore data just because you don’t like what it’s telling you.
30
Table 7.1: The Rules of Experimentation
Neck, Entrepreneurship. © SAGE Publications, 2018.
31
How would you go about finding out what kinds of testing are required for an entrepreneurial product or service?
Neck, Entrepreneurship. © SAGE Publications, 2018.
32
7.5 Types of Experiments
33
Neck, Entrepreneurship. © SAGE Publications, 2018.
34
Three Types of Experiments
Trying out new experiences
Taking apart products, processes, and ideas
Testing ideas through pilots and prototypes
Types of Experiments
Neck, Entrepreneurship. © SAGE Publications, 2018.
35
Examples of Trying Out New Experiences
Living in a different country
Working in multiple industries
Developing a new skill
Types of Experiments
35
Neck, Entrepreneurship. © SAGE Publications, 2018.
36
Examples of Taking Apart Products, Processes, and Ideas
Disassembling a product
Visually mapping out a process
Deconstructing an idea
Types of Experiments
Neck, Entrepreneurship. © SAGE Publications, 2018.
37
Examples of Testing Ideas Through Pilots and Prototypes
Building a prototype
Piloting a new process
Launching a new venture on the market
Types of Experiments
Figure 7.3: Three Types of Experiments
Neck, Entrepreneurship. © SAGE Publications, 2018.
38
7.6 The Power of Storyboarding
39
Neck, Entrepreneurship. © SAGE Publications, 2018.
40
What Is Storyboarding?
Storyboarding is an early form of prototyping that provides a high-level view of thoughts and ideas arranged in sequence in the form of drawings, sketches, or illustrations.
The Power of Storyboarding
Figure 7.4: Storyboard
Neck, Entrepreneurship. © SAGE Publications, 2018.
41
Source: Retrieved from http://www.fastcodesign.com/1672917/the-8-steps-to-creating-a-great-storyboard. Credit: ©iStockphoto.com/Rudimencial
Neck, Entrepreneurship. © SAGE Publications, 2018.
42
Reasons for Storyboarding
Engaging
Simple
Effective in communicating position
“A picture is worth a thousand words”
Inexpensive
Elicits questions from participants
The Power of Storyboarding
Figure 7.5: The Problem–Solution–Benefit Framework
Neck, Entrepreneurship. © SAGE Publications, 2018.
43
The Power of Storyboarding
Neck, Entrepreneurship. © SAGE Publications, 2018.
44
Basic Storyboarding
What is the problem your customer is experiencing?
What are you offering as a solution to the problem?
How will your customer benefit from your product/service offering?
The Power of Storyboarding
The problem: Many women (even those in well-paid jobs) cannot afford designer dresses to wear to special occasions. “I want to wear a designer dress, but they are very expensive and I would probably only wear it once. ”
The solution: Give women access to designer dresses by creating an online business renting designer dresses for one tenth of the original cost. “I get access to the latest dresses but I get to rent for the night rather than buy!”
The benefit: The rental model gives many more women the opportunity to wear designer dresses, which they could have never afforded before; and provides designers with an opportunity to build their brand because their dresses are being showcased by a larger demographic of young, fashionable women. “This service would let me feel like a movie star
44
Figure 7.6: Storyboard of an Idea to Boost Student Engagement
Neck, Entrepreneurship. © SAGE Publications, 2018.
45
Table 7.2: Early Questions Generated by Storyboarding
Neck, Entrepreneurship. © SAGE Publications, 2018.
46
Do you think people with artistic training or talent have an advantage in storyboarding? Why or why not?
Neck, Entrepreneurship. © SAGE Publications, 2018.
47
Create a Storyboard and One Simple Experiment
Neck, Entrepreneurship. © SAGE Publications, 2018.
48
Mindshift: Create a Storyboard and One Simple Experiment
By this point in the book, we are sure you have at least one idea, if not hundreds, floating around in your mind. We hope you’ve developed a practice of writing down your ideas. Now it’s time to take one of your ideas and draw it in action using a storyboard format. The simplest format is the four-quadrant version depicted in Figure 7.6—the storyboard to boost student engagement.
Neck, Entrepreneurship. © SAGE Publications, 2018.
49
Mindshift: Create a Storyboard and One Simple Experiment
Artistic talent is not required. Simply focus on visually representing the four aspects of your idea: problem, solution, organization, and outcomes/benefits. As you create your sketches, questions will probably arise related to your idea. Once you’ve completed the storyboard, write a list of all the questions that you have, now that you have envisioned your idea in action. It’s okay if you have a long list. As a matter of fact, the longer the better.
Neck, Entrepreneurship. © SAGE Publications, 2018.
50
Mindshift: Create a Storyboard and One Simple Experiment
Once you have your questions, identify the top three questions you want to answer first, and develop an action-oriented experiment to answer each question. Be specific: What’s the question the experiment is designed to examine? What do think you’ll find (i.e., what is your hypothesis)? Next, conduct the experiments. What did you learn? How will you build this learning into the next iteration?
Neck, Entrepreneurship. © SAGE Publications, 2018.
51
1. At the outset of this exercise, how did you feel about being asked to create a storyboard? Do you think people with artistic training or talent have an advantage in storyboarding? Why or why not?
Neck, Entrepreneurship. © SAGE Publications, 2018.
52
MINDSHIFT: CRITICAL THINKING
2. Is your list of questions longer or shorter than you expected? How easy or difficult was it to translate your top three questions into experiments?
Neck, Entrepreneurship. © SAGE Publications, 2018.
53
MINDSHIFT: CRITICAL THINKING
3. What did you learn from this exercise that surprised you?
Neck, Entrepreneurship. © SAGE Publications, 2018.
54
MINDSHIFT: CRITICAL THINKING