8

Test Idea Framework Create a Data-Driven Test Idea

from Start to Finish

Qualitative data pool What customer feedback, user research, or other qualitative input do we have?

Hypothesis Write out what you expect to happen based on the change you’re testing with a clear, data-informed rationale.

Secondary metrics What secondary metrics will we track as part of the experiment?

Next Steps • Prioritize this experiment against other ideas we have.

• Determine the level of effort to design, build, and QA the experiment.

• Communicate plans for the experiment.

Targeting Which audiences will be a part of the experiment?

Duration What is the longest the experiment will need to run, based on our statistical significance calculations?

Quantitative data pool Which analytics data do we have to support our decision of what to test?

Top metric Top metric you’re using to measure success: revenue, NPS, or something else.

Test location Pick a specific page, flow, or experience that could affect the top metric.

Proposed change Describe a change you’d like to test.

OPTIMIZELY TESTING TOOLKIT 2016

What problem are we trying to solve? Describe a high-friction experience, an underperforming landing page, or another strategic issue you’re working on.