agreed
JasperZ511626
presentationnote-industry.docxWrite a 1 to 2 page summary/review for the presentation given by Mehron Beheshti from State Farm in class on Monday September 24. The summary/report is to be uploaded in the course spaces as a Word or PDF document. This should include a short summary of what you learned from the presentation and how what you learned will be useful for future plans (e.g. directions for career exploration, areas for future study, how the material applies to career goals you are already pursuing, etc.). Your report may also include comments on aspects of the presentation (graphics, tables, etc.) you particularly liked (or disliked) and what you liked (or disliked) about those particular aspects.
Personal Future study and career: Business analytics.
This is a statistic course. This presentation mainly talks about testing.
Do you know or do you think you know?
Incorporating business experiments into strategy development.
Setting the context
designed testing is a way of treating the business environment like a controlled experimental setting, randomly assigning strategies to customers as in a clinical drug study or lab;
benefits:
clear, accurate data
innovators, on testing
Jeff Bezos, amazon founder.
How do business use testing:
Product& strategy development
Real-time website
Examples:
1. Food lion:
2. EBay
3. Kohl’s: will opening stores an hour later lead to a drop in sales?
4. Capital one
Typical Testing evolution
None---A/B test--- “one factor at a time” (OFAT)---Multivariate(MVT)---MVT w/overate analysis-continuous testing
--First three: Pick” one winner for all” increased business insight while progressing left to right.
-- (MVT): Enables customer or segment level strategy/message optimization
State Farm examples:
Version A: lick leads to Drive Safe& Save” page
Version B: click leads to “agent search” page
1. Which mailer generated the highest lift in life insurance sales?
Version A: simple, version, no images
Version B: colorful, has several images and a big picture, and also a photo of the agent.
--winner is version A (45% greater life in sales) Version B is kind of like an advertisement.
2. Claims Contact Center: Out of scope calls
Can proactive sharing of contact info reduce misdirected or “out of scope” Claims calls?
Multivariate Discount double click test
-explored 7 different page features as tested “factors” (text, graphics, layout, images, colors, etc.)
-results in 384 distinct web page variations (with a subset of 200+ valid combinations)
-only needed to physically test 58 combinations.
Regulation:
Perceived barrier:
-our industry is too heavily regulated for this
-even if allowed, the regulators would never understand it
-it’s not “fair” to randomly assign strategies to customers in a test
response:
-Didn’t stop finance, pharma, and others from successful transition to DOE
-Successful testing industries proactively educate regulators on key concepts
-current strategy isn’t “fair” either
Complication
Perceived barrier:
-Too complicated
-complexity of analysis
-too difficult to execute
-too expensive
Response:
-Nonsense-no more complicated than predictive modeling, just new to your firm
-ignoring complex relationships doesn’t mean they aren’t there
-start small, manageable
-design according to cost constraints
Organizational Tradition
Perceived barrier:
-we tried it once we saw no improvement (i.e. learned nothing)
-“we intuitively know our business”
Response: -you don’t hit a home run every time
-you learned that those strategies were equivalent
-often gain freshman real insights that defy historical logic or perceived wisdom
-designed testing consistently outperforms consultant-provided solutions
putting it into practice
controllable environment
business goal:
-interested in causation
-uncertainty regarding best strategy
-optimization among multiple strategies
How to prepare before graduation
Class: take a stand-alone DOE class, even if you need to go to the engineering department, for example, to find one
Consulting: even if it’s not an experiment, the skills developed during consulting engagements are critical to DOE discussions
Practice: watch for opportunities to conduct your own statistical experiments
Independent research:
