week 5 Discussion
Data Driven Decision Making Week 5: Communication Visualization Monitoring
1
Tracking outcomes
Tracking outcome
When you make a decision, it is critical to answer:
What is the desired outcome?
How will that outcome be measured / monitored?
Week 4 Email-Essay asked for a quantified desired outcome…
Week 5 Email-essay
For this week’s essay, you will write an email to your boss describing the desired outcome for the decision made in one of the cases (Salesforce, Netflix) and how that outcome will be measured.
Specify:
Desired outcome/project objective
what metrics/KPIs will be used and why
targets for KPIs
what it means if a particular metric or KPI goes up or down
what teams should have access to / be responsible for monitoring which KPIs
This week’s essay will not have slides associated with it, instead will use Tableau to build a Dashboard or Outcome Report.
What metrics should be tracked?
Tracking Performance: Key Performance Indicators (KPIs): Best Practices
Quantifiable measurements agreed upon in advance which reflect achievement of marketing goals and brand strategy.
Set KPIs using SMART criteria:
Is our objective Specific?
Can we Measure progress towards that goal?
Is the goal realistically Attainable?
How Relevant is the goal to our organization?
What is the Time-frame for achieving this goal?
Slide from Uwe Hohgrawe
Storytelling
Why Storytelling
Engage your audience for better:
Attention
Interaction
Buy-in/motivation
Recall
Freytag’s Pyramid
By No machine-readable author provided. BrokenSegue assumed (based on copyright claims). - No machine-readable source provided. Own work assumed (based on copyright claims)., Public Domain, https://commons.wikimedia.org/w/index.php?curid=1357598
Based on Greek and Shakespearean drama
Freytag’s Pyramid
By No machine-readable author provided. BrokenSegue assumed (based on copyright claims). - No machine-readable source provided. Own work assumed (based on copyright claims)., Public Domain, https://commons.wikimedia.org/w/index.php?curid=1357598
Based on Greek and Shakespearean drama
Resolution
Freytag’s Pyramid
By No machine-readable author provided. BrokenSegue assumed (based on copyright claims). - No machine-readable source provided. Own work assumed (based on copyright claims)., Public Domain, https://commons.wikimedia.org/w/index.php?curid=1357598
Background, Context
Business Problem
A business case study
Follow up metrics say “Success!”
Next steps
Data analysis
General Solution
Implementation proposal
Timeline proposal
Insight/cause!!!!
Implementation
Resolution
Connect your audience to the material
Motivate why should your audience should care
Build some mystery/conflict – What went wrong? What’s the solution?
Resolve the mystery – Answer what went wrong / what’s the solution
Call to action – End with clear next steps
Many more advice in tips in this week’s video:
”Learning Data Science: Tell Stories with Data” by Doug Rose from Lynda.com
When to use storytelling techniques
Presentations
Explanations
Emails
Thinking about how you’re telling a story is one way to improve your communication skills
Writing: Guide your audience
Explain more! Use transitions and definitions!
What does this mean? Define your terms in their context.
“In education, data-driven decision making (DDDM) refers that teachers, education researchers and school administrators are collecting and analyzing data (achievement test data, outcome and satisfaction data) to guide them making future plans.“
”Accuracy of test scores”? ”Accuracy of sentiment analysis”?
How/why? How does one point follow from another?
Use transitions, both transition phrases + transitional explanations.
If you need info on transitions, see Writing Resources > Writing Mechanics > Transitions
“[Gap] was struggling because of the decline in sales for two years. Because of this, he hoped to eliminate the position of creative director and use big data analytics system in each of the firm’s fashion brand.”
Why should the reader care?
Storytelling caveats…
Just because it makes a good story doesn’t make it so
Compelling narratives generally only have room for one or two explanations
Examples
Origin stories and just-so stories ( “unverifiable narrative explanation for a cultural practice, a biological trait, or behavior of humans or other animals” (Wikipedia editors, 2018))
Malcolm Gladwell/Freakonomics stories
The rigor put into showing conventional wisdom* is incorrect often does not match the rigor in in the explanation they provide, e.g. 90s drop in crime/”broken windows policing”/abortion
Case studies/post-mortems
Often qualitative research with small n, e.g. Kodak in the Bartlett book
Survivorship bias
* “what the community as a whole or particular audiences find acceptable” – John Kenneth Galbraith
Wikipedia contributors. (2018, April 22). Just-so story. In Wikipedia, The Free Encyclopedia. Retrieved 18:40, August 7, 2018, from https://en.wikipedia.org/w/index.php?title=Just-so_story&oldid=837666699
Data visualization
17
Questions to guide you
Who is your audience?
What is the goal of the visualization? What information do you want it to convey?
How do I help the viewers understand that?
What is the data?
Audience
Personal/team visualizations
Examples: exploratory data analysis, visualization of analysis results
General audience visualizations
Examples: presentations, dashboards
19
Personal/team visualizations
You should be finding ways to visualize your data every step of the way, from the exploratory data analysis phase, moving on to modeling/decision making.
Audience: You or others familiar with the data and the methods.
Goals:
Sanity check – on data and analyses
Find errors
Understand analyses
Understand results
Find hypotheses, guide later analyses
Share results
20
Personal/team visualizations
Plot your data every step of the way
Want to understand it’s distribution and how it varies with co-variates (e.g. time)
21
Personal/team visualizations
Plot your data every step of the way
Want to understand it’s distribution and how it varies with co-variates (e.g. time)
Example, S&P returns data
S&P Returns data
22
Personal/team visualizations
Plot your data every step of the way
Want to understand it’s distribution and how it varies with co-variates (e.g. time)
Example, S&P returns data
Plot time series to see how it changes over time
S&P Returns data
23
Personal/team visualizations
Plot your data every step of the way
Want to understand it’s distribution and how it varies with co-variates (e.g. time)
Example, S&P returns data
Plot time series to see how it changes over time
Plot histogram to see what data looks like
S&P Returns data
24
S&P Returns data
See what the numbers look like, if the data set is too large, still worth taking a peak at the actual values of a subset
Label your axes and title your graphs. You’ll thank yourself later.
Help yourself find plots later
Save with clear labels in a folder
Put in a presentation
Don’t need to be perfect
25
From my own analyses
Graphs based on properties of a neural recording.
Personal visualization
26
From my own analyses
Graphs based on properties of a neural recording.
For more on this, watch the
“Data visualization example from my own data” video in this week’s supplemental material
Personal visualization
27
Personal/team visualizations
Look at the raw data!
Raw data numbers
Plot distributions of variables and time series first to get a sense of the data
Look at the “cleaned up data”, e.g. filtered.
Calculate metrics that are relevant to your understanding of the data and asking business (or science) questions
Industry standard or of your own design
Slice and dice the data by relevant categories / variables
If you have multiple datasets, run a standard set of plots across all of them
You (or you+team) are your audience. YOU need to be able to understand your graphs. You’re likely to have complex graphs that your eyes adapt to at this stage. Make them informative so you can parse data quickly.
If you are working with a team, standardize plots/metrics so you can more easily exchange data
28
General audience visualizations
Audience: Different team, executives, MeetUp group, publication
Think: What background knowledge do they have? How much will I need to grab their attention?
First: Define your goal, what information do you want to share with your audience? What story do you want to tell?
29
General audience visualizations Presentations + other static graphs
Video: Data Visualization: Best Practices with Amy Balliett (Lynda.com)
Infographic maker, dresses up graphics more than you’ll generally need to
Lots of good advice, though I don’t agree with everything
For my data, we wanted to put a summary of the amplitude change with burst spike number across recordings into our publication
Story – For some recordings we see big amplitude changes with burst spike number, for others we do not
From my own analyses
Graphs based on properties of a neural recording.
How to summarize?
Primary Story
For some recordings (neurons) amplitude decreases with burst number, for some recordings not
Layer 5 neurons show this decrease more consistently than Layer 2/3 neurons
Secondary story
Amplitude shows a lot of variability
From Allen et al, 2018, JNP
Made in Matlab + Illustrator
Public visualization
31
From my own analyses
Graphs based on properties of a neural recording.
How to summarize?
Primary Story
For some recordings (neurons) amplitude decreases with burst number, for some recordings not
Layer 5 neurons show this decrease more consistently than Layer 2/3 neurons
Secondary story
Amplitude shows a lot of variability
From Allen et al, 2018, JNP
Made in Matlab + Illustrator
For more on this, watch the
“Data visualization example from my own data” video in this week’s supplemental material
Public visualization
32
Data visualization caveats…
Data viz can lie/mislead just like storytelling
A graph is a story…
Examples in supplemental materials
Pie charts
Other misleading graphs
Tools for visualizing data
Language/tool you are manipulating your data in
Excel, Matlab, R, Python
Visualizations from your company’s data sources/services/software
Tableau
Google Analytics
Preparing for publication / more formal settings
Illustrator commonly used (cost $$$)
Inkscape good alternative (free)
34
Dashboards and Outcome Reports
Car Dashboard
What you need to know, immediately, right in front of you
By By Aaron Logan - from http://www.lightmatter.net/gallery/albums.php, CC BY 1.0, https://commons.wikimedia.org/w/index.php?curid=13552
Performance Dashboards
Slide modified from Uwe Hohgrawe
Visibility across functions & geographies
Various perspectives
Fast identification of issues
Tracking Performance: KPIs integrated in One View - Pharma Example: Analytics + SMR + PMR + CI + Forecasting + + +
Audit data: IMS/WK
Attitudinal data: Primary market research
Ex-factory
Forecasts/Budget
Activity by specialty
CRM system or promotional audit
CRM system
Slide from Uwe Hohgrawe
38
Example: Zignal Labs
”Social media listening”
Product tracks a company’s reputation online
Big focus on actionable data visualization
Real time data + visualization
Pricing began at $18k/year in 2015 (Rayson, 2015)
Based on volume too
https://zignallabs.com/products/zignal-command-center/
Rayson, S. (2015) SMToolbox: Real Time Insights and Media Analytics From Zignal Labs. Social Media Today. Retrieved from https://www.socialmediatoday.com/technology-data/steve-rayson/2015-09-30/smtoolbox-real-time-insights-and-media-analytics-zignal-labs
Zignal Dashboard: CCO & Executive Team
https://zignallabs.com/solution/cco-and-cxo/
Zignal Dashboard: Communications & PR
https://zignallabs.com/solution/communications-and-pr/
Zignal Dashboard: Marketing
https://zignallabs.com/solution/marketing/
Building a dashboard
Who designs it?
Requirements from stakeholder teams (e.g. product, executive)
Detailed specifications – Data analyst/scientist, product team member, outside vendor
Who builds it?
Outside vendor
IT/Engineering – Often understaffed. Build a good relationship with them.
Where’s the data from?
Data may internal and/or external
E.g. in Pharma: national in-market sales, national and/or World Pharma & Healthcare information, brand centric information that was generated via primary (customized) & secondary Market Research, other publically available information
Slide modified from Uwe Hohgrawe
Different dashboards for different audiences
Role
the ‘Standard User’ (regular hands on user that analyses the data, creates and delivers reports),
the ‘Brand User’ (Brand Team user who requires a direct link to ‘his’ market share performance data), and
the ‘Executive User’ (helicopter view for the Exec with key information that provides flexibility to analysed the data further).
Geography
Project/product
Slide modified from Uwe Hohgrawe
Outcome Reports
Monthly, quarterly, yearly report on a Project’s Outcome
Format can be a lot like a Dashboard Report, as highlights most important information highlighted and visually appealing
Information not up to the moment
Think, what data summary would my boss like to see?
Week 5 Homework Dashboard/Outcome Report exercise
(2) Dashboard or Outcome Report - Tableau
This week in class we reviewed how companies track and visuals key metrics and KPIs. For projects in which metrics can be measured in an automated fashion, these may be visualized with constantly updated Dashboards. (e.g. Netflix viewing data). For projects in which metrics cannot be calculated in an automated manner, and/or are only calculated quarterly/yearly, an Outcome Report may be produced. Based on this week's email-essay and your knowledge of the Salesforce or Netflix case, use Tableau to mock up Dashboards or an Outcome Report using simulated data.
Week 5 Homework Dashboard/Outcome Report exercise
1. Simulated data. Do not go looking for data. You are making your own!
I want you to simulate 2 KPIs for your case over some period of time. See Simulating Data section above for tutorial video and example code.
For a Dashboard, you'd probably have the numbers hourly to daily. For an Outcome Report, you'll probably simulate values on a monthly, quarterly, or yearly basis.
Make realistic simulated data. I don't need you to get the numbers precisely right, but I'd like to see them within an order of magnitude of reality and the variation in the numbers to make sense (e.g. if you think the trend of your KPI will be increasing, make sure the trend of your KPI is increasing)
Week 5 Homework Dashboard/Outcome Report exercise
2. Create a Dashboard or Outcome Report for the KPIs in Tableau
Using what you learned from the visualization material this week, create a Dashboard or Outcome Report in Tableau for the data. See Simulating Data section above for tutorial video that includes using Tableau.
The purpose of these graphs is to allow the viewer to quickly assess the outcomes (as far as KPIs) for the project. Make the graphs easy to interpret - use legends, plot in an easy to read way, etc
Make minimum 2 graphs in Tableau (min one for each KPI)
Week 5 Homework Dashboard/Outcome Report exercise
Submission
For this assignment, please take screenshots or make a PDF of your Tableau visualizations. Your submission can be in PDF or PPT(X) format. You may also include the link to your Tableau project. You will describe and explain your Outcome Report or Dashboard in part (3), which will be a separate submission. (I will grade them next to each other, though.) I don't need to see your code/Excel notebook/etc.
Week 5 Homework Dashboard/Outcome Report exercise
(3) Dashboard or Outcome Report - Describe and explain
In place of a short answer this week, I want you to describe an explain your Dashboard or Outcome Report mock-ups. Please include:
Definitions of the KPIs/metrics you included in your mock-ups
Why you included each KPI/metric
Why a metric was relevant to one or both teams
Why you choose to display the metric in the way you choose
Week 5 Presentations
If you signed up to present in Week 6:
Present your dashboards!
Can make slides or live demo Tableau
Week 5 Homework
Week 5 Homework - Read/Watch/Review
Data Visualization
Watch (req.): Data Visualization: Best Practices with Amy Balliett (Lynda.com)
Keep in mind presenter works in Infographics, which is a different audience than presenting to your co-workers/bosses
Part 4 on Illustrator is optional
Read (req.): “Save the Pies for Dessert” by Stephen Few, Perceptual Edge
Read (opt.): Crowdsourcing Graphical Perception: Using Mechanical Turk to Assess Visualization Design
Video (req.): “How to spot a misleading graph” - Lea Gaslowitz
Video (opt): Data visualization example from my own data: data_viz_may2018.mp4
Storytelling
Watch (req.): Tell Stories with Data with Doug Rose (Lynda.com)
Read (optional): "Data Mining Reveals the Six Basic Emotional Arcs of Storytelling" from MIT Technology Review
Textbook
Optional Leadership reading that was meant for last week, see Blackboard
Other materials on supplemental for reference
53
Week 5 Homework - Assignments
Email-Essay, Dashboard/Outcome Report exercise - Tableau + Explanation– Due Monday 11:59p
Topic: Discuss a point you found interesting from the either the reading or the video.
Primary post - min. 250 words and one citation - Due Saturday 11:59p
Two Secondary posts - (responses to other posts, or responses to responses to your post) – min. 50 words – Due Monday 11:59p
The 90s drop in crime explained in best-selling non-fiction books
Story 1 (mid-1990s): "[social problems] are contagious in the same way that an infectious disease is contagious" so the implementation of broken windows policing = explanation for 90s decrease in crime in NYC.
Popularized by Malcolm Gladwell's The Tipping Point, a book and a 1996 Accessibility score: Low Click to improve New Yorker article [PDF from commonlit.org].
Story 2 (2004/5): "The result [of legalized abortion in the US]... was a huge reduction in the number of children who would have been at greater than average risk of becoming criminals during the 1990's. Growing up as an unwanted child is itself a risk factor... and the women who had abortions were disproportionately likely to be unmarried teenagers with low incomes and poor education -- factors that also increase the risk." + Some other forces
Popularized by Steven D. Levitt and Stephen J. Dubner in the 2005 book Freakonomics [library link to full book], Chapter 4, Accessibility score: Low Click to improve Where Have All The Criminals Gone? [PDF of chapter from library] and extending into Chapter 5, Accessibility score: Low Click to improve What Makes a Perfect Parent? [PDF of chapter from library], based on an academic paper Levitt had published the year before.
Confession: I find Levitt and Dubner's poor reasoning style to so frustrating that I could not finish rereading these chapters. They position themselves as critical thinkers overturning conventional wisdom and yet their stories of why everyone else is wrong and they're right do not hold up to scrutiny. A critique of many of their stories can be found in "Freakonomics: What Went Wrong?" by Andrew Gelman and Kaiser Fung in American Scientist.
Current story: Unclear + Many forces were likely involved.
Vox's summary: "Why did crime plummet in the US?" (Imperfect but covers pros/cons for many theories.)
What Caused the Crime Decline? by Oliver Roeder, Lauren-Brooke "L.B." Eisen, Julia Bowling, an exhaustive report from the Brennan Center for Criminal Justice at NYU Law. Long and thorough. Summaries here and here.
Many others have also reviewed.
-60.00%
-40.00%
-20.00%
0.00%
20.00%
40.00%
60.00%
1920 1930 1940 1950 1960 1970 1980 1990 2000 2010 2020 2030% r
et ur
n
Year
S&P 500
-60.00%
-40.00%
-20.00%
0.00%
20.00%
40.00%
60.00%
192019301940195019601970198019902000201020202030
%
r
e
t
u
r
n
Year
S&P 500
-60.00%
-40.00%
-20.00%
0.00%
20.00%
40.00%
60.00%
1920 1930 1940 1950 1960 1970 1980 1990 2000 2010 2020 2030% r
et ur
n
Year
S&P 500
-60.00%
-40.00%
-20.00%
0.00%
20.00%
40.00%
60.00%
192019301940195019601970198019902000201020202030
%
r
e
t
u
r
n
Year
S&P 500
0
5
10
15
20 -0 .6
-0 .5
-0 .4
-0 .3
-0 .2
-0 .1 0 0. 1
0. 2
0. 3
0. 4
0. 5
0. 6
M or e
Fr eq
ue nc
y
Bin, % return
Histogram
Frequency
0
5
10
15
20
-
0
.
6
-
0
.
5
-
0
.
4
-
0
.
3
-
0
.
2
-
0
.
10
0
.
1
0
.
2
0
.
3
0
.
4
0
.
5
0
.
6
M
o
r
e
F
r
e
q
u
e
n
c
y
Bin, % return
Histogram
Frequency
-60.00%
-40.00%
-20.00%
0.00%
20.00%
40.00%
60.00%
1920 1930 1940 1950 1960 1970 1980 1990 2000 2010 2020 2030% r
et ur
n
Year
S&P 500
-60.00%
-40.00%
-20.00%
0.00%
20.00%
40.00%
60.00%
192019301940195019601970198019902000201020202030
%
r
e
t
u
r
n
Year
S&P 500
0
5
10
15
20 -0 .6
-0 .5
-0 .4
-0 .3
-0 .2
-0 .1 0 0. 1
0. 2
0. 3
0. 4
0. 5
0. 6
M or e
Fr eq
ue nc
y
Bin, % return
Histogram
Frequency
0
5
10
15
20
-
0
.
6
-
0
.
5
-
0
.
4
-
0
.
3
-
0
.
2
-
0
.
10
0
.
1
0
.
2
0
.
3
0
.
4
0
.
5
0
.
6
M
o
r
e
F
r
e
q
u
e
n
c
y
Bin, % return
Histogram
Frequency
1.4
-0.2
0.2
0.6
1
no rm
al iz
ed m
ea n
V E
1 2 3 4 5 6 spike number in burst
Mean extracellular spike amplitude by spike number in burst, normalized to mean non-burst spike amplitude
c
layer 2/3 layer 5