5316 U1 D1: Data Analytics Lifecycle

Tierainie

Chapter2DataAnalyticsLifecycle.pdf

Home >Human Resource Management homework help >5316 U1 D1: Data Analytics Lifecycle

c02.indd 02:16:20:PM 12/08/2014 Page 25

2 Data Analytics Lifecycle

Key Concepts

Discovery Data preparation

Model planning Model execution

Communicate results Operationalize

Data Science and Big Data Analytics : Discovering, Analyzing, Visualizing and Presenting Data, edited by Education Services EMC, and Education Services, EMC, John Wiley & Sons, Incorporated, 2015. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/capella/detail.action?docID=1908952. Created from capella on 2023-04-13 20:50:46.

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 26

26 DATA ANALYTICS LIFECYCLE

Data science projects differ from most traditional Business Intelligence projects and many data analysis

projects in that data science projects are more exploratory in nature. For this reason, it is critical to have a

process to govern them and ensure that the participants are thorough and rigorous in their approach, yet

not so rigid that the process impedes exploration.

Many problems that appear huge and daunting at first can be broken down into smaller pieces or

actionable phases that can be more easily addressed. Having a good process ensures a comprehensive and

repeatable method for conducting analysis. In addition, it helps focus time and energy early in the process

to get a clear grasp of the business problem to be solved.

A common mistake made in data science projects is rushing into data collection and analysis, which

precludes spending sufficient time to plan and scope the amount of work involved, understanding require-

ments, or even framing the business problem properly. Consequently, participants may discover mid-stream

that the project sponsors are actually trying to achieve an objective that may not match the available data,

or they are attempting to address an interest that differs from what has been explicitly communicated.

When this happens, the project may need to revert to the initial phases of the process for a proper discovery

phase, or the project may be canceled.

Creating and documenting a process helps demonstrate rigor, which provides additional credibility

to the project when the data science team shares its findings. A well-defined process also offers a com-

mon framework for others to adopt, so the methods and analysis can be repeated in the future or as new

members join a team.

2.1 Data Analytics Lifecycle Overview The Data Analytics Lifecycle is designed specifically for Big Data problems and data science projects. The

lifecycle has six phases, and project work can occur in several phases at once. For most phases in the life-

cycle, the movement can be either forward or backward. This iterative depiction of the lifecycle is intended

to more closely portray a real project, in which aspects of the project move forward and may return to

earlier stages as new information is uncovered and team members learn more about various stages of the

project. This enables participants to move iteratively through the process and drive toward operational-

izing the project work.

2.1.1 Key Roles for a Successful Analytics Project In recent years, substantial attention has been placed on the emerging role of the data scientist. In October

2012, Harvard Business Review featured an article titled “Data Scientist: The Sexiest Job of the 21st Century”

[1], in which experts DJ Patil and Tom Davenport described the new role and how to find and hire data

scientists. More and more conferences are held annually focusing on innovation in the areas of Data Science

and topics dealing with Big Data. Despite this strong focus on the emerging role of the data scientist specifi-

cally, there are actually seven key roles that need to be fulfilled for a high-functioning data science team

to execute analytic projects successfully.

Figure 2-1 depicts the various roles and key stakeholders of an analytics project. Each plays a critical part

in a successful analytics project. Although seven roles are listed, fewer or more people can accomplish the

work depending on the scope of the project, the organizational structure, and the skills of the participants.

For example, on a small, versatile team, these seven roles may be fulfilled by only 3 people, but a very large

project may require 20 or more people. The seven roles follow.

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 27

2.1 Data Analytics Lifecycle Overview 27

FIGURE 2-1 Key roles for a successful analytics project

● Business User: Someone who understands the domain area and usually benefits from the results.

This person can consult and advise the project team on the context of the project, the value of the

results, and how the outputs will be operationalized. Usually a business analyst, line manager, or

deep subject matter expert in the project domain fulfills this role.

● Project Sponsor: Responsible for the genesis of the project. Provides the impetus and requirements

for the project and defines the core business problem. Generally provides the funding and gauges

the degree of value from the final outputs of the working team. This person sets the priorities for the

project and clarifies the desired outputs.

● Project Manager: Ensures that key milestones and objectives are met on time and at the expected

quality.

● Business Intelligence Analyst: Provides business domain expertise based on a deep understanding

of the data, key performance indicators (KPIs), key metrics, and business intelligence from a reporting

perspective. Business Intelligence Analysts generally create dashboards and reports and have knowl-

edge of the data feeds and sources.

● Database Administrator (DBA): Provisions and configures the database environment to support

the analytics needs of the working team. These responsibilities may include providing access to

key databases or tables and ensuring the appropriate security levels are in place related to the data

repositories.

● Data Engineer: Leverages deep technical skills to assist with tuning SQL queries for data manage-

ment and data extraction, and provides support for data ingestion into the analytic sandbox, which

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 28

28 DATA ANALYTICS LIFECYCLE

was discussed in Chapter 1, “Introduction to Big Data Analytics.” Whereas the DBA sets up and config-

ures the databases to be used, the data engineer executes the actual data extractions and performs

substantial data manipulation to facilitate the analytics. The data engineer works closely with the

data scientist to help shape data in the right ways for analyses.

● Data Scientist: Provides subject matter expertise for analytical techniques, data modeling, and

applying valid analytical techniques to given business problems. Ensures overall analytics objectives

are met. Designs and executes analytical methods and approaches with the data available to the

project.

Although most of these roles are not new, the last two roles—data engineer and data scientist—have

become popular and in high demand [2] as interest in Big Data has grown.

2.1.2 Background and Overview of Data Analytics Lifecycle The Data Analytics Lifecycle defines analytics process best practices spanning discovery to project

completion. The lifecycle draws from established methods in the realm of data analytics and decision

science. This synthesis was developed after gathering input from data scientists and consulting estab-

lished approaches that provided input on pieces of the process. Several of the processes that were

consulted include these:

● Scientific method [3], in use for centuries, still provides a solid framework for thinking about and

deconstructing problems into their principal parts. One of the most valuable ideas of the scientific

method relates to forming hypotheses and finding ways to test ideas.

● CRISP-DM [4] provides useful input on ways to frame analytics problems and is a popular approach

for data mining.

● Tom Davenport’s DELTA framework [5]: The DELTA framework offers an approach for data analytics

projects, including the context of the organization’s skills, datasets, and leadership engagement.

● Doug Hubbard’s Applied Information Economics (AIE) approach [6]: AIE provides a framework for

measuring intangibles and provides guidance on developing decision models, calibrating expert

estimates, and deriving the expected value of information.

● “MAD Skills” by Cohen et al. [7] offers input for several of the techniques mentioned in Phases 2–4

that focus on model planning, execution, and key findings.

Figure 2-2 presents an overview of the Data Analytics Lifecycle that includes six phases. Teams commonly

learn new things in a phase that cause them to go back and refine the work done in prior phases based

on new insights and information that have been uncovered. For this reason, Figure 2-2 is shown as a cycle.

The circular arrows convey iterative movement between phases until the team members have sufficient

information to move to the next phase. The callouts include sample questions to ask to help guide whether

each of the team members has enough information and has made enough progress to move to the next

phase of the process. Note that these phases do not represent formal stage gates; rather, they serve as

criteria to help test whether it makes sense to stay in the current phase or move to the next.

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 29

2.1 Data Analytics Lifecycle Overview 29

FIGURE 2-2 Overview of Data Analytics Lifecycle

Here is a brief overview of the main phases of the Data Analytics Lifecycle:

● Phase 1—Discovery: In Phase 1, the team learns the business domain, including relevant history

such as whether the organization or business unit has attempted similar projects in the past from

which they can learn. The team assesses the resources available to support the project in terms of

people, technology, time, and data. Important activities in this phase include framing the business

problem as an analytics challenge that can be addressed in subsequent phases and formulating ini-

tial hypotheses (IHs) to test and begin learning the data.

● Phase 2—Data preparation: Phase 2 requires the presence of an analytic sandbox, in which the

team can work with data and perform analytics for the duration of the project. The team needs to

execute extract, load, and transform (ELT) or extract, transform and load (ETL) to get data into the

sandbox. The ELT and ETL are sometimes abbreviated as ETLT. Data should be transformed in the

ETLT process so the team can work with it and analyze it. In this phase, the team also needs to famil-

iarize itself with the data thoroughly and take steps to condition the data (Section 2.3.4).

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 30

30 DATA ANALYTICS LIFECYCLE

● Phase 3—Model planning: Phase 3 is model planning, where the team determines the methods,

techniques, and workflow it intends to follow for the subsequent model building phase. The team

explores the data to learn about the relationships between variables and subsequently selects key

variables and the most suitable models.

● Phase 4—Model building: In Phase 4, the team develops datasets for testing, training, and produc-

tion purposes. In addition, in this phase the team builds and executes models based on the work

done in the model planning phase. The team also considers whether its existing tools will suffice for

running the models, or if it will need a more robust environment for executing models and workflows

(for example, fast hardware and parallel processing, if applicable).

● Phase 5—Communicate results: In Phase 5, the team, in collaboration with major stakeholders,

determines if the results of the project are a success or a failure based on the criteria developed in

Phase 1. The team should identify key findings, quantify the business value, and develop a narrative

to summarize and convey findings to stakeholders.

● Phase 6—Operationalize: In Phase 6, the team delivers final reports, briefings, code, and technical

documents. In addition, the team may run a pilot project to implement the models in a production

environment.

Once team members have run models and produced findings, it is critical to frame these results in a

way that is tailored to the audience that engaged the team. Moreover, it is critical to frame the results of

the work in a manner that demonstrates clear value. If the team performs a technically accurate analysis

but fails to translate the results into a language that resonates with the audience, people will not see the

value, and much of the time and effort on the project will have been wasted.

The rest of the chapter is organized as follows. Sections 2.2–2.7 discuss in detail how each of the six

phases works, and Section 2.8 shows a case study of incorporating the Data Analytics Lifecycle in a real-

world data science project.

2.2 Phase 1: Discovery The first phase of the Data Analytics Lifecycle involves discovery (Figure 2-3). In this phase, the data science

team must learn and investigate the problem, develop context and understanding, and learn about the

data sources needed and available for the project. In addition, the team formulates initial hypotheses that

can later be tested with data.

2.2.1 Learning the Business Domain Understanding the domain area of the problem is essential. In many cases, data scientists will have deep

computational and quantitative knowledge that can be broadly applied across many disciplines. An example

of this role would be someone with an advanced degree in applied mathematics or statistics.

These data scientists have deep knowledge of the methods, techniques, and ways for applying heuris-

tics to a variety of business and conceptual problems. Others in this area may have deep knowledge of a

domain area, coupled with quantitative expertise. An example of this would be someone with a Ph.D. in

life sciences. This person would have deep knowledge of a field of study, such as oceanography, biology,

or genetics, with some depth of quantitative knowledge.

At this early stage in the process, the team needs to determine how much business or domain knowledge

the data scientist needs to develop models in Phases 3 and 4. The earlier the team can make this assessment

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 31

2.2 Phase 1: Discovery 31

the better, because the decision helps dictate the resources needed for the project team and ensures the

team has the right balance of domain knowledge and technical expertise.

FIGURE 2-3 Discovery phase

2.2.2 Resources As part of the discovery phase, the team needs to assess the resources available to support the project. In

this context, resources include technology, tools, systems, data, and people.

During this scoping, consider the available tools and technology the team will be using and the types

of systems needed for later phases to operationalize the models. In addition, try to evaluate the level of

analytical sophistication within the organization and gaps that may exist related to tools, technology, and

skills. For instance, for the model being developed to have longevity in an organization, consider what

types of skills and roles will be required that may not exist today. For the project to have long-term success,

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 32

32 DATA ANALYTICS LIFECYCLE

what types of skills and roles will be needed for the recipients of the model being developed? Does the

requisite level of expertise exist within the organization today, or will it need to be cultivated? Answering

these questions will influence the techniques the team selects and the kind of implementation the team

chooses to pursue in subsequent phases of the Data Analytics Lifecycle.

In addition to the skills and computing resources, it is advisable to take inventory of the types of data

available to the team for the project. Consider if the data available is sufficient to support the project’s

goals. The team will need to determine whether it must collect additional data, purchase it from outside

sources, or transform existing data. Often, projects are started looking only at the data available. When

the data is less than hoped for, the size and scope of the project is reduced to work within the constraints

of the existing data.

An alternative approach is to consider the long-term goals of this kind of project, without being con-

strained by the current data. The team can then consider what data is needed to reach the long-term goals

and which pieces of this multistep journey can be achieved today with the existing data. Considering

longer-term goals along with short-term goals enables teams to pursue more ambitious projects and treat

a project as the first step of a more strategic initiative, rather than as a standalone initiative. It is critical

to view projects as part of a longer-term journey, especially if executing projects in an organization that

is new to Data Science and may not have embarked on the optimum datasets to support robust analyses

up to this point.

Ensure the project team has the right mix of domain experts, customers, analytic talent, and project

management to be effective. In addition, evaluate how much time is needed and if the team has the right

breadth and depth of skills.

After taking inventory of the tools, technology, data, and people, consider if the team has sufficient

resources to succeed on this project, or if additional resources are needed. Negotiating for resources at the

outset of the project, while scoping the goals, objectives, and feasibility, is generally more useful than later

in the process and ensures sufficient time to execute it properly. Project managers and key stakeholders have

better success negotiating for the right resources at this stage rather than later once the project is underway.

2.2.3 Framing the Problem Framing the problem well is critical to the success of the project. Framing is the process of stating the

analytics problem to be solved. At this point, it is a best practice to write down the problem statement

and share it with the key stakeholders. Each team member may hear slightly different things related to

the needs and the problem and have somewhat different ideas of possible solutions. For these reasons, it

is crucial to state the analytics problem, as well as why and to whom it is important. Essentially, the team

needs to clearly articulate the current situation and its main challenges.

As part of this activity, it is important to identify the main objectives of the project, identify what needs

to be achieved in business terms, and identify what needs to be done to meet the needs. Additionally,

consider the objectives and the success criteria for the project. What is the team attempting to achieve by

doing the project, and what will be considered “good enough” as an outcome of the project? This is critical

to document and share with the project team and key stakeholders. It is best practice to share the statement

of goals and success criteria with the team and confirm alignment with the project sponsor’s expectations.

Perhaps equally important is to establish failure criteria. Most people doing projects prefer only to think

of the success criteria and what the conditions will look like when the participants are successful. However,

this is almost taking a best-case scenario approach, assuming that everything will proceed as planned

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 33

2.2 Phase 1: Discovery 33

and the project team will reach its goals. However, no matter how well planned, it is almost impossible to

plan for everything that will emerge in a project. The failure criteria will guide the team in understanding

when it is best to stop trying or settle for the results that have been gleaned from the data. Many times

people will continue to perform analyses past the point when any meaningful insights can be drawn from

the data. Establishing criteria for both success and failure helps the participants avoid unproductive effort

and remain aligned with the project sponsors

2.2.4 Identifying Key Stakeholders Another important step is to identify the key stakeholders and their interests in the project. During

these discussions, the team can identify the success criteria, key risks, and stakeholders, which should

include anyone who will benefit from the project or will be significantly impacted by the project. When

interviewing stakeholders, learn about the domain area and any relevant history from similar analytics

projects. For example, the team may identify the results each stakeholder wants from the project and the

criteria it will use to judge the success of the project.

Keep in mind that the analytics project is being initiated for a reason. It is critical to articulate the pain

points as clearly as possible to address them and be aware of areas to pursue or avoid as the team gets

further into the analytical process. Depending on the number of stakeholders and participants, the team

may consider outlining the type of activity and participation expected from each stakeholder and partici-

pant. This will set clear expectations with the participants and avoid delays later when, for example, the

team may feel it needs to wait for approval from someone who views himself as an adviser rather than an

approver of the work product.

2.2.5 Interviewing the Analytics Sponsor The team should plan to collaborate with the stakeholders to clarify and frame the analytics problem. At the

outset, project sponsors may have a predetermined solution that may not necessarily realize the desired

outcome. In these cases, the team must use its knowledge and expertise to identify the true underlying

problem and appropriate solution.

For instance, suppose in the early phase of a project, the team is told to create a recommender system

for the business and that the way to do this is by speaking with three people and integrating the product

recommender into a legacy corporate system. Although this may be a valid approach, it is important to test

the assumptions and develop a clear understanding of the problem. The data science team typically may

have a more objective understanding of the problem set than the stakeholders, who may be suggesting

solutions to a given problem. Therefore, the team can probe deeper into the context and domain to clearly

define the problem and propose possible paths from the problem to a desired outcome. In essence, the

data science team can take a more objective approach, as the stakeholders may have developed biases

over time, based on their experience. Also, what may have been true in the past may no longer be a valid

working assumption. One possible way to circumvent this issue is for the project sponsor to focus on clearly

defining the requirements, while the other members of the data science team focus on the methods needed

to achieve the goals.

When interviewing the main stakeholders, the team needs to take time to thoroughly interview the

project sponsor, who tends to be the one funding the project or providing the high-level requirements.

This person understands the problem and usually has an idea of a potential working solution. It is critical

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 34

34 DATA ANALYTICS LIFECYCLE

to thoroughly understand the sponsor’s perspective to guide the team in getting started on the project.

Here are some tips for interviewing project sponsors:

● Prepare for the interview; draft questions, and review with colleagues.

● Use open-ended questions; avoid asking leading questions.

● Probe for details and pose follow-up questions.

● Avoid filling every silence in the conversation; give the other person time to think.

● Let the sponsors express their ideas and ask clarifying questions, such as “Why? Is that correct? Is this

idea on target? Is there anything else?”

● Use active listening techniques; repeat back what was heard to make sure the team heard it correctly,

or reframe what was said.

● Try to avoid expressing the team’s opinions, which can introduce bias; instead, focus on listening.

● Be mindful of the body language of the interviewers and stakeholders; use eye contact where appro-

priate, and be attentive.

● Minimize distractions.

● Document what the team heard, and review it with the sponsors.

Following is a brief list of common questions that are helpful to ask during the discovery phase when

interviewing the project sponsor. The responses will begin to shape the scope of the project and give the

team an idea of the goals and objectives of the project.

● What business problem is the team trying to solve?

● What is the desired outcome of the project?

● What data sources are available?

● What industry issues may impact the analysis?

● What timelines need to be considered?

● Who could provide insight into the project?

● Who has final decision-making authority on the project?

● How will the focus and scope of the problem change if the following dimensions change:

● Time: Analyzing 1 year or 10 years’ worth of data?

● People: Assess impact of changes in resources on project timeline.

● Risk: Conservative to aggressive

● Resources: None to unlimited (tools, technology, systems)

● Size and attributes of data: Including internal and external data sources

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 35

2.2 Phase 1: Discovery 35

2.2.6 Developing Initial Hypotheses Developing a set of IHs is a key facet of the discovery phase. This step involves forming ideas that the team

can test with data. Generally, it is best to come up with a few primary hypotheses to test and then be

creative about developing several more. These IHs form the basis of the analytical tests the team will use

in later phases and serve as the foundation for the findings in Phase 5. Hypothesis testing from a statisti-

cal perspective is covered in greater detail in Chapter 3, “Review of Basic Data Analytic Methods Using R.”

In this way, the team can compare its answers with the outcome of an experiment or test to generate

additional possible solutions to problems. As a result, the team will have a much richer set of observations

to choose from and more choices for agreeing upon the most impactful conclusions from a project.

Another part of this process involves gathering and assessing hypotheses from stakeholders and domain

experts who may have their own perspective on what the problem is, what the solution should be, and how

to arrive at a solution. These stakeholders would know the domain area well and can offer suggestions on

ideas to test as the team formulates hypotheses during this phase. The team will likely collect many ideas

that may illuminate the operating assumptions of the stakeholders. These ideas will also give the team

opportunities to expand the project scope into adjacent spaces where it makes sense or design experiments

in a meaningful way to address the most important interests of the stakeholders. As part of this exercise,

it can be useful to obtain and explore some initial data to inform discussions with stakeholders during the

hypothesis-forming stage.

2.2.7 Identifying Potential Data Sources As part of the discovery phase, identify the kinds of data the team will need to solve the problem. Consider

the volume, type, and time span of the data needed to test the hypotheses. Ensure that the team can access

more than simply aggregated data. In most cases, the team will need the raw data to avoid introducing

bias for the downstream analysis. Recalling the characteristics of Big Data from Chapter 1, assess the main

characteristics of the data, with regard to its volume, variety, and velocity of change. A thorough diagno-

sis of the data situation will influence the kinds of tools and techniques to use in Phases 2-4 of the Data

Analytics Lifecycle. In addition, performing data exploration in this phase will help the team determine

the amount of data needed, such as the amount of historical data to pull from existing systems and the

data structure. Develop an idea of the scope of the data needed, and validate that idea with the domain

experts on the project.

The team should perform five main activities during this step of the discovery phase:

● Identify data sources: Make a list of candidate data sources the team may need to test the initial

hypotheses outlined in this phase. Make an inventory of the datasets currently available and those

that can be purchased or otherwise acquired for the tests the team wants to perform.

● Capture aggregate data sources: This is for previewing the data and providing high-level under-

standing. It enables the team to gain a quick overview of the data and perform further exploration on

specific areas. It also points the team to possible areas of interest within the data.

● Review the raw data: Obtain preliminary data from initial data feeds. Begin understanding the

interdependencies among the data attributes, and become familiar with the content of the data, its

quality, and its limitations.

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 36

36 DATA ANALYTICS LIFECYCLE

● Evaluate the data structures and tools needed: The data type and structure dictate which tools the

team can use to analyze the data. This evaluation gets the team thinking about which technologies

may be good candidates for the project and how to start getting access to these tools.

● Scope the sort of data infrastructure needed for this type of problem: In addition to the tools

needed, the data influences the kind of infrastructure that’s required, such as disk storage and net-

work capacity.

Unlike many traditional stage-gate processes, in which the team can advance only when specific criteria

are met, the Data Analytics Lifecycle is intended to accommodate more ambiguity. This more closely reflects

how data science projects work in real-life situations. For each phase of the process, it is recommended

to pass certain checkpoints as a way of gauging whether the team is ready to move to the next phase of

the Data Analytics Lifecycle.

The team can move to the next phase when it has enough information to draft an analytics plan and

share it for peer review. Although a peer review of the plan may not actually be required by the project,

creating the plan is a good test of the team’s grasp of the business problem and the team’s approach

to addressing it. Creating the analytic plan also requires a clear understanding of the domain area, the

problem to be solved, and scoping of the data sources to be used. Developing success criteria early in the

project clarifies the problem definition and helps the team when it comes time to make choices about

the analytical methods being used in later phases.

2.3 Phase 2: Data Preparation The second phase of the Data Analytics Lifecycle involves data preparation, which includes the steps to

explore, preprocess, and condition data prior to modeling and analysis. In this phase, the team needs to

create a robust environment in which it can explore the data that is separate from a production environment.

Usually, this is done by preparing an analytics sandbox. To get the data into the sandbox, the team needs to

perform ETLT, by a combination of extracting, transforming, and loading data into the sandbox. Once the

data is in the sandbox, the team needs to learn about the data and become familiar with it. Understanding

the data in detail is critical to the success of the project. The team also must decide how to condition and

transform data to get it into a format to facilitate subsequent analysis. The team may perform data visualiza-

tions to help team members understand the data, including its trends, outliers, and relationships among

data variables. Each of these steps of the data preparation phase is discussed throughout this section.

Data preparation tends to be the most labor-intensive step in the analytics lifecycle. In fact, it is common

for teams to spend at least 50% of a data science project’s time in this critical phase. If the team cannot obtain

enough data of sufficient quality, it may be unable to perform the subsequent steps in the lifecycle process.

Figure 2-4 shows an overview of the Data Analytics Lifecycle for Phase 2. The data preparation phase is

generally the most iterative and the one that teams tend to underestimate most often. This is because most

teams and leaders are anxious to begin analyzing the data, testing hypotheses, and getting answers to some

of the questions posed in Phase 1. Many tend to jump into Phase 3 or Phase 4 to begin rapidly developing

models and algorithms without spending the time to prepare the data for modeling. Consequently, teams

come to realize the data they are working with does not allow them to execute the models they want, and

they end up back in Phase 2 anyway.

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 37

2.3 Phase 2: Data Preparation 37

FIGURE 2-4 Data preparation phase

2.3.1 Preparing the Analytic Sandbox The first subphase of data preparation requires the team to obtain an analytic sandbox (also commonly

referred to as a workspace), in which the team can explore the data without interfering with live produc-

tion databases. Consider an example in which the team needs to work with a company’s financial data.

The team should access a copy of the financial data from the analytic sandbox rather than interacting with

the production version of the organization’s main database, because that will be tightly controlled and

needed for financial reporting.

When developing the analytic sandbox, it is a best practice to collect all kinds of data there, as team

members need access to high volumes and varieties of data for a Big Data analytics project. This can include

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 38

38 DATA ANALYTICS LIFECYCLE

everything from summary-level aggregated data, structured data, raw data feeds, and unstructured text

data from call logs or web logs, depending on the kind of analysis the team plans to undertake.

This expansive approach for attracting data of all kind differs considerably from the approach advocated

by many information technology (IT) organizations. Many IT groups provide access to only a particular sub-

segment of the data for a specific purpose. Often, the mindset of the IT group is to provide the minimum

amount of data required to allow the team to achieve its objectives. Conversely, the data science team

wants access to everything. From its perspective, more data is better, as oftentimes data science projects

are a mixture of purpose-driven analyses and experimental approaches to test a variety of ideas. In this

context, it can be challenging for a data science team if it has to request access to each and every dataset

and attribute one at a time. Because of these differing views on data access and use, it is critical for the data

science team to collaborate with IT, make clear what it is trying to accomplish, and align goals.

During these discussions, the data science team needs to give IT a justification to develop an analyt-

ics sandbox, which is separate from the traditional IT-governed data warehouses within an organization.

Successfully and amicably balancing the needs of both the data science team and IT requires a positive

working relationship between multiple groups and data owners. The payoff is great. The analytic sandbox

enables organizations to undertake more ambitious data science projects and move beyond doing tradi-

tional data analysis and Business Intelligence to perform more robust and advanced predictive analytics.

Expect the sandbox to be large. It may contain raw data, aggregated data, and other data types that are

less commonly used in organizations. Sandbox size can vary greatly depending on the project. A good rule

is to plan for the sandbox to be at least 5–10 times the size of the original datasets, partly because copies of

the data may be created that serve as specific tables or data stores for specific kinds of analysis in the project.

Although the concept of an analytics sandbox is relatively new, companies are making progress in this

area and are finding ways to offer sandboxes and workspaces where teams can access datasets and work

in a way that is acceptable to both the data science teams and the IT groups.

2.3.2 Performing ETLT As the team looks to begin data transformations, make sure the analytics sandbox has ample bandwidth

and reliable network connections to the underlying data sources to enable uninterrupted read and write.

In ETL, users perform extract, transform, load processes to extract data from a datastore, perform data

transformations, and load the data back into the datastore. However, the analytic sandbox approach differs

slightly; it advocates extract, load, and then transform. In this case, the data is extracted in its raw form and

loaded into the datastore, where analysts can choose to transform the data into a new state or leave it in

its original, raw condition. The reason for this approach is that there is significant value in preserving the

raw data and including it in the sandbox before any transformations take place.

For instance, consider an analysis for fraud detection on credit card usage. Many times, outliers in this

data population can represent higher-risk transactions that may be indicative of fraudulent credit card

activity. Using ETL, these outliers may be inadvertently filtered out or transformed and cleaned before being

loaded into the datastore. In this case, the very data that would be needed to evaluate instances of fraudu-

lent activity would be inadvertently cleansed, preventing the kind of analysis that a team would want to do.

Following the ELT approach gives the team access to clean data to analyze after the data has been loaded

into the database and gives access to the data in its original form for finding hidden nuances in the data.

This approach is part of the reason that the analytic sandbox can quickly grow large. The team may want

clean data and aggregated data and may need to keep a copy of the original data to compare against or

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 39

2.3 Phase 2: Data Preparation 39

look for hidden patterns that may have existed in the data before the cleaning stage. This process can be

summarized as ETLT to reflect the fact that a team may choose to perform ETL in one case and ELT in another.

Depending on the size and number of the data sources, the team may need to consider how to paral-

lelize the movement of the datasets into the sandbox. For this purpose, moving large amounts of data is

sometimes referred to as Big ETL. The data movement can be parallelized by technologies such as Hadoop

or MapReduce, which will be explained in greater detail in Chapter 10, “Advanced Analytics—Technology

and Tools: MapReduce and Hadoop.” At this point, keep in mind that these technologies can be used to

perform parallel data ingest and introduce a huge number of files or datasets in parallel in a very short

period of time. Hadoop can be useful for data loading as well as for data analysis in subsequent phases.

Prior to moving the data into the analytic sandbox, determine the transformations that need to be

performed on the data. Part of this phase involves assessing data quality and structuring the datasets

properly so they can be used for robust analysis in subsequent phases. In addition, it is important to con-

sider which data the team will have access to and which new data attributes will need to be derived in the

data to enable analysis.

As part of the ETLT step, it is advisable to make an inventory of the data and compare the data currently

available with datasets the team needs. Performing this sort of gap analysis provides a framework for

understanding which datasets the team can take advantage of today and where the team needs to initiate

projects for data collection or access to new datasets currently unavailable. A component of this subphase

involves extracting data from the available sources and determining data connections for raw data, online

transaction processing (OLTP) databases, online analytical processing (OLAP) cubes, or other data feeds.

Application programming interface (API) is an increasingly popular way to access a data source [8]. Many

websites and social network applications now provide APIs that offer access to data to support a project

or supplement the datasets with which a team is working. For example, connecting to the Twitter API can

enable a team to download millions of tweets to perform a project for sentiment analysis on a product, a

company, or an idea. Much of the Twitter data is publicly available and can augment other datasets used

on the project.

2.3.3 Learning About the Data A critical aspect of a data science project is to become familiar with the data itself. Spending time to learn the

nuances of the datasets provides context to understand what constitutes a reasonable value and expected

output versus what is a surprising finding. In addition, it is important to catalog the data sources that the

team has access to and identify additional data sources that the team can leverage but perhaps does not

have access to today. Some of the activities in this step may overlap with the initial investigation of the

datasets that occur in the discovery phase. Doing this activity accomplishes several goals.

● Clarifies the data that the data science team has access to at the start of the project

● Highlights gaps by identifying datasets within an organization that the team may find useful but may

not be accessible to the team today. As a consequence, this activity can trigger a project to begin

building relationships with the data owners and finding ways to share data in appropriate ways. In

addition, this activity may provide an impetus to begin collecting new data that benefits the organi-

zation or a specific long-term project.

● Identifies datasets outside the organization that may be useful to obtain, through open APIs, data

sharing, or purchasing data to supplement already existing datasets

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 40

40 DATA ANALYTICS LIFECYCLE

Table 2-1 demonstrates one way to organize this type of data inventory.

TABLE 2-1 Sample Dataset Inventory

Dataset

Data

Available

and

Accessible

Data Available, but

not Accessible

Data to

Collect

Data to Obtain

from Third

Party Sources

Products shipped ●

Product Financials ●

Product Call Center

Data

●

Live Product

Feedback Surveys

●

Product Sentiment

from Social Media

●

2.3.4 Data Conditioning Data conditioning refers to the process of cleaning data, normalizing datasets, and performing trans-

formations on the data. A critical step within the Data Analytics Lifecycle, data conditioning can involve

many complex steps to join or merge datasets or otherwise get datasets into a state that enables analysis

in further phases. Data conditioning is often viewed as a preprocessing step for the data analysis because

it involves many operations on the dataset before developing models to process or analyze the data. This

implies that the data-conditioning step is performed only by IT, the data owners, a DBA, or a data engineer.

However, it is also important to involve the data scientist in this step because many decisions are made in

the data conditioning phase that affect subsequent analysis. Part of this phase involves deciding which

aspects of particular datasets will be useful to analyze in later steps. Because teams begin forming ideas

in this phase about which data to keep and which data to transform or discard, it is important to involve

multiple team members in these decisions. Leaving such decisions to a single person may cause teams to

return to this phase to retrieve data that may have been discarded.

As with the previous example of deciding which data to keep as it relates to fraud detection on credit

card usage, it is critical to be thoughtful about which data the team chooses to keep and which data will

be discarded. This can have far-reaching consequences that will cause the team to retrace previous steps

if the team discards too much of the data at too early a point in this process. Typically, data science teams

would rather keep more data than too little data for the analysis. Additional questions and considerations

for the data conditioning step include these.

● What are the data sources? What are the target fields (for example, columns of the tables)?

● How clean is the data?

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 41

2.3 Phase 2: Data Preparation 41

● How consistent are the contents and files? Determine to what degree the data contains missing or

inconsistent values and if the data contains values deviating from normal.

● Assess the consistency of the data types. For instance, if the team expects certain data to be numeric,

confirm it is numeric or if it is a mixture of alphanumeric strings and text.

● Review the content of data columns or other inputs, and check to ensure they make sense. For

instance, if the project involves analyzing income levels, preview the data to confirm that the income

values are positive or if it is acceptable to have zeros or negative values.

● Look for any evidence of systematic error. Examples include data feeds from sensors or other data

sources breaking without anyone noticing, which causes invalid, incorrect, or missing data values. In

addition, review the data to gauge if the definition of the data is the same over all measurements. In

some cases, a data column is repurposed, or the column stops being populated, without this change

being annotated or without others being notified.

2.3.5 Survey and Visualize After the team has collected and obtained at least some of the datasets needed for the subsequent

analysis, a useful step is to leverage data visualization tools to gain an overview of the data. Seeing high-level

patterns in the data enables one to understand characteristics about the data very quickly. One example

is using data visualization to examine data quality, such as whether the data contains many unexpected

values or other indicators of dirty data. (Dirty data will be discussed further in Chapter 3.) Another example

is skewness, such as if the majority of the data is heavily shifted toward one value or end of a continuum.

Shneiderman [9] is well known for his mantra for visual data analysis of “overview first, zoom and filter,

then details-on-demand.” This is a pragmatic approach to visual data analysis. It enables the user to find

areas of interest, zoom and filter to find more detailed information about a particular area of the data, and

then find the detailed data behind a particular area. This approach provides a high-level view of the data

and a great deal of information about a given dataset in a relatively short period of time.

When pursuing this approach with a data visualization tool or statistical package, the following guide-

lines and considerations are recommended.

● Review data to ensure that calculations remained consistent within columns or across tables for a

given data field. For instance, did customer lifetime value change at some point in the middle of data

collection? Or if working with financials, did the interest calculation change from simple to com-

pound at the end of the year?

● Does the data distribution stay consistent over all the data? If not, what kinds of actions should be

taken to address this problem?

● Assess the granularity of the data, the range of values, and the level of aggregation of the data.

● Does the data represent the population of interest? For marketing data, if the project is focused on

targeting customers of child-rearing age, does the data represent that, or is it full of senior citizens

and teenagers?

● For time-related variables, are the measurements daily, weekly, monthly? Is that good enough? Is

time measured in seconds everywhere? Or is it in milliseconds in some places? Determine the level of

granularity of the data needed for the analysis, and assess whether the current level of timestamps

on the data meets that need.

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 42

42 DATA ANALYTICS LIFECYCLE

● Is the data standardized/normalized? Are the scales consistent? If not, how consistent or irregular is

the data?

● For geospatial datasets, are state or country abbreviations consistent across the data? Are personal

names normalized? English units? Metric units?

These are typical considerations that should be part of the thought process as the team evaluates the

datasets that are obtained for the project. Becoming deeply knowledgeable about the data will be critical

when it comes time to construct and run models later in the process.

2.3.6 Common Tools for the Data Preparation Phase Several tools are commonly used for this phase:

● Hadoop [10] can perform massively parallel ingest and custom analysis for web traffic parsing, GPS

location analytics, genomic analysis, and combining of massive unstructured data feeds from mul-

tiple sources.

● Alpine Miner [11] provides a graphical user interface (GUI) for creating analytic workflows, including

data manipulations and a series of analytic events such as staged data-mining techniques (for exam-

ple, first select the top 100 customers, and then run descriptive statistics and clustering) on Postgres

SQL and other Big Data sources.

● OpenRefine (formerly called Google Refine) [12] is “a free, open source, powerful tool for working

with messy data.” It is a popular GUI-based tool for performing data transformations, and it’s one of

the most robust free tools currently available.

● Similar to OpenRefine, Data Wrangler [13] is an interactive tool for data cleaning and transformation.

Wrangler was developed at Stanford University and can be used to perform many transformations on

a given dataset. In addition, data transformation outputs can be put into Java or Python. The advan-

tage of this feature is that a subset of the data can be manipulated in Wrangler via its GUI, and then

the same operations can be written out as Java or Python code to be executed against the full, larger

dataset offline in a local analytic sandbox.

For Phase 2, the team needs assistance from IT, DBAs, or whoever controls the Enterprise Data Warehouse

(EDW) for data sources the data science team would like to use.

2.4 Phase 3: Model Planning In Phase 3, the data science team identifies candidate models to apply to the data for clustering,

classifying, or finding relationships in the data depending on the goal of the project, as shown in Figure 2-5.

It is during this phase that the team refers to the hypotheses developed in Phase 1, when they first became

acquainted with the data and understanding the business problems or domain area. These hypotheses help

the team frame the analytics to execute in Phase 4 and select the right methods to achieve its objectives.

Some of the activities to consider in this phase include the following:

● Assess the structure of the datasets. The structure of the datasets is one factor that dictates the tools

and analytical techniques for the next phase. Depending on whether the team plans to analyze tex-

tual data or transactional data, for example, different tools and approaches are required.

● Ensure that the analytical techniques enable the team to meet the business objectives and accept or

reject the working hypotheses.

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 43

2.4 Phase 3: Model Planning 43

● Determine if the situation warrants a single model or a series of techniques as part of a larger analytic

workflow. A few example models include association rules (Chapter 5, “Advanced Analytical Theory

and Methods: Association Rules”) and logistic regression (Chapter 6, “Advanced Analytical Theory

and Methods: Regression”). Other tools, such as Alpine Miner, enable users to set up a series of steps

and analyses and can serve as a front-end user interface (UI) for manipulating Big Data sources in

PostgreSQL.

FIGURE 2-5 Model planning phase

In addition to the considerations just listed, it is useful to research and understand how other analysts

generally approach a specific kind of problem. Given the kind of data and resources that are available, evalu-

ate whether similar, existing approaches will work or if the team will need to create something new. Many

times teams can get ideas from analogous problems that other people have solved in different industry

verticals or domain areas. Table 2-2 summarizes the results of an exercise of this type, involving several

domain areas and the types of models previously used in a classification type of problem after conducting

research on churn models in multiple industry verticals. Performing this sort of diligence gives the team

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 44

44 DATA ANALYTICS LIFECYCLE

ideas of how others have solved similar problems and presents the team with a list of candidate models to

try as part of the model planning phase.

TABLE 2-2 Research on Model Planning in Industry Verticals

Market Sector Analytic Techniques/Methods Used

Consumer Packaged

Goods

Multiple linear regression, automatic relevance determination (ARD), and

decision tree

Retail Banking Multiple regression

Retail Business Logistic regression, ARD, decision tree

Wireless Telecom Neural network, decision tree, hierarchical neurofuzzy systems, rule

evolver, logistic regression

2.4.1 Data Exploration and Variable Selection Although some data exploration takes place in the data preparation phase, those activities focus mainly on

data hygiene and on assessing the quality of the data itself. In Phase 3, the objective of the data exploration

is to understand the relationships among the variables to inform selection of the variables and methods

and to understand the problem domain. As with earlier phases of the Data Analytics Lifecycle, it is impor-

tant to spend time and focus attention on this preparatory work to make the subsequent phases of model

selection and execution easier and more efficient. A common way to conduct this step involves using tools

to perform data visualizations. Approaching the data exploration in this way aids the team in previewing

the data and assessing relationships between variables at a high level.

In many cases, stakeholders and subject matter experts have instincts and hunches about what the data

science team should be considering and analyzing. Likely, this group had some hypothesis that led to the

genesis of the project. Often, stakeholders have a good grasp of the problem and domain, although they

may not be aware of the subtleties within the data or the model needed to accept or reject a hypothesis.

Other times, stakeholders may be correct, but for the wrong reasons (for instance, they may be correct

about a correlation that exists but infer an incorrect reason for the correlation). Meanwhile, data scientists

have to approach problems with an unbiased mind-set and be ready to question all assumptions.

As the team begins to question the incoming assumptions and test initial ideas of the project sponsors

and stakeholders, it needs to consider the inputs and data that will be needed, and then it must examine

whether these inputs are actually correlated with the outcomes that the team plans to predict or analyze.

Some methods and types of models will handle correlated variables better than others. Depending on what

the team is attempting to solve, it may need to consider an alternate method, reduce the number of data

inputs, or transform the inputs to allow the team to use the best method for a given business problem.

Some of these techniques will be explored further in Chapter 3 and Chapter 6.

The key to this approach is to aim for capturing the most essential predictors and variables rather than

considering every possible variable that people think may influence the outcome. Approaching the prob-

lem in this manner requires iterations and testing to identify the most essential variables for the intended

analyses. The team should plan to test a range of variables to include in the model and then focus on the

most important and influential variables.

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 45

2.4 Phase 3: Model Planning 45

If the team plans to run regression analyses, identify the candidate predictors and outcome variables

of the model. Plan to create variables that determine outcomes but demonstrate a strong relationship to

the outcome rather than to the other input variables. This includes remaining vigilant for problems such

as serial correlation, multicollinearity, and other typical data modeling challenges that interfere with the

validity of these models. Sometimes these issues can be avoided simply by looking at ways to reframe a

given problem. In addition, sometimes determining correlation is all that is needed (“black box prediction”),

and in other cases, the objective of the project is to understand the causal relationship better. In the latter

case, the team wants the model to have explanatory power and needs to forecast or stress test the model

under a variety of situations and with different datasets.

2.4.2 Model Selection In the model selection subphase, the team’s main goal is to choose an analytical technique, or a short list

of candidate techniques, based on the end goal of the project. For the context of this book, a model is

discussed in general terms. In this case, a model simply refers to an abstraction from reality. One observes

events happening in a real-world situation or with live data and attempts to construct models that emulate

this behavior with a set of rules and conditions. In the case of machine learning and data mining, these

rules and conditions are grouped into several general sets of techniques, such as classification, association

rules, and clustering. When reviewing this list of types of potential models, the team can winnow down the

list to several viable models to try to address a given problem. More details on matching the right models

to common types of business problems are provided in Chapter 3 and Chapter 4, “Advanced Analytical

Theory and Methods: Clustering.”

An additional consideration in this area for dealing with Big Data involves determining if the team will

be using techniques that are best suited for structured data, unstructured data, or a hybrid approach. For

instance, the team can leverage MapReduce to analyze unstructured data, as highlighted in Chapter 10.

Lastly, the team should take care to identify and document the modeling assumptions it is making as it

chooses and constructs preliminary models.

Typically, teams create the initial models using a statistical software package such as R, SAS, or Matlab.

Although these tools are designed for data mining and machine learning algorithms, they may have limi-

tations when applying the models to very large datasets, as is common with Big Data. As such, the team

may consider redesigning these algorithms to run in the database itself during the pilot phase mentioned

in Phase 6.

The team can move to the model building phase once it has a good idea about the type of model to try

and the team has gained enough knowledge to refine the analytics plan. Advancing from this phase requires

a general methodology for the analytical model, a solid understanding of the variables and techniques to

use, and a description or diagram of the analytic workflow.

2.4.3 Common Tools for the Model Planning Phase Many tools are available to assist in this phase. Here are several of the more common ones:

● R [14] has a complete set of modeling capabilities and provides a good environment for building

interpretive models with high-quality code. In addition, it has the ability to interface with databases

via an ODBC connection and execute statistical tests and analyses against Big Data via an open

source connection. These two factors make R well suited to performing statistical tests and analyt-

ics on Big Data. As of this writing, R contains nearly 5,000 packages for data analysis and graphical

representation. New packages are posted frequently, and many companies are providing value-add

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 46

46 DATA ANALYTICS LIFECYCLE

services for R (such as training, instruction, and best practices), as well as packaging it in ways to make

it easier to use and more robust. This phenomenon is similar to what happened with Linux in the late

1980s and early 1990s, when companies appeared to package and make Linux easier for companies

to consume and deploy. Use R with file extracts for offline analysis and optimal performance, and use

RODBC connections for dynamic queries and faster development.

● SQL Analysis services [15] can perform in-database analytics of common data mining functions,

involved aggregations, and basic predictive models.

● SAS/ACCESS [16] provides integration between SAS and the analytics sandbox via multiple data

connectors such as OBDC, JDBC, and OLE DB. SAS itself is generally used on file extracts, but with

SAS/ACCESS, users can connect to relational databases (such as Oracle or Teradata) and data ware-

house appliances (such as Greenplum or Aster), files, and enterprise applications (such as SAP and

Salesforce.com).

2.5 Phase 4: Model Building In Phase 4, the data science team needs to develop datasets for training, testing, and production purposes.

These datasets enable the data scientist to develop the analytical model and train it (“training data”), while

holding aside some of the data (“hold-out data” or “test data”) for testing the model. (These topics are

addressed in more detail in Chapter 3.) During this process, it is critical to ensure that the training and test

datasets are sufficiently robust for the model and analytical techniques. A simple way to think of these

datasets is to view the training dataset for conducting the initial experiments and the test sets for validating

an approach once the initial experiments and models have been run.

In the model building phase, shown in Figure 2-6, an analytical model is developed and fit on the train-

ing data and evaluated (scored) against the test data. The phases of model planning and model building

can overlap quite a bit, and in practice one can iterate back and forth between the two phases for a while

before settling on a final model.

Although the modeling techniques and logic required to develop models can be highly complex, the

actual duration of this phase can be short compared to the time spent preparing the data and defining the

approaches. In general, plan to spend more time preparing and learning the data (Phases 1–2) and crafting

a presentation of the findings (Phase 5). Phases 3 and 4 tend to move more quickly, although they are more

complex from a conceptual standpoint.

As part of this phase, the data science team needs to execute the models defined in Phase 3.

During this phase, users run models from analytical software packages, such as R or SAS, on file extracts

and small datasets for testing purposes. On a small scale, assess the validity of the model and its results.

For instance, determine if the model accounts for most of the data and has robust predictive power. At this

point, refine the models to optimize the results, such as by modifying variable inputs or reducing correlated

variables where appropriate. In Phase 3, the team may have had some knowledge of correlated variables

or problematic data attributes, which will be confirmed or denied once the models are actually executed.

When immersed in the details of constructing models and transforming data, many small decisions are often

made about the data and the approach for the modeling. These details can be easily forgotten once the

project is completed. Therefore, it is vital to record the results and logic of the model during this phase. In

addition, one must take care to record any operating assumptions that were made in the modeling process

regarding the data or the context.

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 47

2.5 Phase 4: Model Building 47

FIGURE 2-6 Model building phase

Creating robust models that are suitable to a specific situation requires thoughtful consideration to

ensure the models being developed ultimately meet the objectives outlined in Phase 1. Questions to con-

sider include these:

● Does the model appear valid and accurate on the test data?

● Does the model output/behavior make sense to the domain experts? That is, does it appear as if the

model is giving answers that make sense in this context?

● Do the parameter values of the fitted model make sense in the context of the domain?

● Is the model sufficiently accurate to meet the goal?

● Does the model avoid intolerable mistakes? Depending on context, false positives may be more seri-

ous or less serious than false negatives, for instance. (False positives and false negatives are discussed

further in Chapter 3 and Chapter 7, “Advanced Analytical Theory and Methods: Classification.”)

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 48

48 DATA ANALYTICS LIFECYCLE

● Are more data or more inputs needed? Do any of the inputs need to be transformed or eliminated?

● Will the kind of model chosen support the runtime requirements?

● Is a different form of the model required to address the business problem? If so, go back to the model

planning phase and revise the modeling approach.

Once the data science team can evaluate either if the model is sufficiently robust to solve the problem

or if the team has failed, it can move to the next phase in the Data Analytics Lifecycle.

2.5.1 Common Tools for the Model Building Phase There are many tools available to assist in this phase, focused primarily on statistical analysis or data mining

software. Common tools in this space include, but are not limited to, the following:

● Commercial Tools:

● SAS Enterprise Miner [17] allows users to run predictive and descriptive models based on large

volumes of data from across the enterprise. It interoperates with other large data stores, has

many partnerships, and is built for enterprise-level computing and analytics.

● SPSS Modeler [18] (provided by IBM and now called IBM SPSS Modeler) offers methods to

explore and analyze data through a GUI.

● Matlab [19] provides a high-level language for performing a variety of data analytics, algo-

rithms, and data exploration.

● Alpine Miner [11] provides a GUI front end for users to develop analytic workflows and interact

with Big Data tools and platforms on the back end.

● STATISTICA [20] and Mathematica [21] are also popular and well-regarded data mining and

analytics tools.

● Free or Open Source tools:

● R and PL/R [14] R was described earlier in the model planning phase, and PL/R is a procedural

language for PostgreSQL with R. Using this approach means that R commands can be exe-

cuted in database. This technique provides higher performance and is more scalable than

running R in memory.

● Octave [22], a free software programming language for computational modeling, has some of

the functionality of Matlab. Because it is freely available, Octave is used in major universities

when teaching machine learning.

● WEKA [23] is a free data mining software package with an analytic workbench. The functions

created in WEKA can be executed within Java code.

● Python is a programming language that provides toolkits for machine learning and analysis,

such as scikit-learn, numpy, scipy, pandas, and related data visualization using matplotlib.

● SQL in-database implementations, such as MADlib [24], provide an alterative to in-memory

desktop analytical tools. MADlib provides an open-source machine learning library of algo-

rithms that can be executed in-database, for PostgreSQL or Greenplum.

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 49

2.6 Phase 5: Communicate Results 49

2.6 Phase 5: Communicate Results After executing the model, the team needs to compare the outcomes of the modeling to the criteria estab-

lished for success and failure. In Phase 5, shown in Figure 2-7, the team considers how best to articulate

the findings and outcomes to the various team members and stakeholders, taking into account caveats,

assumptions, and any limitations of the results. Because the presentation is often circulated within an

organization, it is critical to articulate the results properly and position the findings in a way that is appro-

priate for the audience.

FIGURE 2-7 Communicate results phase

As part of Phase 5, the team needs to determine if it succeeded or failed in its objectives. Many times

people do not want to admit to failing, but in this instance failure should not be considered as a true

failure, but rather as a failure of the data to accept or reject a given hypothesis adequately. This concept

can be counterintuitive for those who have been told their whole careers not to fail. However, the key is

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 50

50 DATA ANALYTICS LIFECYCLE

to remember that the team must be rigorous enough with the data to determine whether it will prove or

disprove the hypotheses outlined in Phase 1 (discovery). Sometimes teams have only done a superficial

analysis, which is not robust enough to accept or reject a hypothesis. Other times, teams perform very robust

analysis and are searching for ways to show results, even when results may not be there. It is important

to strike a balance between these two extremes when it comes to analyzing data and being pragmatic in

terms of showing real-world results.

When conducting this assessment, determine if the results are statistically significant and valid. If they

are, identify the aspects of the results that stand out and may provide salient findings when it comes time

to communicate them. If the results are not valid, think about adjustments that can be made to refine and

iterate on the model to make it valid. During this step, assess the results and identify which data points

may have been surprising and which were in line with the hypotheses that were developed in Phase 1.

Comparing the actual results to the ideas formulated early on produces additional ideas and insights that

would have been missed if the team had not taken time to formulate initial hypotheses early in the process.

By this time, the team should have determined which model or models address the analytical challenge

in the most appropriate way. In addition, the team should have ideas of some of the findings as a result of the

project. The best practice in this phase is to record all the findings and then select the three most significant

ones that can be shared with the stakeholders. In addition, the team needs to reflect on the implications

of these findings and measure the business value. Depending on what emerged as a result of the model,

the team may need to spend time quantifying the business impact of the results to help prepare for the

presentation and demonstrate the value of the findings. Doug Hubbard’s work [6] offers insights on how

to assess intangibles in business and quantify the value of seemingly unmeasurable things.

Now that the team has run the model, completed a thorough discovery phase, and learned a great deal

about the datasets, reflect on the project and consider what obstacles were in the project and what can be

improved in the future. Make recommendations for future work or improvements to existing processes, and

consider what each of the team members and stakeholders needs to fulfill her responsibilities. For instance,

sponsors must champion the project. Stakeholders must understand how the model affects their processes.

(For example, if the team has created a model to predict customer churn, the Marketing team must under-

stand how to use the churn model predictions in planning their interventions.) Production engineers need

to operationalize the work that has been done. In addition, this is the phase to underscore the business

benefits of the work and begin making the case to implement the logic into a live production environment.

As a result of this phase, the team will have documented the key findings and major insights derived

from the analysis. The deliverable of this phase will be the most visible portion of the process to the outside

stakeholders and sponsors, so take care to clearly articulate the results, methodology, and business value

of the findings. More details will be provided about data visualization tools and references in Chapter 12,

“The Endgame, or Putting It All Together.”

2.7 Phase 6: Operationalize In the final phase, the team communicates the benefits of the project more broadly and sets up a pilot

project to deploy the work in a controlled way before broadening the work to a full enterprise or ecosystem

of users. In Phase 4, the team scored the model in the analytics sandbox. Phase 6, shown in Figure 2-8,

represents the first time that most analytics teams approach deploying the new analytical methods or

models in a production environment. Rather than deploying these models immediately on a wide-scale

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 51

2.7 Phase 6: Operationalize 51

basis, the risk can be managed more effectively and the team can learn by undertaking a small scope, pilot

deployment before a wide-scale rollout. This approach enables the team to learn about the performance

and related constraints of the model in a production environment on a small scale and make adjustments

before a full deployment. During the pilot project, the team may need to consider executing the algorithm

in the database rather than with in-memory tools such as R because the run time is significantly faster and

more efficient than running in-memory, especially on larger datasets.

FIGURE 2-8 Model operationalize phase

While scoping the effort involved in conducting a pilot project, consider running the model in a

production environment for a discrete set of products or a single line of business, which tests the model

in a live setting. This allows the team to learn from the deployment and make any needed adjustments

before launching the model across the enterprise. Be aware that this phase can bring in a new set of team

members—usually the engineers responsible for the production environment who have a new set of

issues and concerns beyond those of the core project team. This technical group needs to ensure that

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 52

52 DATA ANALYTICS LIFECYCLE

running the model fits smoothly into the production environment and that the model can be integrated

into related business processes.

Part of the operationalizing phase includes creating a mechanism for performing ongoing monitoring

of model accuracy and, if accuracy degrades, finding ways to retrain the model. If feasible, design alerts

for when the model is operating “out-of-bounds.” This includes situations when the inputs are beyond the

range that the model was trained on, which may cause the outputs of the model to be inaccurate or invalid.

If this begins to happen regularly, the model needs to be retrained on new data.

Often, analytical projects yield new insights about a business, a problem, or an idea that people may

have taken at face value or thought was impossible to explore. Four main deliverables can be created to

meet the needs of most stakeholders. This approach for developing the four deliverables is discussed in

greater detail in Chapter 12.

Figure 2-9 portrays the key outputs for each of the main stakeholders of an analytics project and what

they usually expect at the conclusion of a project.

● Business User typically tries to determine the benefits and implications of the findings to the

business.

● Project Sponsor typically asks questions related to the business impact of the project, the risks and

return on investment (ROI), and the way the project can be evangelized within the organization (and

beyond).

● Project Manager needs to determine if the project was completed on time and within budget and

how well the goals were met.

● Business Intelligence Analyst needs to know if the reports and dashboards he manages will be

impacted and need to change.

● Data Engineer and Database Administrator (DBA) typically need to share their code from the ana-

lytics project and create a technical document on how to implement it.

● Data Scientist needs to share the code and explain the model to her peers, managers, and other

stakeholders.

Although these seven roles represent many interests within a project, these interests usually overlap,

and most of them can be met with four main deliverables.

● Presentation for project sponsors: This contains high-level takeaways for executive level stakehold-

ers, with a few key messages to aid their decision-making process. Focus on clean, easy visuals for the

presenter to explain and for the viewer to grasp.

● Presentation for analysts, which describes business process changes and reporting changes. Fellow

data scientists will want the details and are comfortable with technical graphs (such as Receiver

Operating Characteristic [ROC] curves, density plots, and histograms shown in Chapter 3 and Chapter 7).

● Code for technical people.

● Technical specifications of implementing the code.

As a general rule, the more executive the audience, the more succinct the presentation needs to be.

Most executive sponsors attend many briefings in the course of a day or a week. Ensure that the presenta-

tion gets to the point quickly and frames the results in terms of value to the sponsor’s organization. For

instance, if the team is working with a bank to analyze cases of credit card fraud, highlight the frequency

of fraud, the number of cases in the past month or year, and the cost or revenue impact to the bank

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 53

2.8 Case Study: Global Innovation Network and Analysis (GINA) 53

(or focus on the reverse—how much more revenue the bank could gain if it addresses the fraud problem).

This demonstrates the business impact better than deep dives on the methodology. The presentation

needs to include supporting information about analytical methodology and data sources, but generally

only as supporting detail or to ensure the audience has confidence in the approach that was taken to

analyze the data.

FIGURE 2-9 Key outputs from a successful analytics project

When presenting to other audiences with more quantitative backgrounds, focus more time on the

methodology and findings. In these instances, the team can be more expansive in describing the out-

comes, methodology, and analytical experiment with a peer group. This audience will be more interested

in the techniques, especially if the team developed a new way of processing or analyzing data that can be

reused in the future or applied to similar problems. In addition, use imagery or data visualization when

possible. Although it may take more time to develop imagery, people tend to remember mental pictures to

demonstrate a point more than long lists of bullets [25]. Data visualization and presentations are discussed

further in Chapter 12.

2.8 Case Study: Global Innovation Network and Analysis (GINA) EMC’s Global Innovation Network and Analytics (GINA) team is a group of senior technologists located in

centers of excellence (COEs) around the world. This team’s charter is to engage employees across global

COEs to drive innovation, research, and university partnerships. In 2012, a newly hired director wanted to

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 54

54 DATA ANALYTICS LIFECYCLE

improve these activities and provide a mechanism to track and analyze the related information. In addition,

this team wanted to create more robust mechanisms for capturing the results of its informal conversations

with other thought leaders within EMC, in academia, or in other organizations, which could later be mined

for insights.

The GINA team thought its approach would provide a means to share ideas globally and increase

knowledge sharing among GINA members who may be separated geographically. It planned to create a

data repository containing both structured and unstructured data to accomplish three main goals.

● Store formal and informal data.

● Track research from global technologists.

● Mine the data for patterns and insights to improve the team’s operations and strategy.

The GINA case study provides an example of how a team applied the Data Analytics Lifecycle to analyze

innovation data at EMC. Innovation is typically a difficult concept to measure, and this team wanted to look

for ways to use advanced analytical methods to identify key innovators within the company.

2.8.1 Phase 1: Discovery In the GINA project’s discovery phase, the team began identifying data sources. Although GINA was a

group of technologists skilled in many different aspects of engineering, it had some data and ideas about

what it wanted to explore but lacked a formal team that could perform these analytics. After consulting

with various experts including Tom Davenport, a noted expert in analytics at Babson College, and Peter

Gloor, an expert in collective intelligence and creator of CoIN (Collaborative Innovation Networks) at MIT,

the team decided to crowdsource the work by seeking volunteers within EMC.

Here is a list of how the various roles on the working team were fulfilled.

● Business User, Project Sponsor, Project Manager: Vice President from Office of the CTO

● Business Intelligence Analyst: Representatives from IT

● Data Engineer and Database Administrator (DBA): Representatives from IT

● Data Scientist: Distinguished Engineer, who also developed the social graphs shown in the GINA

case study

The project sponsor’s approach was to leverage social media and blogging [26] to accelerate the col-

lection of innovation and research data worldwide and to motivate teams of “volunteer” data scientists

at worldwide locations. Given that he lacked a formal team, he needed to be resourceful about finding

people who were both capable and willing to volunteer their time to work on interesting problems. Data

scientists tend to be passionate about data, and the project sponsor was able to tap into this passion of

highly talented people to accomplish challenging work in a creative way.

The data for the project fell into two main categories. The first category represented five years of idea

submissions from EMC’s internal innovation contests, known as the Innovation Roadmap (formerly called the

Innovation Showcase). The Innovation Roadmap is a formal, organic innovation process whereby employees

from around the globe submit ideas that are then vetted and judged. The best ideas are selected for further

incubation. As a result, the data is a mix of structured data, such as idea counts, submission dates, inventor

names, and unstructured content, such as the textual descriptions of the ideas themselves.

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 55

2.8 Case Study: Global Innovation Network and Analysis (GINA) 55

The second category of data encompassed minutes and notes representing innovation and research

activity from around the world. This also represented a mix of structured and unstructured data. The

structured data included attributes such as dates, names, and geographic locations. The unstructured

documents contained the “who, what, when, and where” information that represents rich data about

knowledge growth and transfer within the company. This type of information is often stored in business

silos that have little to no visibility across disparate research teams.

The 10 main IHs that the GINA team developed were as follows:

● IH1: Innovation activity in different geographic regions can be mapped to corporate strategic

directions.

● IH2: The length of time it takes to deliver ideas decreases when global knowledge transfer occurs as

part of the idea delivery process.

● IH3: Innovators who participate in global knowledge transfer deliver ideas more quickly than those

who do not.

● IH4: An idea submission can be analyzed and evaluated for the likelihood of receiving funding.

● IH5: Knowledge discovery and growth for a particular topic can be measured and compared across

geographic regions.

● IH6: Knowledge transfer activity can identify research-specific boundary spanners in disparate

regions.

● IH7: Strategic corporate themes can be mapped to geographic regions.

● IH8: Frequent knowledge expansion and transfer events reduce the time it takes to generate a corpo-

rate asset from an idea.

● IH9: Lineage maps can reveal when knowledge expansion and transfer did not (or has not) resulted in

a corporate asset.

● IH10: Emerging research topics can be classified and mapped to specific ideators, innovators, bound-

ary spanners, and assets.

The GINA (IHs) can be grouped into two categories:

● Descriptive analytics of what is currently happening to spark further creativity, collaboration, and

asset generation

● Predictive analytics to advise executive management of where it should be investing in the future

2.8.2 Phase 2: Data Preparation The team partnered with its IT department to set up a new analytics sandbox to store and experiment on

the data. During the data exploration exercise, the data scientists and data engineers began to notice that

certain data needed conditioning and normalization. In addition, the team realized that several missing

datasets were critical to testing some of the analytic hypotheses.

As the team explored the data, it quickly realized that if it did not have data of sufficient quality or could

not get good quality data, it would not be able to perform the subsequent steps in the lifecycle process.

As a result, it was important to determine what level of data quality and cleanliness was sufficient for the

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 56

56 DATA ANALYTICS LIFECYCLE

project being undertaken. In the case of the GINA, the team discovered that many of the names of the

researchers and people interacting with the universities were misspelled or had leading and trailing spaces

in the datastore. Seemingly small problems such as these in the data had to be addressed in this phase to

enable better analysis and data aggregation in subsequent phases.

2.8.3 Phase 3: Model Planning In the GINA project, for much of the dataset, it seemed feasible to use social network analysis techniques to

look at the networks of innovators within EMC. In other cases, it was difficult to come up with appropriate

ways to test hypotheses due to the lack of data. In one case (IH9), the team made a decision to initiate a

longitudinal study to begin tracking data points over time regarding people developing new intellectual

property. This data collection would enable the team to test the following two ideas in the future:

● IH8: Frequent knowledge expansion and transfer events reduce the amount of time it takes to

generate a corporate asset from an idea.

● IH9: Lineage maps can reveal when knowledge expansion and transfer did not (or has not) result(ed)

in a corporate asset.

For the longitudinal study being proposed, the team needed to establish goal criteria for the study.

Specifically, it needed to determine the end goal of a successful idea that had traversed the entire journey.

The parameters related to the scope of the study included the following considerations:

● Identify the right milestones to achieve this goal.

● Trace how people move ideas from each milestone toward the goal.

● Once this is done, trace ideas that die, and trace others that reach the goal. Compare the journeys of

ideas that make it and those that do not.

● Compare the times and the outcomes using a few different methods (depending on how the data is

collected and assembled). These could be as simple as t-tests or perhaps involve different types of

classification algorithms.

2.8.4 Phase 4: Model Building In Phase 4, the GINA team employed several analytical methods. This included work by the data scientist

using Natural Language Processing (NLP) techniques on the textual descriptions of the Innovation Roadmap

ideas. In addition, he conducted social network analysis using R and RStudio, and then he developed social

graphs and visualizations of the network of communications related to innovation using R’s ggplot2

package. Examples of this work are shown in Figures 2-10 and 2-11.

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 57

2.8 Case Study: Global Innovation Network and Analysis (GINA) 57

FIGURE 2-10 Social graph [27] visualization of idea submitters and finalists

FIGURE 2-11 Social graph visualization of top innovation influencers

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 58

58 DATA ANALYTICS LIFECYCLE

Figure 2-10 shows social graphs that portray the relationships between idea submitters within GINA.

Each color represents an innovator from a different country. The large dots with red circles around them

represent hubs. A hub represents a person with high connectivity and a high “betweenness” score. The

cluster in Figure 2-11 contains geographic variety, which is critical to prove the hypothesis about geo-

graphic boundary spanners. One person in this graph has an unusually high score when compared to the

rest of the nodes in the graph. The data scientist identified this person and ran a query against his name

within the analytic sandbox. These actions yielded the following information about this research scientist

(from the social graph), which illustrated how influential he was within his business unit and across many

other areas of the company worldwide:

● In 2011, he attended the ACM SIGMOD conference, which is a top-tier conference on large-scale data

management problems and databases.

● He visited employees in France who are part of the business unit for EMC’s content management

teams within Documentum (now part of the Information Intelligence Group, or IIG).

● He presented his thoughts on the SIGMOD conference at a virtual brownbag session attended by

three employees in Russia, one employee in Cairo, one employee in Ireland, one employee in India,

three employees in the United States, and one employee in Israel.

● In 2012, he attended the SDM 2012 conference in California.

● On the same trip he visited innovators and researchers at EMC federated companies, Pivotal and

VMware.

● Later on that trip he stood before an internal council of technology leaders and introduced two of his

researchers to dozens of corporate innovators and researchers.

This finding suggests that at least part of the initial hypothesis is correct; the data can identify innovators

who span different geographies and business units. The team used Tableau software for data visualization

and exploration and used the Pivotal Greenplum database as the main data repository and analytics engine.

2.8.5 Phase 5: Communicate Results In Phase 5, the team found several ways to cull results of the analysis and identify the most impactful

and relevant findings. This project was considered successful in identifying boundary spanners and

hidden innovators. As a result, the CTO office launched longitudinal studies to begin data collection efforts

and track innovation results over longer periods of time. The GINA project promoted knowledge sharing

related to innovation and researchers spanning multiple areas within the company and outside of it. GINA

also enabled EMC to cultivate additional intellectual property that led to additional research topics and

provided opportunities to forge relationships with universities for joint academic research in the fields of

Data Science and Big Data. In addition, the project was accomplished with a limited budget, leveraging a

volunteer force of highly skilled and distinguished engineers and data scientists.

One of the key findings from the project is that there was a disproportionately high density of innova-

tors in Cork, Ireland. Each year, EMC hosts an innovation contest, open to employees to submit innovation

ideas that would drive new value for the company. When looking at the data in 2011, 15% of the finalists

and 15% of the winners were from Ireland. These are unusually high numbers, given the relative size of the

Cork COE compared to other larger centers in other parts of the world. After further research, it was learned

that the COE in Cork, Ireland had received focused training in innovation from an external consultant, which

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 59

2.8 Case Study: Global Innovation Network and Analysis (GINA) 59

was proving effective. The Cork COE came up with more innovation ideas, and better ones, than it had in

the past, and it was making larger contributions to innovation at EMC. It would have been difficult, if not

impossible, to identify this cluster of innovators through traditional methods or even anecdotal, word-of-

mouth feedback. Applying social network analysis enabled the team to find a pocket of people within EMC

who were making disproportionately strong contributions. These findings were shared internally through

presentations and conferences and promoted through social media and blogs.

2.8.6 Phase 6: Operationalize Running analytics against a sandbox filled with notes, minutes, and presentations from innovation activities

yielded great insights into EMC’s innovation culture. Key findings from the project include these:

● The CTO office and GINA need more data in the future, including a marketing initiative to convince

people to inform the global community on their innovation/research activities.

● Some of the data is sensitive, and the team needs to consider security and privacy related to the data,

such as who can run the models and see the results.

● In addition to running models, a parallel initiative needs to be created to improve basic Business

Intelligence activities, such as dashboards, reporting, and queries on research activities worldwide.

● A mechanism is needed to continually reevaluate the model after deployment. Assessing the ben-

efits is one of the main goals of this stage, as is defining a process to retrain the model as needed.

In addition to the actions and findings listed, the team demonstrated how analytics can drive new

insights in projects that are traditionally difficult to measure and quantify. This project informed investment

decisions in university research projects by the CTO office and identified hidden, high-value innovators.

In addition, the CTO office developed tools to help submitters improve ideas using topic modeling as part

of new recommender systems to help idea submitters find similar ideas and refine their proposals for new

intellectual property.

Table 2-3 outlines an analytics plan for the GINA case study example. Although this project shows only

three findings, there were many more. For instance, perhaps the biggest overarching result from this project

is that it demonstrated, in a concrete way, that analytics can drive new insights in projects that deal with

topics that may seem difficult to measure, such as innovation.

TABLE 2-3 Analytic Plan from the EMC GINA Project

Components of

Analytic Plan GINA Case Study

Discovery Business

Problem Framed

Tracking global knowledge growth, ensuring effective knowledge

transfer, and quickly converting it into corporate assets. Executing on

these three elements should accelerate innovation.

Initial Hypotheses An increase in geographic knowledge transfer improves the speed of

idea delivery.

Data Five years of innovation idea submissions and history; six months of

textual notes from global innovation and research activities

(continues)

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 60

60 DATA ANALYTICS LIFECYCLE

Components of

Analytic Plan GINA Case Study

Model Planning

Analytic Technique

Social network analysis, social graphs, clustering, and regression

analysis

Result and Key Findings 1. Identified hidden, high-value innovators and found ways to share

their knowledge

2. Informed investment decisions in university research projects

3. Created tools to help submitters improve ideas with idea

recommender systems

Innovation is an idea that every company wants to promote, but it can be difficult to measure innovation

or identify ways to increase innovation. This project explored this issue from the standpoint of evaluating

informal social networks to identify boundary spanners and influential people within innovation sub-

networks. In essence, this project took a seemingly nebulous problem and applied advanced analytical

methods to tease out answers using an objective, fact-based approach.

Another outcome from the project included the need to supplement analytics with a separate data-

store for Business Intelligence reporting, accessible to search innovation/research initiatives. Aside from

supporting decision making, this will provide a mechanism to be informed on discussions and research

happening worldwide among team members in disparate locations. Finally, it highlighted the value that

can be gleaned through data and subsequent analysis. Therefore, the need was identified to start formal

marketing programs to convince people to submit (or inform) the global community on their innovation/

research activities. The knowledge sharing was critical. Without it, GINA would not have been able to

perform the analysis and identify the hidden innovators within the company.

Summary This chapter described the Data Analytics Lifecycle, which is an approach to managing and executing

analytical projects. This approach describes the process in six phases.

1. Discovery

2. Data preparation

3. Model planning

4. Model building

5. Communicate results

6. Operationalize

Through these steps, data science teams can identify problems and perform rigorous investigation of

the datasets needed for in-depth analysis. As stated in the chapter, although much is written about the

analytical methods, the bulk of the time spent on these kinds of projects is spent in preparation—namely,

TABLE 2-3 Analytic Plan from the EMC GINA Project (Continued)

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 61

Bibliography 61

in Phases 1 and 2 (discovery and data preparation). In addition, this chapter discussed the seven roles

needed for a data science team. It is critical that organizations recognize that Data Science is a team effort,

and a balance of skills is needed to be successful in tackling Big Data projects and other complex projects

involving data analytics.

Exercises 1. In which phase would the team expect to invest most of the project time? Why? Where would the

team expect to spend the least time?

2. What are the benefits of doing a pilot program before a full-scale rollout of a new analytical method-

ology? Discuss this in the context of the mini case study.

3. What kinds of tools would be used in the following phases, and for which kinds of use scenarios?

a. Phase 2: Data preparation

b. Phase 4: Model building

Bibliography [1] T. H. Davenport and D. J. Patil, “Data Scientist: The Sexiest Job of the 21st Century,” Harvard

Business Review, October 2012.

[2] J. Manyika, M. Chiu, B. Brown, J. Bughin, R. Dobbs, C. Roxburgh, and A. H. Byers, “Big Data: The Next

Frontier for Innovation, Competition, and Productivity,” McKinsey Global Institute, 2011.

[3] “Scientific Method” [Online]. Available: http://en.wikipedia.org/wiki/ Scientific_method.

[4] “CRISP-DM” [Online]. Available: http://en.wikipedia.org/wiki/ Cross_Industry_Standard_Process_for_Data_Mining.

[5] T. H. Davenport, J. G. Harris, and R. Morison, Analytics at Work: Smarter Decisions, Better Results,

2010, Harvard Business Review Press.

[6] D. W. Hubbard, How to Measure Anything: Finding the Value of Intangibles in Business, 2010,

Hoboken, NJ: John Wiley & Sons.

[7] J. Cohen, B. Dolan, M. Dunlap, J. M. Hellerstein and C. Welton, MAD Skills: New Analysis Practices

for Big Data, Watertown, MA 2009.

[8] “List of APIs” [Online]. Available: http://www.programmableweb.com/apis.

[9] B. Shneiderman [Online]. Available: http://www.ifp.illinois.edu/nabhcs/ abstracts/shneiderman.html.

[10] “Hadoop” [Online]. Available: http://hadoop.apache.org.

[11] “Alpine Miner” [Online]. Available: http://alpinenow.com.

[12] “OpenRefine” [Online]. Available: http://openrefine.org.

[13] “Data Wrangler” [Online]. Available: http://vis.stanford.edu/wrangler/.

[14] “CRAN” [Online]. Available: http://cran.us.r-project.org.

[15] “SQL” [Online]. Available: http://en.wikipedia.org/wiki/SQL.

[16] “SAS/ACCESS” [Online]. Available: http://www.sas.com/en_us/software/ data-management/access.htm.

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed

c02.indd 02:16:32:PM 12/08/2014 Page 62

62 DATA ANALYTICS LIFECYCLE

[17] “SAS Enterprise Miner” [Online]. Available: http://www.sas.com/en_us/software/ analytics/enterprise-miner.html.

[18] “SPSS Modeler” [Online]. Available: http://www-03.ibm.com/software/products/ en/category/business-analytics.

[19] “Matlab” [Online]. Available: http://www.mathworks.com/products/matlab/.

[20] “Statistica” [Online]. Available: https://www.statsoft.com.

[21] “Mathematica” [Online]. Available: http://www.wolfram.com/mathematica/.

[22] “Octave” [Online]. Available: https://www.gnu.org/software/octave/.

[23] “WEKA” [Online]. Available: http://www.cs.waikato.ac.nz/ml/weka/.

[24] “MADlib” [Online]. Available: http://madlib.net.

[25] K. L. Higbee, Your Memory—How It Works and How to Improve It, New York: Marlowe &

Company, 1996.

[26] S. Todd, “Data Science and Big Data Curriculum” [Online]. Available: http://stevetodd .typepad.com/my_weblog/data-science-and-big-data-curriculum/.

[27] T. H Davenport and D. J. Patil, “Data Scientist: The Sexiest Job of the 21st Century,” Harvard

Business Review, October 2012.

C op

yr ig

ht ©

2 01

5. J

oh n

W ile

y &

S on

s, In

co rp

or at

ed . A

ll rig

ht s

re se

rv ed