Data Management, Analytics, and Business Intelligence 3 one page
IT for Management: On-Demand Strategies for Performance, Growth, and Sustainability
Eleventh Edition
Turban, Pollard, Wood
Chapter 3
Data Management, Business Intelligence, and Data Analytics
Learning Objectives (1 of 5)
2
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
Data management is the practice of securely, efficiently, and cost-effectively.
Collecting
Keeping
Using data
The goal of data management is
To help people, organizations, and connected things
Optimize the use of data
Within the bounds of policy and regulation
So that they can make decisions and take actions that maximize the benefit to the organization
3
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
The work of data management has a wide scope, covering factors such as how to:
Create, access, and update data across a diverse data tier
Store data across multiple clouds and on premises
Provide high availability and disaster recovery
Use data in a growing variety of apps, analytics, and algorithms
Ensure data privacy and security
Archive and destroy data in accordance with retention schedules and compliance requirements
4
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
A formal data management strategy addresses
The activity of users and administrators
The capabilities of data management technologies
The demands of regulatory requirements
The needs of the organization to obtain value from its data
5
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
Today data is a kind of capital.
It is an economic factor of production in digital goods and services
Just as an automaker can’t manufacture a new model if it lacks the necessary financial capital
It can’t make its cars autonomous if it lacks the data to feed the onboard algorithms
This new role for data has implications for competitive strategy as well as for the future of computing
Strong management practices and a robust management system are essential for every organization
Regardless of size or type
6
Copyright ©2018 John Wiley & Sons, Inc.
7
Copyright ©2018 John Wiley & Sons, Inc.
8
Copyright ©2018 John Wiley & Sons, Inc.
9
Copyright ©2018 John Wiley & Sons, Inc.
10
Copyright ©2018 John Wiley & Sons, Inc.
11
Copyright ©2018 John Wiley & Sons, Inc.
12
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
What is Data?
The quantities, characters, or symbols on which operations are performed by a computer
Which may be stored and transmitted in the form of electrical signals and recorded on magnetic, optical, or mechanical recording media
What is Big Data?
Big Data is a collection of data that is huge in volume
Yet growing exponentially with time
It is a data with so large size and complexity that none of traditional data management tools can store it or process it efficiently
13
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
14
Copyright ©2018 John Wiley & Sons, Inc.
Big Data In 5 Minutes | What Is Big Data?| Introduction To Big Data |Big Data Explained
https://www.youtube.com/watch?v=bAyrObl7TYE
Data Management
Bytes(8 Bits)
0.1 bytes: A binary decision
1 byte: A single character
10 bytes: A single word
100 bytes: A telegram OR A punched card
15
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
Kilobyte (1000 Bytes)
1 Kilobyte: A very short story
2 Kilobytes: A Typewritten page
10 Kilobytes: An encyclopedic page OR A deck of punched cards
50 Kilobytes: A compressed document image page
100 Kilobytes: A low-resolution photograph
200 Kilobytes: A box of punched cards
500 Kilobytes: A very heavy box of punched cards
16
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
Megabyte (1 000 000 Bytes)
1 Megabyte: A small novel OR A 3.5 inch floppy disk
2 Megabytes: A high resolution photograph
5 Megabytes: The complete works of Shakespeare OR 30 seconds of TV-quality video
10 Megabytes: A minute of high-fidelity sound OR A digital chest X-ray
20 Megabytes: A box of floppy disks
50 Megabytes: A digital mammogram
100 Megabytes: 1 meter of shelved books OR A two-volume encyclopedic book
200 Megabytes: A reel of 9-track tape OR An IBM 3480 cartridge tape
500 Megabytes: A CD-ROM OR The hard disk of a PC
17
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
Gigabyte (1 000 000 000 Bytes)
1 Gigabyte: A pickup truck filled with paper OR A symphony in high-fidelity sound OR A movie at TV quality
2 Gigabytes: 20 meters of shelved books OR A stack of 9-track tapes
5 Gigabytes: An 8mm Exabyte tape
10 Gigabytes:
20 Gigabytes: A good collection of the works of Beethoven OR 5 Exabyte tapes OR A VHS tape used for digital data
50 Gigabytes: A floor of books OR Hundreds of 9-track tapes
100 Gigabytes: A floor of academic journals OR A large ID-1 digital tape
200 Gigabytes: 50 Exabyte tapes
18
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
Terabyte (1 000 000 000 000 Bytes)
1 Terabyte: An automated tape robot OR All the X-ray films in a large technological hospital OR 50000 trees made into paper and printed OR Daily rate of EOS data (1998)
2 Terabytes: An academic research library OR A cabinet full of Exabyte tapes
10 Terabytes: The printed collection of the US Library of Congress
50 Terabytes: The contents of a large Mass Storage System
19
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
Petabyte (1 000 000 000 000 000 Bytes)
1 Petabyte: 5 years of EOS data (at 46 mbps)
2 Petabytes: All US academic research libraries
20 Petabytes: Production of hard-disk drives in 1995
200 Petabytes: All printed material OR Production of digital magnetic tape in 1995
20
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
Exabyte (1 000 000 000 000 000 000 Bytes)
5 Exabytes: All words ever spoken by human beings
Zettabyte (1 000 000 000 000 000 000 000 Bytes)
Yottabyte (1 000 000 000 000 000 000 000 000 Bytes)
Xenottabyte (1 000 000 000 000 000 000 000 000 000 Bytes)
Shilentnobyte (1 000 000 000 000 000 000 000 000 000 000 Bytes)
Domegemegrottebyte (1 000 000 000 000 000 000 000 000 000 000 000 Bytes)
21
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
Examples Of Big Data
Following are some of the Big Data examples-
The New York Stock Exchange generates about one terabyte of new trade data per day.
22
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
Social Media
The statistic shows that 500+terabytes of new data get ingested into the databases of social media site Facebook, every day
This data is mainly generated in terms of photo and video uploads, message exchanges, putting comments etc.
23
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
A single Jet engine can generate 10+terabytes of data in 30 minutes of flight time. With many thousand flights per day, generation of data reaches up to many Petabytes.
24
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
Data Growth over the years
25
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
There are three types of data
Structured
Any data that can be stored, accessed and processed in the form of fixed format is termed as a 'structured' data
Computer science has achieved greater success in developing techniques for working with such kind of data and also deriving value out of it
However, There are problems when the size of such data grows to a huge extent
Typical sizes are being in the rage of multiple zettabytes
Data stored in a relational database management system is one example of a 'structured' data
26
Copyright ©2018 John Wiley & Sons, Inc.
Data Management
Unstructured
Any data with unknown form or the structure is classified as unstructured data
In addition to the size being huge
Un-structured data poses multiple challenges in terms of its processing for deriving value out of it
A typical example of unstructured data is a heterogeneous data source containing a combination of simple text files, images, videos etc.
Organizations have wealth of data available
But they don't know how to derive value out of it
This data is in its raw form or unstructured format
27
Copyright ©2018 John Wiley & Sons, Inc.
28
Copyright ©2018 John Wiley & Sons, Inc.
Examples Of Un-structured Data
Data Management
Data Management
Semi-structured
Semi-structured data can contain both the forms of data. We can see semi-structured data as a structured in form but it is actually not defined with e.g. a table definition in relational DBMS
Example of semi-structured data is a data represented in an XML file.
Examples Of Semi-structured Data
29
Copyright ©2018 John Wiley & Sons, Inc.
Database Technologies: Databases
Collections of data sets or records stored in a systematic way
Stores data generated by business apps, sensors, operations, and transaction-processing systems (TPS)
The data in databases are extremely volatile
Medium and large enterprises typically have many databases of various types
30
Copyright ©2018 John Wiley & Sons, Inc.
Database Technologies: Databases
Data in databases are volatile because they can be updated millions of times every second
Especially if they are transaction processing systems (TPS)
The data is constantly changing over time
Transactions, updates, deletions, and database maintenance all contribute to data volatility
31
Copyright ©2018 John Wiley & Sons, Inc.
Database Technologies: Databases
32
Copyright ©2018 John Wiley & Sons, Inc.
Database Technologies: Data Warehouses
Integrate data from multiple databases and data silos,
Organize them for complex analysis, knowledge discovery, and to support decision making
May require formatting processing and/or standardization
Loaded at specific times making them non-volatile and ready for analysis
33
Copyright ©2018 John Wiley & Sons, Inc.
Database Technologies: Data Marts
Small-scale data warehouses that support a single function or one department
Enterprises that cannot afford to invest in data warehousing may start with one or more data marts
34
Copyright ©2018 John Wiley & Sons, Inc.
Database Technologies: BI
Business Intelligence (BI)
Tools and techniques that process data and conduct statistical analysis for insight and discovery
Used to discover meaningful relationships in the data, keep informed of real time, detect trends, and identify opportunities and risks
35
Copyright ©2018 John Wiley & Sons, Inc.
Database Technologies: BI
36
Copyright ©2018 John Wiley & Sons, Inc.
Database Technologies: BI
Business Intelligence (BI)
Business intelligence is the process by which enterprises use strategies and technologies for analyzing current and historical data
With the objective of improving strategic decision-making and providing a competitive advantage
37
Copyright ©2018 John Wiley & Sons, Inc.
Database Technologies: BI
Business Intelligence (BI)
Business intelligence systems combine
Data gathering
Data storage
Knowledge management
Data analysis
To evaluate and transform complex data into meaningful, actionable information
Which can be used to support more effective strategic, tactical, and operational insights and decision-making
38
Copyright ©2018 John Wiley & Sons, Inc.
Database Technologies: BI
Business Intelligence (BI)
Business intelligence environments consist of a variety of
Technologies
Applications
Processes
Strategies
Products
Technical architectures
To enable the collection, analysis, presentation, and dissemination of internal and external business information
39
Copyright ©2018 John Wiley & Sons, Inc.
Database Technologies: BI
Business intelligence technologies use
Advanced statistics and predictive analytics
To help businesses draw conclusions from data analysis
Discover patterns
Forecast future events in business operations
Business intelligence reporting is not a linear practice
It is a continuous, multifaceted cycle of data access, exploration, and information sharing
Common business intelligence functions include:
40
Copyright ©2018 John Wiley & Sons, Inc.
Database Technologies: BI
Data mining:
Sorting through large datasets using databases, statistics, and machine learning to identify trends and establish relationships
Querying:
A request for specific data or information from a database
Data preparation:
The process of combining and structuring data in order to prepare it for analysis
Reporting:
Sharing operating and financial data analysis with decision-makers so they can draw conclusions and make decisions
41
Copyright ©2018 John Wiley & Sons, Inc.
Database Technologies: BI
Benchmarking:
Comparing current business processes and performance metrics to historical data to track performance against industry bests
Descriptive analytics:
The interpretation of historical data to draw comparisons and better understand changes that have occurred in a business
Statistical analysis:
Collecting the results from descriptive analytics and applying statistics in order to identify trends
Data visualization:
Provides visual representations such as charts and graphs for easy data analysis
42
Copyright ©2018 John Wiley & Sons, Inc.
Database Management Systems (DBMS)
43
Copyright ©2018 John Wiley & Sons, Inc.
Database Management Systems (DBMS)
Integrate with data collection systems such as TPS and business applications
Transaction Processing Systems (TPS) process the company's business transactions and thus support the operations of an enterprise
A TPS records a non-inquiry transaction itself, as well as all of its effects, in the database and produces documents relating to the transaction
Organized way to store, access, and manage data
44
Copyright ©2018 John Wiley & Sons, Inc.
Database Management Systems (DBMS)
Stores data in tables consisting of columns and rows, similar to the format of a spreadsheet
Standard database model adopted by most enterprises
DBMS Functions provide an accurate and consistent view of data throughout the enterprise
It enables the organization to make informed, actionable decisions that support the business strategy
45
Copyright ©2018 John Wiley & Sons, Inc.
Database Management Systems (DBMS)
Functions performed by a DBMS to help create such a view are:
Data filtering and profiling:
Process and store data efficiently
Inspect the data for errors, inconsistencies, redundancies, and incomplete information
Data integrity and maintenance:
Correct, standardize, and verify the consistency and integrity of the data.
Data synchronization:
Integrate, match, or link data from disparate sources
46
Copyright ©2018 John Wiley & Sons, Inc.
Database Management Systems (DBMS)
Functions performed by a DBMS to help create such a view are:
Data security:
Check and control data integrity over time.
Data access:
Provide authorized access to data in both planned and ad hoc ways within acceptable time
Today’s computing hardware is capable of crunching through huge datasets that were impossible to manage a few years back
Making them available on-demand via wired or wireless networks
47
Copyright ©2018 John Wiley & Sons, Inc.
Database Technologies: SQL
Relational Database Management Systems (DBMS)
Provides access to data using a declarative language
Declarative language
Simplifies data access by requiring that users only specify what data they want to access without defining how they will be achieved
Structured Query Language (SQL) is an example of declarative language:
SELECT column_name(s)
FROM table_name
WHERE condition
48
Copyright ©2018 John Wiley & Sons, Inc.
Relational Database
A relational database is a type of database that stores and provides access to data points that are related to one another
Relational databases are based on the relational model
An intuitive, straightforward way of representing data in tables
In a relational database, each row in the table is a record with a unique ID called the key
The columns of the table hold attributes of the data
Each record usually has a value for each attribute
Making it easy to establish the relationships among data points
49
Copyright ©2018 John Wiley & Sons, Inc.
Relational Database
50
Copyright ©2018 John Wiley & Sons, Inc.
Relational Database Concepts
https://www.youtube.com/watch?v=NvrpuBAMddw
Relational Database
51
Copyright ©2018 John Wiley & Sons, Inc.
OLTP and OLAP Systems
Online Transaction Processing and Online Analytics Processing
Online Transaction Processing (OLTP)
Designed to manage transaction data, which are volatile
Break down complex information into simpler data tables
Strike a balance between transaction-processing efficiency and query efficiency
Cannot be optimized for data mining
52
Copyright ©2018 John Wiley & Sons, Inc.
OLTP and OLAP Systems
When most business transactions occur
For example when an item is sold or returned
An order is sent or cancelled
A payment or deposit is made
Changes are made immediately to the database
These online changes are additions, updates, or deletions
The database management systems (DBMSs) record and process such transactions in the database
Support queries and reporting
53
Copyright ©2018 John Wiley & Sons, Inc.
OLTP and OLAP Systems
DBMSs are referred to as online transaction-processing (OLTP) systems
OLTP is a database design that breaks down complex information into simpler data tables
It strikes a balance between transaction-processing efficiency and query efficiency
OLTP databases process millions of transactions per second
54
Copyright ©2018 John Wiley & Sons, Inc.
OLTP and OLAP Systems
Online Transaction Processing and Online Analytics Processing
Online Analytics Processing (OLAP)
A means of organizing large business databases
Divided into one or more cubes that fit the way business is conducted
Databases cannot be optimized for
Data mining
Complex online analytics-processing (OLAP) systems
Decision support
These limitations led to the introduction of data warehouse technology
55
Copyright ©2018 John Wiley & Sons, Inc.
OLTP and OLAP Systems
Data warehouses and data marts are optimized for
OLAP
Data mining
BI
Decision support
OLAP is a term used to describe the analysis of complex data from the data warehouse
Databases are optimized for extremely fast transaction processing and query processing
Data warehouses are optimized for analysis
56
Copyright ©2018 John Wiley & Sons, Inc.