Information class
fislal
is_3003_chapter_6.pptxIS 3003
Chapter 6
1
The Globe and Mail
It is the largest newspaper in Canada; is considered the NYT and USA Today all in one; It wants to enjoy the subscribership of every household in Canada, but it has a problem with trying to house the information needed on all the homes in Canada
Its problem with storing and retrieving information started with the practice of storing everything on a mainframe that was hard to use for retrieving and analyzing; this led to large amounts of data relating to specific areas being downloaded to smaller computers which created pockets of data processing interests that became unwieldy; data was not integrated and difficult to analyze without the relationships to other data being available
2
There were inconsistent database systems in use, such as MS Access, SQL Server, Foxbase Pro, and even Excel; updating information was difficult because the latest information was back on the mainframe, and had to be downloaded again to get to new info
Getting to potential new subscribers was almost impossible; even information on existing subscribers was a problem security-wise, because it was stored in many places thus causing potential security breaches with inconsistent controls
3
In 2002, Globe and Mail acquired SAP NetWeaver BW, a data warehousing platform (platform versus program: platform is a capability to build several programs, whereas a program is just that); the general definition of NetWeaver is that it is an application builder from SAP for integrating business processes and databases from a number of sources while exploiting the leading Web services technologies;
All information is now aggregated within the SAP NetWeaver system in which applications for analyzing and using database items can be built and changed quickly; additionally, the database is available quickly for warehousing and mining purposes
4
The investment paid for itself in one year
This example points out the extreme importance of data management; management decision making was also enhanced by more effective use of available info
Boost in efficiency was caused by:
Making the globe and mail data easier to locate and assemble
The SAP NetWeaver system integrated info from all sources available to the paper
Duplications were eliminated, and synchronization of data sources was achieved
5
A database is – based on computer files, records and items; contains data on people, places, and things; the archetypical database is the phone book which is a record of people who use phones that are listed (accessible to the public)
Most non-computer databases, namely paper records, rolodex files, folders, notes, etc. are sequential and can only be reviewed in a certain order
6
Data and file structures
Databases are usually organized as relational databases
These are 2-dimensional tables where each subject, e.g., customers, is organized as a set on entities (records), which contain things like names of each customer, address, phone, and each entity has a set of attributes (as mentioned above – names, addresses, etc. associated with each customer; so, each set of rows is a group of entities (records); each set of columns is a set of attributes; each entity has one or more attributes
Each relational file may link to another file so that its data are available to that file as needed, and so on
Records are also called tuples
7
Relational database
8
Each record contains an attribute referred to as a key field to which the database software will go to retrieve, update, delete, or rearrange that record; the key field may also be called the primary key field; if a key field is duplicated as in the case of a name, another field will uniquely identify one from the other by, possibly the SSN; the SSN then becomes the primary key field
sometimes “foreign keys” are implanted in a record for purposes of allowing the DBMS software to veer away from the current file of say parts records to look at information about the suppliers of such parts
9
| Part_number | Part_name | Unit_price | Assembly_ref | Supplier_number |
10
FOREIGN KEY
Entity-relationships
An entity-relationship diagram is a diagram of how the various files relate to one another
Normalization in DBMS systems
is the process of streamlining
redundant data
Referential integrity – refers to
keeping the relationships between
coupled tables consistent
11
The DBMS database management system (DBMS) is a specific type of software for creating, storing, organizing, and accessing data from a database
MS Access is a DBMS for desktops
MS SQL Server is a DBMS for large mainframes
IBM DB2 is a DBMS for large mainframes
Oracle
MYSQL was a popular open-source DBMS for mid-range and large scale systems until it was bought out by Oracle as a way of eliminating competition
12
In order to construct a DBMS, one must first define the data that goes into the system; this is done through a data dictionary; MS Access has a rudimentary capability
Most all DBMS have a data manipulation language used to add, change, and delete data into and from the database
In order to use the system, the vendor has to provide a structured query language (SQL); it is a form of data manipulation language
All DBMS have a report generator facility to print and in many cases to simulate data changes over time, or some other parameter (what-if games)
13
Non-relational databases are designed for large data sets that allow simple queries over very large record sizes
The phone companies use such databases for managing telephone information; NYNEX (Verizon) has a telephone database of over 20 M numbers that it can search in less than ½ second for a specific number; that’s why 411 information is so efficient
Cloud databases are also popular; Amazon operates a very large capability using software from all the standard vendors; pricing is based on usage which means amount of storage, and time of storage
14
Big data problems
Big data is a new term in our technology lexicon; it refers to very large numbers of records with very large numbers of attributes; the size is thought to be in excess of billions to trillions of records
Data warehouse is a database that stores data of potential interest to decision makers;
Data mart is a subset of the data warehouse database; it is a highly focused portion of the organization’s data placed in a separate database for a specific purpose
15
Business intelligence infrastructure
HADOOP – is an open source software framework managed by the Apache Software Foundation; it enables distributed parallel processing of huge amounts of data across low cost computers
In-memory computing is the use of a computer’s RAM (random access memory), main volatile memory, for data storage; the standard method is for disk storage
The advantage of in-memory storage is for much faster than using disk memory; the issue is that there has to be a lot of RAM; if this additional amount is available, much more can be achieved
16
Data mining
Online analytical processing (OLAP) – a methodology that allows multi-dimensional data analysis; it allows users to view data in several ways
Data mining – is all about searching very large databases to infer rules and future behaviours, forecasting the effects of these decisions; information includes associations, sequences, classifications, clusters, and forecasts
17
Associations – one-to-many connection; occurrences linked to a single event, such as a fiscal start point
Sequences – events linked over time, and possibly by progressions of activity
Classification – patterns that describe the group to which the an item belongs, e.g., a limited partnership, that is a high risk investment vehicle, that uses several strategies, that makes use of derivatives and leverage, is called a hedge fund
Clustering – no groups have yet been defined, but items tend to congregate around some item; a graph of many events that seem to fall close to each other, define a cluster
Forecasting – uses a series of values obtained from basic formulas, and uses them to forecast trends, i.e., what other values will be at some future date
18
19
Text and web mining
Text mining – a methodology where text is analyzed for key elements (words and/or phrases), and discover patterns and relationships; patterns may be a recurrence of context, associations, clustering, etc.; relationships may be associations to a single event, or attribute
Sentiment analysis – mining of text comments (phrases, sentences, or paragraphs) to detect favorable or unfavorable opinions about specific subjects
Web mining – a technique that examines traffic, word usage, trends of various types, associations, etc. to determine what people are interested in, and what triggers their interests
20
Information policy
Information policy – specifies the organization’s rules for sharing, disseminating, acquiring, standardizing, classifying, and inventorying information
Data administration – is an element of the organization that is responsible for the policies and procedures by which data is managed as an organizational resource
The Gartner Group, a marketing consultant company, reported recently that 25% of all data collected and stored by large companies is inaccurate; even the best data cannot be reduced to information, directly or inferred, without accurate collection; data quality is a problem; a new requirement in many companies, therefore, is the data quality audit; this is a structured survey of the accuracy and completeness of the database
21
