EMR dashboard designs attached is the grading Rubric

ISEM545 Healthcare Data

Lecture 8

Data storage, retrieval, and archiving

Hoyt, Chapter 7, 13

Lecture Objectives

• Analyze various methods available for healthcare data storage and retrieval at every level from point of entry to long term archiving

• Survey models of healthcare data storage

• Including specialized storage such as clinical registries

• Compare local vs mirrored vs cloud data storage

• Evaluate data security issues for the storage methods above

Data Storage - Lifecycle

Data Lifecycle

Create

Store

Use

Share

Archive

Destroy

What is a Data Warehouse?

Database Definitions

• Data Warehousing • What is a Data Warehouse?

• A data warehouse is a database designed for query and analysis rather than for transaction processing (Star Schema)

• It usually contains historical data derived from transactional data, but it can include data from other sources.

• It separates analysis workload from transaction workload and enables an organization to consolidate data from several sources.

• In addition to a relational database, a data warehouse includes an extraction, transportation, transformation, and loading (ETL) solution, an online analytical processing (OLAP) engine, client analysis tools, and other applications that manage the process of gathering data and delivering it to business users.

Star Design (Pharmacy)

6

Sales Quantity

Amount=SalePrice*Quantity

Fact Table

Drug

Patient Location

Sales Data

Dimension Tables

Dimension Tables

 Dimension table may be used in multiple places if the data warehouse contains multiple fact tables

 Dimension tables are usually small and change relatively slowly

 Rows in a dimension table establish one-to-many relationships with the fact table

 Data in a dimension is usually hierarchical in nature

7

Fact Tables

 Fact tables contain business event details for summarization.

 Fact tables are often very large, containing hundreds of millions of rows and consuming hundreds of gigabytes or multiple terabytes of storage.

 Dimension tables contain records that describe facts, the fact table can be reduced to columns for dimension foreign keys and numeric fact values.

 Text and blob data are typically not stored in the fact table.

 Multiple fact tables are used in data warehouses that address multiple business functions, such as sales, inventory, and finance.

 Each business function should have its own fact table and will probably have some unique dimension tables

8

9

Database Definitions

• Data Mart • Data warehouse and data mart

are sometimes used incorrectly as synonyms:

• A data warehouse is a central repository for all an organization's data.

• A data mart, however, meets the particular demands of a specific group of users within the organization, such as human resource management (HRM).

• Generally, an organization's data marts are subsets of the organization's data warehouse.

Database Definitions

• Data Lake • A data lake is a storage repository that

holds a vast amount of raw data in its native format until needed

• A hierarchical data warehouse stores data in files or folders; a data lake uses a flat architecture to store data

• Each data element in a lake is assigned a unique identifier and tagged with a set of metadata tags

• When a question arises, the data lake can be queried for any relevant data, and that smaller set of data can then be analyzed more efficiently to answer the question

Database Definitions

• Big data can be described by the following characteristics:

• Volume • The quantity of generated and

stored data • Variety

• The type and nature of the data • Velocity

• The speed at which the data is generated and processed to meet demand

• Variability • Inconsistency in the data set can

ruin processes to handle and manage it

• Veracity • The quality of captured data can

vary greatly, affecting accurate analysis

Data Storage Architecture

Data Storage Architecture

Data Storage Architecture

Data Storage

Temporary

• Private/Individual • Organizational • Public/Open

Working

• Local • Offsite • Cloud

Archival

Data Storage - Temporary

• Computer needs to: • Capture incoming data

elements • Add/check appropriate

metadata • Tags • Date/time • Provenance

• Perform any initial manipulations

• Concatenation • Sorting • Flags

• Raw data is erased after storage in a working file

Data Storage - Working

Organization keeps online what it needs for operations

Lake may be very large

Marts are likely sources

for local manipulation

“Single source of truth” is a

challenge

Storage media are on

Local servers Local PCs/thin

clients Bootleg drives

Data Storage - Archival

• Individual drive/media • Organization servers

Local

• Mirror site • Hard media • Cloud

Off-site

Data Storage – Disaster Recovery

• Problem: What to do when local media is unavailable? • Can be temporary

• Most organizations track downtime and correct for upgrades and scheduled maintenance

• May have local cache • Often bootleg files

• May revert to paper • Effective and low cost but requires data entry later

• Requires policies for frequency, content, destruction of any backups

Data Storage – Sources

Vendor storage

(drives, media, or cloud)

Data Marts

EMR entry Dx device entry

Mirror site

Imaging files

Data Storage – Strengths & Weaknesses

Local • Readily

available • Physically

secure • Fast file transfer • No data loss IF

working

Mirrored • Hard wire

connection • Physical security

separate • Longer data travel

time • Minimal data losses

Cloud • Internet

connection • Physical security in

others’ hands • Backup schedule

critical to lower losses

Common Use of the Cloud

Data Storage – Cloud Types

What do you do with “old” data?

How old is too old?

What do you do with old

data?

• Old lab test results • Old office visit notes • Old imaging studies • Old referral communications • Legacy system files BUT…… You may want to see them sometime in the future, OR You may be legally required to keep them.

Replacement EMRs

• In 2015, the number of clinicians looking for a replacement EMR was greater than the number looking for their first EMR!!

• Success of Meaningful Use • M&A often forces EMR change • Existing products often fail to

meet expectations • What to do with legacy system data?

• Convert data onto new system • Time and money! • Complexity of mapping • Limit scope

• Archive the rest

Data Archiving Considerations

• Cost savings possible with archiving tools

• Data discovery: Where are all the data now?

• Data visibility: Simplified search and access

• Data management: One common archive (lake)

• Reduced backup time: Already archived old data

• Reporting: Review access to archived data

• Accident protection: HIPAA compliant archive

• Data governance: Organizational guidelines

• Server consolidation: Recovers space on actives

• Meets record retention requirements: Read-only

• Reduce breach chance: fewer silos and duplicate data

Legacy Data Planning

Inventory Current State of Data • System Inventory

1. First, identify systems to be addressed so we understand the scope of the project, including systems in both ambulatory and acute care.

• System Prioritization 2. Second, consider the

decommissioning schedule based on go-live dates, accounts receivable wind-down schedules, system failure risks and data conversion or abstraction plans.

• Financial Forecasts 3. Third, layer maintenance costs, IT

labor burden and potential compliance penalties associated with those systems.

Legacy Data Planning

• Clearly state the problem to be solved

PROBLEM

• Summarize the system inventory, financial forecast and system prioritization documents

FINDINGS

• Identify those impacted i.e., HIM, legal, IT, finance, clinicians, etc. and key requirements for storage

STAKEHOLDERS

• Review options for decommissioning legacy software

OPTIONS

• Identify best path forward

RECOMMENDATIONS

Legacy Data Planning Team

Executive Sponsor

Project Manager

Subject Matter Experts

Technical Resources

Data Governance Committee

Special Topic: Spoilation

• Spoliation is destruction or significant alteration of evidence that denies opposing parties their due rights.

• You may think of intentional document shredding, video destruction or information deletion, but you should consider electronic health record (EHR) security.

• Spoliation in a legal suit against a healthcare organization could occur due to unreliable EHR systems that make clinical data impossible to locate or retrieve.

• To be compliant with HIPAA requirements, healthcare organizations are charged with collecting, storing and providing retrievable protected health information.

• Systems that store active or historical patient data, must safeguard against spoliation as well as unintentional record damage or loss.

• Especially during litigation and eDiscovery where healthcare organizations are expected to be able to produce clinical and other health data in a reasonable time-frame. Courts may allow mistaken and negligent conduct to form the basis of a claim for destruction of evidence.

• Being cited for spoliation could result not only in severe sanctions and fines, but also in public embarrassment.

Bottom Line

Older data can be archived when no longer relevant to priority clinical or administrative processes

01 Data from legacy systems is challenging • Running >=2 systems is not

effective or efficient • Data must be available if

needed for long periods

02 Data archiving software is available to help

03