Research08
vivi_5488
Chapter9.pptxMaking BI Relevant
BI Relevance
Relevance goes beyond personalization to consider using information to streamline work flows and empower decision makers
Examples
From centralized control to empowered users
From mandated devices to bring your own devices (BYOD)
Self service BI
From reactive to proactive IT
Source: Howson
2
Relevance (Continued)
From delivering a lot of data with confusing access methods to delivering tailored, targeted applications
From call center agents without BI to call centers that take appropriate action empowered by readily available data
From giving BI to the teams who scream loudest to balancing priorities to improve responsiveness and the bottom line
3
Source: Howson
User Adoption Fosters BI Relevance
“If my {insert role here} don’t use these tools my investment will amount to nothing more than an expensive paperweight” – A sales region manager
User adoption means
Conducting user acceptance testing (UAT) to business’ satisfaction
Systematically analyzing stakeholder needs for an effective BI application “roll out”
Effectively delivering training, communication and post implementation support
Tracking BI application use and effectiveness
It ensures BI is fully adopted for a high return on investment (ROI)
User adoption is extremely important but it tends to be an afterthought
4
Ensuring Active BI User Adoption
Treat user adoption as an important project deliverable and start planning for it in the project initiation phase
Assign dedicated resources in both the business and IT to support it
Procure active sponsor support:
Adjust the incentive structure to support BI user adoption
Have your sponsor explain this change and how it will be tracked
Ask your sponsors to use their BI applications!
5
Ensuring Active BI User Adoption (Cont’d)
While training is important, also consider:
“Day one” operational use including “level 1, level 2 and level 3” escalation and service level agreements so issue resolution expectations are set
Ongoing mentoring
Recordings (i.e. “Brainsharks”)
Fostering Communities of Practice
Communications (newsletters, websites, other) to share success stories and best practices
Record and report progress to relevant stakeholders:
Share usage statistics
“Lift” attributable to the BI application
6
A Tale of Two BI User Adoptions
A large company’s management
Emphasized BI user adoption
Supported delaying projects to accommodate user adoption
Developed and implemented everything they delivered considering user adoption
What happened?
Another large company’s management
Focused on delivering on a tight schedule
Considered BI user acceptance testing and training in isolation
What happened?
7
A Tale of Two BI User Adoptions (Cont’d)
A large company’s management
Emphasized BI user adoption
Supported delaying projects to accommodate user adoption
Developed and implemented everything they delivered considering user adoption
This group earned a reputation for superior delivery
Another large company’s management
Focused on delivering on a tight schedule
Considered BI user acceptance testing and training in isolation
Reactive planning for a user adoption strategy tended to be an afterthought
8
User Acceptance Testing (UAT) in Context
White box testing ensures a developed or updated artifact works in isolation
“White box” because a it is like a light is shined on the contents of the artifact, or “box”
Also called “unit testing”
Black box testing ensures a white box tested artifact works in the larger system
“Black box” because the light in the box is now “turned off” and the focus turns to how the “box” fits into the system
Also called “integration testing” because the unit is integrated into the system
Regression testing ensures the new or updated artifact (a.k.a. “box”) does not adversely impact parts of the system that worked before adding it
User Acceptance Testing (UAT) requires a “clean” and accepted set of integration and regression tests to commence
It is driven by users to ensure the product meets their requirements
A successful, or “clean,” UAT certifies the product is ready for implementation and full operation
9
Bad, Mediocre and Good UAT
10
| Bad | Mediocre | Good |
| “Spot testing” an EIS the night before go live at a reinsurer The cosmetics company whose UAT smelled bad because the developers also conducted UAT and this let to a “single point of failure” | A primary insurer set up a testing room and then they left UAT testing to the business with little guidance At a healthcare insurer the QA team was brand new so their seasoned business partners “walked all over them.” | A reinsurance company hired a senior consultant to conduct UAT who assumed ownership for IT and drive UAT with the business An IT service company’s QA Lead knew the system so well she was respected. Her team wrote UAT scripts and drove UAT with the business for IT. |
Don’t Fast Track User Acceptance Testing (UAT)
Do not implement a BI application until users have accepted UAT results
Since it occurs just before implementation UAT schedules tend to be shortened
Executing UAT cycles in parallel (fast tracking) is often suggested to save time
Unfortunately, users need to experience a system’s modules in sequence and since fast tracking impacts this sequence, it will not inspire confidence
Because modules tend to be reviewed out of sequence, often before development fixes are completed it is probable rework and delays will result
Allocate sufficient time for UAT and be ready to request extensions or consider a “pilot” or “soft launch”
11
BI “Pilots” and “Soft Launches”
User acceptance testing (UAT) will probably be delayed
If UAT is delayed it is reasonable to consider a “pilot” or “soft launch”
These are releases of BI applications to production that are not fully accepted, with a promise to finalize UAT during the “pilot” or “soft launch”
The decision to do this is usually taken late in the project when it is clear the team will not “make the date” because the business requires more testing time
This enables subject matter experts (SMEs) to start using the system in context and to escalate issues while the team finishes testing and builds
12
BI “Pilots” and “Soft Launches” Continued
Tend to occur with new BI applications
With any new application, BI delivery teams have insufficient credibility to request extensions
When a new BI application is installed nobody is using it yet
If the new BI application is standalone, this reinforces the point
After implementation “pilot” or “soft” launches are not viable because
Users require availability
Rolling back complex, new functionality is more difficult within operational systems
13
A Final Word on BI “Pilots” and “Soft Launches”
Purists hate these because it implies
Users were not given adequate time to test and accept their systems
Poor planning
Don’t plan on a soft launch, conduct sufficient due diligence instead
Do keep this concept in your managerial toolbox
14
Save Credibility With Pre-UAT Tests
Quality assurance (QA) professionals tend to not understand the business aspects of BI applications, instead they work from approved test scripts and business requirements
Since QA probably does not understand the business context of what they are testing, they cannot validate their test results from a business perspective
This often leads QA teams to approve and transition BI applications to UAT that are working according to specifications but obviously incorrect from a business perspective
Business users who find such obvious issues tend to protest, QA responds by confirming they approved QA’s test scripts and requirements and conflict ensues
All this undermines the BI team’s credibility
Introduce a “pre UAT” cycle conducted by hybrid business/IT people prior to UAT to catch obvious business context issues QA would not notice. Add an additional fix build for pre-UAT
15
User Acceptance Test (UAT) Concluded
Without successful UAT you cannot reconfirm relevance and expect users to adopt your BI application
Depending on the company the concept of UAT may or may not be mature
A UAT team’s approach and confidence will dictate their ability to manage and meet user expectations
The best UAT is proactive, dedicated, planned and structured which, like user adoption in general, starts at the beginning of your BI initiative
16
The Role of Incentives
Resistance to change is a barrier to success
To remove this barrier, integrate BI into achieving high levels of performance
Financial compensation is a prominent incentive. Other Incentives
A desire to win
A desire to do a better job
Removal of frustration
Misaligned incentives produce undesirable results
17
Source: Howson
Reference List
Howson, C. (2014). Successful business intelligence: Unlock the value
of BI and big data. New York. McGraw Hill Education.
ISBN: 9780071809184
18
