Problem Definition Task

Evidence-based Management

MGMT 7250

TOPIC 2: Asking questions to identify problems & solutions

Plan for the day

-- Recap -- EBM in action -- What is evidence, after all? -- Common misconceptions -- The importance of asking questions -- Problem identification -- Looking for solutions -- The PICOC framework for framing questions

Problem Definition Report

• Deadline: Extended from 16 August to 23 August • EBM case scenarios: Use as basis, but you need to make modifications to define your own problem

What do we mean by strategic choice?

• Choice between alternative paths/practices/structures/strategies/actions faced within and by organizations (including the ‘status quo’) that will help best achieve the organization’s goals • HR practices • Marketing practices • Organizational structure • Business strategies • Etc

Evidence?

• Evidence refers to information, facts or data, supporting or contradicting a claim, assumption, or hypothesis.

Choice of practice/structure/ strategy/action

Presence/absence of specific individual, organizational, environmental factors

Achievement/non- achievement of a goal

Leads to Leads toClaim/ assumption:

Does the evidence support this claim? What, according to the evidence, is the impact of a factor/strategic alternative on achievement of a goal?

Critical thinking

Descriptive/exploratory Causal

What is the question? Descriptive question to observe patterns/facts (e.g. what is, when is, how many, to what degree….)

Causal question to explain patterns/facts (What factors/strategic alternatives cause differences in achievement of goals?

What are the alternative answers/claims to answers?

Alternative descriptions Alternative explanations/solutions

What support exists for each alternative claim?

Stakeholder evidence, professional opinion and organisational data

Organisational data, scientific evidence

How good is support for each claim?

Critical appraisal of above sources of evidence

Critical appraisal of above sources of evidence

What is EBP in Management?

• Practitioner experience and judgment • The scientific literature: findings from empirical studies that are published in peer-reviewed journals. • Organisational data, facts and figures gathered from an organisation. • Stakeholder views and concerns or perspectives of people who might be affected by the decision

EBM in action: CISCO and scientific evidence

Case: M&A at CISCO • What works on average?

• 70% or more of M&A fail to deliver • Odds against success are great

• What are the conditions under which these results would hold? • Systematic examination of what went right and what went wrong in other M&A

• M&A between similar-sized companies rarely works • Works best with geographically proximate companies • Works best with culturally compatible firms

• Result: • Between 1993 and 1998, it acquired one firm per quarter • Successfully ‘digested’ 57 international companies

8

EBM in action: Harrah’s and organisational evidence

Experiments at Harrah’s Hotels and Casinos • Conventional wisdoms: • Rides, etc attract families to gaming venues • Building large and fancy facilities that look like Venice

or Paris are the best way to attract customers • Advertising on radio and television is among the best

ways to attract traffic and revenue • Increasing the ‘hold’ from slot machines will drive

people out

9

EBM in action: Harrah’s and organisational evidence

Experiments at Harrah’s Hotels and Casinos • COO Gary Loveman, on leave associate prof at Harvard • Conducted a series of small experiments to examine this received

wisdom • Results of experiments:

• Families with small children have little time or money • Investing money on employee selection and retention and data analytics

together paid off more than spending large sums on fancy facilities • Direct mail worked better than media ads • You could vary the hold, even increase it, depending on the machine’s

location

10

Common misconceptions

Common misconceptions

• Doesn’t enable rapid decision-making and innovation • Practitioners can’t use their experience -it’s mostly about scientific/academic evidence • Each organization is unique • It’s data-driven decision-making • It’s about the truth and proving things • It’s about making perfectly-informed decisions using all the necessary information • Gathering the evidence will give you the Answer • It’s about new ‘cutting edge’ ideas and studies • Experts/academics know all about the evidence so you just need to ask them • It’s an academic versus practitioner thing

Evidence on EBM

• Still too new to do reliable studies • But we know from scientific studies that: • Forecasts based on the opinion of many professionals is more accurate than forecasts based on one opinion • Professional decisions informed by data or statistical analysis is more accurate • Knowledge from scientific evidence is often more accurate than professional opinion (e.g. HR experts views from Week 1) • Decisions based on many sources of evidence are more effective • Assessment of decisions improves performance and learning

The importance of asking questions

Critical thinking in EBM

Descriptive/exploratory Causal

What is the question? Descriptive question to observe patterns/facts (e.g. what is, when is, how many, to what degree….)

Causal question to explain patterns/facts (What factors/strategic alternatives cause differences in achievement of goals?

What are the alternative answers/claims to answers?

Alternative descriptions Alternative explanations/solutions

What support exists for each alternative claim?

Stakeholder evidence, professional opinion and organisational data

Organisational data, scientific evidence

How good is support for each claim?

Critical appraisal of above sources of evidence

Critical appraisal of above sources of evidence

Imagine you own a building and the tenants are complaining that the elevator is too slow and they have to wait a long time. What potential solutions could you offer them?

Adapted from: Wedell-Wedellsborg, T. (2017), Are you solving the right problems?, Harvard Business Review, January- February

Activity: Slow elevator problem

16

Activity: Slow elevator problem

17

Why asking is so important

• Misdiagnosis and solutioneering (interventions in the absence of a well defined problem) can be common problems in management leading to wastage in organisations. • If you don’t ask the right questions, you won’t get the most useful answers • Should be an answerable question • Need to frame the questions so that they can be answered using evidence (using the PICOC framework)

PICOC: A tool to specify an answerable Q

• The PICOC framework is a method we can use to help us frame the problem in terms of the organizational context – • becomes very important for acquisition of evidence (Acquire) and evaluation of evidence (appraise).

• PICOC is a method used to describe the five elements of a searchable question.

Note: When you search a database, the I and O elements should be used in your search.

Critical thinking in EBM

Descriptive/exploratory Causal

What is the question? Descriptive question to observe patterns/facts (e.g. what is, when is, how many, to what degree….)

Causal question to explain patterns/facts (What factors/strategic alternatives cause differences in achievement of goals?

What are the alternative answers/claims to answers?

Alternative descriptions Alternative explanations/solutions

What support exists for each alternative claim?

Stakeholder evidence, professional opinion and organisational data

Organisational data, scientific evidence

How good is support for each claim?

Critical appraisal of above sources of evidence

Critical appraisal of above sources of evidence

PICOC Population Who?

Type of employee, subgroup, people who may be affected by the outcome

Intervention What or how? Management technique/method, factor, independent variable

Comparison Compared to what? Alternative intervention, factor, variable (Could be the ‘status quo’, ie. Doing nothing)

Outcome What are you trying to accomplish/improve/change?

Objective, purpose, goal, dependent variable

Context In what kind of organization/circumstances?

Type of organization, sector, relevant contextual factors

Application

Imagine you are a consultant, your client is a board of a large Canadian health-care organisation. The board of directors has plans for a merger with a smaller healthcare organisation in a nearby town. According to the board the objective of the merger is to integrate the back offices of the organisations (ICT, finance, facilities, HR) to achieve economies of scale. However it’s been said that the organisational culture differs widely between the two organisations. The board asked you if this culture difference can impede the successful outcome of a merger. Most of them intuitively sense that cultural differences matter but they want evidence-based advice.

PICOC

• P= all organizations • I= merger/integration • C= status quo (i.e. no merger/integration) • O= economies of scale • C= back office functions, healthcare, different organisational culture, unequal size, Canadian context

• Answerable question: • What is known in the scientific literature about • the influence of differences in organisational culture (Context) • on the impact on economies of scale (O) • of the integration of back-office functions during a merger (I) • between two healthcare organisations of unequal size? (Context)

The “Comparison” is hidden here. How?

The process of asking questions

The Center of Evidence-based Management has delineated 4 steps in this process which include asking: 1. What is the problem to be addressed or

solved? 2. What is the evidence for the assumed

problem? 3. What is the preferred solution? 4. What is the evidence for the preferred

solution? Other interventions?

Types of questions to ask Example Questions about effects Example Questions about non-effects Does it work? Does it work better than ....? Does it have an effect on ....? What is the success factor for ....? What is required to make it work ...? Will it do more good than harm?

Needs: What do people want or need? Attitude: What do people think or feel? Experience: What are peoples’ experiences? Prevalence: How many / often do people / organizations ...? Procedure: How can we implement ...? Process: How does it work? Economics: How much does it cost?

Questions about Effects

Causal & confirmatory questions: • Does team-building work? • What are the costs and benefits of self-steering teams? • What are the success factors for culture change? • Does management development improve the performance of managers? • Does employee participation prevent resistance to change? • How do employees feel about 360 degree feedback? • What is the effect on task performance of organisational commitment?

Quantitative research methods

• Closed ended questionnaires • Experimental methods • Structured interviews • Non-participant structured observation • Secondary data analysis • Meta-analysis

Questions about non-effects

Descriptive questions: are questions about quantity and frequency, differences and benchmarking. • What is the incidence of the reporting of health and safety incidents? Exploratory questions: are questions about needs and experiences, exploration of processes, procedures, dynamics. • What are employees’ experiences of performance management practices? Premise: there are different ways (methods) to acquire evidence for these different types of questions. Exploratory questions are addressed more adequately using qualitative research methods, and causal questions- quantitative methods.

Qualitative research methods

• Open-ended questionnaires • Unstructured interviews • Semi-structured interviews • Focus groups • Participant observation

Problem & solution identification

Step 1. Problem framing & identification

• Why is this a problem? • What are the symptoms? • What is the effect on the organisation if nothing is done? • Why does the problem exist and what is the assumed major cause? • What is the assumed causal mechanism? • Who are the stakeholders concerned? Business areas? • Which of the four sources of evidence have been consulted? • What is the evidence for the assumed cause?

Surfacing causal assumptions

• In the framing of the problem, assumptions are often hidden. • Once the problem has been identified, it is important to think about your own understanding of the problem. • Does an effect really exist or is it just popular opinion?

• Is the assumed cause of the problem, the problem itself, and organisational consequences causally plausible? • Is the causal mechanism plausible (links the cause, problem, and consequence)?

Activity: Low organisational commitment (1)

• An organization’s annual staff climate survey found that levels of organizational commitment were low. • Managers were concerned that this explained some of the observed low performance and a higher levels of absenteeism among staff.

Activity: Low organisational commitment (2)

? Cause Low commitment

? Organisational consequences

Purpose: The purpose of this activity is to surface assumptions linking assumed cause(s) to an assumed problem, and its outcomes Instructions: Surface your assumptions and ideas about potential causes and consequences, are they the same as management’s? Skill development: Meta-cognitive skills, critical thinking, asking, causal reasoning, communication skills.

Step 2. The evidence for the assumed cause of the problem • Consult four sources of evidence and gauge support for assumed problem and cause. üStart with professional evidence – your own and consult experts üThen consult organisational data üScientific evidence üAffected stakeholders

• Are there any relevant theories that can provide insights especially in relation to the causal mechanisms? • If we were interested in understanding causes of low job commitment we may turn to psychological contract theory.

Questions: Organisational commitment

Professional evidence • Have I/we seen organizational commitment problems before? What happened? • Based on our experience, is the level of organizational commitment there a problem? • What do I/we believe about causes of low organizational commitment? • How relevant and applicable and trustworthy is my/our expertise?

Organisational evidence • What actually is the organizational commitment level? • Are there patterns or trends in commitment? • Do data show how organizational commitment is a problem? • Do data show that low organizational commitment is causing problems? • How relevant and applicable and trustworthy are our organizational data?

Questions : Organisational commitment

Scientific evidence • What are the average rates of organizational commitment in my sector and location – is the level here ‘low’?

• What does research evidence suggest are the causes of low organizational commitment?

• What does research evidence suggest are the consequences of low organizational commitment?

• How relevant and applicable and trustworthy are the scientific findings?

Other questions • Are there valid and reliable measures of commitment? • Is commitment different from previous concepts (e.g., satisfaction, engagement)? • Are general claims made about the importance and role of commitment reasonable and accurate given the scientific evidence?

Questions : Organisational commitment

Stakeholder evidence • How do employees feel about and view the organizational commitment ‘problem’?

• Do they think low organizational commitment has negative consequences?

• What do managers think about the problem? • Do customers or clients or service users have a view?

• How relevant and applicable and trustworthy are stakeholder concerns evidence?

Can you classify the types of questions asked?

Step 3. Identifying solutions

• What is the preferred solution? • Is this feasible? • Where is the evidence to support it?

Activity continued

Preferred Solution? Key causes

Low commitment

Purpose: The purpose of this activity is surface assumptions regarding interventions likely to address the key causes of low commitment Instructions: In pairs discuss what your preferred solution to the problem is, then do a quick search on Google scholar and explore some of the scientific literature Skill development: Meta-cognitive skills, critical thinking, causal reasoning, asking, acquiring

Step 4. Consulting the four sources

Professional evidence • Have I/we seen interventions to increase organizational commitment before? What happened? • What do I/we believe about organizational commitment interventions? • Based on our experience, is the level of organizational commitment here a problem? What are costs & benefits of intervening? • How relevant and applicable and trustworthy is my/our expertise?

Organisational evidence • What attempts to enhance organizational commitment are currently in place and are they working?

• What else is happening that might be affecting organizational commitment?

• Are there relationships between organizational commitment and other data? Employee type? Shift? Department?

• How relevant and applicable and trustworthy are our organizational data?

Step 4. Consulting the four sources (2)

Scientific evidence • What does research evidence from systematic reviews suggest are major causes of low organizational commitment? • What does research evidence from systematic reviews suggest as effective interventions to increase organizational commitment? • How relevant and applicable and trustworthy are the scientific findings?

Stakeholder evidence • How do employees feel about and view the solutions? • What do managers think about the solutions? • What alternative explanations and proposed solutions do others have? • How relevant and applicable and trustworthy are stakeholder concerns evidence?

Problem Definition Report • Due: 16 August 2019 via Wattle Turnitin facility by 23.59 (There will be no penalty till 19 August, 00:00)

• Weight: 30%

• Problem Definition Report

• For this report, please select a topic that is of interest to you and an organisation of your choice. Ideally the report will provide some useful insights and recommendations to the organization.

• This topic will form the basis of the CAT Application Report that contributes to the summative assessment for this course. Remember you can change the topic for the final CAT Application Report, though this is not recommended since you will be duplicating work if you change your topic--so choose your topic wisely.

• Instructions: Identify a complex organisational issue or problem. Write a report of no more than 500 words describing the problem in the following terms:

• 1. Define the problem including brief background and context

• 2. Justify the problem. Why is it relevant/important to the organisation?

• 3. Formulate questions about the causes of the problem that are best answered using scientific evidence

• 4. Formulate questions about potential solutions to the problem that are best answered using scientific evidence