Answer five questions in requirement . Each question requires more than 300 words， a total of 1,500 words

Welcome to MGMT 7250

TOPIC 1: An introduction to Evidence- based Management

Dr. Chirag Kasbekar [email protected]

Plan for today

1. Why EBM and… what is it? 2. What is evidence, after all? 3. Common misconceptions: Some reasons why we hesitate to adopt

EBM

A little about me

Dr. Chirag Kasbekar PhD in Organisation and Management Goizueta Business School, Emory University, USA Research The evolution of industries, over time and over geographic space: (1) Why industries come to be distributed over geographic space the way they

do and how this affects entrepreneurial and organizational outcomes and processes

(2) Why industries and organizations evolve over time the way they do and the impact this has on entrepreneurial and organizational outcomes and processes

(3) How the spatial distribution of industries influences their temporal evolution and vice versa.

3

A little about me

Industry experience

Helped set up and launch an entrepreneurial software business that provided services and products to some of the largest companies in India, including the Tata Group, the Aditya Birla Group, and the Godrej Group.

4

Contact

Three ways: -- Consultation (Wed 11am-12pm) in Room 1075, LF Crisp Building -- Last 15 minutes of each seminar -- Email (least preferred) Email address: [email protected]

5

Course assessments

• Problem definition report (30%): due 16/08/2019 • CAT application report (45%): due 18/10/2019 • Self-reflective assessment (PART B: 25%): due 24/10/2019

Mahara – e-portfolio • The use of the e-portfolio in Wattle as a resource to do the self-reflective assessment.

• We highly recommend the use of the journaling tool of e-portfolio to record your reflections on the course, its outcomes, topics and topic learning objectives, contents, theories, frameworks, assessments, the challenges and opportunities, the strengths and weaknesses of your approach to study. E-portfolio in that sense serves a repository of all these reflections which will serve as resources or raw materials to do the self- reflective assessment.

• Guidance on using e-portfolio:

• For each topic each week we recommend you initiate an entry in your e-portfolio. The entry can relate to the planning, monitoring, and evaluating of learning of contents, theories, readings, assessments, skills and capabilities aimed at in each topic. Students will then have the option to drag and drop relevant entries from anywhere in their e-portfolio to address the structured questions posed in the online self-reflective assessment on Wattle.

• We have developed a set of Topic learning questions to assist students use meta-cognition when reflecting and in populating their journal entries. These questions are indicative and help students to understand how they are thinking about the course content and topics, their approach to them, etc. It is recommended that students consider and answer the questions in the course outline every week to guide learning for every topic.

Library & CEBMa resources

• https://anulib.anu.edu.au/find-access/e-resources-databases • Support, guidance and training on how to use e-databases • https://www.cebma.org/ • Critical Appraisal module

Why EBM and… what is it?

Group Activity – Beliefs, Research, & Practice

1. “On average, encouraging employees to participate in decision making is more effective than setting performance goals.”

2. “Being very intelligent is actually a disadvantage for performing well on a low-skilled job”

3. “Most errors in performance appraisals can be eliminated by providing training that describes the kinds of errors made and suggesting ways to improve them”

4. “Once employees have mastered a task, they perform better when they are told to ‘do their best’ than when they are given specific, difficult performance goals”

5. “Companies with vision statements perform better than those without them.”

Group Activity – Beliefs, Research, & Practice

1. Discuss the statements on the following slide and decide whether you agree with them, disagree with them or are not sure

2. Put down three reasons for your decision

Human perception and its perils

• The process by which people make sense of their complex environment

12

Strategic choice and managerial perception

13Hambrick and Mason (1984)

What do we mean by strategic choice?

• Choice between alternative paths/practices/structures/strategies/actions faced within and by organizations (including the ‘status quo’) that will help best achieve the organization’s goals • HR practices • Marketing practices • Organizational structure • Business strategies • Etc

What sort of questions underlie this choice?

• What are the individual, organizational and environmental factors that would lead to successful (or unsuccessful) achievement of goals? • Which alternative path/practice/structure/strategy/action would lead to better achievement of goals?

Choice of practice/structure/ strategy/action

Presence/absence of individual, organizational, environmental factors

Achievement/non- achievement of a goal

Leads to? Leads to?

Leads to?

Another way to look at this: Identifying patterns of differences/variation

• Are there differences across individuals/teams/organizations/contexts/time periods in the achievement of a particular outcome/goal? • What factors cause this variation? • Does choice of strategic solution/alternative explain differences in outcomes?

0 .2

.4 .6

.8 1

M ul

tip lie

r o f t

he m

or ta

lit y

ra te

0 10 20 30 40 Competitive experience with Post-Civil War firms

Source: Kahneman (2016), HBR

Bias: Consistently

inaccurate

Errors in identifying patterns: Bias (Systematic) vs. Noise (Unsystematic)

Noise: Inconsistently inaccurate

17

The need

• To reduce individual and group cognitive bias • To stop Solutioneering, fads and quick fixes • To isolate what works and what doesn’t • To test strong beliefs and the wrong beliefs

• Decision awareness through meta-cognitive skills • Decision accuracy through critical thinking + best available evidence • In executing EBP steps with leadership skills, demonstrate decision accountability

EBM as a corrective lens

19

The EBM lens can help correct some of the biases before decisions are made

Hambrick and Mason (1984)

Meta-cognition and critical thinking are key

• Be aware of biases and presuppositions/assumptions • Be critical in your evaluation of arguments and reasons • Basing decisions on evidence is a logical result of critical thinking

Group Activity – Beliefs, Research, & Practice

1. Choose one of the statements from the earlier activity that you got wrong

2. Critique your arguments for your decision on that statement 3. What sort of questions do you ask to critique the arguments?

Better reasoning

• Is it an argument (or does it contain one)? • What is it an argument for (that is, what is the conclusion)? • What support does it provide (that is, what are the support statements)? • How strong is the support (that is, how good is the argument)?

(p. 6, Wright, 1982)

Critical thinking

Descriptive/exploratory Causal

What is the question? Descriptive question to understand patterns (e.g. what is, when is, how many, to what degree….)

Causal question to explain patterns (What factors/strategic alternatives cause differences in achievement of goals?

What are the alternative answers/claims to answers?

Alternative descriptions Alternative explanations/solutions

What support exists for each alternative claim?

Stakeholder evidence, professional opinion and organisational data

Organisational data, scientific evidence

How good is support for each claim?

Critical appraisal of above sources of evidence

Critical evidence of above sources of evidence

Evidence-based practice in management (EBMgt) is about making decisions through the conscientious, explicit, and judicious use of four sources of information

(Briner, Denyer, & Rousseau, 2009: 19).

What is Evidence-Based Practice (EBP) in Management?

24

What is evidence, after all?

Evidence?

• Evidence refers to information, facts or data, supporting or contradicting a claim, assumption, or hypothesis.

Choice of practice/structure/ strategy/action

Presence/absence of specific individual, organizational, environmental factors

Achievement/non- achievement of a goal

Leads to Leads toClaim/ assumption:

Does the evidence support this claim? What, according to the evidence, is the impact of a factor/strategic alternative on achievement of a goal?

Types of evidence

• Practitioner experience and judgment • The scientific literature: findings from empirical studies that are published in peer-reviewed journals. • Organisational data, facts and figures gathered from an organisation. • Stakeholder views and concerns or perspectives of people who might be affected by the decision

Generalisable Idiosyncratic

High internal and external validity In-depth and anecdotal insights

Important for questions about cause and effect

Important for descriptive (non-causal or non-effects) questions about non- effects, processes

Largely derived from quantitative methods

Qualitative methods

Large N or samples Small n or samples

Different types of evidence

Core Principles

• Managers are more effective in decision-making and problem-solving when they draw upon more than one source of evidence. • Relying on one source of evidence only (intuition and personal and professional judgement) will not provide sufficient information to adequately address most management problems

• The quality of our decisions is likely to improve the more we are able to make use of trustworthy evidence • Managers need critical thinking skills to use evidence to make better decisions

Some key concepts

• Primary research: involves using research methods to generate evidence for the purpose at hand.

• Secondary research: involves using existing information or evidence that has been generated for some purpose other than the one at hand.

• Trustworthiness of claim and evidence: refers to the extent to which evidence can be relied on

• Reliability: is about measurement accuracy and the degree to which information is biased by error – systematic or unsystematic error. Without reliability you cannot have validity.

• Validity: is concerned with uncovering truth, and determining the soundness of the explanations being offered. Two kinds of error can creep in when trying to make sure an account accurately represents the social phenomena (validity): • Type I error (false positives): Believing something to be true when it is false • Type II error (false negatives): Believing something to be false when it is true

EBM in action: CISCO and scientific evidence

Case: M&A at CISCO • What works on average?

• 70% or more of M&A fail to deliver • Odds against success are great

• What are the conditions under which these results would hold? • Systematic examination of what went right and what went wrong in other M&A

• M&A between similar-sized companies rarely works • Works best with geographically proximate companies • Works best with culturally compatible firms

• Result: • Between 1993 and 1998, it acquired one firm per quarter • Successfully ‘digested’ 57 international companies

31

EBM in action: Harrah’s and organisational evidence

Experiments at Harrah’s Hotels and Casinos • Conventional wisdoms:

• Rides, etc attract families to gaming venues • Building large and fancy facilities that look like Venice

or Paris are the best way to attract customers • Advertising on radio and television is among the best

ways to attract traffic and revenue • Increasing the ‘hold’ from slot machines will drive

people out

32

EBM in action: Harrah’s and organisational evidence

Experiments at Harrah’s Hotels and Casinos • COO Gary Loveman, on leave associate prof at Harvard • Conducted a series of small experiments to examine this received

wisdom • Results of experiments:

• Families with small children have little time or money • Investing money on employee selection and retention and data analytics

together paid off more than spending large sums on fancy facilities • Direct mail worked better than media ads • You could vary the hold, even increase it, depending on the machine’s

location

33

34 Developed by Alessandra Capezio & Patrick L’Espoir Decosta Design: Camilo Potocnjak-Oxman

Activity 1. Imagine you are an Human Resource manager in a large public sector organisation. Would you regard the following statement from a management consultant as evidence of the effectiveness of 360 degree feedback?

“It is common knowledge that 360 feedback is the best way to do a reliable performance assessment .”

• Yes

• No

2. Would you regard the outcome of the annual employee satisfaction survey as evidence?

• Yes

• No

3. Imagine you are the Chief Executive Officer of a major company. Would you consider as evidence concerns expressed by the surrounding community regarding chemical waste disposal, as reported by the local paper?

• Yes

• No

Common misconceptions

Common misconceptions

• Doesn’t enable rapid decision-making and innovation • Practitioners can’t use their experience -it’s mostly about scientific/academic evidence

• It’s data-driven decision-making • It’s about the truth and proving things • It’s about making perfectly-informed decisions using all the necessary information

• Gathering the evidence will give you the Answer • It’s about new ‘cutting edge’ ideas and studies • Experts/academics know all about the evidence so you just need to ask them • It’s an academic versus practitioner thing