Order 1125645: Factors Influencing Social Sustainable in Small and Medium-Sized Enterprises in India: A Case Study of Food Industry of India

ARTICLE IN PRESS JID: EOR [m5G; May 28, 2018;10:55 ]

European Journal of Operational Research 0 0 0 (2018) 1–14

Contents lists available at ScienceDirect

European Journal of Operational Research

journal homepage: www.elsevier.com/locate/ejor

Invited Review

The characteristics of problem structuring methods: A literature

review

Chris M. Smith a , ∗, Duncan Shaw b

a Alliance Manchester Business School, The University of Manchester, G28 Sackville Street Building, Sackville Street, Manchester, M1 3BB, UK b Alliance Manchester Business School, Humanitarian and Conflict Research Institute (HCRI), The University of Manchester, H24 Sackville Street Building,

Sackville Street, Manchester, M1 3BB, UK

a r t i c l e i n f o

Article history:

Received 22 April 2016

Accepted 2 May 2018

Available online xxx

Keywords:

Problem structuring methods

OR methodology

Soft OR

Literature review

a b s t r a c t

Problem structuring methods (PSMs) are a class of qualitative operational research (OR) modelling ap-

proaches that were first developed approximately 40 years ago. Different definitions of PSMs have been

proposed, some focusing on the types of problems that PSMs typically address, others on how they

address these problems. Despite this, there is no clear framework for what characteristics need to be

present in an approach to warrant it being regarded as a PSM. This presents a challenge to understanding

what constitutes a PSM and the acceptance of new PSMs. This exploratory paper develops a framework

from a literature review to identify similarities between PSMs. The framework reflects that PSMs hold

different philosophical assumptions to traditional OR and, thus, the framework is structured according to

the four pillars of ontological, epistemological, axiological and methodological assumptions an approach

makes. Across these assumptions, the framework poses 13 questions to determine if an approach could

be a PSM. The effectiveness of the framework is understood by applying it to eight OR approaches to see

if it successfully identifies PSMs.

© 2018 Elsevier B.V. All rights reserved.

1

f

M

r

f

E

i

t

s

i

p

w

e

r

P

o

R

a

d

r

t

u

(

t

a

V

B

2

r

P

m

t

o

n

s

t

&

h

0

. Introduction

Problem structuring methods (PSMs) are qualitative approaches

or making progress with ill-structured problems ( Rosenhead &

ingers, 2001b ). PSMs sit within operational research (OR) but

epresent an alternative paradigm for problem-solving, distinct

rom ‘traditional quantitative OR’ ( Rosenhead & Mingers, 2001b ).

ach PSM is distinctive, but this paper searches for their similar-

ties. It has been 40 years since PSMs emerged ( Kirby, 20 0 0 ), but

here is still no detailed characterisation of the features that are

hared by PSMs. As Ackermann (2012 , p. 656) writes, “whilst it

s believed that they have similar characteristics and aim to sup-

ort a particular type of problems there is not agreement as to

hich method[ologies] do and do not comply”. There is no gen-

rally accepted definition because the original PSMs were not de-

ived from a common starting point. While the theory of individual

SMs has advanced, the universal understanding of PSM method-

logy has been more stagnant ( Westcombe, Franco, & Shaw, 2006 ).

ecognising this, Eden and Ackermann (2006) call for research to

nalyse across PSMs, but the response to this has been sparse, with

∗ Corresponding author. E-mail addresses: [email protected] (C.M. Smith),

[email protected] (D. Shaw).

P

T

s

o

a

ttps://doi.org/10.1016/j.ejor.2018.05.003

377-2217/© 2018 Elsevier B.V. All rights reserved.

Please cite this article as: C.M. Smith, D. Shaw, The characteristics of pr

of Operational Research (2018), https://doi.org/10.1016/j.ejor.2018.05.00

ecent publications focussing on mixing PSMs together and with

raditional approaches ( Kotiadis & Mingers, 2014 ); developing eval-

ation frameworks ( Midgley et al., 2013 ); or defining a single PSM

Yearworth & White, 2014 ).

Clarity on the similarities of PSM characteristics is fundamen-

al to the advancement of the field. For example, a definition can

ssess the status of approaches that claim to be PSMs, such as

isioning Choices ( O’Brien & Meadows, 2006 ), WASAN ( Shaw &

lundell, 2010 ), DPSIR ( Bell, 2012 ) and Wuli–Shili–Renli ( Li & Zhu,

014 ). Thus, this paper develops and tests a framework of inter-

elated questions that can be used to assess the veracity of the

SM claims that such methods make. The classification may not

atter for established OR/non-OR approaches, but the classifica-

ion of new methods will matter if they compromise the identity

f the PSM label, rendering the term meaningless.

The seminal work on PSMs by Rosenhead (1989) in ‘Ratio-

al Analysis for a Problematic World’ identified that PSMs con-

tituted a new paradigm of analysis when compared with tradi-

ional OR. This, and the subsequent edition of the book ( Rosenhead

Mingers, 2001b ), became the consistent naming convention for

SMs, with some approaches considered PSMs and others not.

his established a form of exclusivity that may have limited the

pace for a rigorous debate concerning the philosophical, the-

retical and methodological characteristics of PSMs. Rosenhead

nd Mingers (2001a) argued that methods associated with

oblem structuring methods: A literature review, European Journal

3

2 C.M. Smith, D. Shaw / European Journal of Operational Research 0 0 0 (2018) 1–14

ARTICLE IN PRESS JID: EOR [m5G; May 28, 2018;10:55 ]

Characteristics of the Quantitative OR

paradigm Characteristics of the Qualitative OR paradigm

Problem formulation in terms of a single

objective and optimization. Multiple

objectives, if recognized, are subjected to

trade-off onto a common scale.

Non-optimizing; seeking alternative solutions which

are acceptable on a separate dimensions, without

trade-offs.

Overwhelming data demands, with

consequent problems of distortion, data

availability and data credibility.

Reduced data demands, achieved by greater

integration of hard and soft data with social

judgements.

Scientization and depoliticization,

assumed consensus.

Simplicity and transparency, aimed at clarifying the

terms of conflict.

People are treated as passive objects. Conceptualizes people as active subjects.

Assumption of a single decision maker

with abstract objective from which

concrete actions can be deduced for

implementation through a hierarchical

chain of command.

Facilitates planning from the bottom-up.

Attempts to abolish future uncertainty, and

pre-take future decisions. Accepts uncertainty, and aims to keep options open.

Fig. 1. Characteristics of quantitative and qualitative OR ( Rosenhead & Mingers, 2001a ).

n

u

t

o

2

s

t

b

n

d

e

m

w

p

P

(

a

T

w

a

(

T

s

o

e

o

t

a

1

b

t

2

r

1 In defining PSMs here, and in Section 4 , the paper recognises that some of the

literature cited predates the emergence of PSMs (in some cases by decades). How-

ever, these early sources set the foundations for the development of OR research,

quantitative OR follow a more objectivist stance and are better

suited to ‘tame’ problems that can be more easily comprehended.

In contrast, PSMs take a subjectivist stance (within an interpre-

tivist paradigm) and are suited to ‘wicked’ problems that are diffi-

cult to specify. Rosenhead and Mingers (2001a) assumed that be-

cause wicked and tame problems are opposites, the assumptions

underpinning the two paradigms should also be diametrically op-

posed. They took the assumptions underpinning traditional OR and

defined the opposing state as assumptions for PSMs ( Fig. 1 ). This

meant traditional OR and PSMs were cast as opposites, which did

not fully recognise the commonalities between some of their un-

derpinning assumptions.

Rosenhead and Mingers (2001a) noted that when these charac-

teristics were developed in the 1970s, they were rather theoretical,

a blueprint for future approaches that may be developed; however,

they have remained a dominant set of assumptions that underpin

PSMs. While the assumptions were a useful starting point in the

1970s, it is opportune to revisit the philosophical, theoretical and

methodological position of PSMs.

Underpinning our framework is a view that (compared to

other problem-solving approaches) PSMs make some unique as-

sumptions about the nature of problems and how to solve them

( Rosenhead & Mingers, 2001b ). These are underpinned by a frame-

work of ideas ( Checkland & Scholes, 1990 ). Our framework is in-

formed by the research methodology work of Guba and Lincoln

(1994; 2005 ). They define four constructs of the qualitative re-

search paradigm: ontology, the form and nature of reality and

what can be known; epistemology, the nature of relationships be-

tween the knower and what can be known; axiology, what is val-

ued in terms of research processes for generating knowledge; and

methodology, how the knower can find out what can be known.

We use these constructs to build the four pillars of our framework.

Based on the literature, the framework identifies a suite of

inter-related, common characteristics of PSMs, some of which are

shared with traditional OR approaches, as expected given that they

are both part of the OR family. All characteristics are included in

the framework, not only those that are unique to PSMs. The re-

mainder of the paper is organised as follows: Section 2 shows

how PSMs have been defined in the literature. Section 3 intro-

duces the methodology used to develop the four pillar framework.

Section 4 introduces the framework. Section 5 tests the useful-

Please cite this article as: C.M. Smith, D. Shaw, The characteristics of pr

of Operational Research (2018), https://doi.org/10.1016/j.ejor.2018.05.00

ess of the framework by applying it to eight OR approaches to

nderstand if it can identify the three PSM approaches contained

herein. Section 6 discusses the findings in relation to existing the-

ry. Section 7 draws conclusions and limitations of this work.

. Defining PSMs

In the 1970s a ‘Crisis in OR’ was identified ( Thunhurst, 1973 )

uggesting the assumptions underpinning existing quantitative OR

echniques were ill equipped to deal with the social problems

eing faced by organisations ( Kirby, 2007 ). Responding to these

ew problem characteristics, new approaches were developed with

ifferent methods of analysis, viewing problems from a differ-

nt philosophical position. Here we call these problem structuring

ethods (PSMs); in the literature PSMs have been defined in three

ays 1 : problem characteristics, method of analysing problems and

hilosophical dimensions. First, the characteristics of problems that

SMs address have been called ‘messy’ ( Ackoff, 1979 ) and ‘wicked’

Rittel, 1972 ). Such problems are pluralistic ( Jackson & Keys, 1984 ),

s stakeholders have divergent views about goals and objectives.

he problems exist in dynamic and complex systems that interact

ith each other ( Ackoff, 1979 ). While these problems are varied,

nd it is difficult to exhaustively list their attributes, Churchman

1967) believes them to share many of the following properties:

hey cannot be exhaustively formulated, every formulation is a

tatement of a solution, there is no stopping rule, there is no true

r false, there is no exhaustive list of operations, there are many

xplanations for the same problem, every problem is a symptom

f another problem, there is no immediate or ultimate test, solu-

ions are ‘one shot’ and every problem is unique. Other authors

dd that problems often lack reliable data ( Mingers & Brocklesby,

997 ) and that standard mathematical techniques are not applica-

le ( Simpson, 1978 ) as problems are defined by a social construc-

ion by actors ( Keys, 2006 ) and require constant negotiation ( Pidd,

009 ). Given their diversity of form and interpretation, problems

arely fit neatly into rigid analytical frameworks ( Checkland, 1983 ).

including PSMs, and thus underpin the assumptions of PSMs.

oblem structuring methods: A literature review, European Journal

3

C.M. Smith, D. Shaw / European Journal of Operational Research 0 0 0 (2018) 1–14 3

ARTICLE IN PRESS JID: EOR [m5G; May 28, 2018;10:55 ]

Fig. 2. Illustration of the coding process for 3 pieces of litrature relating to Pillar 2.

P

i

g

i

i

1

t

t

t

e

s

2

a

&

u

d

a

p

p

t

m

o

i

c

a

d

o

s

d

p

t

c

a

m

c

3

l

t

l

t

t

i

P

t

e

p

F

p

s

y

i

a

p

i

(

u

o

f

c

w

Second, some definitions focus on how PSMs analyse a problem.

SMs build models of situations ( Franco, 2013 ), where a model

s an integrated representation of a situation that supports ne-

otiation or develops new understanding. The models are qual-

tative ( Ackermann, 2012 ), often representing data from differ-

ng worldviews ( Mingers, 2011 ). PSMs reject reductionism ( Ackoff,

979 ), where individual elements are optimised independently of

he whole. Instead, they manage complexity ( Rosenhead, 2006 ) by

aking a holistic approach and seeking emergent system proper-

ies ( Checkland, 1981 ). PSMs see problems as systems in which

lements are connected by interrelationships rather than static

napshots. Therefore, PSMs explore systemic issues ( Midgley et al.,

013 ), aiming to build shared understanding and commitment

cross stakeholders ( Ackermann, 2012 ) through facilitation ( Franco

Montibeller, 2010 ), participation ( Rosenhead, 1996 ) and stim-

lating dialogue ( Mingers & White, 2010 ) through a structured

ecomposition of issues. Rather than relying on the analysis of

bstract data ( Mingers, 20 0 0 ), a social process of learning takes

lace through which actions are agreed upon ( Pidd, 2009 ). Finally,

hilosophical definitions centre on PSMs offering an alternative

o ‘traditional OR’, which assumes that reality can be objectively

odelled to identify efficient ways of achieving well-specified

bjectives ( Rosenhead & Mingers, 2001b ). In contrast, PSMs take

nterpretivist and social constructivist views that situations are

onstructed differently by different people are therefore subjective

nd require participation ( Rosenhead & Mingers, 2001a ).

These three classes of definition do not provide a sufficiently

etailed classification of similarities across PSMs as they are based

n anecdotal evidence from these papers (i.e. the definitions are

tated in the papers but not as the product of a research project

esigned to discover the characteristics of PSMs). Also, those pa-

ers do not aim to explain a classification system of PSM charac-

eristics. This paper aims to offer a comprehensive framework to

Please cite this article as: C.M. Smith, D. Shaw, The characteristics of pr

of Operational Research (2018), https://doi.org/10.1016/j.ejor.2018.05.00

lassify PSMs that brings together a breadth of characteristics from

cross the PSM literature base by integrating understanding from

ultiple authors and providing a structure through which PSMs

an be understood.

. Methodology to identify the pillars of PSMs from the

iterature

To aid readability of this section, Fig. 2 summarises and illus-

rates the coding process. While the table presents each stage in a

inear fashion the process was more cyclical, particularly between

he axial and relational coding, where the formation of the rela-

ional codes helped to contextualise the axial codes and the result-

ng questions. Fig. 2 is illustrative of the process focussing only on

illar 2 which yields 3 of the 13 questions. The figure only shows

hree sources of literature, therefore not all open codes informing

ach axial code are shown.

To explore the similarities among PSMs, we undertook a com-

rehensive literature review to identify common characteristics.

irst a set of search terms were identified to return a set of pa-

ers to review from the literature. We started with just problem

tructuring methods, however wanted to expand the search be-

ond this generic term. Therefore the five approaches identified

n Rosenhead and Mingers (2001b) were considered. Of these five

pproaches, three (Soft Systems Methodology, Strategic Choice Ap-

roach, Strategic Options Development and Analysis/Journey Mak-

ng) dominate written literature on PSMs with the other two

Drama Theory and Robustness Analysis) having a less than 10%

sage rate as found in Munro and Mingers (2002) in a survey

f practitioners, and having a less expansive literature base with

ewer papers and fewer authors contribution to their theoreti-

al and methodological development. Therefore these approaches

ere dropped from the search terms. We revisit both Drama

oblem structuring methods: A literature review, European Journal

3

4 C.M. Smith, D. Shaw / European Journal of Operational Research 0 0 0 (2018) 1–14

ARTICLE IN PRESS JID: EOR [m5G; May 28, 2018;10:55 ]

Fig. 3. Search terms in Google Scholar, EJOR, JORS and other sources of literature.

F

t

I

1

t

t

t

r

p

T

O

(

c

d

o

f

t

t

d

c

u

l

w

t

d

P

t

M

l

g

i

i

c

a

a

i

t

r

w

i

i

u

i

p

Theory and Robustness Analysis later on in the Discussion as a fur-

ther test of the framework.

Google Scholar searches made for the terms problem structuring

methods , soft systems methodology , strategic choice approach , strate-

gic options development and analysis or journey making . This identi-

fied 12,600 unique entry articles (some articles were returned for

multiple key words). This data set was too large for thorough anal-

ysis; therefore, the search was filtered by focussing on two key

OR journals: the European Journal of Operational Research (EJOR)

and the Journal of the Operational Research Society (JORS) ( Fig. 3 ).

These journals were selected because they are the leading OR jour-

nals with a remit to develop theoretical contributions in PSMs. Re-

moving duplications reduced this further. Additional key sources

of highly-cited literature were also included based on references

from the reduced set of articles (see last row of Fig. 3 ). These are

not grouped by search term as they emerged through the review

process. This produced a more manageable data set from which to

identify key characteristics of PSMs 2 .

The process of identifying the common characteristics of PSMs

from the literature followed a three-stage coding process of open

coding, axial coding and relational coding. During open coding,

each paper was read to identify the characteristics of PSMs they

contained. Where the same (or very similar) characteristics were

found in different papers they were grouped together in open

codes. This broke down the data to allow comparisons across

the literature ( Strauss & Corbin, 1990 ). To illustrate using Fig. 2 ,

in Checkland (1985a) we identified “both (systematically) desir-

able and (culturally) feasible” and Ackermann (2012) “Changes that

are both culturally feasible and systematically desirable identified”.

Here, repetition was removed to create a more usable dataset by

grouping these though open coding as ‘desirable and feasible out-

comes’. This gave a comprehensive list of common characteristics

of PSMs extracted from the literature.

Next, axial coding identified connections across the open codes.

Axial coding clusters open codes to understand how the open

codes are related and highlights patterns of interaction. This iden-

tified characteristics that were prevalent in the literature and, thus,

the characteristics to be included in the framework. For example,

there were several open codes relating to the political feasibility

of outcomes; therefore, this was identified as an axial code (see

Fig. 2 ). Dual coding (axial and open) in this way aided understand-

ing of what was meant by political feasibility through the open

codes. That is, political feasibility included, for example, seek buy-in

through accommodation and desirable and feasible outcomes . This led

to a diverse interpretation of what political feasibility means for a

PSM. Each axial code was then developed into a question that em-

bodied the open codes. These questions could be asked of an ap-

proach to see if that PSM feature is present or not in an approach.

2 Space prohibits us from citing every source used in this paper. An additional

bibliography is provided as a supplement, and is available from the EJOR online

system.

r

s

a

t

Please cite this article as: C.M. Smith, D. Shaw, The characteristics of pr

of Operational Research (2018), https://doi.org/10.1016/j.ejor.2018.05.00

or example, for political feasibility, the question created was Does

he approach aim to develop buy-in to politically feasible outcomes?

n total we identified 13 axial codes from the data set resulting in

3 questions, each addressing a particular feature of PSMs.

Relational coding identified overarching relational theory within

he axial codes. Relational codes are core categories through which

he theory can be understood. This procedure validates the rela-

ionships across each group, filling in categories that need further

efinement and development ( Strauss & Corbin, 1990 ).

Rosenhead (1989) suggested that PSMs constitute a new

aradigm of analysis, an alternative way of viewing the world.

o understand what paradigm means for PSMs we reviewed the

R and PSM literature and found Mingers (2003) and Shaw

2006) already drew on the highly cited authors Guba and Lin-

oln (1989, 1994, 2005 ) who identify four constructs that un-

erpin a paradigm: ontology, epistemology, axiology and method-

logy. Mingers (2003) operationalized three of these constructs

or OR, stating that they represent the most general characteris-

ics that OR approaches share (p561) – thus signalling the po-

ential of Guba and Lincoln’s work to OR/PSMs. Analysis of the

ata led to the emergence of these four constructs (relational

odes) and naming them using Guba & Lincoln’s terms helped

s to understand how the identified characteristics of PSMs re-

ated to philosophical constructs that underpin PSMs. Thus we

ere able to broaden our conceptualisation of the data using this

heory, leading to a more systematic approach to understand the

ata and a more holistic way to understand the characteristics of

SMs.

The first construct, ontology, guides users on the form and na-

ure of reality and what is there that can be known about it. For

ingers, this translates into OR by identifying the types of prob-

ems to which an approach can be applied, aspects to model and

eneral system characteristics required to apply an approach. This

dentifies the first relational code of the framework, the character-

stic and scope of the system modelled by the PSM, called systems

haracteristics .

Epistemology considers the relationship between the knower

nd what can be known. Mingers operationalized epistemology

s how knowledge is created using an approach, by whom, and

dentifying goals of this. Thus, our second relational code defines

he knowledge and involvement of stakeholders to ensure that the

equired breadth/depth of insight is available. Axiology considers

hat is valued or considered right. Mingers operationalized ax-

ology as judging the value of the intervention and the insight

t produces. Hence, the third relational code represents the val-

es of model building through the contribution of model build-

ng to the discovery of new knowledge. Mingers published his pa-

er in 2003, but in 2005, Guba and Lincoln added a fourth theo-

etical construct, methodology, which considers how the inquirer

hould go about finding knowledge. Here, methodology is oper-

tionalized as the structured process of analysis and modelling

hat an approach takes to formally build and represent knowledge.

oblem structuring methods: A literature review, European Journal

3

C.M. Smith, D. Shaw / European Journal of Operational Research 0 0 0 (2018) 1–14 5

ARTICLE IN PRESS JID: EOR [m5G; May 28, 2018;10:55 ]

Fig. 4. The theoretical construction of four pillars.

T

s

m

K

a

o

m

t

c

t

i

i

c

t

w

t

R

e

q

U

t

i

t

o

i

n

r

f

o

l

t

m

4

e

s

P

w

f

j

fi

s

T

I

P

4

t

t

m

t

s

(

a

t

t

a

m

s

t

b

a

a

s

o

a

f

p

F

p

D

w

s

P

i

s

p

a

e

t

p

T

c

e

hus, the fourth relational code represents the structured analy-

is of knowledge using formalised rules and its representation in

odels.

Next, the questions were evaluated using five measures from

eeney and Raiffa (1993) which they developed to evaluate the

ppropriateness of criteria in MCDA interventions: completeness,

perationality, decomposability, absence of redundancy, and mini-

um size. Completeness requires that all attributes of concern to

he decision maker are included. Operationality requires that the

riteria are specific enough to compare and evaluate actions effec-

ively. Decomposability requires that the performance of an action

n one criterion can be judged independently of its performance

n other criteria. Absence of redundancy requires that two or more

riteria do not represent the same thing. Minimum size requires

hat there are not too many criteria, as this would make the frame-

ork too large and impractical ( Goodwin & Wright, 2004 ). We use

hese five measures because we have a similar aim to Keeney and

aiffa – namely, to understand the usability of a set of criteria for

valuating an option – in our case – the option of whether the

uestions can identify if an approach has characteristics of PSMs.

sing these principles led to a set of 13 mutually exclusive ques-

ions being developed, each aligned to one of four constructs, mak-

ng the four pillar framework. Finally, these questions were ex-

ernally reviewed by two academics who actively research PSMs,

ne being a developer of a new qualitative OR approach identified

n Section 1 of this paper. This helped to verify the appropriate-

ess and applicability of these questions to current PSMs and the

ange of newly developed approaches, increasing confidence in the

ramework. In the discussion we consider the interrelated nature

f these questions and if all 13 are needed.

To build the framework each relational code is defined as a pil-

ar, each question is developed from an axial code which is linked

o one of these four pillars. The four pillars of PSMs are sum-

arised in Fig. 4 .

. Introducing the four pillar framework

We now introduce the 13 characteristics identified from the lit-

rature review. Each feature aligns to one of the four pillars and

pecifies a question that, when asked of an approach, uncovers if a

SM characteristics is present. If all 13 characteristics are present

ithin an approach we suggest it may be eligible for consideration

or being a PSM, however the measure of proof may be higher than

ust exhibiting all 13 characteristics. While the framework speci-

es characteristics of PSMs, we expect some characteristics to be

hared with non-PSMs as they may be characteristics of wider OR.

he questions are numbered and in italics throughout this section.

Please cite this article as: C.M. Smith, D. Shaw, The characteristics of pr

of Operational Research (2018), https://doi.org/10.1016/j.ejor.2018.05.00

n Section 5 , we apply these questions to a range of PSMs and non-

SMs to test the framework.

.1. Pillar 1: Systems characteristics

OR approaches build models that reflect a system of elements

hat interact with each other. Elements included in a model reflect

he ontological assumptions of the modelling approach.

The first characteristic of PSMs is that they build a model, or

odels. Moreover, each analytical approach should be designed

o model a system that has been identified. For example, soft

ystems methodology (SSM) investigates human activity systems

Checkland & Scholes, 1990 ) and strategic options development

nd analysis (SODA) analyses strategically important causal rela-

ionships ( Eden & Ackermann, 2001 ). PSMs should clearly iden-

ify the analytical approach that is being applied to identify and

nalyse a model. Question 1: Does the approach identify a system to

odel?

Next, we consider the assumptions about the nature of the

ystem being modelled. Here we identify opposing positions be-

ween traditional OR and PSMs. These two views are summarised

y Franco and Montibeller (2010) : In traditional OR, problem situ-

tions are assumed to exist as external realities; in PSM, problems

re socially constructed entities that depend on how participants

ubjectively interpret the world. For PSMs, actors construct their

wn interpretation of reality so that multiple subjective realities

re inputs to be modelled ( White, 2009 ). Hence, PSMs ‘move away

rom “objectively” modelling the external world towards modelling

eoples’ concepts and beliefs about the world’ ( Mingers, 1992 , p3).

or PSMs, inputs to a model are the subjective understanding of

articipants about the external world they perceive. Question 2:

oes the approach model participants’ subjective interpretations of the

orld?

Another aspect of a PSM is how the approach tries to under-

tand the system through modelling. Ackermann (2012) states that

SMs focus on managing (rather than reducing) complexity, look-

ng at the whole picture and not breaking problems into con-

tituent parts. Rather than isolating parts and studying them inde-

endently, PSMs advocate ‘holism’ to concentrate on the whole and

nalyse the relationships between parts to identify emergent prop-

rties ( Preece, Shaw, & Hayashi, 2013 ). Traditional OR approaches

ake a more mechanistic view that assumes that phenomena are

redictable and inherently understandable ( Jackson & Keys, 1984 ).

he mechanistic view leads to reductionism ( Ackoff, 1979 ), where

ause-and-effect relationships are measured assuming that knowl-

dge of all these individual relationships would lead to knowledge

oblem structuring methods: A literature review, European Journal

3

6 C.M. Smith, D. Shaw / European Journal of Operational Research 0 0 0 (2018) 1–14

ARTICLE IN PRESS JID: EOR [m5G; May 28, 2018;10:55 ]

v

D

m

f

d

c

D

c

4

e

o

a

j

t

w

w

P

e

p

e

n

‘

t

t

w

h

p

t

t

i

t

8

p

c

b

a

c

i

p

i

l

P

s

e

t

s

h

c

t

s

p

i

p

i

b

c

g

o

o

of the entire system. Question 3: Does the approach seek to build a

holistic understanding of the system?

4.2. Pillar 2: Knowledge and involvement of stakeholders

OR techniques use models to construct and represent knowl-

edge about an area of concern. The way in which knowledge is

created reflects an approach’s epistemological assumptions. Knowl-

edge creation is aided by the representations of a problem situa-

tion in a model. The form the model takes affects the knowledge-

creation process. PSM models take a qualitative form, are often dia-

grammatic ( Ackermann, 2012 ) and represent differing perspectives.

This is in contrast to the quantitative representations of reality that

typify traditional OR models and represent a more objective stand-

point. Question 4: Does the approach build a qualitative model?

The next characteristic is the process of eliciting knowledge

to build a PSM model. Franco and Montibeller (2010) argue that

building a model can be done in two forms, expert and facilitator.

In expert form, the problem situation faced by a client is given to

the OR consultant, who builds a model to develop a (quasi-) op-

timal solution. In facilitator form, the consultant jointly develops

a model through participant interaction, possibly in a group work-

shop. Checkland and Winter (2005) suggest that some PSMs can

also be implemented in two modes. First, is formal facilitation of a

group (which they call Mode 1). The facilitator is a process expert

and facilitates the elicitation of the participants’ knowledge in the

application of the approach ( Phillips & Phillips, 1993 ). Alternatively,

a participant structures their own thinking using the principles of

the approach (perhaps via an interviewer), so the user is both a

process and content expert and uses the approach to facilitate their

own thinking processes (called Mode 2). Between these two modes

is self-facilitation where a group guides themselves through a pre-

defined process using prompts ( Johnson & Johnson, 2002 ). With

facilitation, the model is the focus for participants exploring the

complexity of issues and beginning to transition their understand-

ing either achieving this clarity in a group or by themselves ( Eden

& Ackermann, 2006 ). Question 5: Does the model building involve

the facilitation of participants?

For PSMs, stakeholder learning is critical ( Checkland, 1985a ).

This arises from participants sharing situational knowledge to build

joint definitions and construct problem resolutions within a model.

A shared model can act as a boundary object offering a shared

language, shared meaning and a common interest ( Franco, 2013 ),

helping stakeholders to understand how their knowledge inter-

relates ( Ackermann, 2012 ). This learning can be done in groups or

individually. For example, participants working as individuals can

also understand the relevance of their own knowledge through a

structured process ( Shaw, Eden, & Ackermann, 2009 ). Traditional

OR will also lead to clients learning about the problem situation;

however, this will be through analysis of the model rather than

through participation in the model building process as described

above. Question 6: Does the model building enhance participants’

learning about the situation?

Finally, in Pillar 2, PSMs prioritise taking actions that are sys-

temically desirable and culturally feasible ( Pidd, 2009 ). PSMs as-

sume that it is better to have a good set of actions that improve

the situation and are politically feasible and implementable rather

than optimal solutions that may not get implemented ( Checkland,

1981 ). Some other OR approaches seek optimal solutions, but these

may never be implemented if political factors do not also inform

the model. Political feasibility can be gained through recognition

of power structures getting buy-in from important stakeholders

( Eden & Ackermann, 1998 ). Stakeholders can explore perceptions

of the problem and find agreement or accommodation between

participants’ conflicting constructions ( Checkland & Scholes, 1990 ).

Participation goes beyond merely consulting stakeholders, and en-

Please cite this article as: C.M. Smith, D. Shaw, The characteristics of pr

of Operational Research (2018), https://doi.org/10.1016/j.ejor.2018.05.00

elops stakeholders into the model building process ( Davis, Mac-

onald, & White, 2010 ) to increase their commitment to imple-

enting the outcome as they then appreciate how their views in-

orm the analysis, with the models reflecting solutions they jointly

eveloped ( Franco & Montibeller, 2010 ). Participation in the pro-

ess over time develops buy-in to feasible outcomes. Question 7:

oes the approach aim to develop buy-in to politically feasible out-

omes?

.3. Pillar 3: The values of model building

An OR approach must have a set of values to guide the mod-

lling and offer a standard against which to judge the quality

f analysis. These will reflect the axiological assumptions of an

pproach. Guba and Lincoln (1989) introduce four measures for

udging the quality and rigour of qualitative research: credibility,

ransferability, dependability and confirmability. These measures

ere used by Shaw (2006) to judge the value of Journey Making

orkshops (similar to SODA) and showed their compatibility with

SMs; therefore, we employ these values in the framework.

Credibility requires the data to accurately reflect stakehold-

rs’ social constructions. PSMs recognise that problems are multi-

erspective, allow a range of distinctive views to be explored and

mbrace conflicting objectives without collapsing them into a fi-

al single function ( Mingers, 2011 ). Instead of trying to define a

real’ or ‘objective’ problem, the focus is on joint problem defini-

ions, which encompasses the main features of individual percep-

ions ( Franco & Montibeller, 2010 ). Where Question 2 is concerned

ith the inputs to a model, this question is concerned with what

appens to those inputs. Question 8 explores if the final model

reserves the different competing logics or social realities of par-

icipants without forcing the model to represent a single objec-

ive reality. For example, different worldviews are accommodated

n SSM. These models are credible to participants as they can iden-

ify their own views as present within the final models. Question

: Is credibility established in models by preserving multiple partici-

ant contributions?

Transferability is the extent to which methodological findings

an be generalised and used in other problem contexts. The model

uilding approach should be suitably generic so it is not limited to

single setting but can be used with a diverse set of problems and

lients. Question 9: Is the model building process suitably generic so

t can be transferred to multiple problem contexts?

Traditional OR methods attempt to show that outputs are de-

endable by demonstrating their economic (substantive) rational-

ty, or when outputs are appropriate to achieve stated goals within

imits imposed by given constraints ( Eden & Ackermann, 1998 ).

SMs are used in situations where a single goal and explicitly-

tated constraints may not exist, as they are constructions of differ-

nt stakeholders. Therefore, in the absence of being able to show

hat outcomes are substantively rational, PSMs need to demon-

trate reliability in outcomes by showing that a logical procedure

as been followed. In part, dependability puts focus on the pro-

ess of collecting data ( Shaw, 2006 ). This is called procedural ra-

ionality, where ‘the procedure itself is the outcome of a publicly

tated reasoning and so can gather cognitive commitment from

articipants’ ( Eden & Ackermann, 1998 , p. 55). Procedural rational-

ty and involving users in the model building process makes the

rocess transparent ( Jackson, 2006 ), potentially increasing partic-

pants’ confidence in the outcomes. Question 10: Does the model

uilding process aim to create confidence in the outcome through pro-

edural rationality?

Confirmability requires that the data in a PSM model is

rounded in the situation being studied and not the facilitator’s

wn constructions. Furthermore, confirmability suggests that the

utcomes are grounded in the content of that model and are

oblem structuring methods: A literature review, European Journal

3

C.M. Smith, D. Shaw / European Journal of Operational Research 0 0 0 (2018) 1–14 7

ARTICLE IN PRESS JID: EOR [m5G; May 28, 2018;10:55 ]

t

s

c

d

(

r

i

w

i

s

c

t

t

4

e

o

f

p

r

b

p

p

s

s

t

c

t

y

Q

e

f

g

e

c

i

2

s

p

b

Q

a

5

s

q

w

p

W

S

s

‘

v

s

o

c

t

a

(

t

o

t

H

O

t

f

a

e

f

t

h

(

g

a

h

b

p

a

e

t

o

c

a

c

b

e

a

a

t

f

s

a

f

s

p

t

s

s

p

t

s

b

t

t

s

s

o

t

v

a

b

s

l

(

f

d

a

c

1

c

raceable to its source (i.e. that a validated audit trail exists of

takeholder views leading to model content leading to model out-

omes). In PSMs, the validation of models is done by participants

uring model building in a process called collaborative inquiry

Champion & Wilson, 2010 ). This ensures that modellers accurately

epresent the views of participants. This is different from the def-

nition of validation applied to traditional OR by Pidd (2009) in

hich validation is a process of assessing the degree to which the

nput-output relation of the model is the same as that of the real

ystem within some defined experimental frame. Confirmability fo-

uses on ensuring a transparent path of inferring findings. Ques-

ion 11: Does the model act as an audit trail that has been validated

hrough collaborative enquiry?

.4. Pillar 4: Structured analysis

OR approaches build and analyse a model and create knowl-

dge, and how an approach structures this reflects its method-

logical basis. Methodologically, PSMs comprise a number of dif-

erent tools that enable different stages of analysis. For exam-

le, SSM has rich pictures and root definitions. These stages offer

ichness to an application, allowing a range of considerations to

e included in the analysis. Because PSMs are applied to wicked

roblems, a multiplicity of different tools can help approach the

roblem from different analytical perspectives to formalise and

tructure the knowledge of participants. The staged approach re-

ults in flexibility within an application, where users can cycle

hrough different analytical tools according to the needs of the

ontext ( Checkland, 1985a ). With a diverse toolset, PSMs can struc-

ure the problem in different formats over several stages of anal-

sis. These different tools are well-documented in the literature.

uestion 12: Does the approach structure knowledge through differ-

nt stages of analyses?

While PSMs have different stages of analysis, they also use dif-

erent types of thinking: divergent and convergent. During diver-

ent thinking, participants are encouraged to think with variety to

xplore diverse issues thus increasing the likelihood of identifying

reative solutions ( Franco & Montibeller, 2010 ). Convergent think-

ng allows participants to identify commonalities in views ( Franco,

013 ) and consolidate the best ideas in preparation for the next

tage ( Shaw, 2003 ). Phillips and Phillips (1993) cite many exam-

les of poor practice in which groups converge and reject ideas

efore they are fully explored which may lead to poorer outcomes.

uestion 13: Does the approach have distinct phases for divergent

nd convergent thinking?

. Testing the four pillars

Section 4 detailed four pillars, characteristics and 13 corre-

ponding questions. This section explores the validity of these 13

uestions by considering if they effectively identify PSMs from

ithin the family of OR approaches.

To demonstrate breadth and variety of application, eight ap-

roaches have been selected. Six approaches are selected using the

illiams (2008) taxonomy of OR methods: From PSMs, we chose

SM ( Checkland, 1981 ), SODA ( Eden & Ackermann, 1998 ) and the

trategic choice approach (SCA) ( Friend & Hickling, 2005 ). From

methods to calculate an attribute of a system’, we chose data en-

elopment analysis (DEA). From ‘methods to replicate or forecast

ystem behaviour’ we chose simulation. From ‘optimisation meth-

ds’, we chose linear programming (LP). To assess if the framework

an distinguish between PSMs and approaches that are reported

o be closely aligned with PSMs ( Mingers & Rosenhead, 2001 ), we

lso chose the viable system model (VSM) and system dynamics

SD).

Please cite this article as: C.M. Smith, D. Shaw, The characteristics of pr

of Operational Research (2018), https://doi.org/10.1016/j.ejor.2018.05.00

In our selection of SD to test the framework, we appreciate

he breadth of use of the approach and so take a perspective

f SD that is closest to it being a PSM. To define this perspec-

ive, we take three recent EJOR papers on SD: Lane, Munro, and

usemann (2016), Thompson et al. (2016) , and Torres, Kunc, and

’Brien (2017) . We refer specifically to these applications (rather

han the widest spectrum of SD applications). Consequently, we re-

er to SD3 in this paper to signal that we are not considering an

ll-encompassing view of SD. As these papers represent the clos-

st form of SD to PSMs, they will provide the toughest test of the

ramework while also giving clarity to the answers to each ques-

ion. Similarly, to give a more specific definition to simulation, we

ave chosen a mainstream approach to discrete event simulation

DES).

Having established the 13 characteristics, we now test their le-

itimacy by applying them to the selected approaches. Importantly,

pplication of these characteristics aims to test the framework we

ave developed, not test the OR approaches to which they are

eing applied. Informing our understanding of the non-PSM ap-

roaches is the established OR literature relating to each of these

pproaches with the exception of SD3, as mentioned above. Differ-

nt amounts of literature for each OR approach was needed across

he 13 questions to be confident when answering them – however,

ur answers were also checked with an expert in each field to be

onfident in the answers.

To conduct this test, there are four issues to consider: whether

generic or unique scale is used to test the legitimacy of each

haracteristic, how many points should be on the scale(s), on what

asis to apply each descriptor and what descriptors are used for

ach point. First, using a unique scale to test each of the 13 char-

cteristics could better reflect the diversity of characteristics and

ddress unique features of each characteristic. However, this op-

ion was rejected because it would be difficult to agree on 13 dif-

erent scales, and the usability of a framework with 13 unique

cales may be low. Instead, we opted for a generic scale to be used

cross all characteristics because it allows users to become more

amiliar with its application, future-proofs the framework and is

ufficient to test the characteristics, which is the focus of this

aper.

Second, we needed to determine a scale to identify if a fea-

ure of PSMs is present within an approach. A binary yes/no

cale was trialled but did not adequately represent the diver-

ity of how techniques were reported in the literature. For exam-

le, when considering question 5, Does the model building involve

he facilitation of participants? , it is not possible to answer con-

istently for all techniques, as some, such as DES, can be built

oth with and without facilitation. Therefore, our scale needed

o represent a wider range of alternatives found in the litera-

ure. We began with a 5-point Likert scale (strongly agree to

trongly disagree) but found it difficult to be confident and con-

istent on what the threshold should be between different points

n the scale, as the differences can be nuanced. This was also

rue of a 4-point scale. We settled on a 3-point scale, which pro-

ided the opportunity for extreme responses (1 or 3) as well

s a middle option (2) when the literature was more equally

alanced.

Third, having established the three point scale, to ensure con-

istent application of the framework to each approach we fol-

owed the validation process reported in Shaw, Smith, and Scully

2017) whereby two researchers independently coded the answers

or each question on each of the approaches. Where there were

iscrepancies between these assessments, increasingly tight rules

nd definitions were agreed between the researchers and the pro-

ess started again. This continues until both researchers had a

00% coding match across all questions and approaches. Here we

onsider one of these principles, namely the threshold required to

oblem structuring methods: A literature review, European Journal

3

8 C.M. Smith, D. Shaw / European Journal of Operational Research 0 0 0 (2018) 1–14

ARTICLE IN PRESS JID: EOR [m5G; May 28, 2018;10:55 ]

Fig. 5. Answer to pillar 1 questions.

Fig. 6. Answer to pillar 2 questions.

c

F

5

m

S

o

c

s

a

i

o

o

b

c

l

m

m

r

2

o

t

r

w

j

f

p

t

allocate each of the three points on the scale. The main difficulty

here was to specify the circumstances under which to allocate a 2

on the scale, as the choice between 1 and 3 for a technique was

often clear. The basis of evaluation was the dominant narrative in

the OR literature, with the exception of SD, for which we used the

SD3 papers. Therefore, a journal paper in which an OR technique

was used in a way that was not consistent with that dominant nar-

rative would not change the evaluation. For example, the majority

of the VSM literature does not use the model as a method of facil-

itation, so the answer would be ‘ no’ to question 5, Does the model

building involve the facilitation of participants ? There is one paper

that uses VSM as a facilitation tool ( Tavella & Papadopoulos, 2015 ),

but this does not represent the dominant use of VSM in the liter-

ature and so would not move the evaluation to ‘ unclear’ , which is

reserved for techniques in which there is no single dominant nar-

rative.

Finally, we needed to identify suitable descriptors for each of

the three points on the scale. To provide evaluation descriptors

that cover all options, we decided that: descriptor 1 should be

a positive response to the characteristic; descriptor 2 should be

neutral; and descriptor 3 should be a negative response. On the

specificity of the descriptor, we trialled 1-Must , 2-May and 3-Must

not , but this language was too restrictive, implying 100% compli-

ance in options 1 and 3. This did not allow for the outlier paper,

not consistent with the dominant narrative, meaning many tech-

niques would be incorrectly classed as 2. We also trialled softer

descriptors (e.g. 1-Mostly , 2-Unclear and 3-Mostly not ), but the

equivocality of ‘unclear’ was not helpful. We finally trialled and

accepted 1 -Yes , 2-Often and 3-No , which (as we describe below)

allowed 1 and 3 to reflect the dominant narrative in the liter-

ature and allowed 2 to reflect when there was not a dominant

narrative.

We now present the application of these questions to the

eight OR approaches. Most of these classifications are unproblem-

atic and have been stated briefly; longer answers are given in

Please cite this article as: C.M. Smith, D. Shaw, The characteristics of pr

of Operational Research (2018), https://doi.org/10.1016/j.ejor.2018.05.00

ases that are more contentious. Summary tables are presented in

igs. 5–8 .

.1. Pillar 1: Systems characteristics

1. Does the approach identify a system to model?

All of the OR approaches are clear about the system being

odelled. SSM models the human activity system ( Checkland &

choles, 1990 ), the “modelling language used for making models

f human activity systems is all the verbs in language; an indi-

ator of logical dependency; indicators of flows, concrete or ab-

tract” ( Checkland, 1981 p. 315). SODA builds cognitive maps that

re designed to represent the way in which a person defines an

ssue ( Eden & Ackermann, 2001 ). The cognitive map is made up

f constructs (nodes) linked to form chains (shown by arrows)

f action-oriented argumentation ( Eden & Ackermann, 1998 ). SCA

uilds several models that represent the interconnectedness of de-

isions with an aim to reduce uncertainty ( Friend, 2001 ). VSM out-

ines five sub-systems that are required for an organisation to re-

ain viable ( Beer, 1981 ). SD3 draws causal loop diagrams based on

ental models of a situation, which are converted into level and

ate equations that can be quantitatively modelled ( Torres et al.,

017 ). DES models show how an entity moves through a system

ver time. A DEA model consists of inputs and outputs from a sys-

em of decision making units (DMUs) that are used to calculate the

elative efficiency of DMUs within the system. LP models are built

ith constraints defining a feasible range, or convex hull. An ob-

ective function is then either maximised or minimised within this

easible range to give an optimum answer for the defined system.

2. Does the approach model participants’ subjective interpretations of

the world?

SSM builds models of the human activity system, in which a

urposeful system is modelled in the systems world from mul-

iple perspectives, so subjectivity is a key feature of what is

oblem structuring methods: A literature review, European Journal

3

C.M. Smith, D. Shaw / European Journal of Operational Research 0 0 0 (2018) 1–14 9

ARTICLE IN PRESS JID: EOR [m5G; May 28, 2018;10:55 ]

Fig. 7. Answer to pillar 3 questions.

Fig. 8. Answer to pillar 4 questions.

e

s

u

j

c

u

t

t

(

a

d

e

t

T

e

c

H

t

m

b

c

t

t

n

e

d

m

a

(

a

u

5

S

q

e

b

l

m

D

u

i

V

a

o

n

i

2

i

a

t

b

r

o

d

s

f

t

i

o

licited ( Checkland, 1981 ). SODA builds models from ‘different

ubjective views of the situation as expressed through individ-

al interviews’ ( Eden, 1995 , p. 304). SCA models represent sub-

ective information ( Friend & Hickling, 2005 ). For SD3, all appli-

ations elicit participants’ interpretation about the problem sit-

ation as the inputs to a model, not an objective representa-

ion of reality. VSM takes a system-in-the-world position in which

he laws underpinning the model, such as requisite variety, exist

Sinn, 1998 ) and objectively model an external reality. DES, DEA

nd LP all build models of external systems that are objectively

escribed.

3. Does the approach seek to build a holistic understanding of the

system?

SSM, SODA , SCA , VSM and SD3 all prioritise the study of whole

ntities before the study of parts. They codify system properties

o represent how the system being studied relates to the whole.

his allows decision-makers to consider systemic properties. For

xample, SCA uses the shaping mode to make judgments about the

onnectedness between one field of choice and another ( Friend &

ickling, 2005 ). VSM analyses information flows and communica-

ion links between different parts of the system ( Beer, 1981 ). SD3

odels seek to understand the whole system, as was demonstrated

y Lane et al. (2016) , who sought to understand the unintended

onsequences of decisions.

DEA and LP do not attempt to gain a holistic understanding of

he situation. These models reduce complexity by breaking the sys-

em into constituent, related parts that are formulated in a mecha-

istic way. The model can only give predefined answers about, for

xample, an optimal solution or a sensitivity analysis, without un-

erstanding how this relates to the whole.

DES is more flexible, a model often may only seek to give a

echanistic understanding of the world with a single output, such

s queuing time, or it can be used in a more holistic way such as

Robinson, 2001 ) where effort s were made on understanding “why

particular change led to an improvement or worsening of the sit-

ation” (p. 909).

Please cite this article as: C.M. Smith, D. Shaw, The characteristics of pr

of Operational Research (2018), https://doi.org/10.1016/j.ejor.2018.05.00

.2. Pillar 2: Knowledge and involvement of stakeholders

4. Does the approach build a qualitative model?

Qualitative models are built in SSM, SODA, SCA and VSM, while

D3 builds both quantitative and qualitative models. In SD3, the

ualitative model shows the interrelationships between different

lements of a system by qualitatively mapping the feedback loops

etween these different elements. Quantitative data is then col-

ected to show the stocks and flows between the different ele-

ents of the system, which is the input for a quantitative model.

EA, DES and LP build objective models to represent the situation

sing quantitative variables that interconnect.

5. Does the model building involve the facilitation of participants?

SSM, as described by Checkland and Scholes (1990) , can be used

n facilitated Mode 1 as well as non-facilitated Mode 2. Likewise,

SM and SODA models can be built in Modes 1 or 2. SCA models

re typically built in Mode 1. All SD3 approaches have elements

f facilitation. DES models are often built in expert mode with

o facilitation; however, there is an established body of literature

n which these models are built using facilitation (e.g. Robinson,

001 ).

The selection of input-output variables to build DEA models

s usually based on the result of conversations between analysts

nd experts in the units being assessed, supported by quantita-

ive analysis ( Casu, Shaw, & Thanassoulis, 2005 ). The model is then

uilt in expert mode. Casu et al. (2005) used Journey Making (de-

ived from SODA) with a group of stakeholders to determine input-

utput variables. This facilitated approach constitutes a different

ata collection technique. However, while the model built with the

takeholders to identify input-output variables was made through

acilitation, the DEA model was built in expert mode. Therefore,

he dominant narrative is that DEA does not build models in facil-

tator mode. Likewise, LP models are not built in a facilitated way.

6. Does the model building enhance participants’ learning about the

situation?

Learning arises from participants sharing knowledge with each

ther, allowing them to acquire and create knowledge by synthe-

oblem structuring methods: A literature review, European Journal

3

10 C.M. Smith, D. Shaw / European Journal of Operational Research 0 0 0 (2018) 1–14

ARTICLE IN PRESS JID: EOR [m5G; May 28, 2018;10:55 ]

s

r

f

t

a

a

c

s

b

a

d

i

a

F

p

1

h

t

c

S

b

t

b

a

c

a

m

1

a

w

t

a

t

T

o

s

b

c

d

a

t

i

w

e

u

t

e

s

t

a

sising views ( Edwards, Ababneh, Hall, & Shaw, 2009 ). SSM does

this by encouraging participants to discuss different worldviews

during group modelling, and encourages learning about the system

( Checkland, 1985b ). SODA enables participants to share knowledge

through the building of composite or group causal maps. Friend

and Hickling (2005) suggest that SCA groups should adopt open

technology so that many can share ideas, allowing participation

to be interactive and learning to be enhanced. Thompson, Howick,

and Belton (2016) show that participants of facilitated SD3 work-

shops experience critical learning incidents during the conceptual-

ization phase. Kotiadis and Mingers (2014) also show how partici-

pants can learn about the problem context during model building

and specification when it is facilitated in DES.

The purpose of the model building phase in VSM, DEA, and LP

is not to be a vehicle for participants to learn through facilitation

although learning may certainly come from that phase. The pur-

pose of that phase is to build a model so that the formal outcome

of the analysis from using these methods can allow greater learn-

ing from the context being analysed. Their focus is for clients to

learn about potential solutions so that wider learning is gained

while focussing on outputs.

7. Does the approach aim to develop buy-in to politically feasible

outcomes?

To build buy-in and enhance political feasibility, approaches in-

crease participation through enveloping stakeholders in the pro-

cess and addressing issues of power within the problem situation.

SSM envelops stakeholders by building different models with them

during the intervention. SODA establishes a joint understanding of

a problem through building shared group maps. These maps are

either a composite of individual cognitive maps or a single map

built by a number of participants. Both ‘can provide a means of

enabling group members to jointly understand the perspectives of

others, reflect on the emergent issues that are surfaced from them

and begin to negotiate an agreed strategic direction’ ( Eden & Ack-

ermann, 1998 p 73). SCA builds shared models to increase under-

standing of a situation. For example, decision graphs represent the

linkages between different decision areas and the focus for the

group, while the different options are represented on a compatibil-

ity grid ( Friend, 2001 ). In addition, SCA integrates a policy stream

that involves managing the conflicting positions of those involved

to develop commitment to the results ( Friend & Hickling, 2005 ).

SD3 authors report aiming to develop buy-in of participants and

searching for outcomes that can be implemented (e.g. Lane et al.,

2016 ). DES often includes participants in the process to develop

fuller recommendations and increase the likelihood that they are

implemented (e.g. Robinson, 2001 ).

VSM considers power in the systems it models with the aim

of understanding business functions rather than increasing buy-

in from powerful stakeholders. The purpose of DEA and LP is not

to explicitly envelop stakeholders or manage power relationships

through their modelling process to build buy-in to the outcomes.

5.3. Pillar 3: Values of model building

8. Is credibility established in models by preserving multiple partici-

pant contributions?

During SSM multiple perspectives are accommodated, preserv-

ing multiple contributions by modelling a range of root definitions

and conceptual models. SODA preserves multiple views in cogni-

tive maps stitching together participant models to form a new

model that encompasses multiple views ( Smith & Shaw, 2018 ). SCA

builds group models via participants writing out their individual

ideas so that competing contributions can be compared, merged or

preserved. This ensures that each participant feels that they partic-

ipated in building the model ( Friend & Hickling, 2005 ).

Please cite this article as: C.M. Smith, D. Shaw, The characteristics of pr

of Operational Research (2018), https://doi.org/10.1016/j.ejor.2018.05.00

The final models in VSM, SD3, DES, DEA and LP typically repre-

ent a single (objective) reality; therefore, their purpose is not to

epresent different social realities. These models may have input

rom several stakeholders, who each start off with different men-

al models of the situation; however, there will be convergence to

single model that represents the issue so competing perspectives

re not retained. In some approaches, the final model can be re-

onfigured, for example, using sensitivity analysis to show different

cenarios. However, these do not represent different social realties

eing embedded in a single model in the same sense as SSM, SODA

nd SCA.

9. Is the model building process suitably generic so it can be trans-

ferred to multiple problem contexts?

All eight OR approaches discussed here have been successfully

eployed in multiple and varied problem situations. For case stud-

es of SSM, see Checkland and Scholes (1990) ; for SODA, see Eden

nd Ackermann (1998) ; for SCA, see Friend and Hickling (2005) .

or DEA, DES and LP, see Williams (2008) ; for SD3, see the three

apers quoted above; for VSM, see Beer (1981) .

0. Does the model building process aim to create confidence in the

outcome through procedural rationality?

PSMs have to demonstrate they are procedurally just without

aving hard data to prove economically that the outcome is ra-

ional; therefore, there is transparency in the model building pro-

ess and involvement by participants. This is explicitly the case for

SM, SODA, SCA, and SD3 which involve participants in the model

uilding process. VSM does not explicitly involve participants in

he model building process and therefore must build confidence

y relying on the strength of the VSM and cybernetic principles.

DEA and LP can show economic rationally through hard data,

nd the reliability of outcomes is accepted based on proof of out-

omes, not just inputs. DES uses a combination of both economic

nd procedural rationality, sometimes involving participants in the

odel building process to increase confidence in outcomes.

1. Does the model act as an audit trail of the decision making process

validated through collaborative enquiry?

The audit trail of models and other artefacts (e.g. reports) for

ll these OR approaches should show the rationale behind how and

hy outcomes and outputs were reached. The process of validating

he audit trail through collaborative enquiry varies according to the

pproach. In SSM, SODA and SCA, participants build models and

he audit trail so will have seen it develop throughout the process.

hus, participants validate the audit trail through intensive collab-

rative enquiry. The audit trail can also be recorded, either through

oftware (such as Decision Explorer (SODA) and STRAD (SCA)), or

y photographing models drawn on paper.

VSM, DEA, DES and LP do not offer the same opportunity for

ollaborative enquiry between stakeholders to continuously vali-

ate an audit trail because the model is likely to have been built by

n expert modeller. For these approaches, validation ensures that

he model accurately and objectively represents the system be-

ng modelled. For example, in DES, although there are instances in

hich the model will be built with participant facilitation thereby

ncouraging collaborative enquiry, validation is completed by sim-

lating a current state of the system and comparing this with his-

orical data ( Greasley & Smith, 2017 ).

SD3 completes validation via both means. For example, Torres

t al. (2017) shared notes with participants during facilitated work-

hops. Quantitative SD models are validated in the same way as

raditional OR, checking the accuracy of outputs objectively against

current state, or validating individual relationships.

oblem structuring methods: A literature review, European Journal

3

C.M. Smith, D. Shaw / European Journal of Operational Research 0 0 0 (2018) 1–14 11

ARTICLE IN PRESS JID: EOR [m5G; May 28, 2018;10:55 ]

5

1

t

o

r

p

a

s

l

e

p

a

o

s

t

d

1

o

t

c

m

r

d

c

i

o

s

n

f

6

a

E

s

f

s

a

a

o

a

t

S

c

(

e

t

o

d

t

f

i

a

t

l

t

(

t

o

t

a

t

(

w

b

W

“

m

t

f

h

f

c

t

t

R

(

o

P

o

t

t

m

t

t

e

c

t

w

1

t

t

2

a

a

e

a

i

t

f

D

m

o

h

S

(

d

m

s

a

s

o

p

T

S

A

c

a

.4. Pillar 4: Structured analysis

2. Does the approach structure knowledge through different stages of

analyses?

SSM, SODA , SCA , SD3, DES, DEA and LP all structure knowledge

hrough different stages of analysis. The approaches give guidance

n the order of using the stages, but they also give flexibility to

evisit or switch between stages. For example, SSM has a 7-stage

rocess ( Checkland & Scholes, 1990 ); SODA can be presented as

step-by-step guide ( Ackermann, Eden, & Brown, 2005 ); SCA has

haping, choosing, comparing and designing phases ( Friend & Hick-

ing, 2005 ); SD3 traditionally has a 6-stage process (see Torres

t al., 2017 ) and DES, DEA and LP have generic phases such as

roblem formulation, and model analysis ( Pidd, 2009 ).

VSM is neither a staged methodology nor a method; it is an

bstract model or blueprint for helping to design the structure of

rganisations ( Mingers & Rosenhead, 2001 ). Some authors propose

taged approaches using principles from VSM, such as viable sys-

ems diagnosis ( Flood & Zambuni, 1990 ); however viable systems

iagnosis is not the focus of this paper.

3. Does the approach have distinct phases for divergent and conver-

gent thinking?

SSM, SODA and SCA all have examples of structuring both types

f thinking: SSM encourages divergent thinking by looking at the

ransformation from different world views. SODA facilitators en-

ourage participants to expand the richness of a cognitive or group

ap. SCA decision graphs help participants to consider how a

ange of issues are connected. Thompson et al. (2016) suggest that

ivergent thinking takes place during SD3 model definition and

onceptualisation stages. DES, DEA and LP employ divergent think-

ng during the model specification/problem formulation stage. All

f these approaches exhibit convergent thinking when participants

elect the most relevant and accurate information to build the fi-

al model. VSM is a model and therefore does not specify different

orms of thinking.

. Discussion

Only the three established PSMs answered yes to all questions

nd, according to the framework, should be classified as a PSM.

ven an interpretation of SD at the soft end of the technique re-

ulted in question 8—about credibility built through preserving dif-

erent participants’ contributions, a key aspect to PSMs—being an-

wered no . Thus, the framework distinguishes the application of SD

s described in the SD3 papers from PSMs. Had we not taken such

soft interpretation of SD, we would expect more answers of often

r no . Section 5 shows that the framework can identify PSMs from

mong other OR approaches.

To triangulate our findings we also applied the framework to

he two PSMs from Rosenhead and Mingers (2001b) not tested in

ection 5 , Robustness Analysis and Drama Theory. Data for this was

aptured from chapters 8 - 11 from the Rosenhead and Mingers

2001b) as well as from Rosenhead (1980) and Bryant (2002) . This

vidence suggests both approaches can answer yes to all 13 ques-

ions. Below we discuss the wider implications and contributions

f the framework along with reflections on the framework and its

evelopment.

First, to understand how the four pillar framework contributes

o the debate on PSMs, Fig. 9 compares the 13 characteristics in the

ramework against the six assumptions of the alternative paradigm

dentified by Rosenhead and Mingers (2001a) from Fig. 1 . The six

ssumptions are shown on the left with the 13 characteristics in

he corresponding row of the right-hand column. The last section

ists five characteristics not covered by any of the original assump-

ions.

Please cite this article as: C.M. Smith, D. Shaw, The characteristics of pr

of Operational Research (2018), https://doi.org/10.1016/j.ejor.2018.05.00

Our findings support the assertion by Rosenhead and Mingers

2001a) that the alternative paradigm makes different assump-

ions to traditional OR regarding problem-solving and the nature

f problems. We also agree with them that defining the charac-

eristics of PSMs as ‘not traditional OR’ is limited as some char-

cteristics of PSMs are shared with hard OR. Thus, we suggest

hat the theoretical assumptions made by Rosenhead and Mingers

2001a) can be further improved to provide a stronger basis on

hich to consider the claims by new approaches that may also

elong to the family of PSMs, such as Visioning Choices, WASAN,

uli–Shili–Renli and DPSIR

All eight approaches answered yes to questions 1 and 9: 1 –

Does the approach identify a system to model?” and; 9 – “Is the

odel building process suitability generic so it can be transferred

o multiple contexts?”. As these questions do not distinguish PSMs

rom non-PSMs, they may seem superfluous to the framework –

owever, the purpose and contribution of this paper is to search

or common characteristics of PSMs of which these are central

haracteristics. That these characteristics may also be characteris-

ics of wider OR does not exclude them from being characteris-

ics of PSMs. Interestingly, neither questions 1 or 9 compare to

osenhead and Mingers’ (2001a) assumptions of the PSM paradigm

Fig. 9 ); This is a difference between the aim of our work and that

f Rosenhead and Mingers, who sought opposing assumptions of

SMs and traditional OR.

Next, there are 5 instances in which the framework answers

ften , all relating to DES. To understand this we found it useful

o think in terms of Pidd’s (1998) two streams of DES projects:

he simulation problem (concerned with the technical aspects of

odel) and the simulation project (concerned with the aims for

he context and modellers). Where a characteristic related to the

echnical aspects of an approach answers were consistent with

ither yes or no (Questions 1, 2, 4, 8, 9, 12 & 13). Where a

haracteristic related to the context or the modeller’s view of

heir/participants role, this brought a diversity of application that

as sometimes best answered with often (Questions 3, 5, 6, 7, 10 &

1). This was not surprising given that DES recognises the poten-

ial for qualitative modelling to make initial sense of the system

o be modelled quantitatively ( Kotiadis & Mingers, 2006; Robinson,

007 ). Like some other traditional OR approaches, DES has scope to

lter its method of application and exhibit some of the softer char-

cteristics of PSMs from the framework. For example, DES mod-

ls have been built using facilitation ( Robinson, 2001 ), and so may

nswer yes to questions from the second pillar. This is not say-

ng that Robinson’s (2001) use of DES constitutes a PSM but that

he epistemological assumptions underpinning this work are dif-

erent to those assumed in the exclusively harder applications of

ES. This may result in the facilitated use of DES answering yes to

ore questions in the framework than a purely hard application

f DES. The inverse is true for the established PSMs, which also

ave examples of non-standard use ( Mingers, 2003 ). For example,

haw et al. (2017) use tools from SODA to model secondary data

a non-standard use). Therefore, validation of their models was not

one through collaborative enquiry, meaning that the framework

ay not classify the application as a PSM. We argue that, given a

pectrum of behaviour relating to the project elements, the adapt-

bility of the framework is a strength. PSMs should not be clas-

ified based on their historic definitions, but on the assumptions

f the approach in context and how it is used and adapted for a

articular application.

It is not surprising that SD3 papers answered only yes and no .

his was due to the clarity provided in the three selected EJOR

D papers, which were chosen to provide a dominant narrative.

wider range of SD papers (with a wider interpretation of SD)

ould return a different result with more often answers if they had

more balanced discussion in a wider literature. Restricting the

oblem structuring methods: A literature review, European Journal

3

12 C.M. Smith, D. Shaw / European Journal of Operational Research 0 0 0 (2018) 1–14

ARTICLE IN PRESS JID: EOR [m5G; May 28, 2018;10:55 ]

Characteris�cs of Rosenhead and Mingers Alterna�ve Paradigm

Characteris�cs from the Four Pillar Framework

Non-op�mizing; seeking alterna�ve solu�ons which are acceptable on separate dimensions, without trade-offs.

Q7: Buy-in to poli�cally feasible outcomes.

Reduced data demands, achieved by greater integra�on of hard and so� data with social judgements.

Q4: Build qualita�ve models.

Q10: Shows procedural ra�onality.

Q11: Builds validated audit trail of decision making.

Q2: Models subjec�ve interpreta�ons.

Q6: Facilitates par�cipants' involvement.

Q8: Credibility through preserving par�cipant contribu�ons.

Facilitates planning from the bo�om-up. Q6: Par�cipant learning. Accepts uncertainty, and aims to keep op�ons open. Q3: Holis�c understanding.

Q1: Iden�fy system to model. Q9: Generic model building

approach. Q12: Different stages of analysis.

Q13: Phases of divergent and convergent thinking.

Simplicity and transparency, aimed at clarifying the terms of conflict.

Not included.

Conceptualizes people as ac�ve subjects.

Fig. 9. Rosenhead and Mingers’ (2001a) assumptions compared with the four pillar framework.

F

(

f

8

7

h

a

t

u

c

t

a

fi

n

r

h

p

o

p

J

f

b

f

t

s

p

a

definition of SD in this way allowed for a more thorough examina-

tion of the framework as we took the view of SD that was closest

as possible to PSMs thus, justifying this action to support the aims

of the paper.

The structure of the framework has a heavy reliance on Guba

and Lincoln (1994, 2005 ) through relational coding and how these

codes are understood for PSMs through Mingers (2003) . We feel

this grounding in higher-level theory is necessary to differentiate

between the pillars and add clarity about the different ways PSMs

are similar to each other (specifically, the entities included in a

model, the process by which knowledge is created, the values of

good research and the structure of enquiry).

We note the circular argument in classifying the three PSMs

using the framework. The literature on these three PSMs led to

the identification of the characteristics and the resulting questions,

which were then used to decide if the PSMs satisfied their own cri-

teria. Critics could argue that this self-referencing approach does

not demonstrate that the established approaches are PSMs, but

merely that they have the same characteristics already identified

in the literature. However, this misunderstands why the framework

was applied to the eight approaches. The framework was devel-

oped to understand common characteristics of PSMs, and the in-

clusion of three PSMs was a test of the framework rather than a

test of the selected approaches. The important finding is the abil-

ity of the framework to identify the PSMs, showing that they have

common and defining characteristics.

The 13 questions each focus on a separate characteristic and

should each be considered independently however, by their nature,

the questions are not discrete – they are interrelated. This is shown

in two ways, first, to understand if an approach has a claim of be-

ing a PSM all 13 questions must be considered. Second, not every

conceivable combination of answer to the 13 questions is possible.

o

Please cite this article as: C.M. Smith, D. Shaw, The characteristics of pr

of Operational Research (2018), https://doi.org/10.1016/j.ejor.2018.05.00

or example, if an approach answers yes for Question 2 then it may

but may not) answer yes for Question 8, however if it answers no

or Question 2 then the approach must also answer no in Question

.

. Conclusion

Through an exploratory review of the literature this paper

as identified characteristics of PSMs that were developed into

theoretical framework by which to clarify the similarities be-

ween PSMs, their underpinning assumptions and understand their

nique identity as a family of OR approaches. It aims to prompt

ritical conversation about PSMs by questioning and expanding on

he dominant assumptions underpinning what it is to be a PSM

s first described 40 years ago. As a result of the framework, con-

dence can be given to claims of being a PSM that are made by

ew or candidate PSMs, establishing the conditions of evidence. By

evisiting the assumptions underpinning PSMs, the framework can

elp to refocus the development of PSM approaches and theory.

We recognise that this work is not without limitations; first, the

aper takes a particular view of PSMs through the selection criteria

f drawing from the two most prominent journals that regularly

ublish methodological and case study based papers (EJOR and

ORS) ( Ranyard, Fildes, & Hu, 2015 ). As Ranyard et al. (2015) also

ound, US-based journals tend to overlook PSMs, even INFORMS

ased journals “have refused to engage with the topic, at least the

ormal problem structuring component” (p11). OMEGA is an excep-

ion in publishing PSM case study papers (although many fewer

uch papers) but most of their PSM papers are written by Euro-

ean researchers who also publish in EJOR and JORS i.e. there isn’t

distinct PSM literature only available outside of the dataset. Sec-

nd, the framework offers a view of PSMs which is a product of

oblem structuring methods: A literature review, European Journal

3

C.M. Smith, D. Shaw / European Journal of Operational Research 0 0 0 (2018) 1–14 13

ARTICLE IN PRESS JID: EOR [m5G; May 28, 2018;10:55 ]

t

p

f

w

a

o

a

t

o

d

o

f

l

t

o

a

c

t

s

o

c

S

f

R

A

A

A

B B

B

C

C

C

C

C

C

C

C

C D

E

E

E

E

E

F

F

F

F

F

G

G

G

G

G

J

J

J

K

K

K

K

K

K

L

L

M

M

M

M

M

M

M

M

M

O

P

P

P

he literature considered and the coding process adopted. We ap-

reciate that some readers may not agree with every aspect of the

ramework or identify aspects that they feel are missing. However,

hile we may have set out to answer ‘what is a PSM’, we re-

lise the resulting answer is more like Rosenhead’s characteristics

f PSMs in Fig. 1 , that is, it is a blueprint to debate. Third, placing

n approach on the 3 point scale was easier for some questions

han for others. This is particularly true where there is a diversity

f application of an approach and, to overcome this spread (as we

id for system dynamics), it is necessary to reduce the spectrum

f use by tightly specifying what is the approach. However as the

ramework was designed to identify characteristics of PSMs it is

ess relevant how the non-PSMs faired as this is not indicative of

he intended use of the framework.

In terms of future work, this framework could be further tested

n a range of qualitative approaches developed since Rosenhead

nd Mingers published their work in 2001 to identify if they sit

omfortably in the same family as the methods that established

he field of PSMs. The authors also repeat the calls of other re-

earchers cited in this paper for more research and development

f theory that spans across PSMs rather than focussing on a spe-

ific approach.

upplementary materials

Supplementary material associated with this article can be

ound, in the online version, at doi: 10.1016/j.ejor.2018.05.003 .

eferences

ckermann, F. (2012). Problem structuring methods “in the Dock”: Arguing the case

for Soft OR. European Journal of Operational Research, 219 (3), 652–658 . ckermann, F. , Eden, C. , & Brown, I. (2005). The practice of making strategy a

step-by-step guide . London: Sage Publications .

ckoff, R. L. (1979). The future of operational research is past. The Journal of the Operational Research Society, 30 (2), 93–104 .

eer, S. (1981). Brain of the firm (2nd ed.). Chichester: John Wiley and Sons Ltd . ell, S. (2012). DPSIR = a problem structuring method? An exploration from the

“Imagine” approach. European Journal of Operational Research, 222 (2), 350–360 . ryant, J. W. (2002). Confrontations in health service management: Insights from

drama theory. European Journal of Operational Research, 142 , 610–624 .

asu, B. , Shaw, D. , & Thanassoulis, E. (2005). Using a group support system to aid input-output identification in DEA. The Journal of the Operational Research Soci-

ety, 56 (12), 1363–1372 . hampion, D. , & Wilson, J. M. (2010). The impact of contingency factors on vali-

dation of problem structuring methods. The Journal of the Operational Research Society, 61 (9), 1420–1431 .

heckland, P. (1981). Systems thinking, systems practice . Chichester: John Wiley &

Sons . heckland, P. (1983). O. R. and the systems movement : Mappings and conflicts. The

Journal of the Operational Research Society, 34 (8), 661–675 . heckland, P. (1985a). Achieving “desirable and feasible” change: An application of

soft systems methodology. The Journal of the Operational Research Society, 36 (9), 821–831 .

heckland, P. (1985b). From optimizing to learning: A development of systems

thinking for the 1990s. The Journal of the Operational Research Society, 36 (9), 757–767. http://doi.org/10.1057/jors.1985.141 .

heckland, P. , & Scholes, J. (1990). Soft systems methodology in action . Chichester: Wiley .

heckland, P., & Winter, M. (2005). Process and content: Two ways of using SSM. The Journal of the Operational Research Society, 57 (12), 1435–1441. http://doi.org/

10.1057/palgrave.jors.2602118 .

hurchman, C. W. (1967). Wicked problems. Management Science, 14 (4), B141–B142 . avis, J. , MacDonald, A. , & White, L. (2010). Problem-structuring methods and

project management: An example of stakeholder involvement using hierarchical process modelling methodology. The Journal of the Operational Research Society,

61 (6), 893–904 . den, C. (1995). On evaluating the performance of “wide-band” GDSS’s. European

Journal of Operational Research, 81 (2), 302–311 . den, C. , & Ackermann, F. (1998). Making strategy: The journey of strategic manage-

ment . Sage Publications .

den, C. , & Ackermann, F. (2001). SODA- the principles. In J. Rosenhead, & J. Mingers (Eds.), Rational analysis for a problematic world revisited (2nd ed.) (pp. 20–42).

Chichester: Wiley . den, C. , & Ackermann, F. (2006). Where next for problem structuring methods. The

Journal of the Operational Research Society, 57 (7), 766–768 .

Please cite this article as: C.M. Smith, D. Shaw, The characteristics of pr

of Operational Research (2018), https://doi.org/10.1016/j.ejor.2018.05.00

dwards, J. S. , Ababneh, B. , Hall, M. , & Shaw, D. (2009). Knowledge management: A review of the field and of OR’s contribution. The Journal of the Operational

Research Society, 60 , S114–S125 . lood, R. L. , & Zambuni, S. A. (1990). Viable systems diagnosis. 1. Application with a

major tourism services group. Systems Practice, 3 (3), 225–248 . ranco, L. A. (2013). Rethinking soft OR interventions: Models as boundary objects.

European Journal of Operational Research, 231 (13), 720–733 . ranco, L. A. , & Montibeller, G. (2010). Facilitated modelling in operational research.

European Journal of Operational Research, 205 (3), 489–500 .

riend, J. K. (2001). The strategic choice approach. In J. Rosenhead, & J. Mingers (Eds.), Rational analysis for a problematic world revisited (2nd ed.) (pp. 115–149).

Chichester: Wiley . riend, J. K. , & Hickling, A. (2005). Planning under pressure: The strategic choice ap-

proach (3rd ed.). Oxford: Elsevier . oodwin, G. , & Wright, G. (2004). Decision analysis for management judgment (3rd

ed.). Chichester: Wiley .

reasley, A. , & Smith, C. M. (2017). Using activity based costing and simulation to reduce cost at a Police communications centre. Policing: An International Journal

of Police Strategies & Management, 40 (2), 426–441 . uba, E. G. , & Lincoln, Y. S. (1989). Fourth generation evaluation . Newbury Park, CA:

Sage . uba, E. G. , & Lincoln, Y. S. (1994). Competing paradigms in qualitative re-

search. In N. K. Denzin, & Y. S. Lincoln (Eds.), Handbook of qualitative research

(pp. 105–117). Thousand Oaks, California: Sage . uba, E. G. , & Lincoln, Y. S. (2005). Paradigmatic controversies, contradictions, and

emerging confluences. In N. K. Denzin, & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (3rd ed.) (pp. 191–216). Thousand Oaks, California: Sage

Publications . ackson, M. C. (2006). Beyond problem structuring methods: Reinventing the future

of OR/MS. The Journal of the Operational Research Society, 57 (7), 868–878 .

ackson, M. C. , & Keys, P. (1984). Towards a system of systems methodologies. The Journal of the Operational Research Society, 35 (6), 473–486 .

ohnson, P. , & Johnson, G. (2002). Facilitating group cognitive mapping of core competencies. In A. S. Huff, & M. Jenkins (Eds.), Mapping strategic knowledge

(pp. 220–236). London: Sage . eeney, R. L. , & Raiffa, H. (1993). Decisions with multiple objectives: Preferences and

value tradeoffs . Cambridge: Cambridge University Press .

eys, P. (2006). On becoming expert in the use of problems structuring methods. The Journal of the Operational Research Society, 57 (7), 822–829 .

irby, M. W. (20 0 0). Operations research trajectories: The Anglo-American experi- ence from the 1940s to the 1990s. Operations Research, 48 (5), 661–670 .

irby, M. W. (2007). Paradigm change in operations research: Thirty years of debate. Operations Research, 55 (1), 1–13 .

otiadis, K. , & Mingers, J. (2006). Combining PSMs with hard OR methods: The

philosophical and practical challenges. The Journal of the Operational Research Society, 57 (7), 856–867 .

otiadis, K. , & Mingers, J. (2014). Combining problem structuring methods with sim- ulation : The philosophical and practical challenges. In S. Brailsford, L. Churilov,

& B. Dangerfield (Eds.), Discrete-event simulation and system dynamics for man- agement decision making (pp. 52–75). Chichester: John Wiley & Sons Ltd .

ane, D. C. , Munro, E. , & Husemann, E. (2016). Blending systems thinking approaches for organisational analysis: Reviewing child protection in England. European

Journal of Operational Research, 251 , 613–623 .

i, Y. , & Zhu, Z. (2014). Soft OR in China: A critical report. European Journal of Oper- ational Research, 232 (3), 427–434 .

idgley, G. , Cavana, R. Y. , Brocklesby, J. , Foote, J. L. , Wood, D. R. R. , & Ahuriri– Driscoll, A. (2013). Towards a new framework for evaluating systemic problem

structuring methods. European Journal of Operational Research, 229 (1), 143–154 . ingers, J. (1992). Recent developments in critical management science. The Journal

of the Operational Research Society, 43 (1), 1–10 .

ingers, J. (20 0 0). Variety is the spice of life: Combining soft and hard OR/MS methods. International Transactions in Operational Research, 7 (6), 673–691 .

ingers, J. (2003). A classification of the philosophical assumptions of management science methods. The Journal of the Operational Research Society, 54 (6), 559–570 .

ingers, J. (2011). Soft OR comes of age—but not everywhere!. Omega, 39 (6), 729–741 .

ingers, J. , & Brocklesby, J. (1997). Multimethodology: Towards a framework for

mixing methodologies. Omega, 25 (5), 489–509 . ingers, J. , & Rosenhead, J. (2001). An overview of related methods. In J. Rosenhead,

& J. Mingers (Eds.), Rational analysis for a problematic world revisited (2nd ed.) (pp. 267–288). Chichester: John Wiley and Sons Ltd .

ingers, J. , & White, L. (2010). A review of the recent contribution of systems think- ing to operational research and management science. European Journal of Oper-

ational Research, 207 (3), 1147–1161 .

unro, I. , & Mingers, J. (2002). The use of multimethodology in practice—results of a survey of practitioners. The Journal of the Operational Research Society, 53 (4),

369–378 . ’Brien, F. , & Meadows, M. (2006). Developing a visioning methodology: Visioning

choices for the future of operational research. The Journal of the Operational Re- search Society, 58 (5), 557–575 .

hillips, L. D. , & Phillips, M. C. (1993). Facilitated work groups: Theory and practice.

The Journal of the Operational Research Society, 44 (6), 533–549 . idd, M. (1998). Computer simulation in management science (4th ed.). Chichester,

UK: Wiley . idd, M. (2009). Tools for thinking: Modelling in management science (3rd ed.). Chich-

ester: Wiley .

oblem structuring methods: A literature review, European Journal

3

14 C.M. Smith, D. Shaw / European Journal of Operational Research 0 0 0 (2018) 1–14

ARTICLE IN PRESS JID: EOR [m5G; May 28, 2018;10:55 ]

S

S

S

T

W

Preece, G. , Shaw, D. , & Hayashi, H. (2013). Using the Viable System Model (VSM) to structure information processing complexity in disaster response. European

Journal of Operational Research, 224 (1), 209–218 . Ranyard, J. C. , Fildes, R. , & Hu, T. (2015). Reassessing the scope of OR practice: The

influences of problem structuring methods and the analytics movement. Euro- pean Journal of Operational Research, 245 (1), 1–13 .

Rittel, H. (1972). On the planning crisis: Systems analysis of the “first and second gen- erations”.

Robinson, S. (2001). Soft with a hard centre : Discrete-event simulation in facilita-

tion. The Journal of the Operational Research Society, 52 (8), 905–915 . Robinson, S. (2007). PSMs: Looking in from the outside. Journal of the Operational

Research Society, 58 (5), 689–691 . Rosenhead, J. (1980). Planning under uncertainty: II. A methodology for robustness

analysis. Journal of the Operational Research Society, 31 (4), 331–341 . Rosenhead, J. (1989). Rational analysis for a problematic world . In J. Rosenhead (Ed.).

Chichester: John Wiley & Sons .

Rosenhead, J. (1996). What’s the problem? An introduction to problem structuring methods. Interfaces, 26 (6), 117–131 .

Rosenhead, J. (2006). Past, present and future of problem structuring. The Journal of the Operational Research Society, 57 (7), 759–765 .

Rosenhead, J. , & Mingers, J. (2001a). A new paradigm of analysis. In J. Rosenhead, & J. Mingers (Eds.), Rational analysis for a problematic world (2nd ed.) (pp. 1–20).

Chichester: Wiley .

Rosenhead, J. , & Mingers, J. (2001b). Rational analysis for a problematic world revis- ited . In J. Rosenhead, & J. Mingers (Eds.). Chichester: Wiley .

Shaw, D. (2003). Evaluating electronic workshops through analysing the brain- stormed ideas. The Journal of the Operational Research Society, 54 , 692–705 .

Shaw, D. (2006). Journey making group workshops as a research tool. The Journal of the Operational Research Society, 57 (7), 830–841 .

Shaw, D. , & Blundell, N. (2010). WASAN: The development of a facilitated method-

ology for structuring a waste minimisation problem. European Journal of Opera- tional Research, 207 (1), 350–362 .

Please cite this article as: C.M. Smith, D. Shaw, The characteristics of pr

of Operational Research (2018), https://doi.org/10.1016/j.ejor.2018.05.00

haw, D. , Eden, C. , & Ackermann, F. (2009). Mapping causal knowledge: How man- agers consider their environment during meetings. International Journal of Man-

agement and Decision Making, 10 (5/6), 321–340 . Shaw, D. , Smith, C. M. , & Scully, J. (2017). Why did Brexit happen ? Using causal

mapping to analyse secondary, longitudinal data. European Journal of Operational Research, 263 (3), 1019–1032 .

impson, M. G. (1978). Those who can’t ? The Journal of the Operational Research Society, 29 (6), 517–522 .

Sinn, J. S. (1998). A comparison of interactive planning and soft systems methodol-

ogy : Enhancing the complementarist position. Systemic Practice and Action Re- search, 11 (4), 435–453 .

mith, C. M. , & Shaw, D. (2018). Horizontal recursion in soft OR. Journal of the Op- erational Research Society, 5682 , 1–14 .

Strauss, A. L. , & Corbin, J. M. (1990). Basics of qualitative research: Grounded theory in practice . Thousand Oaks, California: Sage .

avella, E. , & Papadopoulos, T. (2015). Expert and novice facilitated modelling: A

case of a viable system model workshop in a local food network. Journal of the Operational Research Society, 66 (2), 247–264 .

Thompson, J. P. , Howick, S. , & Belton, V. (2016). Critical Learning Incidents in sys- tem dynamics modelling engagements. European Journal of Operational Research,

249 (3), 945–958 . Thunhurst, C. (1973). Who does operational research operate for. In Proceedings of

the annual conference of operational research society Torquay, UK .

Torres, J. P. , Kunc, M. , & O’Brien, F. (2017). Supporting strategy using system dynam- ics. European Journal of Operational Research, 260 (3), 1081–1094 .

Westcombe, M. , Franco, L. A. , & Shaw, D. (2006). Where Next for PSMs: A grassroots revolution? The Journal of the Operational Research Society, 57 (7), 776–778 .

White, L. (2009). Understanding problem structuring methods interventions. Euro- pean Journal of Operational Research, 199 (3), 823–833 .

illiams, T. (2008). Management science in practice . Chichester: Wiley .

Yearworth, M. , & White, L. (2014). The non-codified use of problem structuring methods and the need for generic constitutive definition. European Journal of

Operational Research, 237 (5), 932–945 .

oblem structuring methods: A literature review, European Journal

3