Application 1 – Analysis and Synthesis of Prior Research

INTERNATIONAL JOURNAL OF

www.elsevier.com/locate/ijproman

International Journal of Project Management 23 (2005) 524–537

PROJECT MANAGEMENT

The impact of project portfolio management on information technology projects

Bert De Reyck a,*, Yael Grushka-Cockayne

a,1 , Martin Lockett

c,2 ,

Sergio Ricardo Calderini a , Marcio Moura

a , Andrew Sloper

b,2

a London Business School, Regent’s Park, London NW1 4SA, United Kingdom

b CVC The Customer Value Company, 48 St. Mary’s Road, Long Ditton, Surrey KT6 5EY, United Kingdom

c Ashridge, Berkhamsted, Hertfordshire HP4 1NS, United Kingdom

Received 27 July 2004; received in revised form 17 September 2004; accepted 25 February 2005

Abstract

The ever-increasing penetration of projects as a way to organise work in many organisations necessitates effective management of

multiple projects. This has resulted in a greater interest in the processes of project portfolio management (PPM), with more and

more software tools being developed to assist and automate the process. Much of the early work on PPM concentrated on the man-

agement of IT projects, largely from the perspective of the management of resources and risk. Many of the recent articles have been

by vendors of the software, promoting the value of the PPM process. However, the claims made in those articles are typically only

supported by anecdotal evidence. In this paper, we assess whether there is a correspondence between the use of PPM processes and

techniques, and improvements in the performance of projects and portfolios of projects. Based on our findings, we introduce a three-

stage classification scheme of PPM adoption, and present a strong correlation between (1) increasing adoption of PPM processes

and a reduction in project related problems, and (2) between PPM adoption and project performance.

� 2005 Elsevier Ltd and IPMA. All rights reserved.

Keywords: Project portfolio management; Managing programmes; Process, procedures; Information technology

1. Introduction

Recently, information technology (IT) has moved be-

yond the implementation of IT applications to an age of

IT-enabled change. The trend towards increasing use of

IT continues and the challenge remains how to better

0263-7863/$30.00 � 2005 Elsevier Ltd and IPMA. All rights reserved. doi:10.1016/j.ijproman.2005.02.003

* Corresponding author. Tel.: +44 20 7706 6884; fax: +44 20 7724

7875.

E-mail addresses: [email protected] (B.D. Reyck), ygrushka.

[email protected] (Y. Grushka-Cockayne), [email protected]

(M. Lockett), [email protected] (S.R. Calderini),

[email protected] (M. Moura), andrew.sloper@custo-

mervalue.co.uk (A. Sloper). 1 Fax: +44 20 7724 7875.

2 Fax: +44 1582 768409.

manage IT projects in order to maximise their economic

benefits. Part of that challenge can be tackled by ‘‘doing

projects right’’ and part by ‘‘doing the right projects’’

[1]. While Project Management concentrates primarily

on the former, Project Portfolio Management, hereafter

referred to as PPM, is focussed on the latter. Contrary

to Project Management, which focuses on single project, and Programme Management, which concerns the man-

agement of a set of projects that are related by sharing a

common objective or client, or that are related through

interdependencies or common resources, PPM considers

the entire portfolio of projects a company is engaged in,

in order to make decisions in terms of which projects are

to be given priority, and which projects are to be added

to or removed from the portfolio (see also Lycett et al. [2]).

B.D. Reyck et al. / International Journal of Project Management 23 (2005) 524–537 525

PPM has largely developed around the following ele-

ments: providing a centralised view of all the projects in

an organisation, enabling a financial and risk analysis of

projects, modelling interdependencies between a family

of projects, incorporating constraints on resources

shared between projects, enabling prioritisation and selection of projects, ensuring accountability and gover-

nance at the portfolio level, allowing for portfolio opti-

misation and providing support in the form of

standardised processes and software tools.

However, despite the relatively extensive literature on

PPM (see Sections 2 and 3), evidence of its value has

been rather anecdotal. It is unclear whether there are

specific PPM elements that add more value than others or indeed, whether they add value at all. It is for these

reasons that we decided to investigate the potential for

increasing business value through the application of

PPM techniques to IT projects.

The first contribution of this paper is the develop-

ment of a classification scheme for the adoption level

of PPM across a diversity of organizations. Secondly,

we identify the impact of the PPM adoption level on project performance by investigating the correspon-

dence between the adoption level and reported project-

related problems on the one hand and observed positive

elements in projects on the other. Finally, we suggest a

phased implementation process for the adoption of

PPM and describe the challenges that organisations

might face in each phase.

The paper is organized as follows. Sections 2 and 3 contain a literature review of the theories, models and

processes presented for PPM, reviewed according to a

historic and a thematic perspective. The historic per-

spective provides a view of how the field has developed

over time, while the thematic perspective summarizes

the main themes identified in the literature. Section 4

describes the objectives and hypotheses of this study,

as well as the methodology used, with the general re- sults presented in Section 5. In Section 6, we present

a classification for adoption levels of PPM and in Sec-

tion 7 we investigate the impact of PPM and project

performance, highlighting the managerial implications

of this analysis. In Section 8, we provide a phased

implementation plan. Section 9 contains a summary

and our conclusions.

2. Literature review: a historic perspective

The field of portfolio management owes its origins to a

seminal paper written in 1952, in which Harry Markowitz

[3] laid down the basis for the Modern Portfolio Theory

(MPT). MPT allows to determine the specific mix of

investments generating the highest return for a given level of risk. Whereas MPT was initially developed for finan-

cial investments, in 1981, McFarlan [4] provided the basis

for the modern field of PPM for IT projects. According to

McFarlan, management should also employ a risk-based

approach to the selection and management of IT project

portfolios. He observed that risk-unbalanced portfolios

could lead an organization to suffer operational disrup-

tions, or leave gaps for competitors to step in. In 1992, Wheelwright and Clark [5] developed a

framework for categorising projects that they called

the Aggregate Project Plan. This plan allows for an

overview of the project portfolio along two dimensions,

(1) the extent of changes made to the product, and (2)

the degree of process change, leading to four categories

of projects (in increasing order of change): derivative

projects, platform projects, breakthrough projects and R&D projects (for complete definitions see [5]). This

framework can be used to identify gaps in the portfolio,

or potential resource shortages.

In the mid-1990s, the field of PPM received increasing

attention. In 1994, a GAO report [6] described a success-

ful company that used portfolio investment techniques

to manage its IT projects. The organisation developed

a set of criteria to evaluate benefits, costs and risks and thus determined the best mix of projects for obtain-

ing a better balance between maintenance and strategic

initiatives. As a result, in three years, the organization

reported a 14-fold increase in the return on investment

from IT projects.

In 1998, Thorp published the ‘‘Information Paradox’’

[7], putting PPM in a broader framework called ‘‘Benefits

Realization’’. According to the author, PPM techniques are fundamental for getting value from IT projects.

In a recent publication, Jeffery and Leliveld [8] report

the results of a survey with 130 senior executives, 90% of

whom were CIOs. The survey identified, among other

things, that 25% of the respondents could be defined

as optimally applying Information Technology Portfolio

Management (ITPM), 45% as having or adopting it and

78% as planning to have or to keep it.

3. Literature review: a thematic perspective

In this section, we review the main themes around

which research on PPM has been developed, namely

the PPM objectives and scope, the pre-conditions for

PPM, the key elements of PPM, the impact on organisa- tions and the problems within organizations associated

with a lack of PPM processes.

PPM objectives and scope. The majority of the litera-

ture in the PPM field provides similar lists of objectives

to be achieved through the adoption of PPM

approaches. Five main goals dominate the literature

[9–11], namely (1) defining goals and objectives, i.e.,

clearly articulating what the portfolio is expected to achieve, (2) understanding, accepting, and making

trade-offs, (3) identifying, eliminating, minimizing and

526 B.D. Reyck et al. / International Journal of Project Management 23 (2005) 524–537

diversifying risk, (4) monitoring portfolio performance,

i.e., understanding the progress that portfolio is making

towards the achievement of the goals and objectives,

and (5) establishing confidence in achieving a desired

objective.

Pre-conditions for PPM. Several papers discuss the pre-conditions that organisations should take into ac-

count, when adopting PPM approaches:

� Organisational strategy: Organisations should have clear strategic imperatives in place, properly commu-

nicated across all departments, to which the PPM

goals are to be aligned to [2]. Matheson and Mathe-

son [11], for example, describe a firm that designed a task force to develop a strategy for its lacklustre

R&D Portfolio, but soon found this to be in vain

since the company lacked a business strategy in the

first place.

� Business leaders� involvement: Another pre-condition for the adoption of PPM approaches is the involve-

ment of top executives who should be able to take a

less siloed view of the portfolio. As Kendall and Rollins [12] highlight, without the full understanding

and support of top executives the constant fight over

resources and reprioritisations will never be

resolved.

� Team skills: Another relevant aspect is the impor- tance of having a project team with relevant finance

and strategy skills. Although most IT professionals

have sufficient knowledge to calculate the net pres- ent value (NPV) or return on investment (ROI) of

a project, as Jeffery and Leliveld [8] noticed, to

build a proper business case for a project, the team

must understand the assumptions behind those cal-

culations, analyse the sensitivity of these results and

evaluate the risks that might impact project returns.

Key elements in PPM. Other papers discuss the key elements that constitute PPM:

� Centralised view of the project portfolio: Widely emphasised in the literature is the need for a centra-

lised view of the organisation�s projects [12]. The first step in the adoption of PPM approaches requires the

preparation of an inventory of current and proposed

projects, preferably through a central area responsi- ble for collecting, analysing and distributing project

information in a common format.

� Financial Analysis: While finance professionals have worked for decades with metrics to capture return

and risk of assets, the use of these methods for IT

projects still seems uncommon for many organisa-

tions [8]. Nonetheless, several techniques have been

created to properly measure the financial value of projects (see Benaroch [13], and McGrath and

MacMillan [14]). Most importantly, however, one

should choose a valuation methodology, be it return

on investment (ROI), internal rate of return (IRR),

net present value (NPV) or economic value added

(EVA), and consistently apply it.

� Risk analysis: McFarlan, [4, p. 142], noted that two of the main reasons for project failure were ‘‘the failure to assess individual project risk and the failure to con-

sider the aggregate risk of the portfolio of projects’’.

In addition, as observed by Markowitz [3], a portfolio

should not be chosen considering only individual

characteristics of the investments, but it should be

built based on the overall risk and reward of the port-

folio. When investment interactions are considered,

one can create portfolios with the same expected return but lower risk than when not taking into

account the interactions.

� Interdependencies: Thorp [7] noticed that one advan- tage of PPM is its ability to reduce inter-program

competition for resources and to turn program over-

laps into productive interdependencies. He identified

the following types of interdependencies: sequential

dependencies, overlapping outcomes, competition for scarce resources, and change bottlenecks.

� Prioritisation, alignment and selection: The selection of projects to compose a portfolio should ensure that

all areas of the organisation�s strategy are properly addressed and that the portfolio is well balanced

[15]. When properly combining portfolio alignment

and balance, organisations should come up with a

very clear picture of which projects should be cut off and which ones should be funded.

� Constraints: Goldman [15] highlights that incorporat- ing constraints is a key step in the portfolio alignment

process. Four types of constraints should be man-

aged: scarce human resources, staff capabilities, bud-

gets and infrastructure.

� Dynamic re-assessment of the portfolio: Jeffery and Leliveld [8] mention that only 26% of the respondents in their survey track financial measures after an

investment is made. As a result, managers ignore

options embedded in the portfolio, which would have

allowed them either to abandon unprofitable projects

before further investments are made, or to expand

successful investments.

� Need for specialized software: The need for special- ized software for PPM is a controversial issue in the literature. Some believe that there is no need

at all, whereas others claim that besides working

as a process change catalyst, specialised software

is indispensable due to the time consuming process

of updating all information needed for the decision

making process. Due to the growing number of

available software tools, several reports have been

issued that evaluate the best software available in the market according to several dimensions (see

e.g. [16]).

B.D. Reyck et al. / International Journal of Project Management 23 (2005) 524–537 527

Impact on organisations. Datz [17] provides a sum-

mary of the main benefits organisations should expect

from adopting PPM approaches. These include: (1)

maximizing the value of IT investments while minimiz- ing risk, (2) improving communication and alignment

between IT and business leaders, and (3) encouraging

business leaders to act as team players, allowing plan-

ners to allocate resources more efficiently and to termi-

nate projects.

Problems within organisations. Kendall and Rollins

[12] list four generic problems associated with a lack

of PPM processes and tools: (1) too many active pro- jects, (2) projects that do not add value, (3) projects

not linked to strategic goals, and (4) an unbalanced

portfolio. Other typical problems [18] include a lack

of coordination between projects, conflicting project

objectives, unexpected resource bottlenecks, late deliv-

ery of projects, lack of commitment from business

leaders, lack of cross-functional working, disappoint-

ment with final project benefits, and resistance to organisational change.

4. Methodology and hypotheses

There seems to be an agreement among experts that

organisations are in different stages of the adoption of

PPM approaches, even when organisations do not

explicitly and formally adopt them. Berinato [19] be-

lieves that there are 5 levels of PPM adoption from

the simplest to the more complex:

1. Put all projects in one database.

2. Prioritise the projects in the database.

3. Divide the projects into two or three budgets based

on type of investment.

4. Automate the repository.

5. Apply Modern Portfolio Theory.

Yet, according to the same author, benefits could be achieved at every level. At level 1, for example, the pro-

ject overview allows spotting redundancies. At level 2,

prioritization allows for an improvement in the relation-

ship between business leaders and IT people, since the

projects are seen as investments with economic value.

At level 3, by separating projects by type, organisations

can apply a more appropriate set of criteria to each type

of investments, facilitating the prioritisation and selec- tion process. At level 4, one of the main benefits is to en-

sure that information will be updated when needed

without people spending too much time collecting it. Fi-

nally, level 5 provides a better balance between risk and

reward.

Jeffery and Leliveld [8] propose a similar framework

comprised of three stages:

1. Defined: Organisations in this stage have defined

and documented the key components of the IT

portfolio and have high-level estimates of costs

and benefits for each element. The IT depart-

ment prioritises projects and has central budget

oversight. 2. Managed: Organisations in this stage have periodic

portfolio reviews with quantified and financial invest-

ment measures. Project data is codified and logged in

a central database. New initiatives are screened, cate-

gorised and prioritised.

3. Optimised: Organisation in this stage distinguish

themselves by the ability to balance and optimise

their IT project portfolio. They are disciplined in get- ting feedback from business leaders and use financial

valuation techniques such as Real Option Analysis

(for full definition see [20] and [21]).

Based on a survey, we developed a new framework

of PPM adoption and identified groups of organisa-

tions at different stages of the adoption of PPM. We

also correlated the PPM adoption level to the benefits the organisations perceived and to the level of prob-

lems they face in managing their projects. We used

an approach similar to the one adopted by Ibbs and

Regato [22], who showed that there is a positive rela-

tion between the level of adoption of project manage-

ment techniques and improvements in cost and

schedule performance in projects. We extended this

methodology to PPM. The aim of the research is to investigate the extent to which organisations view

their internal projects as discrete projects or as a

coherent portfolio of investments, and the value they

get from taking this perspective. We examined two

hypotheses:

Hypothesis 1. The adoption level of PPM processes and

techniques varies across organisations, allowing for

classification of organisations according to their level

of adoption. In addition, the PPM processes and

techniques adopted by organisations having a higher

adoption level are supersets in that they comprise the

processes and techniques also adopted by organisations

in lower categories, but also include more enhanced processes and methods.

Hypothesis 2. Higher adoption levels of PPM methods

and techniques result in fewer project related problems,

and increased value gained from information technology

projects.

In order to test the hypotheses stated above we de-

signed a survey with 5 sections. Section 1 collects

demographic information from the respondents. Sec-

tion 2 is designed to solicit information about the

PPM adoption level of each organisation, and is devel-

oped around the main elements that are part of the

528 B.D. Reyck et al. / International Journal of Project Management 23 (2005) 524–537

PPM approach, namely a centralised view of projects,

financial analysis, risk analysis, interdependencies,

constraints, overall analysis at portfolio level, prioriti-

sation, selection, accountability and governance, opti-

misation and specialized software. The majority of

the questions in this section were closed questions with a scale from 1 (‘‘don�t have any or don�t plan to have’’) to 5 (‘‘Always use’’). Section 3 provides respondents

with a set of issues commonly identified in the litera-

ture as associated to the lack of PPM. The scale

adopted is the same as Section 2 (1–5 scale with 1

being ‘‘not at all’’ and 5 being ‘‘to a great extent’’).

Section 4 provides respondents with a list of PPM ap-

proaches with a question whether they had imple- mented them and if yes, what had been the impact in

their organisation. The scale used in this section to rep-

resent a negative vs. positive impact is from �2 (‘‘sig- nificant negative impact’’) to +2 (‘‘Significant positive

impact’’), with two extra options ‘‘do not use’’ and

‘‘use but too early to assess’’. The questions in Sections

3 and 4 serve as proxies for the return on investment of

the projects in the IT portfolio, where we expect to see a negative correlation between the PPM adoption level

and the problems experienced, and a positive correla-

tion between the adoption level and the positive effects

observed. Finally, Section 5 solicits the challenges that

companies faced in the implementation of PPM. Here

a 1–5 scale is used, 1 representing ‘‘not at all’’ and 5

‘‘to a great extent’’. Note that since the survey is self-

assessment based, the results are subjective, which may affect the validity of the analysis and conclusions.

See Appendix A for a small sample of the survey ques-

tions. The full survey is available upon request.

An online survey was sent to 125 companies, either

through existing contacts or via existing agreements 3 .

Of these, we received 31 responses, a response rate of

25%. In addition, a mass mailing of 350 emails was

sent to IT departments within public organisations. In this case the response rate was disappointing, since

only 3 responses came from this source, a 1% response

rate, probably due to the fact that the survey never

reached an appropriate person in the organisation.

Our results therefore focus on the sample obtained

from the 125 companies with which we already had

contacts.

The sample includes medium and big organisations with turnover varying from less than £100 million

(19%) to more than £15 billion (4%). The number of

employees varies from less than 1500 (21%) to more

than 100,000 (6%). The majority of the respondents

are from the IT sector (55%), with others from Business

3 Existing agreements include the Impact Programme and Prog-M

branch of the Association for Project Management. For details of their

activities see www.impact-sharing.com and www.apm.org.uk

respectively.

Operations (15%), General Management (15%), Strat-

egy (12%) and Finance (3%). Geographically, the

respondents are predominantly from the UK (40%),

with Europe accounting for 15% and the rest of the

world for 45%.

Using the terminology of Turner and Cochrane [23], we can classify the majority of projects these firms are

engaged in as either type 2 or type 3 projects.

5. Adoption of PPM processes: key results

Our main findings in terms of PPM adoption include

the following: Centralised view. The vast majority of the respon-

dents have a centralised view of its projects and have a

central point responsible for collecting, analysing and

distributing information:

� 93% have an inventory of current and proposed sig- nificant projects and 100% of the organisations that

do not have it plan to have it in the future; � 79% claim that they have a central point responsible for collecting, analysing and distributing information

regarding the portfolio of projects and 71% of the

organisations that do not have one plan to have in

the future.

Financial analysis. Our research identified that almost

all respondents use at least one technique to calculate the financial worth of its projects, with Payback being

the most common (93% ) and EVA the least common

(31%).

Risk analysis. The results of our survey show that the

complexity of projects and technological risks are the

main types of risks monitored by most organisations:

� Approximately 92% evaluate the complexity of pro- jects, including technology risks, cash flow and

organisational changes.

� Approximately 80% evaluate team expertise, market and environmental risks and management

commitment.

� 83% who do not evaluate team expertise intend to do so in the future and 80% who do not evaluate envi-

ronmental risks intend to do so in the future.

Interdependencies. We found that 84% of the respon-

dents take into account project interdependencies. 85%

look at cross-project dependencies while 82% considers

implementation bottlenecks.

Constraints at portfolio level. Among the firms sur-

veyed, budget and financial constraints are widely and

frequently taken into account, 91% of the respondents consider them, including 83% who observe them fre-

quently or always. We also noted that 100% of the

Table 1

PPM elements that organisation plan to implement

PPM elements Already use (%) Plan to use (%)

Centralised view Have and inventory of current and proposed significant projects 93 7

Financial analysis Use payback 93 0

Use ROI 85 3

Use NPV 68 16

Use IRR 65 18

Use of economic value added (EVA) 31 28

Use real options 37 14

Risk analysis Evaluate complexity of the project, including technology risks, cash

flow and organisational changes

92 8

Evaluate team expertise, market and environmental risks and

management commitment

80 17

Management of overall risk analysis of project portfolio 62 35

Interdependencies Take into account project interdependencies, divided occasionally,

frequently or always in similar proportions

84 N/A

Considers implementation bottlenecks 82 N/A

Constraints at portfolio level Budget and financial constraints are taken into account 91 9

Evaluation of staff capabilities to implement projects 74 26

Overall analysis Management of project diversification 33 33

Management of risk vs. reward analysis of project portfolio 59 32

Management of the financial analysis of project portfolio 74 21

Categorisation, selection,

accountability and governance

Use categorisation to balance project mix; 47 N/A

Align the project portfolio with organisation�s strategy and with organisation�s IT architecture

82 N/A

Have top management involved 91 N/A

Have business leaders accountable; 85 N/A

Use of regular project portfolio reporting 79 21

Optimisation Centrally tracking of the benefits of projects 50 41

Comparison of outcome of projects with their original targets 68 32

Analysis of the impact of individual new projects to the overall

portfolio

62 35

Annually (or more frequently) prioritisation of the overall project

portfolio

76 21

Regularly review and revision of the project portfolio 71 24

Specialized software Use of specialized software to manage portfolio of projects 29 47

B.D. Reyck et al. / International Journal of Project Management 23 (2005) 524–537 529

respondents who do not yet consider these constraints,

plan to do so in the future.

Overall analysis at portfolio level. Our research identi-

fied that an overall analysis at the portfolio level is not

widely implemented. Only 33% confirm examining

diversification to reduce portfolio risk, while 59% ana-

lyse risk-reward at the portfolio level; Categorisation, selection accountability and gover-

nance. The results show that the categorisation of pro-

jects does not seem to be a common practice among

respondents; 47% use categorisation to balance project

mix. On the other hand, accountability and strategic

alignment seems to be high and increasing; about

82% align the project portfolio with the organisation�s strategy and IT architecture, and consider the cus- tomer impact of project portfolio results.

Specialized software. Only 30% of the respondents use

specialized software to manage their portfolio of pro-

jects; of those who do not use any, 67% intend to use

in the future.

The survey results for Section 2 of the survey (PPM

adoption) are summarized in Table 1.

6. A classification on the adoption of project portfolio management

We grouped the organisations with different levels

of PPM adoption using cluster analysis, a multivariate

statistical methodology that classifies elements into rel-

atively homogenous groups, minimizing the variance

in use of PPM approaches within the groups and

maximizing the variance across the groups. Using SPSS, a k-means cluster analysis with Ward�s method [24] was performed. As the solution depends on the

Table 2

Adoption level analysis per stage

PPM elements Stage I Stage II Stage III

Centralisation of

project control

Most organisations have an

inventory of projects and

frequently use it, but project

control is largely decentralised

Frequently have a central point

responsible for collecting, analysing

and distributing project information

in a common format

Almost always have a central point

to control projects, which is

extensively used

Financial analysis Some financial analysis is

undertaken with special attention

to Payback Period and ROI

Financial tools are frequently used to

evaluate projects. The most utilised

are ROI, Payback period, NPV and

IRR

The financial analysis is always

done. Several tools are frequently

used

Risk analysis Occasionally risks are evaluated.

In most cases the attentions is in

financing/cash flow risks

All aspects of risks are to some extent

considered. Most of the focus is in the

complexity of the project and

technology risk

An extensive risk analysis is

performed. Attention is devoted to

project complexity, technological

risks, team experience and cash

flow risks

Interdependencies Some consideration of overlaps

and duplication of project results

Cross-project dependencies and

implementation bottlenecks are

frequently considered

Interdependencies are frequently

managed. Significant attention

given to cross-project dependencies

Constraints Little constraint analysis. The

only exception is the control of

the budget/financial capacity

Frequently evaluate budget/financial

capacity and competition for scarce

resources. Other constraints, such as,

staff capabilities to implement

projects are occasionally evaluated

Budget/financial capacity

constraints are always evaluated.

Other aspects such as staff

capabilities and competition for

scarce resources are frequently

managed

Overall portfolio

analysis

Very little analysis at the

portfolio level

More concern about portfolio

analysis. The portfolio financial

analysis is frequently managed and

risk vs. reward evaluation is

occasionally undertaken

Almost always the portfolio is

evaluated in terms of overall risk

and financial value. Frequently

portfolio diversification is

considered

Categorisation,

selection,

accountability

and

governance

Occasionally have top

management involved in project

selection. Also some effort is

spent in aligning the project

portfolio to the organisation�s strategy

More attention is devoted to the

project portfolio alignment to

strategy and IT structure. Also, there

is frequent involvement of top

management in the project selection

process

Significant alignment of the

portfolio to the organisation�s strategy. Systematic review of

projects at specific stages. Top

management frequently involved in

the project selection process and

business leaders are accountable

for project results

Optimisation Very few processes to optimise

the portfolio are in place. Some

effort is spent in generating

regular project portfolio

reporting

Frequently have regular project

portfolio reporting and annually, or

more frequently, the overall project

portfolio is prioritised

In general, processes to optimise

the portfolio are frequently

applied. Project outcomes are

always compared with the original

targets and project benefits are

frequently centrally tracked

Specialised software Not used Occasionally use specialised software

to manage the project portfolio

Use is more frequent than in the

Stage II, but it is still occasionally

employed

Overall adoption

level a

1.63 2.59 3.52

a Overall adoption level is calculated as the average response level whereby the scale is: 1, do not use; 2, use occasionally; 3, frequently use; 4, always

use.

4 The level of adoption of PPM in this classification refers to current

situation. Hence, survey answers ‘‘do not use and do not plan to use’’

and ‘‘do not use but plan to use’’ represent the same current level ‘‘do

not use’’.

530 B.D. Reyck et al. / International Journal of Project Management 23 (2005) 524–537

initial partition of the data, the analysis was repeated

several times with different starting values. The ANO-

VA analysis rejected the hypothesis, at a 95% confi-

dence level, that there was no significant difference

among the cluster-specific means. The best quantita-

tive and qualitative results were attained splitting

our sample in three groups. Within each cluster, the

adoption was measured using a scale from 1 (‘‘do

not use’’) to 4 (‘‘always use’’) 4 . The level of adoption

Stage III: 0.58

Stage II:0.28

Stage I:0.13

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

1.00 1.50 2.00 2.50 3.00 3.50 4.00

Adoption Level

O rg

a n

is a ti

o n

i m

p a c t

le v e l

Fig. 1. Impact per stage of PPM adoption. (I) Adoption level scale: (1)

do not use; (2) use occasionally; (3) frequently use; (4) always use. (II)

Organisation impact level scale: �1, significant negative impact; 0, no impact; +1, significant positive impact.

Stage I: 0.80

Stage II: 0.70

Stage III: 0.53 0.50

0.55

0.60

0.65

0.70

0.75

0.80

0.85

1.00 1.50 2.00 2.50 3.00 3.50 4.00

Adoption Level

P ro

je c t

p ro

b le

m l e v e l

Fig. 2. Level of project problems per stage of PPM adoption. (I)

Adoption level scale: (1) Do not use; (2) Use occasionally; (3)

Frequently use; (4) Always use. (II) Project problem level scale: (0) Do

not have problems; (1) Have problems to a great extent.

B.D. Reyck et al. / International Journal of Project Management 23 (2005) 524–537 531

of PPM elements for each cluster was quantified by

averaging the respondents� scores to all questions. See Appendix B for the detailed scores. After having

defined the groups and quantified their level of use

of PPM techniques, we studied each group in order

to understand how they differ from each other. The

conclusions appear in Table 2 below.

7. The impact of project portfolio management

Once groups were identified and scores quantified, an

analysis of the impact of different levels of adoption of

PPM in organisations was performed.

Level of adoption vs. organisational impact. To verify

that the incremental use of PPM methods has a positive impact in organisations, we studied each groups� an- swers to the questions in the survey�s fourth section and compared them to groups� average levels of adop- tion of PPM previously identified in the cluster analysis.

Fig. 1 clearly demonstrates a significant positive rela-

tionship between the level of adoption and the impact

generated.

Four aspects appeared to most organisations as factors of greatest positive impact. First, the return

of what should be the first step of any PPM imple-

mentation was extremely valued by almost all respon-

dents: 90% of the managers who maintain an

inventory of projects claimed that the practice has

been responsible for a positive impact. Second, 88%

of the respondents who align the project portfolio to

a clear statement of the organisation�s objective said that this process has positively impacted their organi-

sation. Among those 52% noted a significant positive

impact. Third, the consolidation of information about

projects and the standardization of project analysis

improved the performance of 89% of the respondents

that implemented it. Fourth and finally, among the

organisations that explicitly considered project interde-

pendencies, 86% of them stressed the positive return

obtained by the process, with 43% claiming significant

success.

Level of adoption vs. level of problems with projects. We also identified the relationship between the use of

PPM processes and the level of problems associated

with the lack of PPM approaches in organisations. Sec-

tion 3 of the survey was used to calculate the groups� averages level of problems and again a strong relation-

ship, this time negative, between the use of PPM tech-

niques and level of problems was identified, as can be

seen in Fig. 2. The analysis of the three clusters identified in our

study showed that the groups were not only different in

terms of number of PPM elements adopted but also in

the intensity in which those were used. Also, the study

of how the use of PPM has impacted organisations and

changed the level of project related problems allows

answering our second hypothesis. First, the results show

that organisations, in general, see projects collectively and apply portfolio management processes. However,

they adopt these methods with different intensities.

Organisations at stage I, for example, scored 1.22 in

the survey sub-section related to analysis at portfolio le-

vel while respondents at stages II and III scored 2.03 and

3.37, respectively. Second, it is clear that the return from

PPM methods is reduced when some PPM approaches

are not adopted. As Fig. 1 shows, organisations at stage I obtained less impact from the implementation of PPM

elements (0.13 in a scale from �1 to 1), than stage II (0.28) or stage III (0.58) organisations. This means that

as organisations increasingly adopt PPM approaches,

the impact is strengthened. Third, the use of PPM tech-

niques not only positively impacts organisations but also

reduces the level of project related problems. As Fig. 2

shows, organisations at stage I found a significantly higher (0.80) level of problems than stage II (0.70) and

stage III (0.53) organisations. Fig. 3 provides an addi-

tional perspective on the difference between the stages

by mapping the main project issues, and their degree of

severity, to the different adoption stages.

0.0

1.0

2.0

3.0

4.0

5.0 Too many projects

Lack of coordination between projects

Lack of alignment of projects to strategy

Conflicting project objectives

People constraints

Financial constraintsLate delivery of projects

Lack of commitment from business leaders

Lack of cross-functional working

Disappointment with final project benefits

Resistance to organisational change

Stage Stage Stage I II III

Fig. 3. Map of project issues per stage.

532 B.D. Reyck et al. / International Journal of Project Management 23 (2005) 524–537

8. A phased implementation plan for PPM

Based on the different PPM adoption stages and the

resulting benefits, we have developed a phased imple-

mentation plan for PPM. This plan is centred around the elements having the biggest impact in organisa-

tions in each stage, the issues managers currently face,

and the challenges these organisations face while

implementing these PPM processes. We elaborate on

the proposed three-stage PPM implementation plan

below.

8.1. Stage I: portfolio inventory

In this stage, the following PPM processes are to be

installed:

� Centralised project administration. � Risk evaluation procedures. � Explicit incorporation of resource constraints. � Increasing business leaders� accountability for project results.

Our study indicated that the greatest positive impact

of PPM can be observed when organisations start to (a)

assess economic and technological risks at the portfolio

level, (b) incorporate resource constraints in their deci-

sion making, and (c) explicitly look for risk diversifica-

tion across the portfolio. Nevertheless, according to our respondents, such organisations still face issues in

terms of lack of commitment from business leaders, lack

of alignment of projects to strategy, lack of coordination

between projects and conflicting project objectives.

Therefore, it was not a surprise to identify that, accord-

ing to the managers of these organisations, one of the

greatest challenges to advance the implementation of

PPM is the lack of a clear company strategy. They also

understand that their teams typically lack relevant train-

ing and appropriate knowledge to measure projects ben-

efits, which can explain the low impact of PPM in improving their assessment of financial worth of the

project portfolio.

8.2. Stage II: portfolio administration

In this stage, the following PPM processes are to be

installed:

� Project categorisation. � Evaluation of customer impact of the project portfo- lio results.

Organisations at this stage have better defined objec-

tives and are better informed about the costs and bene-

fits of their projects. Two aspects are especially

important for creating additional benefits from PPM, namely (1) processes to categorise projects and (2) better

evaluation of customer impact of the project portfolio

results. However, a large part of respondents of this

group also said that they lack resources to analyse pro-

ject data, suffered from people and financial constraints

as well as an overload of projects.

As organisations evolved from stage I to II, they still

encounter the same level of problems with excessive number of projects and people constraints, as seen in

Fig. 4. These organisations also cited lack of resources,

lack of appropriate ways of measuring project benefits

and high staff turnover as the main challenges to further

implement PPM.

0.0

1.0

2.0

3.0

4.0

5.0

Lack of knowledge of portfolio management techniques

Staff turnover

Frequent project scope changes

Lack of resources (people/financial) to analyse project data

Lack of a clear company strategy

Lack of relevant training Lack of appropriate way to measure

project benefits

Lack of commitment of business leaders

Lack of knowledge to evaluate risks

Lack of financial skills of project/other staff

Lack of cross-functional communication

Stage I Stage II Stage III

Fig. 4. Map of PPM challenges per stage.

B.D. Reyck et al. / International Journal of Project Management 23 (2005) 524–537 533

8.3. Stage III: portfolio optimisation

In this stage, the following PPM processes are to be

installed:

� A project portfolio committee. � Assessment of the financial worth of the portfolio. � Management of project interdependencies. � Tracking project benefits.

The most optimistic findings came from stage III

organizations. The respondents in this stage claimed

that almost all aspects of PPM had significantly in-

creased the return on investment in projects. Among

the top of the list it is observed the importance of a

project portfolio committee. 45% of these companies have a committee composed by top management and

56% composed of senior general management. This

stresses the importance of top executives managing

the portfolio. Also, 89% of these committees meet at

least once a month.

Moreover, it is relevant to notice how strongly these

organisations think about investments at the portfolio

level. Respondents at this stage devoted high importance to the management of project interdependencies, align-

ment of the project portfolio with the organisation�s objectives and assessment of the financial worth of the

portfolio. Interestingly, this group was the only one that

obtained a fair return from the effort of the use of PPM

software. This reinforces the thesis that the use of PPM

software is only justifiable when the other aspects of

PPM are already in place. Although the level of project-related problems for

organisations in this stage is typically lower than at

other stages, managers of organisations at stage III said

that they still have problems with people and financial

constraints and late delivery of projects that are corre-

lated with high number of changes in project scope

and lack of cross-functional communication and work,

as highlighted in Fig. 4 above.

9. Conclusions

Based on a survey of 34 medium-to large-size com-

panies from the UK, Europe and the rest of the

world, we developed a framework for the adoption

of Project Portfolio Management (PPM) processes at

organisations. Using a cluster analysis, we identified

three stages of PPM adoption, and investigated which of the elements of PPM are typically present in the

organisations at the different adoption levels, including

a centralised view of projects, financial analysis, risk

analysis, managing interdependencies, incorporating

constraints, enabling an overall analysis at portfolio

level, project prioritisation and selection, and the use

of specialized software tools.

We correlated the PPM adoption level to the benefits the organisations perceived and to the level of problems

they face in managing their projects, and showed that an

increased PPM adoption level has a significant positive

impact on the return on the projects in the portfolio,

and a significant negative impact on the number of pro-

ject-related problems reported.

We also provide a phased implementation plan

for organisations to evolve through three stages of PPM adoption: from Portfolio Inventory through

Portfolio Administration to Portfolio Optimisation

534 B.D. Reyck et al. / International Journal of Project Management 23 (2005) 524–537

with increasing positive impact, both reducing the

organisational problems associated with projects and

enhancing return on investment. However, we have

also shown that organisations do not have to imple-

ment all the elements of PPM to create benefits. Even

organisations at stage I benefited from the few PPM elements they had implemented. According to our

Section 2: Projects and portfolio management in your organ

Section 3: Issues associated with projects

Appendix A. Sample of survey questions

results, value could be enhanced by properly choosing

the right elements to adopt. For example, our results

showed that implementing specialised project portfolio

management software will not add any value unless all

other major processes have also been adopted, and

therefore only make sense for organisations at stage III of PPM adoption.

isation

Section 4: Impact of using project portfolio management

Section 5: Challenges of implementing project portfolio management

Appendix B

Section 2 average scores by adoption stage

Questions Average score a

Stage 1 Stage II Stage III Total

Have an inventory of current and proposed significant projects? 2.75 3.19 3.90 3.29 Have a central point responsible for collecting, analysing and

distributing project information in a common format?

2.50 3.14 3.80 3.19

ROI 2.13 3.00 3.60 2.97

NPV 1.88 2.90 3.25 2.69

IRR 1.71 2.88 3.13 2.61

Economic value added (EVA) 1.17 2.00 3.25 2.00 (continued on next page)

B.D. Reyck et al. / International Journal of Project Management 23 (2005) 524–537 535

Appendix B (continued)

Questions Average score a

Stage 1 Stage II Stage III Total

Real options 1.25 2.20 3.40 2.36

Payback period 2.63 2.92 3.67 3.07

Complexity of project 2.25 3.13 3.90 3.15

Technology risk 2.38 3.25 3.90 3.24

Expertise and experience of the team 1.63 2.87 3.90 2.88 Market and environmental risks 2.13 2.20 3.60 2.61

Financing/cash-flow 2.88 2.80 3.80 3.12

Organisational change 1.88 2.73 3.30 2.70

Management commitment 1.57 2.71 3.60 2.74

Cross-project dependencies 1.43 2.69 3.80 2.76

Overlaps and duplication 1.63 2.44 3.67 2.58

Implementation bottlenecks, e.g., too much change in one area 1.25 2.88 3.30 2.62

Competition for scarce resources 1.13 2.94 3.40 2.65

Organisational capacity to absorb change 1.25 2.13 3.30 2.27

Staff capabilities to implement projects 1.13 2.38 3.50 2.41

Budgets/financial capacity 2.38 3.31 3.90 3.26

Financial analysis of project portfolio 1.50 2.64 3.70 2.69

Overall risk analysis of project portfolio 1.13 1.87 3.40 2.15 Risk vs. reward analysis of project portfolio 1.25 2.00 3.10 2.16

Project diversification 1.00 1.63 3.29 1.95

Categorization of projects to ensure a balanced mix, e.g., short-term

vs. long term, supply side vs. market side, etc.

1.20 1.88 3.11 2.23

Alignment of the project portfolio with organisation�s strategy 1.63 3.13 3.80 2.97 Alignment of the project portfolio with organisation�s IT architecture 1.38 2.81 3.70 2.74 Customer impact of the project portfolio results 1.38 2.44 3.50 2.50 Have top management involved in the project selection process 2.38 3.33 3.60 3.18

Have business leaders accountable for project results 1.50 2.44 3.50 2.53

Have systematic reviews of projects at specific stages, e.g., stage gates 1.63 2.67 3.70 2.73

Centrally track the benefits of projects? 1.00 1.62 3.40 2.03

Compare the outcome of projects with their original targets? 1.25 2.06 3.90 2.41

Have regular project portfolio reporting? 1.63 2.88 3.60 2.79

Analyse the impact of individual new projects to the overall portfolio? 1.13 2.13 3.20 2.21 Annually (or more frequently) prioritise the overall project portfolio? 1.25 3.13 3.80 2.88

Regularly review and revise the project portfolio (eg several times a

year)?

1.25 2.64 3.40 2.53

Use specialized software to manage your portfolio of projects? 1.00 1.67 2.38 1.73 a Scale: 1, do not use; 2, use occasionally;, frequently use; 4, always use.

536 B.D. Reyck et al. / International Journal of Project Management 23 (2005) 524–537

References

[1] Linenberg Y, Stadlker Z, Arbuthnot S. Optimising organisational

performance by managing project benefits. PMI Global Congress

2003, Europe.

[2] Lycett M, Rassau A, Danson J. Programme management: a

critical review. Int J Proj Manag 2004;22(4):289–99.

[3] Markowitz H. Portfolio selection. J Financ 1952;7:77–91.

[4] McFarlan FW. Portfolio approach to information systems.

Harvard Bus Rev 1981(September–October):142–51.

[5] Wheelwright SC, Clark KB. Creating project plans to focus

product development. Harvard Bus Rev 1992;70(2): 67–83.

[6] GAO. Improving mission performance through strategic informa-

tion management: learning from leading organisations; May 1994.

B.D. Reyck et al. / International Journal of Project Management 23 (2005) 524–537 537

[7] Thorp J. The information paradox – realizing the business

benefits of IT. Toronto: McGraw-Hill; 1999.

[8] Jeffery M, Leliveld I. Best practices in IT portfolio management.

Sloan Manag Rev 2003;45.

[9] Dickinson MW, Thornton AC, Graves S. Technology portfolio

management: optimizing interdependent project over multiple

time periods. IEEE Trans Eng Manag 2001;48(4):518–27.

[10] CIO Council. A summary of first practices and lessons learned in

ITPM; March 2002.

[11] Matheson D, Matheson J. The smart organization. Boston: Har-

vard Business School Press; 1998.

[12] Kendall G, Rollins S. Advanced project portfolio management

and the PMO. Florida: J. Ross Publishing; 2003.

[13] Benaroch M. Managing information technology investment risk:

a real options perspective. J Manag Inform Sys 2002;19(2):43–84.

[14] McGrath GR, Macmillan IC. Assessing technology projects using

real options reasoning. Res Technol Manag 2000(July–August):

35–49.

[15] GoldmanC. Aligndrive–expertadvice.CIOMag1999(January):5.

[16] Lawrie G, Metcalfe D, Hollaway E. Grading Service Automation

Apps for Corporate IT. Forrester 2003(Sep).

[17] Datz T. Portfolio management – how to do it right. CIO Mag

2003:2.

[18] Payne H. Management of multiple simultaneous projects:

a state-of-the-art review. Int J Proj Manag 1995;13(3):

163–8.

[19] Berinato S. Do the math. CIO Mag 2001:3.

[20] Dixit AK, Pindyck RS. Investment under uncertainty. Prince-

ton: Princeton University Press; 1994.

[21] Trigeorgis L. Real options: managerial flexibility and strategy in

resource allocation. Massachusetts: MIT Press; 1996.

[22] Ibbs W, Reginato J. Quantifying the value of project manage-

ment. PMI, Newton Square, PaPA 2002.

[23] Turner JR, Cochrane RA. Goals-and-methods matrix: coping

with projects with ill defined goals and/or methods of achieving

them. Int J Proj Manag 1993;11(2):102–293.

[24] SPSS Base 12.0 Users�s Guide. Prentice Hall; 2003 [Chapter 33].