Transforming Nursing

REVIEW Open Access

Practices of decision making in priority setting and resource allocation: a scoping review and narrative synthesis of existing frameworks Brayan V. Seixas1* , François Dionne2 and Craig Mitton3,4

Abstract

Background: Due to growing expenditures, health systems have been pushed to improve decision-making practices on resource allocation. This study aimed to identify which practices of priority setting and resource allocation (PSRA) have been used in healthcare systems of high-income countries.

Methods: A scoping literature review (2007–2019) was conducted to map empirical PSRA activities. A two-stage screening process was utilized to identify existing approaches and cluster similar frameworks. That was complemented with a gray literature and horizontal scanning. A narrative synthesis was carried out to make sense of the existing literature and current state of PSRA practices in healthcare.

Results: One thousand five hundred eighty five references were found in the peer-reviewed literature and 25 papers were selected for full-review. We identified three major types of decision-making framework in PSRA: 1) Program Budgeting and Marginal Analysis (PBMA); 2) Health Technology Assessment (HTA); and 3) Multiple-criteria value assessment. Our narrative synthesis indicates these formal frameworks of priority setting and resource allocation have been mostly implemented in episodic exercises with poor follow-up and evaluation. There seems to be growing interest for explicit robust rationales and ample stakeholder involvement, but that has not been the norm in the process of allocating resources within healthcare systems of high-income countries.

Conclusions: No single dominate framework for PSRA appeared as the preferred approach across jurisdictions, but common elements exist both in terms of process and structure. Decision-makers worldwide can draw on our work in designing and implementing PSRA processes in their contexts.

Keywords: Priority setting, Resource allocation, Frameworks, Health economics, Efficiency

Introduction Priority setting and resource allocation (PSRA) practices constitute processes and rationales applied to the pivotal task of determining how resources (i.e., money, people,

time) are allocated within healthcare systems [1]. The volume of activity by type of intervention is not simply the result of the aggregation of individual clinical deci- sions but rather is predominantly the result of a budget- ing decision. Given a pre-determined budget, decisions are made on the amount of funding budgeted for each type of intervention or area of care. Trade-offs then be- come a space not only for demonstration of clinical value but also of dispute for the scare resources. In

© The Author(s). 2021 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

* Correspondence: [email protected] 1Department of Health Policy and Management, Fielding School of Public Health, University of California, Los Angeles (UCLA), Los Angeles, USA Full list of author information is available at the end of the article

Seixas et al. Health Economics Review (2021) 11:2 https://doi.org/10.1186/s13561-020-00300-0

essence, this is the practice of deciding what is covered and what is not, with the aim of reducing spending on low value activity. A PSRA framework has at least two components: a

mechanism to assess the value of interventions and a mechanism to guide the prioritization activity (i.e., making trade-offs). Other work has focused on approaches employed for the assessment of the value of health care in- terventions [2]. One common approach to value assess- ment is multi-criteria decision analysis (MCDA) [3, 4], which can be viewed as a competitor to the more trad- itional approach of the incremental cost effectiveness ratio (ICER) based on quality adjusted life years (QALYs) [5–8]. While it is critically important to understand value assess- ment, value assessment is not a choice making exercise per se. Value assessment needs hence to be set within a framework for decision making, i.e., the processual space and institutional environment where choices are made [9]. As previous research has indicated [10, 11], resource

allocation in healthcare systems are often carried out based on historical decisions, without any explicit ration- ale and proper consideration of opportunity costs. Yet, the growing budgetary pressures observed in virtually every high-income country has led researchers and decision-makers to pursue novel methodologies to prioritize investment options and allocate the scarce existing resources. Thus, the present study aims to as- sess the evolution of the PSRA field understanding which practices have been developed and implemented. Previous systematic reviews on PSRA are available in

the literature, with distinct nuances. Some focus on priority-setting at the macro or meso levels [12], some focus on hospitals [13], and others emphasize frame- works implemented in low- and middle-income coun- tries (LMIC) [12, 14]. The results are diverse and include a varied collection of case-studies. The distinct methodologies described in the literature as PSRA strat- egies have been employed in virtually every level of gov- ernance and type of healthcare setting. As each particular setting has specific contexts and goals, these decision-making approaches get different procedural for- mats and involve different stakeholders (for example, while a PSRA initiative may involve clinicians, adminis- trators and other healthcare professionals within the realm of a single healthcare organization, a PSRA exer- cise may involve legislators, bureaucrats and representa- tives of the general public at a government level). In the realm of publicly funded healthcare systems, the PSRA initiatives play a vital role helping decision-makers to improve budgetary and financial management, ensuring legitimacy, fairness and transparency while also adding value to decisions on resource allocation [15, 16]. Effect- ive public financial management depends on explicit and formal PSRA approaches [17, 18].

The current study focuses on existing mechanisms, pro- cesses or frameworks to guide prioritization. Specifically, the objective was to identify frameworks that have been employed in real-world settings in high-income countries. Our intent was not to identify every single implementation of a given framework but rather to report on instances where key frameworks were utilized. We do not intend to provide a meta-analysis of all available evidence, but rather touch upon the most relevant aspects for reporting a collection of frame- works used in practice for priority setting and resource allo- cation decision-making across countries. A variety of methodologies have been described in the health economics literature under the PSRA umbrella terminology. They may have markedly different procedures and rationales, such as Health Technology Assessment (HTA) and Programme Budgeting and Marginal Analysis (PBMA). However, as long as they are used to assess the value of existing investment al- ternatives and to guide the choice-making process, they have been deemed PSRA strategies in the literature. Given that our goal is to map the literature to identify existing frame- works with some evidence of empirical use, we did not chal- lenge authors in their categorization of any given methodology as a PSRA framework. We narrowed down the analysis for high-income

countries mainly because the urge for efficient resource allocation in these settings is a response to a critical his- torical trend of unsustainable growth of health expendi- tures, which makes the motivation and goal very different from LMICs, whose health systems are often underfunded and where PSRA strategies have the object- ive of achieving universal healthcare coverage. In addition, a recent review was conducted in LMIC coun- tries [14] and turned up very limited empirical applica- tions which was our focus here. The overall research question guiding this study was:

which decision-making frameworks have been developed and implemented to set priorities and allocate resources within healthcare systems of high-income countries? To answer this question, we conducted a scoping review of the peer-reviewed scientific literature, a gray literature review and horizontal scanning, and then a narrative synthesis to make sense of obtained data and dialogue with other pieces from the literature.

Methods A scoping review was conducted focusing on frameworks used for PSRA in high-income countries. A comprehensive search of the peer-reviewed literature published between 2007 and 2019 was conducted using Ovid MEDLINE, an ex- tensive database of public health journals with a platform for building searching strategies. The literature review search strategy is outlined in Appendix 1. Given that the study ob- jective was to identify frameworks with actual value for im- plementation in healthcare systems (and not simply a

Seixas et al. Health Economics Review (2021) 11:2 Page 2 of 11

historical view of the field), searching older papers would be limited in describing current practice and also would overlap with previous studies. In total, 1585 titles and abstracts were found, once du-

plicates were removed. We then used a two-step screening process. First, the 1585 abstracts were screened by one primary reviewer and two secondary reviewers based on the inclusion/exclusion criteria found in Appendix 2. Re- viewer 1 (BVS) reviewed all 1585 articles while reviewer 2 (CM) reviewed 227 articles and reviewer 3 (FD) reviewed 71 articles. The agreement rate between reviewer 1 and reviewers 2 and 3 was over 90%. Discrepancies were han- dled conservatively, resulting in a total of 92 abstracts ini- tially being screened ‘in’. Following this, one of the two senior reviewers (CM) took a further detailed read of the abstracts and pared the list down to 25 relevant articles. This second stage screen in the main excluded papers that initially appeared to be an empirical study but were in fact discussing some aspect of priority setting without an ac- tual case study or implementation of a framework. We then applied a data extraction tool (see Appendix 3) to identify the relevant information. Figure 1 presents a step flow diagram depicting the screening process. Note that because of the objective and design of this review

(which focuses on revealing existing formal PSRA frame- works, thus including only papers that provided a full de- scription of a framework employed in a real-world setting), it

is likely that articles presenting relevant information on current practices of decision-making in priority-setting have not been captured. Thus, we also conducted horizontal scanning and con-

ducted searches in the gray literature. The major search strategy here consisted of exploring the websites of reput- able HTA agencies and other relevant international orga- nizations for presentations, guidelines, working papers or any other pertinent piece of gray literature. We looked at the following organizations: Health Technology Assess- ment International (HTAi); International Network of Agencies for Health Technology Assessment (INAHTA); International Society for Pharmacoeconomics and Out- comes Research (ISPOR); European Network for Health Technology Assessment (EUnetHTA); International Health Economics Association (IHEA); Agency for Health Research and Quality (AHRQ); Canadian Agency for Drugs and Technologies in Health (CADTH); Kaiser International Health Group; and Blue Cross Blue Shield Association. Furthermore, we also perused other existing literature reviews and their references in order to obtain a deeper and richer view of PSRA field. These documents were not included in our primary data analysis as they were not empirical studies per se but provide important context and further insight with respect to PSRA. As discussed in the Introduction, a PSRA framework

constitutes a formal process to determine the available

Fig. 1 Screening 115 steps resulting in final full-text article review.

Seixas et al. Health Economics Review (2021) 11:2 Page 3 of 11

options for investment and a rationale for choice mak- ing. These frameworks can be applied at any level of governance or type of healthcare organization. No fur- ther a priori criteria were defined to determine whether or not a study constitutes a PSRA initiative, allowing cat- egories to emerge from the data. In other words, we did not aim to challenge previously published authors in their judgment on whether or not certain approaches constitute PSRA frameworks, but rather we aimed to document what has been done in the field. In addition, we focused on describing the set of exist-

ing PSRA frameworks rather than reporting all the in- stances a given methodology is employed. We ultimately aim to provide a helpful overview of available practices for decision-makers and those interested in the process of priority setting and resource allocation in health care.

Results The 25 papers that met the inclusion criteria and were subject to data extraction provide relevant information from ten countries: Australia [19–21]; Austria [22]; Canada [23–27]; Israel [28]; Korea [29, 30]; New Zealand [31]; Norway [32]; Sweden [33, 34]; UK [35–42]; US [43]. The list of frameworks identified in our view pro- vides a reasonable summary of PSRA frameworks devel- oped and implemented in health care systems in high- income countries.

Emerging classification system Based on the information obtained through our extrac- tion tool (Appendix 3), three major umbrella categories emerged to make sense of possible grouping similarities among the identified PSRA initiatives: 1) PBMA frame- works; 2) HTA-related frameworks; and 3) multi-criteria value assessment frameworks. Before we delve into how the identified practices fit these categories, let us better define each classification category. First, Program Budgeting and Marginal Analysis

(PBMA) is a deliberative framework used to assist decision-makers in determining what to fund and what not to fund. It can be used to achieve specific goals (such as deal with existing budgetary deficits or direct resources to capital investments) or as a routine practice of decision making in resource allocation. PBMA pro- vides the process and structure within which a specific value assessment approach can be applied. The steps of the process are usually as follows [44]: 1) determine the aim and scope of the priority setting exercise; 2) exam- ine how resources are currently spent; 3) form a multi- disciplinary committee to identify the relevant decision criteria; 4) identify proposals for changes to the current spending pattern, either for investment (increased spending) or disinvestment (reduced spending), and in all cases the focus is on marginal analysis or change to

the status quo; 5) the impact of each proposals is then assessed by the committee using the pre-identified cri- teria; 6) make decisions based on relative value trade- offs; 7) provide an opportunity for appeal based on pre- defined guidelines; and 8) evaluate the process and make adjustments to refine the process as necessary. The process steps are made known throughout the organization and key stakeholders are involved in pro- posal generation as much as possible. Public members or patient representatives can also be engaged to provide input on the relevant criteria or the criteria weights. The chosen criteria and their respective weights typically vary across jurisdictions, reflecting local values and prefer- ences. A PBMA framework can be applied at virtually any organizational setting, e.g., a system level, an individ- ual hospital or a unit level within a given hospital. Second, Health Technology Assessment (HTA) is for-

mally defined by INAHTA as “the systematic evaluation of the properties and effects of a health technology, ad- dressing the direct and intended effects of this technol- ogy, as well as its indirect and unintended consequences, and aimed mainly at informing decision making regard- ing health technologies” [45]. Similarly, the European Network for Health Technology Assessment states that “HTA is a multidisciplinary process that summarises in- formation about the medical, social, economic and eth- ical issues related to the use of a health technology in a systematic, transparent, unbiased, and robust manner” [46] Note that this type of endeavor is distinguished from priority-setting, which is the choice making activ- ities that decision makers undertake in determining what health care services to fund and what not to fund [47]. The purpose of HTA per se is not to realize or imple- ment trade-offs, pointing out investment and disinvest- ment opportunities [48]. Rather, HTA in itself is a tool to produce evidence that helps inform the management of technologies. Harris et al. [49] state that HTA is a valuable tool for decision making and its use may lead to disinvestment but it is not a framework specifically intended for assessing trade-offs, broadly speaking. It is possible to conceive, however, a broader framework of priority setting predominantly based on HTA. Third, the last category of priority-setting framework

emerged directly from the data analysis of identified em- pirical PSRA activities and refers to organization-wide priority-setting processes with explicit use of multiple criteria for value assessment. Under this category some sort of explicit and formal consideration of multiple cri- teria was carried out, which could be a formal MCDA (multi-criteria decision analysis) tool, a discrete-choice methodology, or some other deliberative process involv- ing criteria assessment. Thus, this category encompasses a wide variety of practices that ultimately have one thing in common: the consideration of multiple criteria and

Seixas et al. Health Economics Review (2021) 11:2 Page 4 of 11

participation of multiple stakeholders in the priority- setting decision-making process. Of note, PBMA frame- works often use MCDA tools and thus could fall under this catch-all category of frameworks that lead to choices based on multiple criteria. Yet, because of its predomin- ance in the literature and its particular underpinning principles, we have chosen herein to refer to PBMA as its own category. Also, the definition of value seems to vary among these studies and was rarely provided.

Classification of identified studies As it can be seen in Table 1, out of the 25 studies, nine were found to have used a formal PBMA framework, three employed an HTA-related framework, and thirteen were broadly classified as multiple-criteria value assess- ment framework. We do not at all claim that these are the full set of applications of PSRA. Rather, this list re- flects current practice and is likely representative of the types of approaches in use at this time. One study falling within the PBMA framework cat-

egory and two falling within multiple-criteria value as- sessment framework explicitly stated applying Accountability for Reasonableness (A4R) as well. Some- times referred to in the literature as a priority-setting framework, A4R consists of a set of principles developed by Daniels and Sabin [50, 51] focusing on the ethical as- pects of choice-making. The underlying thinking is that there is inescapable uncertainty in decision making in health care and therefore it is critical to the acceptability of the decisions that the prioritization process be per- ceived as a fair process by the stakeholders. The original framework had four elements or conditions that deter- mine the perceived fairness of a given process: relevance, publicity, revision and enforcement. Gibson and col- leagues [52] then proposed a fifth condition: empower- ment. A4R can essentially be used with any approach to decision making in that, however the decisions are made on resource allocation, the ethical conditions of A4R can be brought to bear to ensure that those decisions are made as fairly as possible.

Key findings and further contextual comments We found that formal PSRA frameworks have been used in virtually all possible levels of governance and adminis- tration (national, state/provincial, regional, hospital) and have served for the prioritization of a wide variety of health services (e.g., from community care, to mental health, drug reimbursement, immunization and specific diseases). Despite this heterogeneity in application, a number of key process characteristics were identified from the selected papers:

� PSRA strategies have employed a variety of criteria to assess value, and do not necessarily use a single,

consistent technique to judge alternatives and summarize preferences;

� A variety of stakeholders were involved in almost every case, such as administrators, government officials and clinicians;

� Decisions were typically not subject to review by external stakeholders (i.e., the general public);

� It was generally not clear from the papers how requests for funding were initiated, nor was it specified if current spending was reviewed as part of the process of decision making (although PBMA, for example, implies such a review);

� Several types of data were reported to inform decision-making, including published literature, clin- ical opinions, economic evaluations, HTAs, and data on disease prevalence;

� There was limited reporting of evaluation in these studies;

� Types of information reported to be drawn on for decision making included published literature, clinical opinions, economic evaluations and data on disease prevalence;

� Some level of political involvement was stated in most cases, although in cases where a more robust framework is described, there seemed to be less political interference;

� Deliberation has largely become the norm, e.g.: “following common practice in decision analysis, validity and consistency of responses was established through panel discussion and deliberation” [19], p. 908];

� It seems that approaches of willingness-to-pay thresholds are being abandoned in light of a greater understanding of the complexities of health care de- cision making, of the limitations of ‘single truth’ evi- dence and of the need for broader stakeholder engagement.

Note that because of the objective and design of this review, which focused on revealing existing formal PSRA frameworks that were employed in a real-world setting, it is a certainty that relevant articles presenting informa- tion on current practices of decision-making in priority- setting were not captured (e.g., think pieces or other non-empirical activity). To reiterate, the purpose was not to capture every framework that has been employed, and in every situation, but rather to provide an indica- tion of key frameworks that have been applied with some consistency across countries. An important contextual factor is that the majority of

PSRA initiatives were found in countries where there is a pre-set limit on how much can be spent and the orga- nizations holding the envelope must find ways to stay within this limit. That is, the total value of services

Seixas et al. Health Economics Review (2021) 11:2 Page 5 of 11

provided over a year is largely determined at the outset as a fixed envelope and providers have to prioritize an- ticipated claims or adjust the fee structure (reimburse- ment level) in order to keep total costs within the envelope. Defechereux et al. [32] summarize the nature of this challenge: “In all health care systems, choices in the allocation of resources are necessary. Public re- sources ( …) are insufficient to provide all possible services”.

Whereas priority-setting approaches based on decision sciences have been shown to be useful and versatile in allocating scarce resources in a wide variety of levels of governance and administration (hospital, regional and national levels [53]) as well as within diverse areas of care (like mental health [54], coronary heart diseases [55], and community care [27]), approaches grounded on welfare economics have been usually employed only to make recommendations regarding the coverage of

Table 1 Practices of priority setting found in the literature

Authors Date Country Level of exercise

Area of application Primary decision maker

Classification category

Peacock et al. [19] 2007 Australia Regional Mental health services Executive team PBMA framework

Galego et al. [20] 2007 Australia Hospital Drugs Executive team Multi-criteria value assessment framework

McDonald et al. [21]

2011 Australia Regional Primary care services Executive team Multi-criteria value assessment framework

Mentzaskis et al. [22]

2014 Austria National General health services Policy makers Multi-criteria value assessment framework (specifically, DCE)

Urquhart et al. [23] 2008 Canada Regional Home and community care Executive team PBMA framework + A4R

Dionne et al. [24] 2009 Canada Regional General health services Executive team PBMA framework

Stafinski et al. [25] 2011 Canada National Health technologies Policy makers HTA-related framework

Mitton et al. [26] 2011 Canada Regional Primary care, community care and public health

Executive team PBMA framework

Cornelissen et al. [27]

2016 Canada Regional Community care Executive team PBMA framework

Greenberg et al. [28]

2009 Israel National Health technologies Policy makers HTA-related framework

Ahn et al. [29] 2012 Korea National Health technologies Policy makers HTA-related framework

Choe et al. [30] 2014 Korea National Vaccines Policy makers Multi-criteria value assessment framework

Ashton et al. [31] 2008 New Zealand

Regional General health services Executive team Multi-criteria value assessment framework

Defechereux et al. [32]

2012 Norway National General health services Policy makers Multi-criteria value assessment framework

Waldau et al. [33] 2010 Sweden Regional General health services Executive team Multi-criteria value assessment framework + A4R

Waldau et al. [34] 2015 Sweden Regional General health services Executive team Multi-criteria value assessment framework + A4R

Bate et al. [35] 2007 UK Regional Orthopedic surgery Executive team PBMA framework

Wilson et al. [36] 2007 UK Regional General health services Executive team PBMA framework

Airoldi et al. [37] 2008 UK National Diabetes Policy makers Multi-criteria value assessment framework

Marsh et al. [38] 2013 UK National Preventative health interventions Policy makers Multi-criteria value assessment framework

Goodwin et al. [39] 2013 UK Regional General health services Executive team PBMA

Airoldi et al. [40] 2013 UK Regional Mental health services Executive team Multi-criteria value assessment framework

Holmes et al. [41] 2018 UK Regional Dental services Executive team PBMA

Vernazza et al. [42] 2019 UK Regional Dental services Executive team Multi-criteria value assessment framework

Canham-Chervak et al. [43]

2010 US National Military injuries Executive team Multi-criteria value assessment framework

Seixas et al. Health Economics Review (2021) 11:2 Page 6 of 11

specific technologies. Cost-effectiveness analysis and cost-utility analysis have been widely used within this re- stricted context of ‘priority-setting’ by HTA agencies all across the globe, such as CADTH in Canada, PBAC and MSAC in Australia, PHARMAC in New Zealand, and NICE in the UK. Additional PSRA frameworks with some evidence of

empirical use are also observed in the literature. Angelis and Kanavos [56], for instance, propose a MCDA-based approach called Advanced Value Framework, which uses five domains of criteria (burden of disease, therapeutic impact, safety profile, innovation level and socioeco- nomic impact) and a MAVT function to aggregate scores. Another framework based on MCDA that was developed to inform decision-making in health care and has been proposed for priority-setting and resource allo- cation is EVIDEM [57]. Airoldi et al. [58] propose the Socio-Technical Allocation of Resources (STAR) for re- source allocation, as it is claimed to be theoretically strong and highly useful for decision-makers. STAR em- ploys models to appraise the “cost-effectiveness of all in- terventions considered for resource reallocation by explicitly applying the theory of health economics to evi- dence of scale, costs, and benefits, with deliberation fa- cilitated through an interactive social process of engaging key stakeholders” [58]. In this so-called ‘social process’, the involved stakeholders produce missing esti- mates of scale, costs, and benefits of the interventions, create visual representations of their relative cost- effectiveness and then interpret them. STAR was used by a Primary Care Trust (a local NHS planning agency) to allocate a fixed budget in 2008 and 2009 [58].

Discussion Our work found 25 studies describing a real-world prac- tice of a formal framework of priority setting and re- source allocation in ten high-income countries. In the process of making sense of all qualitative data generated by the scoping review through our previously designed extraction tool, we created a classification system that grouped identified studies in three categories: PBMA framework, HTA-related framework, and multi-criteria value assessment framework. Unlike the first two cat- egories that refer to studies with explicit mention to PBMA and HTA components, the last category had some residual component and had a loose common fea- ture, i.e., the use of multiple criteria and multiple stake- holder involvement in the process of decision-making (which could involve a formal MCDA tool or a DCE, for example). Although PBMA and HTA have clearly differ- ent goals and rationales, both approaches have been paradigmatically deemed frameworks of priority setting and resource in the health economics and health policy literature [1, 11–13, 59].

This study contributes to the literature not only in identifying which formal strategies of priority setting and resource allocation have been developed and imple- mented in healthcare systems of high-income countries, but also reveals important issues for the field of health economics and health policy. First, it indicates that for- mal decision-making processes with explicit and legitim- ate rationales are seemingly still episodic and have not turned into routine practice. Second, it reveals that al- though several important initiatives have been tried, evi- dence from evaluation is rare and there is still much to be learned about which practices are more successful and which system characteristics might be associated with them. Third, our findings suggest that the conven- tional extra-welfarist position that supports the mechan- istic employment of a single value measure like the incremental cost-effectiveness ratio (ICER) has been los- ing space in favor of decision-making approaches that incorporate multiple criteria and combine multiple ac- tors’ views. Fourth, a broad set of types of evidence is being used, moving beyond the traditionally gold- standard randomized controlled-trial and even peer- reviewed observational studies towards incorporating ex- perts’ opinions and patients’ perspectives. Fifth, we no- ticed that appeals mechanism or review process for final decisions, a key element within A4R, are virtually absent from the empirical strategies of priority setting and re- source allocation. In a review focusing on resource allocation and dis-

investment, Polisena et al. [11] found 14 studies, all in high-income countries. Two of them reported use of HTA to propose disinvestments whereas the majority described applications of PBMA. Studies reported initia- tives at the national level (basically the HTA approaches towards disinvestment), at the regional level (health au- thorities) and at a single health care unit or department. Another review carried out by Barasa et al. [13] with particular interest in formal PSRA initiatives in hospital settings revealed a small number of studies, which were mostly based in high-income countries. Almost all of these exercises addressed allocation of resources among hospital departments (usually based on PBMA or MCDA) or decision-making regarding acquisition of specific technologies (employing CEA/CUA). Hipgrave et al. [12] and Wiseman et al. [14] conducted

systematic reviews emphasizing PSRA endeavours in low- and middle-income countries. They both point out that relatively little information is known about practices of decision-making in priority-setting within health care systems of developing countries in comparison to high- income settings. The majority of reports identified by Wiseman et al. [14] involve global or regional efforts of Global Cost-effectiveness Analysis (GCEA) using cost per DALY averted, as the WHO frameworks to identify

Seixas et al. Health Economics Review (2021) 11:2 Page 7 of 11

the most cost-effective interventions to achieve the Mil- lennium Development Goals and Universal Coverage. In almost all of these cases, options for disinvestment were not considered alongside options for investment. Both re- views identified ranking of alternatives based on MCDA tools, including initiatives in Ghana and Nepal. In addition, other approaches were identified, such as case- studies of multi-criteria frameworks based on A4R in Tanzania and Uganda and the applications of the Invest- ment Case approach in India, Indonesia and Philippines. In another review of the published literature con-

ducted by Cromwell et al. [10] to find examples of ‘real- world’ priority setting exercises that used explicit criteria to guide decision-making, several case-studies were iden- tified, mainly in Canada and UK. The most common ap- proaches identified were PBMA and MCDA applied in various settings, e.g., national level, health authorities, hospitals and for specific disease programmes. A range of criteria were identified, with effectiveness and equity appearing most often. Relying upon the evidence found in previous literature

reviews on priority-setting, Hipgrave et al. [12] comments that “the overarching conclusion was that even in high- income settings where participatory, accountable and ra- tional approaches to health priority-setting should be achievable, the process and outcomes of such exercises have been unsatisfactory”. The evidence from the pub- lished literature is usually about specific case-studies and very rarely report a systematic and continuous use of for- mal PSRA frameworks. A 2017 review [59] aiming to understand ‘how have systematic priority setting ap- proaches influenced policy making’ concludes that “while systematic approaches have the potential to improve healthcare priority setting; most have not been adopted in routine policy making”. Having said that, Tsourapas and Frew [60] found that PBMA applications specifically have shown much success across countries and particularly in Canada. Thus, it seems that, as it is often the case, individ- ual details on implementation and indeed the individuals involved play a key part in achieving success or not. Our review has a few limitations. First, as it focuses on

high-income countries, we might have failed to capture some interesting and auspicious approaches being per- formed elsewhere. Yet, given the limited existing litera- ture on PSRA frameworks in other settings, and our own knowledge of the field over the last 20 years, that is unlikely to be the case. Second, due to the hues of grounded theory in analyzing the massive amount of qualitative data found in the literature review (as no rigid and formal a priori analytical framework was used to categorize and critique the practices deemed as prior- ity setting and resource allocation initiatives), the bound- aries among the frameworks are not always so clear. Similarly, their definition as a type of PSRA endeavor

can be debatable, as opposed to a value assessment framework, for example. These and other possible in- consistencies are not due to our analysis, rather it is a manifestation of a broad literature that not always oper- ates with clear and robust theoretical underpinning ideas around priority setting and value assessment. Third, the review is limited to existing empirical initiatives, which means that it is possible that promising and insightful theoretical frameworks have been left out. Lastly, our re- view has been limited to works published in the English language. It is possible that some relevant work had been captured by our search strategy for that reason.

Conclusion The unsustainable growth of health expenditures in high-income countries has led researchers and decision- makers to pursue efficiency in managing existing re- sources. The present work sought to identify which for- mal decision-making frameworks of priority setting and resource allocation have been developed and imple- mented in healthcare systems. We found three major categories of initiatives in this realm: PBMA, HTA and other multiple-criteria value assessment frameworks. Most were presented as an episodic management exer- cise, lacking information on evaluation and further im- plementation in routine practice. In terms of future research, our work indicates a few im-

portant areas for further exploration. First, the epistemo- logical boundaries between priority setting, value assessment and health technology assessment are not always clear. There seems to be space for a robust and extensive theoretical work aiming to establish these definitions in an interactive way, de- termining the nature of each endeavor with an explicit refer- ence to the ontological frontiers that delimitate them. Such epistemological enterprise would have to be conducted with a clear view of its operational implications, in terms of prac- tices and institutions. Second, more emphasis should be put on evaluation of implemented practices of decision-making. Very few studies present and discuss evaluation findings. This points out not only to the need of more focus on evalu- ating the existing PSRA practices but also to the need of de- veloping novel evaluative tools in this realm. Third, as most papers present case-studies of PSRA initiatives that were im- plemented for a particular purpose (and usually not even evaluated), it is of high importance to establish a converging agenda for the development of PSRA frameworks that can be turned into routine processes. As virtually every health care organization is making decisions on what to fund and what not to fund, PSRA is most certainly hap- pening but this is very often done in non-explicit and informal manners. Ensuring decisions are consistently made on reasonable, formal and agreed bases is ex- pected to result in more efficient, equitable and legit- imate allocation of the scarce resources available.

Seixas et al. Health Economics Review (2021) 11:2 Page 8 of 11

Appendix 1 Table 2 Literature Review Search Strategy

Database: MEDLINE (OVID)

Ovid MEDLINE(R) In-Process & Other Non-Indexed Citations & and Ovid MEDLINE(R) < 1946 to Present>

Search Name: 2017 Res M10

Date: Nov.18, 2017

Search Strategy:

1 (framework or frameworks).tw,kw. (213264)

2 (tool or tools).tw,kw. (588532)

3 case stud$.mp. (89455)

4 (approach or approaches).tw,kw. (1447206)

5 or/1–4 [Frameworks] (2153708)

6 *resource allocation/ (3486)

7 *health care rationing/ (6485)

8 *health priorities/ (5178)

9 or/6–8 (13437)

10 5 and 9 (2152)

11 comment/ or editorial/ or letter/ or news/ (1858162)

12 10 not 11 (2083)

13 limit 12 to yr = “2007 -Current” (1077)

14 limit 13 to English language (998)

15 commissioning.mp. (3703)

16 5 and 15 (923)

17 comment/ or editorial/ or letter/ or news/ (1858162)

18 16 not 17 (911)

19 limit 18 to yr = “2007 -Current” (723)

20 limit 19 to English language (718)

21 20 not 14 (707)

Appendix 2 Table 3 Inclusion and Exclusion Criteria of Scoping Literature Review

Inclusion Criteria Exclusion Criteria

Empirical study or exercise of priority setting in so far as it involves choice making at any level in the health system (e.g. national, provincial/state, regional, single organization)

Priority setting for non-health care settings (animal, environmental, edu- cation, etc.)

Exercises that involve priority setting for health research

Reviews, commentaries or think pieces (although they may be kept for broader context)

Studies in low- or middle-income country (LMICs)

Presentation of an actual framework for decision making in relation to priority setting (e.g., CEA in and of itself, or MCDA in and of itself, does not constitute a framework for priority setting)

Procurement, supply management, other purely financial mechanisms for cost containment

Descriptions of bedside or strictly clinically focused priority setting/ rationing, including organ donation

Descriptions of only a single aspect of priority setting, even if empirically focused (e.g. public engagement, evaluation activity) where the whole process or framework is not described

Seixas et al. Health Economics Review (2021) 11:2 Page 9 of 11

Abbreviations A4R: Accountability for Reasonableness; AHRQ: Agency for Health Research and Quality; CADTH: Canadian Agency for Drugs and Technologies in Health; CEA: Cost-Effectiveness Analysis; CUA: Cost-Utility Analysis; EUnetHTA: European Network for Health Technology Assessment; HTA: Health Technology Assessment; HTAi: Health Technology Assessment International; IHEA: International Health Economics Association; INAH TA: International Network of Agencies for Health Technology Assessment; ISPOR: International Society for Pharmacoeconomics and Outcomes Research; MCDA: Multi-Criteria Decision Analysis; NHS: National Health Service (United Kingdom); NICE: National Institute for Health and Care Excellence; PBAC: Pharmaceutical Benefits Advisory Committee; PBMA: Program Budgeting and Marginal Analysis; PSRA: Priority Setting and Resource Allocation; QALY: Quality-Adjusted Life Years; RCT: Randomized Controlled Trial

Acknowledgements The authors acknowledge the financial support received from the US National Pharmaceutical Council for the development of this work.

Authors’ contributions BVS developed the study design, created the qualitative methodology, conducted data collection, data management and data analysis, and wrote the first manuscript draft. CM developed the study design, supervised data

collection and data analysis, and reviewed the manuscript. All authors read and approved the manuscript.

Funding This work was funded by the US National Pharmaceutical Council.

Availability of data and materials Data are publicly available.

Ethics approval and consent to participate Not applicable.

Consent for publication Not applicable.

Competing interests The authors declare that they have no competing interests.

Author details 1Department of Health Policy and Management, Fielding School of Public Health, University of California, Los Angeles (UCLA), Los Angeles, USA. 2Prioritize Consulting Inc., Vancouver, Canada. 3Center for Clinical Epidemiology and Evaluation, Vancouver, Canada. 4School of Population and Public Health, University of British Columbia (UBC), Vancouver, Canada.

Received: 9 September 2020 Accepted: 16 December 2020

References 1. Mitton C, Donaldson C. Priority Setting Toolkit : Guide to the Use of

Economics in Healthcare Decision Making (1). London, GB: Wiley; 2009. [cited 2017 Apr 18]. Available from: http://site.ebrary.com/lib/alltitles/ docDetail.action?docID=10158782.

2. Seixas BV, Dionne F, Conte T, Mitton C. Assessing value in health care: using an interpretive classification system to understand existing practices based on a systematic review. BMC Health Serv Res. 2019;19:560.

3. Marsh K, Lanitis T, Neasham D, Orfanos P, Caro J. Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature. PharmacoEconomics. 2014;32:345–65.

4. Thokala P, Devlin N, Marsh K, Baltussen R, Boysen M, Kalo Z, et al. Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA emerging good practices task force. Value Health. 2016;19:1–13.

5. Diamantopoulos A, Sawyer LM, Lip GYH, Witte KK, Reynolds MR, Fauchier L, et al. Cost-effectiveness of an insertable cardiac monitor to detect atrial fibrillation in patients with cryptogenic stroke. Int J Stroke. 2016;11:302–12.

6. Stout NK, Lee SJ, Schechter CB, Kerlikowske K, Alagoz O, Berry D, et al. Benefits, harms, and costs for breast cancer screening after US implementation of digital mammography. J Natl Cancer Inst. 2014;106: dju092.

7. Bristow RE, Santillan A, Salani R, Diaz-Montes TP, Giuntoli RL, Meisner BC, et al. Intraperitoneal cisplatin and paclitaxel versus intravenous carboplatin and paclitaxel chemotherapy for stage III ovarian cancer: a cost- effectiveness analysis. Gynecol Oncol. 2007;106:476–81.

8. Harron K, Mok Q, Dwan K, Ridyard CH, Moitt T, Millar M, et al. CATheter Infections in CHildren (CATCH): a randomised controlled trial and economic evaluation comparing impregnated and standard central venous catheters in children. Health Technol Assess. 2016;20:1–219 vii–xxviii.

9. Mitton C, Donaldson C. Health care priority setting: principles, practice and challenges. Cost Eff Resour Alloc. 2004;2:3.

10. Cromwell I, Peacock SJ, Mitton C. ‘Real-world’ health care priority setting using explicit decision criteria: a systematic review of the literature. BMC Health Serv Res. 2015;15:164.

11. Polisena J, Clifford T, Elshaug AG, Mitton C, Russell E, Skidmore B. Case studies that illustrate disinvestment and resource allocation decision-making processes in health care: a systematic review. International journal of technology assessment in health care. Cambridge. 2013;29:174–84.

12. Hipgrave DB, Alderman KB, Anderson I, Soto EJ. Health sector priority setting at meso-level in lower and middle income countries: lessons learned, available options and suggested steps. Soc Sci Med. 2014;102:190– 200.

Appendix 3 Table 4 Extraction Tool

IDENTIFICATION

1. In what country did the implementation take place?

2. What level or part of the system did the priority setting implementation take place? (e.g., national, provincial, regional, hospital, community care, etc.)

3. What was the scope or context of the implementation? (e.g., drugs, vaccines, disease area, across disease areas, across sectors, etc.)

DECISION-MAKING

4. Who establishes the strategic guidance for the organization and how specific is it (i.e. how much room is there for interpretation)?

5. How does the organization establish priorities and make decisions on where to increase or reduce spending?

• Is there any formal process or framework that is used for that purpose or is it done on a case by case basis?

• How are requests for funding initiated? Who do they go to?

• Is current spending typically reviewed as part of the process?

• Are the organization’s priority setting decisions subject to review by external stakeholder(s)?

6. What stakeholders were involved in the decision-making process? (e.g., researchers, policy makers, public members, patients, clinicians)

7. What types of evidence/information are taken into account? (e.g., epidemiological evidence, clinical evidence, economic evidence, expert opinion, patient reported outcomes)

EVALUATION

8. Was there a reported discrepancy between ‘recommendations’ and actions taken?

9. Was the implementation formally evaluated and if so what were the findings of the evaluation (specifically, were health outcomes impact assessed)? How successful was the implementation? What are the key lessons learned?

10. What was the level of political involvement? Were notions of equity explicitly considered?

Seixas et al. Health Economics Review (2021) 11:2 Page 10 of 11

13. Barasa EW, Molyneux S, English M, Cleary S. Setting healthcare priorities in hospitals: a review of empirical studies. Health Policy Plan. 2015;30:386–96.

14. Wiseman V, Mitton C, Doyle-Waters MM, Drake T, Conteh L, Newall AT, et al. Using economic evidence to set healthcare priorities in low-income and lower-middle-income countries: a systematic review of methodological frameworks. Health Econ. 2016;25:140–61.

15. Allen R, Schiavo-Campo S, Garrity TC. Assessing and Reforming Public Financial Management: The World Bank; 2003. [cited 2020 Dec 14]. Available from: https://elibrary.worldbank.org/doi/abs/10.1596/0-8213-5599-6.

16. Kioko SN, Marlowe J, Matkin DST, Moody M, Smith DL, Zhao ZJ. Why public financial management matters. J Public Adm Res Theory. 2011;21:i113–24.

17. Public expenditure management handbook. [Internet]. [cited 2020 Dec 14]. Available from: https://elibrary.worldbank.org/doi/pdf/10.1596/0-8213-4297-5.

18. Alkaraan F. Public financial management reform: an ongoing journey towards good governance. J Financ Rep Account. 2018;16:585–609.

19. Peacock SJ, Richardson JRJ, Carter R, Edwards D. Priority setting in health care using multi-attribute utility theory and programme budgeting and marginal analysis (PBMA). Soc Sci Med. 2007;64:897–910.

20. Gallego G, Taylor SJ, Brien JE. Priority setting for high cost medications (HCMs) in public hospitals in Australia: a case study. Health Policy. 2007;84:58–66.

21. McDonald J, Ollerenshaw A. Priority setting in primary health care: a framework for local catchments. Rural Remote Health. 2011;11:1714.

22. Mentzakis E, Paolucci F, Rubicko G. Priority setting in the Austrian healthcare system: results from a discrete choice experiment and implications for mental health. J Ment Health Policy Econ. 2014;17:61–73.

23. Urquhart B, Mitton C, Peacock S. Introducing priority setting and resource allocation in home and community care programs. J Health Serv Res Policy. 2008;13(Suppl 1):41–5.

24. Dionne F, Mitton C, Smith N, Donaldson C. Evaluation of the impact of program budgeting and marginal analysis in Vancouver Island health authority. J Health Serv Res Policy. 2009;14:234–42.

25. Stafinski T, Menon D, McCabe C, Philippon DJ. To fund or not to fund. Pharmacoeconomics. 2011;29:771–80.

26. Mitton C, Dionne F, Damji R, Campbell D, Bryan S. Difficult decisions in times of constraint: Criteria based Resource Allocation in the Vancouver Coastal Health Authority: University of British Columbia; 2011. [cited 2017 Nov 5]. Available from: https://open.library.ubc.ca/cIRcle/collections/ facultyresearchandpublications/52383/items/1.0223748.

27. Cornelissen E, Mitton C, Davidson A, Reid C, Hole R, Visockas A-M, et al. Fit for purpose? Introducing a rational priority setting approach into a community care setting. J Health Organ Manag. 2016;30:690–710.

28. Greenberg D, Siebzehner MI, Pliskin JS. The process of updating the National List of health Services in Israel: is it legitimate? Is it fair? International journal of technology assessment in health care. Cambridge. 2009;25:255–61.

29. Ahn J, Kim G. Hae sun Suh, sang moo Lee. Social values and healthcare priority setting in Korea. J of Health Org and Mgt. 2012;26:343–50.

30. Choe YJ, Han OP, Cho H, Bae G-R, Chun B-C, Kim J-H, et al. Prioritization of the introduction of new vaccines to the national immunization program in the Republic of Korea. Vaccine. 2014;32:6049–53.

31. Ashton T, Tenbensel T, Cumming J, Barnett P. Decentralizing resource allocation: early experiences with district health boards in New Zealand. J Health Serv Res Policy. 2008;13:109–15.

32. Defechereux T, Paolucci F, Mirelman A, Youngkong S, Botten G, Hagen TP, et al. Health care priority setting in Norway a multicriteria decision analysis. BMC Health Serv Res. 2012;12:39.

33. Waldau S, Lindholm L, Wiechel AH. Priority setting in practice: participants opinions on vertical and horizontal priority setting for reallocation. Health Policy. 2010;96:245–54.

34. Waldau S. Bottom-up priority setting revised. A second evaluation of an institutional intervention in a Swedish health care organisation. Health Policy. 2015;119:1226–36.

35. Bate A, Donaldson C, Ray H. Resource allocation in orthopaedics: economic evaluation to priority setting. Clin Orthop Relat Res. 2007;457:49–56.

36. Wilson E, Sussex J, Macleod C, Fordham R. Prioritizing health technologies in a primary care trust. J Health Serv Res Policy. 2007;12:80–5.

37. Airoldi M, Bevan G, Morton A, Oliveira M, Smith J. Requisite models for strategic commissioning: the example of type 1 diabetes. Health Care Manage Scie. 2008;11:89–110.

38. Marsh K, Dolan P, Kempster J, Lugon M. Prioritizing investments in public health: a multi-criteria decision analysis. J Public Health (Oxf). 2013;35:460–6.

39. Goodwin E, Frew EJ. Using programme budgeting and marginal analysis (PBMA) to set priorities: reflections from a qualitative assessment in an English primary care trust. Soc Sci Med. 2013;98:162–8.

40. Airoldi M. Disinvestments in practice: overcoming resistance to change through a sociotechnical approach with local stakeholders. J Health Polit Policy Law. 2013;38:1149–71.

41. Holmes RD, Steele JG, Exley C, Vernazza CR, Donaldson C. Use of programme budgeting and marginal analysis to set priorities for local NHS dental services: learning from the north east of England. J Public Health (Oxf). 2018;40(4):e578-e585. https://doi.org/10.1093/pubmed/fdy075.

42. Vernazza CR, Taylor G, Donaldson C, Gray J, Holmes R, Carr K, et al. How does priority setting for resource allocation happen in commissioning dental services in a nationally led, regionally delivered system: a qualitative study using semistructured interviews with NHS England dental commissioners. BMJ Open. 2019;9:e024995.

43. Canham-Chervak M, Hooper TI, Brennan FH, Craig SC, Girasek DC, Schaefer RA, et al. A systematic process to prioritize prevention activities: sustaining Progress toward the reduction of military injuries. Am J Prev Med. 2010;38: S11–8.

44. Peacock S, Ruta D, Mitton C, Donaldson C, Bate A, Murtagh M. Using economics to set pragmatic and ethical priorities. BMJ. 2006;332:482–5.

45. INAHTA [Internet]. [cited 2018 Jul 12]. Available from: www.inahta.org. 46. Submission FAQs for Industry - Pharmaceuticals [Internet]. EUnetHTA. [cited

2019 Jul 19]. Available from: https://www.eunethta.eu/frequently-asked- questions-for-the-pharmaceutical-industry/.

47. Mooney GH, Russell EM, Weir RD. Choices for health care: a practical introduction to the economics of health provision. Macmillan; 1986.

48. Mitton C, Seixas BV, Peacock S, Burgess M, Bryan S. Health technology assessment as part of a broader process for priority setting and resource allocation. Appl Health Econ Health Policy. 2019;17:573–6.

49. Harris C, Green S, Elshaug AG. Sustainability in Health care by Allocating Resources Effectively (SHARE) 10: operationalising disinvestment in a conceptual framework for resource allocation. BMC Health Serv Res. 2017; [cited 2018 Jan 27];17. Available from: https://www.ncbi.nlm.nih.gov/pmc/ articles/PMC5590199/.

50. Daniels N, Sabin J. Setting limits fairly: can we learn to Share medical resources? 1 edition. Oxford. New York: Oxford University Press; 2002.

51. Daniels N. Accountability for reasonableness. BMJ. 2000;321:1300–1. 52. Gibson JL, Singer DKM, PA. Evidence, Economics and Ethics: Resource

Allocation in Health Services Organizations: Healthcare Quarterly; 2005. [cited 2017 May 29]. Available from: http://www.longwoods.com/content/1 7099.

53. Edwards RT, Charles JM, Thomas S, Bishop J, Cohen D, Groves S, et al. A national Programme budgeting and marginal analysis (PBMA) of health improvement spending across Wales: disinvestment and reinvestment across the life course. BMC Public Health. 2014;14:837.

54. Mooney G. Priority setting in mental health services. Appl Health Econ Health Policy. 2002;1:65–74.

55. Haas M, Viney R, Kristensen E, Pain C, Foulds K. Using programme budgeting and marginal analysis to assist population based strategic planning for coronary heart disease. Health Policy. 2001;55:173–86.

56. Angelis A, Kanavos P. Multiple criteria decision analysis (MCDA) for evaluating new medicines in health technology assessment and beyond: the advance value framework. Soc Sci Med. 2017;188:137–56.

57. Goetghebeur M, Wagner M, Khoury H, Levitt R, Erickson L, Rindress D. PMC50 evidence ANDVALUE: impact on decision making—the EVIDEM framework and potential applications. Value Health. 2008;11:A183–4.

58. Airoldi M, Morton A, Smith JAE, Bevan G. STAR—people-powered prioritization: a 21st-century solution to allocation headaches. Med Decis Mak. 2014;34:965–75.

59. Kapiriri L, Razavi D. How have systematic priority setting approaches influenced policy making? A synthesis of the current literature. Health Policy. 2017;121:937–46.

60. Tsourapas A, Frew E. Evaluating ‘success’ in Programme budgeting and marginal analysis: a literature review. J Health Serv Res Policy. 2011;16:177–83.

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Seixas et al. Health Economics Review (2021) 11:2 Page 11 of 11