OSH 3301 Unit IV Article Critique

Assessing the effectiveness of fire prevention strategies Mark Taylora, Deb Appletonb, Guy Keenc and John Fieldingd

aDepartment of Computer Science, Liverpool John Moores University, Liverpool UK; bStrategy and Performance, Merseyside Fire and Rescue Service, Liverpool UK; cCommunity Risk Management, Merseyside Fire and Rescue Service, Liverpool UK; dBusiness Intelligence Manager, Merseyside Fire and Rescue Service, Liverpool UK

ABSTRACT The authors examined the process of assessing the effectiveness of fire prevention within a Fire and Rescue Service in north west England, demonstrating how this was applied in practice. The approach to fire prevention strategy assessment included overall fire and rescue service performance, performance in relation to different population segments, cost per head performance, and multi-agency collaboration to support an effectiveness-based rather than outcomes-based performance assessment.

IMPACT This case study examination of the effectiveness of fire prevention strategies shows that fire prevention activities have a clear impact on the reduction of the number of fires. Major lessons for other fire and rescue services include that referrals to partner agencies were appropriate to assist in reducing factors associated with fire risk. In addition, concentrating fire prevention on the elderly was important given the proportion of fire fatalities and injuries involving this group. Finally, analysis of the effectiveness of fire prevention activities can inform fire prevention strategies.

KEYWORDS Effectiveness; fire; fire and rescue services; prevention; strategy

Introduction

McKevitt (2015) said that defining and measuring effec- tiveness at the organizational level is complex. Typi- cally, public sector organizations are assessed by output measures, rather than by effectiveness measures (Ammons & Roenigk, 2015). In terms of public management theory, there is a need for public sector organizations to move towards performance measures that assess effectiveness rather than just outputs—since, ultimately, it is important that public sector organizations are actually effective in what they do. English fire and rescue services are mainly assessed through outcome measures (FRNF, 2018). UK fire and rescue services are currently expanding their scope of operations through the ‘Safe and Well initiat- ive’ (Safe and Well, 2017), which is an evolution of the Home Fire Safety Check initiative (HFSC, 2017). The initiative involves health and social care prevention initiatives, as well as just fire prevention. This has the potential to improve fire prevention effectiveness by not only advising potentially vulnerable individuals of how to avoid house fires, but also overall to reduce the impact of factors such as alcohol consumption and smoking that have been linked with house fire inci- dence (Higgins, Taylor, Jones, & Lisboa, 2013). An important aspect of fire prevention is fitting and check- ing smoke alarms. Operational smoke alarms prevent or reduce many fires and the associated injuries or fatalities (Tannous et al., 2017). Costs associated with home fire safety checks can include internal costs

such as staff salaries, and on-costs, marketing costs, and travel costs, and external costs such as home fire safety material (batteries, smoke alarms, information booklets, calling cards, and fire blankets) (Tannous et al., 2107).

Puolokainen (2017) stated that the desired fire and rescue service outcomes that include the emergencies prevented or suppressed and, ultimately, the human life and property preserved are not easy to measure. Evaluations of the UK Home Fire Safety Check initiative in Wales identified a link between the installation of smoke alarms and reductions in dwelling fires and non-fatal casualties. The benefits of the Home Fire Safety Check initiative were found to far outweigh its costs (Welsh Government Social Research, 2013). Tannous et al. (2017), in a study in New South Wales, Australia, commented that analysis of the cost of fire within the region, combined with measurements of the success of similar home fire safety check pro- grammes internationally, demonstrated the home fire safety check programme’s cost-effectiveness in New South Wales. Weinholt and Andersson Granberg (2015) commented that a common tool for evaluating policy initiatives in the public sector is cost-benefit analysis (CBA), where all the benefits and costs are monetized and compared to each other, and that this approach was beginning to be applied to collabor- ations between emergency services.

The academic challenge of the research reported in this paper is to examine how a fire and rescue service

© 2019 CIPFA

PUBLIC MONEY & MANAGEMENT 2019, VOL. 39, NO. 6, 418–427 https://doi.org/10.1080/09540962.2019.1579439

can move from output-based performance measures to effectiveness-based measures and, in practical terms, how fire prevention strategies can be assessed in terms of their effectiveness. The potential impact of this research is the ability to inform future fire pre- vention approaches, based upon assessment of the effectiveness of current approaches. The originality of the research presented in this paper is the detailed examination of the process of assessing the effective- ness of fire prevention within a UK fire and rescue service. Although the UK National Audit Office (NAO, 2015) has argued that, despite the budget reductions experienced by UK fire and rescue services the sector had coped well, there is still a need to develop a deeper understanding of the effectiveness of current fire prevention activities, in order for future fire preven- tion activities to be as effective and efficient as poss- ible, given the decreasing level of funding available.

Literature review

Fire prevention approaches

UK fire and rescue services generally adopt an inte- grated risk management planning approach that involves the development of operational strategies to reducing risk within the community. This is typically achieved by combining prevention, protection and emergency response, on a risk-assessed basis in order to improve the safety of the community (MFRSIRM, 2018). Jennings (2013) concluded that research into fire in the community residential context has remained fragmented and isolated by discipline. Various fire pre- vention initiatives have been introduced in different communities around the world in an attempt to reduce fire incidence, morbidity and mortality. These initiatives have included education and training of chil- dren, parents, and certain high-risk populations such as the elderly (Jonsson et al., 2013; Wuschke, Clare, & Garis, 2013) in community-based programmes (Lehna et al., 2015; Warda et al., 1999). Andrews, Ashworth, and Meier (2014) commented that on the community service side, firefighters seek to build commitment to fire prevention. This is an area with fewer rules and much more discretion on how to approach the task, it is also an area in which the relationship between the service provider and client may hold the key to success. However, the research conducted by Andrews et al. (2014) focused more on equality and diversity in fire and rescue services rather than actual performance management.

Typically, the fitting or checking of smoke alarms is a major fire prevention initiative (Tannous et al., 2017) undertaken by fire and rescue services. Fires detected by smoke alarms tend to be discovered more rapidly and are associated with a reduced risk of death and property damage (Saramago et al., 2014). However,

Jennings (2013) warned that when developing fire pre- vention strategies based upon risk levels, the dynamic of fire risk may vary depending on the level of analysis. Care needs to be taken to avoid ecological fallacy (failure in reasoning that arises when an inference is made about an individual based on aggregate data for a group) when moving from higher-level statistical data to making conclusions about household or indi- vidual behaviour or risk.

The UK fire and rescue services’ safe and well initiat- ive involves not only fire prevention, but also health and social care prevention initiatives involving referrals to other agencies such as smoking cessation, and alcohol management. In terms of assessing the useful- ness of referrals to other agencies (such as the UK NHS or local council) via a home fire safety check or safe and well visit, the quality-adjusted life year (QALY) measure can be beneficial (Prieto & Sacristan, 2003). The QALY is a measure of the state of health of a person or group in which the benefits, in terms of length of life, are adjusted to reflect the quality of life. One QALY is equal to one year of life in perfect health. QALYs are cal- culated by estimating the years of life remaining for a patient following a particular treatment or intervention, and weighting each year with a quality-of-life score (on a 0 to 1 scale). It is often measured in terms of the person’s ability to carry out the activities of daily life, and freedom from pain and mental disturbance. This can be used to assess generic (and banded) levels of the person’s ability to carry out the activities of daily life (for example with regard to dementia patients). An important aspect of fire prevention approaches, such as the home fire safety check or safe and well visit, is targeting of resources towards those most at risk of fire. This includes vulnerable groups, such as the elderly, those with disabilities, and those with life- styles and behaviours (for example binge drinking and smoking) that can increase the risk of fire (Chainey, 2013; Corcoran, Higgs, & Anderson, 2013; Higgins et al., 2013; Taylor, Higgins, Lisboa, Jarman, & Hussain, 2016; Turner et al., 2017).

Fire prevention effectiveness

Devlin and Parkin (2004) stated that evidence on cost- effectiveness is used in many countries to inform decisions about the allocation of public funds to public services and products. Saramago et al. (2014) commented that assessing cost-effectiveness, as well as effectiveness, is important in a public sector system operating under a fixed budget restraint. Abdel- Maksoud, Elbanna, Mahama, and Pollanen (2015) stated that there can be issues regarding whether the development of performance measures is properly aligned with the objectives of public organizations. Kloot (2009) commented that fire and rescue service

PUBLIC MONEY & MANAGEMENT 419

performance assessments need to examine both oper- ational and financial performance outcomes.

Hastie and Searle (2016) commented that there are striking inequalities in the way in which dwelling fire incidence is distributed through society. Jennings (2013) found that fires can be comparable in cost to crimes and therefore warrant the significant attention of the public and governments at all levels. Warda, Tenenbein, and Moffact (1999) concluded that the effectiveness of fire prevention approaches in redu- cing fire incidence, injuries and deaths needs further study. Educational programmes, for example, are widespread, but their relative effectiveness has not been properly established. Clark and Smith (2015) commented that further research is required into how fire risk is perceived and mitigated against by individuals, in particular with regard to the impor- tance of clear communication in fire prevention strat- egies, including communicating the causes of fire. Studies of fire prevention interventions have reported variable success in reducing fire injury. Typically, this is measured in terms of the rates of fire incidence, injury and fatality before and after the fire prevention inter- vention. Typically, such measurements do not take into account socio-economic groupings, and do not include the use of control groups. Jaaskelainen et al. (2012) commented that, in general, measurement of service operations has proven to be a significant challenge.

McNamara (2017) discussed the difficulties associ- ated with determining fire prevention strategy priori- ties. Previous studies of fire prevention effectiveness have not really analysed whether targeting high-risk age groups achieves a greater reduction in injuries or produces a larger increase in the implementation of fire safety practices than alternative age prioritizations. Typically, studies of individual fire prevention pro- grammes may be unreliable due to inconsistent design or performance measures, lack of randomness or controls, and inadequate sample size or follow-up time. Murphy and Greenhalgh (2013) commented upon the changing nature of performance manage- ment regimes adopted by the UK government for fire and rescue services. Murphy and Greenhalgh (2013) found that a particular focus of performance review for English fire and rescue services was the integrated risk management plan, however, they also advocated that further assessment of performance management regarding efficiency and effectiveness was required. This is an important area of research, given that UK fire and rescue services need to meet the challenges of continuously improving their services. Carvalho, Fernandes, Lambert, and Lapsley (2006) and Adcroft and Willis (2005) stated that the delivery of public services can be undermined by the time, effort and management expertise deployed to cope with ever more elaborate forms of performance measures.

A study by the Welsh Government concluded that targeting was essential for successful home fire safety check initiatives. To reduce the number of dwelling fires, home fire safety checks need to be targeted at those most at risk, such as vulnerable populations (Welsh Government Social Research, 2013). Clare, Garis, Plecas, and Jennings (2012) commented that public education, and targeted home visits by fire and rescue services, have produced promising results via a range of outcome measures, from reduction in rates of fires and fire-related casualty, through to increased presence of working smoke alarms when residences were audited. Clare et al. (2012) used pre- and post-home visit comparisons via percentages and ANOVA to analyse for significant difference in fire inci- dence rates following home visits in British Columbia, Canada, by fire and rescue service staff.

Overall, previous research (Jaaskelainen et al., 2012; Murphy & Greenhalgh, 2013) has indicated that it can be difficult to conclusively deem any given fire preven- tion strategy to be effective, given the general absence of appropriate randomized controls across studies, without which positive results have to be interpreted with caution. The contribution of the research reported in this paper is the development of an approach to assess the effectiveness of fire prevention strategies, in order to inform future fire prevention approaches.

Research method

Merseyside fire and rescue service staff and staff from a university in the north west region of the UK were involved in the assessment of fire prevention strategy effectiveness over a six-month period. Fire and rescue service staff involved in the assessment of fire preven- tion strategy effectiveness included the community risk manager, the director of strategy and performance, the corporate information and systems manager, and the business intelligence manager.

The business intelligence manager performed the primary data collection. University staff and the business intelligence analyst undertook the analysis of the data following discussions regarding the analysis approach to be adopted involving the community risk manager, the director of strategy and performance, the corporate information and systems manager, and the business intelligence manager within the fire and rescue service studied. The research was internally driven by the fire and rescue service’s director of strat- egy and performance, and the community risk manager. Collection of data was mainly an internal exercise for the organization managed by the corpor- ate information and systems manager, and undertaken by the business intelligence manager, apart from the population data that was gathered by the university staff from the UK Office for National Statistics (ONS). In order to address issues of ethics, subjective bias

420 M. TAYLOR ET AL.

and objectivity to preserve the validity and sanctity of the data gathered, all the work undertaken conformed to the codes of ethics and professional conduct in oper- ation at Merseyside fire and rescue service and the uni- versity involved in the research.

Fire prevention within the area covered by the fire and rescue service was mostly conducted through the home fire safety check initiative (HFSC, 2017). Referrals to other agencies carried out during home fire safety checks (and later expanded) during safe and well (Safe and Well Visit Principles, 2017) visits included: smoking cessation referrals, alcohol reduction referrals, bowel cancer referrals, and falls risk assessment referrals. Barton and Valero-Silva (2013) had commented upon the positive impact on service users of successful multi-agency working.

The research questions posed by the work reported in this paper were:

. How can the effectiveness of overall fire and rescue service provision be assessed?

. How can the effectiveness of a given fire prevention strategy be assessed?

. How can the effectiveness and of the targeting of fire prevention be assessed?

The research questions arose from the need of the fire and rescue service studied to assess effectiveness of operations. Previously English fire and rescue ser- vices were mainly assessed through outcome measures (FRNF, 2018); however, a new inspection approach by HM Inspectorate of Constabularies, Fire and Rescue Ser- vices (HMICFRS, 2018) for the 45 fire and rescue ser- vices in England places greater emphasis on measures of efficiency and effectiveness. During the period of the study, Merseyside fire and rescue service had not had an inspection under the new inspectorate. In addition, given the reductions in the rates of fire in England during the last 10 years (during a period of reducing budgets), fire prevention, and in particular the targeting of fire prevention are increasingly important aspects of fire and rescue service operations.

Our research questions are important since the con- tinuing budget reductions experienced by fire and rescue services in the UK necessitates a deeper under- standing of the effectiveness of current fire prevention activities, in order for future fire prevention activities to be as effective and efficient as possible given the decreasing level of funding available. Funding for Mer- seyside fire and rescue service decreased to £59.490 million for 2017/2018, from £73.576 million for 2004/ 2005. Merseyside has had the largest budget reductions of any English fire and rescue service (MFRS, 2018). The main objective of the research was to develop an approach that could assess effectiveness of fire and rescue service operations at different levels:

at an overall operational level, and at the level of oper- ational fire prevention.

Study of the outcomes from an intervention might typically use random samples from intervention and control groups. However, in the case of fire prevention, due to duty-of-care requirements, it would not be appropriate, having identified vulnerable groups, to apply such fire prevention in some areas within the region and not others, in order to assess the relative impact on the number of fire incidents, injuries and fatalities. In addition, in the case of fire prevention, typi- cally home fire safety checks and safe and well visits would be conducted across the areas in a region over a relatively long period of time (due to the number of visits involved, and the limited numbers of fire safety officers). For these reasons, analysis of effectiveness was more appropriate over a longer time period, than shorter term analyses. The research approach adopted was appropriate given the time periods over which fire prevention activities are conducted, and the time taken for the impact of fire prevention approaches to be measurable in terms of the frequency of production of fire statistics. Data for the analyses undertaken in order to answer the research questions posed was obtained from Merseyside fire and rescue service and the ONS.

The conceptual contribution of the research is the development of an approach to assessing the effective- ness of fire prevention strategies, based upon analysis of the approach to targeting fire prevention, and multi- agency collaboration. A limitation of the research pre- sented here is that the results are only relevant to one fire and rescue service serving a relatively small popu- lation, and therefore the applicability and generalizabil- ity to a wider population may be limited. However, due to the variable nature of the populations served by different fire and rescue services and the different area geographies, and the different funding levels, meaning- ful comparisons of effectiveness between different fire and rescue services would be difficult to interpret.

Research results

Assessing the effectiveness of overall fire and rescue service provision

The first stage in the examination of fire prevention strategy effectiveness was to determine what quantifi- able measures could potentially be used to assess the effectiveness of fire and rescue service provision. Dis- cussions with the relevant staff within Merseyside fire and rescue service identified the following potential measures:

. Overall number of fire incidences, injuries and fatal- ities per year within the area covered by the fire and rescue service.

PUBLIC MONEY & MANAGEMENT 421

. Number of fire incidences, injuries and fatalities per year within the different population segments within the area covered by the fire and rescue service.

. Proportion of individuals within each different popu- lation segment per 100,000 of population within the area covered by the fire and rescue service involved in a fire incidence, injury and fatality.

These measures could be used to assess overall fire and rescue service performance, performance in relation to different population segments, and per- formance in relation to vulnerable population seg- ments. These measures address the research question about how to assess the effectiveness of overall fire and rescue service provision. Table 1 shows the overall number of fire incidences, injuries and fatalities in the period 2006 to 2016 within the area covered by Merseyside fire and rescue service.

In terms of overall fire and rescue service effective- ness indicated by how many fire incidences, injuries

and fatalities occurred over the 10-year period 2006 to 2016, there was a clear downward trend in the number of fire incidences. In the other 44 fire and rescue services in England, the number of fire inci- dents had also been on a general downward trend since 2004 (HO, 2017). This data also answers our research question about how to assess the effective- ness of overall fire and rescue service provision because, by comparing fire-incidence rates over time, a significant reduction in fire incidences was identified. In terms of the pattern of fire injuries and fatalities, due to their relatively small numbers, a few fires could sig- nificantly affect the figures, which makes interpreting the data more difficult. Table 2 shows the overall number of fire incidences, injuries and fatalities by population segments within the Merseyside area from 2006 to 2016. Merseyside fire and rescue service had identified the elderly (those aged 65+) as being the most vulnerable population segment in terms of fire risk (as had other previous research: for

Table 2. Overall number of fire incidences, injuries and fatalities by population segments in the Merseyside area: 2006 to 2016. (Data regarding dwelling occupants involved in a fire incidence is typically only recorded for fire injuries and fatalities.)

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Fire incidences (all ages)

1384 1318 1315 1337 1248 1183 1090 1221 1023 1084 1049

Fire injuries (age 65+)

35 31 35 24 25 43 36 52 40 35 25

Fire injuries (age < 65)

94 68 70 91 101 102 71 93 77 73 75

Fire fatalities (age 65+)

4 5 5 3 3 4 1 4 6 10 6

Fire fatalities (age < 65)

2 6 5 3 4 1 2 6 3 5 4

Figure 1. Fire instances in the Merseyside area: 2006 to 2016.

Table 1. Overall number of fire incidences, injuries and fatalities in Merseyside: 2006 to 2016. 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Fire incidences 1384 1318 1315 1337 1248 1183 1090 1221 1023 1084 1049 Fire injuries 129 100 106 115 126 145 110 146 118 110 104 Fire fatalities 6 11 10 6 7 5 3 10 9 15 10

422 M. TAYLOR ET AL.

example Mulvaney et al., 2009; and Corcoran, Higgs, Rohde, & Chhetri, 2011). In the period 2006 to 2016 for the Merseyside area, 56% of fire fatality victims were over the age of 65.

This data also answers our research question about how to assess the effectiveness of overall fire and rescue service provision because, by examining fire fatal- ity and injury rates in greater detail via different ident- ified population segments, it was possible to examine the differences in frequency between different popu- lation segments over time. Table 3 shows the proportion of individuals within each different population segment involved in fire incidences, injuries and fatalities by population segment per 100,000 of population in the Merseyside area in the period 2006 to 2016.

Data regarding dwelling occupants involved in a fire is typically only recorded for fire injuries and fatalities. This data also answers our research question about how to assess the effectiveness of overall fire and rescue service provision by examining the proportions of different identified population groups involved in fire incidences. This allows an examination of the rela- tive rates of fire injuries and fatalities between different population segments over time. In particular, the pro- portion of elderly individuals involved in fire injuries and fatalities was significantly higher than the pro- portion of the non-elderly population.

For the purposes of calculating population pro- portions, the population estimate figures for Mersey- side were as shown in table 4 (Nomis, 2017).

The population estimate figures for Merseyside resi- dents aged 65+ were as shown in table 5 (Nomis, 2017).

There appeared to be a clear downward trend in the overall proportion of the Merseyside population involved in a fire incidence over the 10-year period studied. The small proportions of those involved in a fire injury and fatality meant that these figures were less clear to interpret. However, there appeared to be a clear indication that those aged 65+ were proportion- ately more likely to be injured or die in a dwelling fire than those aged less than 65.

Assessing fire incidences, injuries and fatalities compared to spend per head

The next effectiveness measures assessed the number of fire incidences, injuries and fatalities compared to the relevant spend per head:

. Fire incidences, injuries and fatalities versus overall spend per head of overall population in the area.

. Fire incidences, injuries and fatalities versus overall fire prevention spend per head of overall population in the area.

. Fire injuries and fatalities versus overall fire preven- tion spend per head of different population seg- ments in the area.

Table 6 shows the overall fire and rescue service spend per head (in £s) and the overall number of fire incidences, injuries and fatalities in the Merseyside area in the period 2006 to 2016.

Table 7 shows the ratio of overall fire and rescue service spend (or cost) (in £000s) to fire incidents/

Table 5. Population figures for Merseyside residents aged 65 + 2006 to 2016 (000s). 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

230 231 232 234 236 239 247 251 256 259 263

Table 3. Proportion of individuals within each different population segment involved in fire incidences, injuries and fatalities in the Merseyside area: 2006 to 2016.

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Fire incidences proportion (all ages)

101.1 96.4 96.1 97.5 90.7 85.7 78.7 88.1 73.5 77.5 74.6

Fire injuries proportion (age 65+)

15.2 13.4 15.1 10.3 10.6 18.0 14.6 20.7 15.6 13.5 9.5

Fire injuries proportion (age < 65)

8.3 6.0 6.2 8.0 8.9 8.9 6.2 8.2 6.8 6.4 6.6

Fire fatalities proportion (age 65+)

1.7 2.2 2.2 1.3 1.3 1.7 0.4 1.6 2.3 3.9 2.3

Fire fatalities proportion (age < 65)

0.2 0.5 0.4 0.3 0.4 0.1 0.2 0.5 0.3 0.4 0.3

Table 4. Population figures for Merseyside 2006 to 2016 (000s). 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

1,368 1,367 1,369 1,372 1,376 1,381 1,386 1,387 1,391 1,398 1,406

PUBLIC MONEY & MANAGEMENT 423

injuries/fatalities. This data also answers the research question: How can the effectiveness of overall fire and rescue service provision be assessed, by examining the spending associated with fire incidents, injuries and fatalities over time, in terms of the cost per fire incident, injury and fatality.

Assessing the effectiveness of fire prevention strategies

Table 8 shows the overall fire prevention spend per head (in £s) and the number of fire incidences, fire incidents involving injuries, and fire incidents involving fatalities in the Merseyside area in the period 2006 to 2016.

This appeared to indicate that overall the spend per head on fire prevention has been effective in reducing the number of fire instances over the 10-year period studied. However, the impact upon fire injuries and fatalities was more difficult to interpret. Table 9 shows the ratio of overall fire prevention spend (or cost) (in £000s) to fire incidents/fire incidents involving injuries/fire incidents involving fatalities.

This data answers our research question about how to assess the effectiveness of a given fire prevention strategy by examining the spending associated with fire prevention compared with the number of fire inci- dents, injuries and fatalities. However, due to fluctu- ations in the spend per head of population, and in the overall spend on fire prevention per year over the period studied, it was difficult to discern an overall

trend or pattern. For example, whether spending more on fire prevention was associated with fewer fire injuries and fatalities, or spending less on fire pre- vention was associated with fewer fire injuries and fatalities year on year.

Assessing the effectiveness of the targeting of fire prevention

Table 10 shows the fire prevention spend per head (in £s) and the number of fire injuries and fatalities by population segment in the Merseyside area in the period 2006 to 2016. (Data regarding dwelling occu- pants involved in a fire incidence is typically only recorded for fire injuries and fatalities.)

This data answers our research question about how to assess the effectiveness of the targeting of fire pre- vention by examining the spending associated with fire prevention for different population groups com- pared with the number of fire incidents, injuries and fatalities for those groups. Overall, fire prevention activities appear to have had a clear impact on the number of fires, which steadily reduced during the period 2006 to 2016, even though the spend per head of the population on fire prevention fluctuated between £13.50 and £16.99 during this period.

The impact of fire prevention on fire injuries and fire fatalities over the period studied was less clear. The number of fire incidents involving injuries fluctuated between 77 and 114 with a mean value of 92.6, and

Table 6. Overall fire and rescue service spend per head (in £s) and overall number of fire incidences, injuries and fatalities in the Merseyside area, 2006 to 2016.

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Spend per head of population

48.72 50.23 51.80 51.07 52.24 51.14 52.63 53.64 52.07 43.37 46.73

Fire incidences 1384 1318 1315 1337 1248 1183 1090 1221 1023 1084 1049 Fire injuries 129 100 106 115 126 145 110 146 118 110 104 Fire fatalities 6 11 10 6 7 5 3 10 9 15 10

Table 8. Overall fire prevention spend per head (in £s) and overall number of fire incidences, fire incidents involving injuries, and fire incidents involving fatalities in the Merseyside area, 2006 to 2016.

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Fire prevention spend per head of population

16.07 15.11 13.72 14.26 13.50 14.73 14.00 15.62 16.99 13.86 16.11

Fire incidences 1384 1318 1315 1337 1248 1183 1090 1221 1023 1084 1049 Fire incidents involving injuries

109 84 85 90 84 112 86 114 91 86 77

Fire incidents involving fatalities

6 11 10 6 7 5 3 9 9 14 7

Table 7. Ratio of overall fire and rescue service spend (or cost) (in £000s) to fire incidents/injuries/fatalities in the Merseyside area, 2006 to 2016.

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Cost/fire incidence ratio 48 52 54 52 58 60 67 61 71 56 63 Cost/fire injuries ratio 517 687 669 609 570 487 663 510 614 551 632 Cost/fire fatalities ratio 11108 6242 7091 11678 10269 14125 24315 7440 8048 4042 6570

424 M. TAYLOR ET AL.

the number of fire incidents involving fatalities fluctu- ated between 3 and 14 with a mean value of 7.9. There did not appear to be an overall trend with either the numbers of fire incidents involving injuries, or the numbers of fire incidents involving fatalities. Nor did there appear to be a link between the spend per head of the population on fire prevention in a given year and the numbers of fire incidents involving injuries, or fire incidents involving fatalities in that year (or the following year).

However, analysis of the factors associated with fire fatalities over the period 2006 to 2016 in the Mersey- side area revealed that:

. 50.5% of fire fatalities involved smoker’s materials, indicating that the referrals to NHS smoking cessa- tion services that are an integral part of the home fire safety checks carried out by Merseyside fire and rescue service were an appropriate and necess- ary aspect of current and future fire prevention.

. 39.8% of fire fatalities involved alcohol consump- tion, indicating that the referrals to NHS alcohol reduction services that are an integral part of the home fire safety checks carried out by Merseyside fire and rescue service were an appropriate and necessary aspect of current and future fire prevention.

. 55.9% of fire fatalities and 21.5% of fire injuries involved those aged over 65, indicating that this age group was the most in need of fire prevention

support, which is the current main focus of fire pre- vention activities carried out by Merseyside Fire and Rescue Services.

. 64.5% of the fire fatalities involved those living alone, indicating that this social group was also the most in need of fire prevention support.

This data also answers the research question about how to assess the effectiveness of the targeting of fire prevention by examining the different population groups and lifestyles and behaviours associated with fire fatalities. Overall, the effectiveness measures described in this research combined what was done by the fire and rescue service studied, the value to the users of the service (the population of Merseyside), and the manner in which the services were delivered (Fryer, Antony, & Ogden, 2009).

Effectiveness of communication strategy

A particular issue with regard to assessing the effective- ness of fire prevention approaches was attempting to understand the communication aspect of fire preven- tion (Andrews et al., 2014). Behavioural insights can potentially aid in determining how best to contact/ influence people with regard to fire prevention accord- ing to the behaviours they display (Clark & Smith, 2015). Typically, communication methods include face- to-face via home fire safety checks or safe and well visits, and leaflets distributed to local residents as part

Table 9. Ratio of overall fire prevention spend per head of population (or cost) (in £000s) to fire incidents/fire incidents involving injuries/fire incidents involving fatalities in the Merseyside area, 2006 to 2016.

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Fire prevention cost/incidence ratio 16 16 14 15 15 17 18 18 23 18 22 Fire prevention cost/incidences involving injuries ratio

202 246 221 217 221 182 226 190 260 225 294

Fire prevention cost/incidences involving fatalities ratio

3664 1878 1878 3261 2654 4068 6468 2407 2626 1384 3236

Table 10. Fire prevention spend per head (in £s) and the number of fire injuries and fatalities by population segment in the Merseyside area, 2006 to 2016.

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Fire prevention spend per head of population (Age 65+)

2.70 2.55 2.32 2.43 2.32 2.55 2.49 2.83 3.12 2.57 3.01

Fire prevention spend per head of population (Age < 65)

13.37 12.56 11.39 11.83 11.18 12.18 11.51 12.49 13.86 11.30 13.11

Fire injuries (Age 65+)

35 31 35 24 25 43 36 52 40 35 25

Fire injuries (Age < 65)

94 68 70 91 101 102 71 93 77 73 75

Fire fatalities (Age 65+)

4 5 5 3 3 4 1 4 6 10 6

Fire fatalities (Age < 65)

2 6 5 3 4 1 2 6 3 5 4

PUBLIC MONEY & MANAGEMENT 425

of a publication scheme under the UK Freedom of Infor- mation Act 2000. In addition, referral to other agencies, such as NHS smoking cessation or alcohol reduction, can provide an effective communication approach.

Conclusions

Evidence on cost-effectiveness is used in many countries to inform decisions about the allocation of public funds to public services and products. In this paper we have examined a fire prevention strategy assessment approach that supports moving from output-based performance measures to effectiveness- based performance measures. The fire prevention strat- egy assessment approach included overall fire and rescue service performance, performance in relation to different population segments (especially the most vulnerable groups such as the elderly), and fire inci- dences, injuries and fatalities compared to spend per head of the population covered by the fire and rescue service concerned.

The key contribution from the research reported in this paper is the development of an approach to assessing the effectiveness of fire prevention strat- egies, based on analysis of the approach to targeting fire prevention, and multi-agency collaboration. Our approach measures overall fire and rescue service per- formance, performance in relation to different popu- lation segments, and performance in relation to vulnerable population segments using fire and rescue service data and UK national statistics and NHS data.

In terms of overall fire and rescue service effective- ness indicated by how many fire incidences, injuries and fatalities occurred over the 10-year period studied, there was a clear downward trend in the number of fire incidences within the region. There also appeared to be a clear downward trend in the overall proportion of the Merseyside population involved in a fire incidence over the 10-year period. The small proportions of those involved in a fire injury or fatality meant that these figures were less easy to interpret. Over the 10-year period studied, the fire and rescue service spend per head on fire preven- tion appeared to have been effective in reducing the number of fire instances.

Given that a significant proportion of the fire fatal- ities over the time period studied involved smoker’s materials and alcohol consumption, this indicated that referrals to NHS smoking cessation and alcohol reduction services were an appropriate and necessary aspect of current and future fire prevention.

The implications for theory and practice of the research reported in this paper are the need to move from output-based performance measures to effective- ness-based performance measures. In this paper we have examined how this can be achieved via detailed

analysis of fire incidences, injuries and fatalities in terms of the different population segments within the area covered by a given fire and rescue service. In addition, it is necessary to analyse the costs associated with fire prevention for the different population seg- ments and, finally, it is important to recognize the need for fire and rescue services to work closely with other public sector agencies.

A limitation of the research presented is that due to the variable nature of the populations served by different fire and rescue services and the different area geographies, and the different funding levels, meaning- ful comparisons of effectiveness between different fire and rescue services would be difficult to interpret. Future research could, however, examine the use of the fire prevention effectiveness assessment measures in other fire and rescue service contexts. It is hoped that the examination of fire prevention strategy assess- ment reported in this paper will be of benefit to other fire and rescue services, both in the UK and elsewhere.

References

Abdel-Maksoud, A., Elbanna, S., Mahama, H., & Pollanen, R. (2015). The use of performance information in strategic decision making in public organizations. International Journal of Public Sector Management, 28(7), 528–549.

Adcroft, A., & Willis, R. (2005). The (un)intended outcome of public sector performance measurement. International Journal of Public Sector Management, 18(5), 386–400.

Ammons, D., & Roenigk, D. (2015). Performance management in local government: is practice influenced by doctrine? Public Performance and Management Review, 38(3), 514– 541.

Andrews, R., Ashworth, R., & Meier, K. (2014). Representative bureaucracy and fire service performance. International Public Management Journal, 17(1), 1–24.

Barton, H., & Valero-Silva, N. (2013). Policing in partnership: a case study in crime prevention. International Journal of Public Sector Management, 26(7), 543–553.

Carvalho, J., Fernandes, M., Lambert, V., & Lapsley, I. (2006). Measuring fire service performance: a comparative study. International Journal of Public Sector Management, 19(2), 165–179.

Chainey, S. (2013). Using the vulnerable localities index to identify priority areas for targeting fire safety services. Fire Safety Journal, 62, 30–36.

Clare, J., Garis, L., Plecas, D., & Jennings, C. (2012). Reduced fre- quency and severity of residential fires following delivery of fire prevention education by on-duty fire fighters: Cluster randomized controlled study. Journal of Safety Research, 43, 123–128.

Clark, A., & Smith, J. (2015). Experiencing a domestic fire: an overview of key findings from a post incident research pro- gramme. Safer Communities, 14(2), 95–103.

Corcoran, J., Higgs, G., Rohde, D., & Chhetri, P. (2011). Investigating the association between weather conditions, calendar events and socio-economic patterns with trends in fire incidence: an Australian case study. Journal of Geographic Systems, 13, 193–226.

Corcoran, J., Higgs, G., & Anderson, T. (2013). Examining the use of a geodemographic classification in an exploratory

426 M. TAYLOR ET AL.

analysis of variations in fire incidence in South Wales, UK. Fire Safety Journal, 62, 37–48.

Devlin, N., & Parkin, D. (2004). Does NICE have a cost-effective- ness threshold and what other factors influence its decisions? A binary choice analysis. Health economics, 13 (5), 437–452.

FRRF (2018) Fire and Rescue National Framework for England, UK Home Office. Retrieved from https://www.gov.uk/ government/publications/fire-and-rescue-national-frame work-for-england–2

Fryer, K., Antony, J., & Ogden, S. (2009). Performance manage- ment in the public sector. International Journal of Public Sector Management, 22(6), 478–498.

Hastie, C., & Searle, R. (2016). Socio-economic and demo- graphic predictors of accidental dwelling fire rates. Fire Safety Journal, 84, 50–56.

HFSC (2017) Home fire safety check. Retrieved from https:// www.merseyfire.gov.uk/aspx/pages/prevention/campaign. aspx

Higgins, E., Taylor, M., Jones, M., & Lisboa, P. (2013). Understanding Community Fire Risk – A Spatial Model for Targeting Fire Prevention Activities. Fire Safety Journal, 62, 20–29.

HMICFRS (2018) HM Inspectorate of Constabularies, Fire and Rescue Services, How we inspect fire and rescue services. Retrieved from https://www.justiceinspectorates.gov.uk/ hmicfrs/fire-and-rescue-services/how-we-inspect-fire-and- rescue-services/

HO (2017) UK Home Office, Fire Statistics: England April 2015 to March 2016. Retrieved from https://assets.publishing. service.gov.uk/government/uploads/system/uploads/ attachment_data/file/611182/fire-statistics-england- 1516-hosb0517.pdf

Jaaskelainen, A., Laihonen, H., Lonnqvist, A., Palvalin, M., Sillanpaa, V., Pekkola, S., & Ukko, J. (2012). A contingency approach to performance measurement in service oper- ations. Measuring Business Excellence, 16(1), 43–52.

Jennings, C. (2013). Social and economic characteristics as determinants of residential fire risk in urban neighbor- hoods: A review of the literature. Fire Safety Journal, 62, 13–19.

Jonsson, A., Runefors, M., Sardqvist, S., & Nilson, F. (2016). Fire- related mortality in Sweden: temporal trends 1952 to 2013. Fire technology, 52(6), 1697–1707.

Kloot, L. (2009). Performance measurement and accountabil- ity in an Australian fire service. International Journal of Public Sector Management, 22(2), 128–145.

Lehna, C., Coty, M., Fahey, E., Williams, J., Scrivener, D., Wishnia, G., & Myers, J. (2015). Intervention study for changes in home fire safety knowledge in urban older adults. Burns, 41(6), 1205–1211.

McKevitt, D. (2015). Debate: Value for money - in search of a definition. Public Money & Management, 35(2), 99–100.

McNamara. (2017). Questioning risk-based fire and life safety education age priorities. Injury Prevention, 23, 73–74.

MFRS (2018) Merseyside Fire and Rescue Authority Budget and Medium Term Financial Plan Resolution 2015/2016 - 2019/20. Retrieved from http://mfra.merseyfire.gov.uk/ documents/s6640/Appendix%20A%20Budget% 20Resolution.pdf

MFRSIRM (2018) Merseyside fire and rescue service Integrated Risk Management Plan. Retrieved from https://www. merseyfire.gov.uk/aspx/pages/IRMP/pdf/IRMP_2017-20_ Update_2017.pdf

Mulvaney, C., Kendrick, D., Towner, E., Brussoni, M., Hayes, M., Powell, J., … Ward, H. (2009). Fatal and non-fatal fire inju- ries in England 1995-2004: time trends and inequalities by age, sex and area deprivation. Journal of Public Health, 31(1), 154–161.

Murphy, P., & Greenhalgh, K. (2013). Performance manage- ment in fire and rescue services. Public Money & Management, 33(3), 225–232.

NAO (2015) Financial sustainability of fire and rescue services, UK National Audit Office HC 491. Retrieved from https:// www.nao.org.uk/report/financial-sustainability-of-fire-and- rescue-services/

Nomis, Official labour market statistics, UK Office for National Statistics. Retrieved from https://www.nomisweb.co.uk/

Prieto, L., & Sacristan, J. (2003). Problems and solutions in calcu- lating quality-adjusted life years (QALYs). Health and Quality of Life Outcomes, 1(80), doi: 10.1186/1477-7525-1-80

Puolokainen, T. (2017). Reforming fire and rescue services: a comparative study of Estonia and Georgia. International Journal of Public Sector Management, 30(3), 227–240.

Saar, I. (2017). The effects of the lower ignition propensity cigarettes standard in Estonia: time-series analysis. Injury prevention, DOI: 10.1136/injuryprev-2016-042187

Safe and Well (2017) Fire Safety in the Home, Safe and Well visit. Retrieved from http://www.fireservice.co.uk/safety/

Safe and Well Visit Principles (2017) Principles for a Safe and Well visit by a Fire and Rescue Service, Public Health England. Retrieved from https://www.england.nhs.uk/wp- content/uploads/2015/09/safe-well-visit-pinciples.pdf

Saramago, P., Cooper, N., Sutton, A., Hayes, M., Dunn, K., Manca, A., & Kendrick, D. (2014). Cost-effectiveness of inter- ventions for increasing the possession of functioning smoke alarms in households with pre-school children: a modelling study. BMC public health, 14(1), 459–471.

Tannous, W., Whybro, M., Lewis, C., Broomhall, S., Ollerenshaw, M., Watson, G., … Franks, E. (2017). Home Fire Safety Checks in New South Wales: an economic evalu- ation of the pilot program. Journal of Risk Research, DOI: 10. 1080/13669877.2017.1281336

Taylor, M., Higgins, E., Lisboa, P., Jarman, I., & Hussain, A. (2016). Community fire prevention via population segmen- tation modelling. Community Development Journal, 51(2), 229–247.

Turner, S., Johnson, R., Weightman, L., Rodgers, S., Arthur, G., Bailey, R., & Lyons, R. (2017). Risk factors associated with unintentional house fire incidents, injuries and deaths in high-income countries: a systematic review. Injury Prevention, 23, 131–137.

Warda, L., Tenenbein, M., & Moffact, M. (1999). House fire injury prevention update. Part II. A review of the effective- ness of preventive interventions. Injury Prevention, 5, 217– 225.

Welsh Government Social Research (2013) Evaluation of the Home Fire Safety Check, Knowledge and Analytical Services, Welsh Government, Rhydycar, Merthyr Tydfil, CF48 1UZ.

Weinholt, A., & Andersson Granberg, T. (2015). New collabor- ations in daily emergency response: Applying cost-benefit analysis to new first response initiatives in the Swedish fire and rescue service. International Journal of Emergency Services, 4(2), 177–193.

Wuschke, K., Clare, J., & Garis, L. (2013). Temporal and geo- graphic clustering of residential structure fires: A theoreti- cal platform for targeted fire prevention. Fire Safety Journal, 62, 3–12.

PUBLIC MONEY & MANAGEMENT 427

Copyright of Public Money & Management is the property of Routledge and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.