Safety Performance Monitoring and Measurement
Liberty University
AVIA 409 - Safety Management Systems
Professor: Andrew Walton
2022
Definition
Safety performance monitoring and measurement is one of three elements that comprise the
safety assurance component of the International Civil Aviation Organisation (ICAO) safety management
system (SMS) framework. Safety assurance consists of processes and activities undertaken by the service
provider to determine whether the SMS is operating according to expectations and requirements.
Safety performance monitoring and measurement represents the means to verify the safety
performance of the organisation and to validate the effectiveness of safety risk controls. (ICAO Doc 9859
- Safety Management Manual)
EUROCONTROL has defined it as a systematic action conducted to detect changes affecting the air
traffic management (ATM) system with the specific objective of identifying that acceptable or tolerable
safety can be met. (EUROCONTROL ESARR3)
Description
Regulatory requirements related to safety management and implementation of an SMS mandate
aviation service providers to develop and maintain the means to verify the safety performance of the
organisation. Safety performance monitoring and measurement enables continuous monitoring and
regular assessment of the safety levels achieved by an organisation during service delivery. The safety
performance shall be verified in reference to the safety policy and approved safety objectives (safety
performance targets) using dedicated safety performance indicators (SPIs). Trends should be analysed
and, consequently, corrective actions should be identified after detecting any deterioration of specified
safety levels.
Safety monitoring could be an important source of information for measuring the effective
functioning of other SMS components. According to EUROCONTROL ESARR4 - Risk Assessment and
Mitigation in ATM, safety monitoring and data collection mechanisms could be specifically developed
and used for the validation of the safety assumptions and requirements identified in the risk
assessment and mitigation processes, as well as for the validation of safety models used in risk
management. Also, safety monitoring and data collection support the development of safety indicators
at national, regional and global level.
Information Sources for Safety Monitoring
Information used to measure an organisastion’s safety performance is generated through its safety
reporting systems, and there are two types of reporting systems:
Mandatory incident or occurrence reporting systems, which require the reporting of certain
types of events, such as serious incidents and accidents; and
Voluntary incident or occurrence reporting systems, which allow for the submission of
information related to observed hazards or inadvertent errors without a legal requirement to do
so.
Other sources of safety information to support safety performance monitoring and measurement may
include:
Safety audits and informal inspections, which focus on the integrity of the organisation's SMS
and its supporting systems.
Safety surveys that examine procedures or processes related to a specific operation.
Safety occurrence reporting
Investigation of safety occurrences
Safety studies, which are rather large analyses encompassing broad safety concerns. An in-
house safety analysis can find arguments and make use of findings and recommendations of
studies carried out at industry-wide or global scale. Because of their nature, safety studies are
more appropriate to address system safety deficiencies rather than identify specific, individual
hazards.
Safety reviews that are conducted during introduction and deployment of new technologies,
change or implementation of procedures, or in situations of a structural change in operations.
Safety reviews are a fundamental component of the management of change. They have a clearly
defined objective that is linked to the change under consideration.
Systematic capture of data reflecting actual day-to-day performance (by means of dedicated
programmes, such as Flight Data Monitoring, Line Operations Safety Audit (LOSA) and Normal
Operations Safety Survey (NOSS))
Continuous monitoring of day-to-day activities related to the delivery of services.
Each of these types of information source may be available to an organisation. However,
specifications about what these sources should be and what they should “look like” are not available -
individual organisations should tailor them to the appropriate scope and scale.
Procedures and Arrangements for Safety Monitoring
The size and structure of the organisation, and the operational environment have to be considered
when setting up safety monitoring arrangements in the organisation. The guidance material developed
by EUROCONTROL suggests the following basic steps for establishing effective safety monitoring and
measurement processes:
Identification of indicators to be monitored
The scope of monitoring should cover operational, technical and organisational (safety management)
aspects. Safety indicators can be quantitative or qualitative, leading (proactive) or lagging (reactive).
Collation of the information for safety monitoring
There should be a systematic collation and evaluation of results from all safety monitoring activities to
ensure that interrelationships can be detected.
Analysis of indicators
The evolution of the indicators should be analysed, trends and related causes and influencing factors
established.
Application of corrective action process
Corrective actions should be determined, taken and followed up wherever the monitoring shows that an
element is approaching a point which may affect safety to an unacceptable extent; coordination with
relevant units/organisations should take place wherever necessary; the indicators and their evolution
should be documented as well as actions taken and their results.
Safety Monitoring Methods and Programmes
Not long ago the safety monitoring processes relied mainly on staff identifying existing or potential
risks and hazards by using their best judgement, experience and skills. Often if unsafe practices are used
regularly as a normal method of operation, it is very likely that personnel involved in these activities
would not recognise the actual and potential threats.
The need to improve the control of human error and develop countermeasures to error in
operational environments lead to the introduction of observation-based methods, such as LOSA and
NOSS and data analysis programmes, such as FDA. These methods are used to systematically capture
data which reflect actual day-to-day performance and provide proactive means for safety monitoring
and improvement.
