Annotated Bibliography on Aviation Maintenance

Abstract:

Voluntary reporting programs such as the Aviation Safety Action Program and Aviation Safety Reporting System exist so that organizations are able to detect problems, trends, and hazards during flight operations and make iterative improvements, constituting proactive safety management. Reporting systems are an integral component of safety management systems for flight schools. Understanding the difference in mishap frequency by phase of flight will help the aviation industry become more aware of when errors are occurring during flight training. This study examined the Operational Mishap and Incident Reports (OMIRs) from a collegiate flight program in the southeastern United States. De-identified data from the OMIRs for 2015-2017 were provided for the study. All reports were classified into seven phases of flight: ground-parked, ground-moving, takeoff and climb, cruise, maneuvers, pattern operations, and descent and landing. There was a significant difference in the frequency of OMIRs by phase of flight. Ground phases, both parked and taxiing, had the highest frequency of reports, followed by descent and landing.

Full text:

Headnote

Abstract

Voluntary reporting programs such as the Aviation Safety Action Program and Aviation Safety Reporting System exist so that organizations are able to detect problems, trends, and hazards during flight operations and make iterative improvements, constituting proactive safety management. Reporting systems are an integral component of safety management systems for flight schools. Understanding the difference in mishap frequency by phase of flight will help the aviation industry become more aware of when errors are occurring during flight training. This study examined the Operational Mishap and Incident Reports (OMIRs) from a collegiate flight program in the southeastern United States. De-identified data from the OMIRs for 2015-2017 were provided for the study. All reports were classified into seven phases of flight: ground-parked, ground-moving, takeoff and climb, cruise, maneuvers, pattern operations, and descent and landing. There was a significant difference in the frequency of OMIRs by phase of flight. Ground phases, both parked and taxiing, had the highest frequency of reports, followed by descent and landing.

1.Introduction

Voluntary incident reporting aims to identify variables that may lead to incidents and accidents in order to make corrective actions for future safety (Runciman, 2001). Given that approximately 34% of all general aviation accidents occur during flight training, and around 95% of those incidents are due to human error (Houston, 2012), the identification of mistakes and patterns of error during flight training through voluntary reporting has potential for a powerful effect on safety in aviation.

The purpose of this study was to determine whether there was a difference in the frequency of OMIRs by phase of flight at a collegiate flight school in the southeastern United States. Phase of flight was classified into seven phases: ground parked, ground moving, takeoff and climb, cruise, maneuvers, pattern operations, and descent and landing. The flight school provided archival data from all OMIRs submitted from January 2015 to December 2017 in the flight school's safety management system (SMS).

Research and awareness is critical to promoting a safer flight-training environment for student pilots and instructors (Houston, 2012). Safety management systems are uncommon at collegiate flight schools. Thus, access to OMIRs provided a unique opportunity to learn about where errors are occurring in flight training. Understanding the variation in incidents and mishaps by phase of flight during flight training can be used to improve awareness for student pilots and flight instructors by highlighting specific phases of flight that incur more OMIRs. This could be the first step in identifying how to reduce the potential for mistakes and incidents that warrant a report. In addition, developing a plan to help reduce the potential of future mishaps will help lead the industry towards a safer flight environment.

This research was limited by the nature of voluntary reporting: it is possible that not all mishaps and incidents were reported. However, the SMS has built in incentives for reporting to increase participation, including immunity for pilots. It is also possible that some situations that warranted a report did not result in an OMIR because the pilot was not aware that the event should be reported through an OMIR. That said, the reporting system was introduced to all new student pilots, and flight instructors help teach new students about the system. Therefore, pilots at the flight school should be aware of the reporting system, when it is appropriate to submit a report, and the benefits to participating in the voluntary reporting system.

1.1. Aviation voluntary reporting systems

Although most aviation accidents have been attributed to human factors (NTSB, 2010), accidents, incidents, and lower-level errors or mishaps can be viewed from either an individual or system approach. The individual perspective focuses on mistakes, potentially caused by fatigue, task overload, or lack of attention. However, the system perspective emphasizes the role of the organization and environment in protecting against mistakes (Reason, 2000). From the organizational perspective, developing a positive culture of safety management is critical, and a central part of this is reporting mistakes so that learning can occur, rather than blame (Reason, 2000).

The aim of voluntary reporting in aviation is the ability to learn from more common small errors, in order to improve safety and avoid more severe accidents. Effective reporting systems in fields from aviation to medicine can enable learning through four essential functions: 1) gathering data that feeds into learning and safety culture rather than punishment or disciplinary action; 2) collecting complete data with closed and open questions; 3) implementing data analysis to translate reports into safety outcomes; and 4) providing constructive rather than critical feedback (Majahjan, 2010). Reporting systems need buy-in from everyone to function well (Majahjan, 2010; Runciman, Merry, & Smith, 2001). To increase participation, anonymity of reports allows for participation without fear of negative career impacts and increases the ability to collect data about causal reasons for errors (Runciman et al., 2001). Alternatively, reporting systems may incentivize participation by offering immunity or partial immunity in exchange for reporting and submission to any training deemed necessary. Regardless of the method, maximizing participation in a reporting system is critical to its effectiveness as a tool to learn from mistakes and proactively manage risk.

In aviation, several reporting systems exist. The Federal Aviation Administration (FAA) established the Aviation Safety Action Program (ASAP) to create voluntary reporting systems through safety partnerships between the FAA and businesses. For example, airlines and unions work with the FAA to promote confidential reporting of unsafe situations or occurrences. While not anonymous, some level of protection from punitive action and confidentiality serves to increase participation. The National Aeronautics and Space Administration (NASA) has oversight of the Aviation Safety Reporting System (ASRS) for the FAA, in order for the FAA, as the regulating agency, to maintain distance from the program. ASRS aims to support human factors research to identify issues and improve safety, is publicly accessible, and accepts submissions from anyone who witnesses a safety incident (e.g. air traffic controllers, pilots, maintenance, flight attendance). One of the central elements is confidentiality and anonymity provided by removing identifiers from data when it becomes part of the database and protection for pilots who submit reports (Cusick, Cortes, & Rodrigues, 2017).

Within organizations, developing a safety culture that places value on voluntary reporting is a central component to a safety management system. Although safety management systems are a recent development at collegiate flight programs, voluntary reporting enables identification of safety issues and iterative improvement. At the collegiate flight program in this study, a voluntary reporting system of OMIRs is used to collect data on safety-related events. This also serves to ensure that everyone involved in the flight program has an outlet to be heard, increasing participation and engagement (Runciman et al., 2001). The OMIRs may be submitted by anyone and serve as the interface to the outside reporting system, ASAP. Therefore, the reports also offer pilots performing operations for the primary business of the flight school immunity, a further incentive for participation. Collection of voluntary reports through the SMS also enables monitoring and safety research.

1.2. Phases of flight

The FAA (2016) breaks flight operations into nine phases of flight: preflight/taxi, takeoff, climb, cruise, descent, maneuvering, approach, landing, and other. While climb, cruise, and descent phases comprise 83% of flight time, only approximately 21.6% of general aviation accidents occur during these phases. In contrast, takeoff and initial climb represent only two percent of flight time but 23% of accidents, and 24% of accidents occur during landing (FAA, 2016). Thus, the takeoff and landing phases of flight alone account for approximately half of the accidents in general aviation.

Pilots are trained to anticipate workload and manage their workload. However, during a given flight, the approach and landing phase is when the pilot workload is highest. Pilots may reach task saturation, which means the workload may exceed the lowest margin of safety (FAA, 2016). This is one reason that a higher percentage of accidents occur during landing: the high workload may result in more human errors.

While the general aviation accident risk is highest in the takeoff and landing phases of flight (FAA, 2016), research is necessary to determine the incidence of mistakes by phase of flight during flight training. The voluntary reporting system consisting of OMIRs at a collegiate flight school provided the means to quantify and explore the differences in mistakes between phases of flight.

2.Methods

We used an ex post facto methodology in order to compare the frequency of OMIRs by phase of flight. An Institutional Review Board exemption (18-168) was approved. Deidentified data maintained the anonymity of individuals who submitted reports, and the study used archival data, having no direct contact with human subjects. The flight school provided de-identified data files, containing all OMIRs submitted from January 2015 through December 2017, or three full calendar years.

The flight phase column in the reports was used as the initial classification of phase of flight. Full reports and descriptions were used to code the 18 different phases of flight found in the dataset into seven categories: (a) ground parked, (b) ground moving, (c) takeoff and climb, (d) cruise, (e) maneuvers, (f) pattern operations, and (g) descent and landing. These categories align closely with the FAA (2016) phases of flight; however, parked and taxiing ground phases were separated, and takeoff and climb were combined. The number of OMIRs for each phase of flight was tallied by month (36 months). Descriptive statistics were calculated in Microsoft Excel. RStudio version 1.1.383 was used to calculate a one-way Analysis of Variance (ANOVA), Tukey's pairwise comparison, and effect size.

3.Results

A total of 689 OMIRs were provided in the archival data; Table 1 shows the breakdown of OMIRs by year and by total flight hours at the flight school. This represents an average reporting rate of approximately one OMIR per 70 flight hours. A total of 18 OMIRs were excluded because the phase of flight was unknown or unclear from the report, resulting in 671 OMIRs that were used for the analyses. Figure 1 and Table 2 depict the 671 total OMIRs by phase of flight. Ground parked had the highest frequency of reports (177), followed by ground-moving (163) and descent and landing (115). The two ground phases together accounted for just over half of the total reports. The other five phases had fewer reports, with the fewest reports during maneuvers (38). Ground parked had the highest standard deviation (3.5), and maneuvers and cruise had the lowest standard deviations (1.1). Figure 2 depicts the average monthly OMIRs submitted by phase of flight with the standard deviation overlaid.

The ANOVA found a significant effect for phase of flight: F (6, 245) = 17.65, p< .01. To determine which phases of flight had different OMIR frequencies, a post hoc analysis was necessary. Tukey's pairwise comparison showed that 11 pairs of phases of flight had different OMIR frequencies at p< .02 (Table 3). Ground parked and ground moving were not different from each other, but were different from other phases: takeoff and climb, pattern operations, maneuvers, and cruise. Descent and landing was different from cruise and maneuvers. The eta-squared was 0.302, which is a small effect size.

4.Discussion

The data on OMIR frequency by month supported the hypothesis that there would be a difference in the frequency of reporting by phase of flight. As illustrated in Tables 2 and 3 and Figure 2, there is a difference in voluntary reporting based on phase of flight. The frequencies of OMIRs from the ground parked and the ground moving phases were significantly higher than other phases of flight, except descent and landing. Descent and landing had a higher frequency than cruise and maneuvering. However, the effect size (eta squared= 0.302) was small, suggesting that even though there is a statistical difference, the difference in the real world is small.

The difference in voluntary reporting underscores the higher frequencies during the ground parked and ground moving phases, which approximately half of the OMIRs referenced. There are several reasons that this may be the case. First, while still on the ground, the preflight examination of the aircraft and startup procedures have checklists, creating a multitude of potential mishaps if they are not properly completed. Second, while all operations necessarily include some phases of flight (e.g. ground parked, ground moving, takeoff and landing), not all flights will include every phase. Third, a self-report by a flight student or a report by a flight instructor would be most common for OMIRs during flight. However, on the ground, there are other people at the flight line that may observe an unsafe event and submit an OMIR, introducing the potential for more reports made by observers. Anyone can submit a report, and there are more observers on the ground.

Descent and landing was also significantly higher than cruise and maneuvering. Again, while almost all flights involve ramp or taxi phase and landing phase, not all operations have a cruise or maneuver phase, depending on the aims of a particular flight lesson. While parked and taxing reports were the most common, all flights involve a landing phase, and this is a heavy workload phase of flight for the pilot, introducing potential for mistakes, as compared to cruise and maneuvers, which are more straightforward and require a lower pilot workload.

When interpreting these results, the number or frequency of OMIRs should not be equated with risk. That is, even though the ground parked and ground moving phases had the highest OMIR frequencies, other phases of flight may have the higher risks. One cannot automatically assume that a higher number of reports is the result of a higher risk level in a specific flight phase.

Future research should evaluate trends in voluntary reporting over time. Elements such as number of daily operations at the flight line and airspace category should also be examined for their impact on frequency of reports. Finally, research relating the severity of the incident or potential risk with OMIR frequency would help to further understand these relationships and risks.

Voluntary incident reporting enables the aviation industry to proactively identify issues to improve safety. Analyzing OMIRs at collegiate flight training programs increases understanding of which phases of flight have more incidents and mishaps: ground parked, ground moving, and descent and landing. This, in turn, can be used by flight instructors to improve training. However, a higher frequency of reports in a specific phase does not necessarily mean that there is a higher associated level of risk.

References

5.References

Cusick, S. K., Cortes, A. I., & Rodrigues, C. C. (2017). Commercial aviation safety. New York: McGraw Hill Professional.