Journal of Aviation/Aerospace Education & Research Volume 32 Number 3 Article 1 2023 Safety in Flight Training - An Analysis of the NTSB Data 2014-2018 Michael F. Walach Ph.D. Montana State University-Bozeman, michael.walach@montana.edu Follow this and additional works at: https://commons.erau.edu/jaaer Part of the Aviation and Space Education Commons, and the Aviation Safety and Security Commons Scholarly Commons Citation Walach, M. F. (2023). Safety in Flight Training - An Analysis of the NTSB Data 2014-2018. Journal of Aviation/Aerospace Education & Research, 32(3). DOI: https://doi.org/10.58940/2329-258X.1954 This Article is brought to you for free and open access by the Journals at Scholarly Commons. It has been accepted for inclusion in Journal of Aviation/Aerospace Education & Research by an authorized administrator of Scholarly Commons. For more information, please contact commons@erau.edu. Walach: Safety in Flight Training - NTSB Data Abstract There were 7,500 safety events in the NTSB data sets from 2013-2018. These events were analyzed using Chi-square, Cramer’s V, and the odds ratio. Major findings in the study determined that while pilots crash aircraft for the same reasons whether they are in a training environment or not, student pilots are typically less likely to be killed, or seriously injured. The aircraft that student pilots fly however, do not share the same relative safety in some event types. Students destroy and substantially damage more aircraft than their non-training counterparts in abnormal runway contact events. The top five causes of safety events for all pilots are loss of control in flight, system component failure of the power plant, abnormal runway contact, fuel related issues, and loss of control on the ground. While the data analyzed in this study cannot explain the causation of these findings, they set the stage for further study of training accidents to determine possible explanations of these differences. Building on findings in similar studies, this researcher suggests that annual flight reviews for general aviation pilots contain more scenario-based simulation under real flight conditions as is found in the training for Part 121 operators. It is theorized that some of the safety found in the training environment may come not just from the supervision of the flight instructor, but also from the repeated practice and attention to safety procedures. General aviation has been plagued with a poor safety record for a long time with little to no progress in reducing safety events, and more importantly, fatalities. It is the hope of this researcher that findings from this study may help others to dig deeper into some of these issues and find areas of focus that may help reduce the risk of injury or death for general aviation pilots. Keywords: flight training, training safety, NTSB accidents Published by Scholarly Commons, 2023 1 Journal of Aviation/Aerospace Education & Research, Vol. 32, No. 3 [2023], Art. 1 Introduction Flight students, and the instructors who train them, work in a high stress, high risk training environment. While fatal accidents in flight training are not common, they are still a risk, and take an average of 17.8 (about 1 per 315,000 hours flown) students every year in the United States (National Transportation Safety Board [NTSB] 2013, 2014, 2015, 2016, 2017, 2018). Students and non-students crash airplanes for the same reasons, but the extent of damage to the airplane and the pilots varies dramatically between the two groups. Examining the severity of injuries and damage to aircraft by accident type can provide a much clearer picture of the risks associated with flight training. To better understand the risk factors related to flight training, general aviation accidents have been examined in this study. By identifying the most common factors in training accidents, steps can then be taken to improve training safety and mitigate training risk. Furthermore, the risks of flying beyond flight training can be better understood, thereby possibly reducing the chance of accidents for newly trained pilots as they begin their journey into aviation. Commercial aviation in the United States is considered the safest form of transportation with very few major accidents and no catastrophic loss of aircraft in just over 11 years (NTSB, 2020). In 2019, the NTSB reported one fatality on a Part 121 flight with more than 19,000,000 flight hours flown (NTSB, 2019). Crew resource management, situational awareness, and a multitude of safety management systems have contributed to this strong safety record (Dizikes, 2020). General aviation, however, does not benefit from such a safe record. General aviation accident rates have not seen much change in the last several years, and loss of control in flight is still the number one cause for general aviation accidents (NTSB, 2014, 2015, 2016, 2017, 2018). https://commons.erau.edu/jaaer/vol32/iss3/1 DOI: 10.58940/2329-258X.1954 2 Walach: Safety in Flight Training - NTSB Data Training flights are of particular interest as a large number of pilots will retire from jobs in the commercial (Part 121) and corporate (Part 135) airlines, and there is a demand for new pilots (Aviation Week Network Staff, 2020). In aviation, good pilots are always trying to mitigate risk as much as possible. As new pilots begin their training to fill the roles in the airline industry, how much risk as they taking by learning to fly airplanes? Are the risks associated with training pilots higher than or different than other general aviation flights? Lee et al. (2017) examined the influencing factors and accident severity in training flights. The study found a fatal accident occurring during training had a 4.05 times higher chance of happening during dual instruction than solo. While both dual and solo flights accident causation was tied to skill deficiency and landing errors, there were more fatal accidents during dual instructional flights. What is surprising in the study, however, is there were more serious accidents when an instructor was present than without. Also, a student was more likely to die in an accident with an instructor present than when flying solo. The Aircraft Owners and Pilots Association (AOPA) published research findings on flight training from the Air Safety Institute (2014). The AOPA report found that fatal accident rates were about half compared to non-training flights. Solo student accidents were most often (80%) during take-off, landing and go-arounds. Less common causes of accidents for students were fuel mismanagement, adverse weather or mechanical failures. They also found that most fatal stall accidents for solo primary student pilots occurred on landing attempts. Dual instruction stalls happened most often while maneuvering or conducting simulated emergency procedures. Houston et al. (2012) studied loss of control accidents during instructional flights. The leading cause of training accidents during the time examined in their study (2000-2009) was “loss of control in flight.” They found the leading causal factor for loss of control in training Published by Scholarly Commons, 2023 3 Journal of Aviation/Aerospace Education & Research, Vol. 32, No. 3 [2023], Art. 1 flights was failure to maintain directional control n=78, failure to maintain airspeed (n=45), inadequate supervision (n=43), stall/spin (n=40), improper procedure (n=35), and weather (n=31). Houston et al. (2012) also found 66% of the instructors in their study had over 1500 hours of flight time, suggesting increased instructor experience has little impact on accident rates. Finally, Houston et al. (2012) determined that student flight hours in a specific type aircraft increased, their risk of accidents decreased, however as instructor hours increased, the risk of an accident increased, rather than decreased. Majumdar et al. (2021) investigated GA loss of control in flight (LOC-I) accidents using a state-based approach. This method considers states that may exist throughout an accident such as mechanical issues that existed pre-flight but were ignored or unnoticed, or a pilots diminished mental or physical state. By digging deeper into the cause of LOC-I accidents they found that 4% of LOC-I crashes were related to mechanical issues with the aircraft while about 20% of LOC-I accidents involved collision with an object or terrain. While their study doesn’t examine training flights specifically, their technique of accident analysis adds a depth of understanding that helps to better code GA accidents. Kalagher et al. (2021) focused on GA accidents related to a loss of situational awareness (SA). They found that GA accidents that were a result of loss of SA were fatal a shocking 59.6% of the time! Students in the training environment can often become task saturated and fall into a state of cognitive overload, all potential contributors to loss of SA. de Voogt et al. (2022) examined accidents in the “go-around” phase of flight. They found that while accident rates in GA had gone down in some areas, the “go-around” phase of flight saw an increase in accident rates. It may be possible that the “go around” is not practiced by GA https://commons.erau.edu/jaaer/vol32/iss3/1 DOI: 10.58940/2329-258X.1954 4 Walach: Safety in Flight Training - NTSB Data pilots much outside of flight training and could be a contributor to GA accidents coded by the NTSB as “abnormal runway contact.” Uitdewilligen and Voogt (2009) examined accidents involving solo student pilots in the United States using NTSB data between 2001-2005. All accidents involving students on a first solo flight in the sample resulted in only minor injuries and one aircraft destroyed. No fatalities or serious injuries were found for the first solo flights. Students’ severity of injury increased with total flight hours, consistent with the findings in the Houston et al. (2012) study. Pilot errors were the leading contributing cause (93%) of accidents with failure to maintain control, correct for crosswind, or decisions/planning errors. Accidents in the landing phase of flight were less likely to result in serious or fatal injuries than accidents happening in other phases of flight. Cruise and maneuvering phases of flight were the most fatal for students. Uitdewilligen and Voogt agree with Houston et al. (2012) and Lee et al. (2013) that students with higher total flight hours have more accidents. They all attribute this to the possibility that students with higher flight hours are having more difficulty reaching proficiency and therefore are more likely to have an accident. Boyd and Dittmer (2016) conducted a similar study to Uitdewilligen and Voogt (2009) and focused only on student solo flights, but they examined accidents from 1994-2013. They restricted their research to solo flights in Cessna 172 aircraft. They found training accidents accounted for 14% of all general aviation accidents. Their findings confirmed the Uitdewilligen and Voogt (2009) study that 90% of solo accidents resulted in only minor or no injuries. The aircraft were not as fortunate as the students however, as 97% of the aircraft involved had substantial damage. Published by Scholarly Commons, 2023 5 Journal of Aviation/Aerospace Education & Research, Vol. 32, No. 3 [2023], Art. 1 Wilson and Sloan (2003) analyzed visual flight rules (VFR) flight into instrument meteorological condition (IMC) accidents. They found VFR into IMC the leading cause of fatal weather-related general aviation accidents. Recommendations from their study include hazardous weather avoidance procedures in training, truthfulness (by flight instructors) about the dangers of VFR into IMC and encouraging private pilot students to pursue an instrument rating. Ison (2014) identified pilot and situational characteristics that have correlations to VFR into IMC conditions. The study found pilots involved in a fatal VFR into IMC crash were 19 times more likely to have received a weather briefing, and 10 times more likely to have been in mountainous terrain. This means pilots involved in these accidents were usually in areas with high terrain (which can hide poor weather) but had received weather data that likely would have warned of poor visibility. He did find however, as a pilots ratings went up, his/her likelihood of VFR into IMC went down (r =-.641). The study suggests a focus on weather education and hazardous pilot attitudes. Randel et al. (2008) detailed loss of control on landing, focusing on unstable approaches, and the decision not to go-around. They found, in landing accidents, loss of control was the largest factor (32.8%). Wind (17.2%), Surface conditions (6%), landing gear malfunction (5.6%) and aircraft configuration (4.7%) and “all other” (33.7%) made up the rest of landing accident causes. The only way to recover from an unstable approach is to go-around. The study recommends better training procedures focused on identifying unstable approaches and when to execute a go-around. None of the studies presented here have examined the most recent NTSB data (2014- 2018). At the time of this study, NTSB data from 2014-2018 was complete and available for analysis. The literature suggests training flight accidents might be different from non-training https://commons.erau.edu/jaaer/vol32/iss3/1 DOI: 10.58940/2329-258X.1954 6 Walach: Safety in Flight Training - NTSB Data flight accidents in severity of injuries, and phase of flight. While the top primary causes of general airplane accidents are the same for training and non-training flights, a deeper analysis needs to be explored to determine if there are differences in causation of accidents for students vs non-students. Research Questions Q1: How do flight-training (student) accidents differ from non-flight-training (non- student) accidents? Q2: How are the risk factors that lead to training accidents associated with the risk factors that lead to non-training accidents? Q3: How do the odds of having a specific type of training accident differ from the odds of having a specific type of non-training accidents? Method National Transportation Safety Board (NTSB) data from 2014 to 2018 was analyzed in SPSS. The 2018 data set contains 1,292 accidents. The data sets for each year from 2014-2017 data are of similar size. To narrow the scope and breadth of this study, a focus was placed on instructional vs non-instructional flights only. The NTSB uses the code “instructional,” for training flights and this is the variable used to determine training vs non-training accidents. Accidents are then further coded by injury type, and damage to aircraft. NTSB codes for accident types are used in this study and include abnormal runway contact, loss of control in flight, loss of control on ground, system component failure power-plant, and fuel related. NTSB codes for injury are used and include fatal, serious, minor, none. Finally, NTSB codes for aircraft damage used include destroyed, substantial, minor, none. The previous NTSB codes and the variables are compared in this study. Descriptive statistics were used to rank accidents by type. Chi Square Published by Scholarly Commons, 2023 7 Journal of Aviation/Aerospace Education & Research, Vol. 32, No. 3 [2023], Art. 1 analysis and Cramer’s V were used to analyze accidents and an odd’s ratio was calculated to determine chance of accident, damage, and injury. Definition of Terms Training flights are defined as flights in which a pilot is receiving instruction from a certified flight instructor (CFI) or is under the supervision of a CFI conducting solo flights as part of a training program. Non-training flights are any general aviation flights in which no formal instruction was being given or supervised by a CFI. Only general aviation Title 14 Code of Federal Aviation Regulations (FAR) Part 91 operations in the United States have been included in this study. All aircraft types including airplanes, rotorcraft, ultralights, and homebuilt aircraft are included in the study sample. The classifications of bodily injury as reported by the NTSB have been used and they include fatal, serious, minor, none. The classifications for aircraft damage as reported by the NTSB have also been used and they include destroyed, substantial, minor, none. NTSB accident categories are also used and include abnormal runway contact, loss of control in the air, loss of control on the ground, system component failure of the power-plant, and fuel related. The term “probable cause” is used by the NTSB to define the most likely cause of an accident based on a review of factual data collected in an investigation. The term “student” is used for any pilot conducting a training flight regardless of the license they were operating under at the time. Some training flights were conducted for primary training while other flights may have been training toward advanced ratings (instrument, multiengine, commercial, etc.) while others could have been check-rides for additional ratings. Every attempt has been made to keep terminology consistent with that reported by the NTSB and any errors or deviations within are solely the fault of the author. https://commons.erau.edu/jaaer/vol32/iss3/1 DOI: 10.58940/2329-258X.1954 8 Walach: Safety in Flight Training - NTSB Data Data Analysis There were a total of 7,509 accidents investigated by the NTSB in the years 2013 through 2018 which were analyzed in this study. The geographic locations of both training and non- training accidents are representative of the population densities of the states within the United States. A correlation test was run to compare accidents rates with state populations (training events r=.790 p=.000 non-training events r=.810 p=.000) demonstrating and even distribution of accidents within all 50 states. First, all general aviation accidents investigated by the NTSB were ranked by the critical event that led to the accident as determined by the NTSB. Next, training accidents were examined, and ranked by critical event determined by the NTSB, as shown in Table 1. Table 1 Top Five Accident Types Top 5 Training Accidents (n=789) Top 5 Non-Training Accidents (n=4,507) Abnormal Runway Contact (n=237) Loss of Control In-Flight (n=1,192) Loss of Control In-Flight (n=192) System Component Failure Powerplant (n=1,170) Loss of Control on Ground (n=181) Loss of Control on Ground (n=1,026) System Component Failure Powerplant Abnormal Runway Contact (n=743) (n=141) Fuel (n=38) Fuel (n=376) All accidents that resulted in fatalities were examined next for all general aviation flights regardless of training or non-training flights as shown in Table 2. The leading causes of death in order are loss of control in flight (n=591), system component failure of the powerplant (141), and controlled flight into terrain (n=103). The category “unknown” accounted for the fourth most Published by Scholarly Commons, 2023 9 Journal of Aviation/Aerospace Education & Research, Vol. 32, No. 3 [2023], Art. 1 common fatalities (n=96). It should be noted, most general aviation aircraft do not have a flight data recording system as would be found on commercial aircraft, making determining causation much more difficult especially when there are no eyewitnesses, and all aboard perished in the crash. Table 2 GA Fatal Accidents by Type Number of Fatal GA Accidents Critical Event 591 Loss of Control In-Flight 141 System Component Failure Powerplant 103 Controlled Flight into Terrain 96 Unknown Next, accidents that resulted in fatalities and defined as training flights by the NTSB were ranked by critical events. Loss of control in flight remained the number one cause of fatalities for training flights, followed by system component failure of the powerplant. Accidents resulting in fatalities and not defined as training flights by the NTSB were rank ordered by critical events. Loss of control in-flight remained the number one cause of fatalities for non-training flights, followed by system component failure of the powerplant. It is worth noting abnormal runway contact resulted in six fatalities for non-training flights; however there were no fatalities resulting from abnormal runway contact for training flights. Unintended flight into Instrument Meteorological Conditions (IMC ranks number five for non-training flights but is not a factor for any accidents for training flights. A Chi Square analysis was performed to determine the level of association in terms of safety risk outcomes in training vs non-training accidents. Additionally, an odds ratio was https://commons.erau.edu/jaaer/vol32/iss3/1 DOI: 10.58940/2329-258X.1954 10 Walach: Safety in Flight Training - NTSB Data calculated to determine the magnitude of risk for each type of injury or aircraft damage between training and non-training flights. Abnormal Runway Contact Abnormal runway contact accidents were compared at each of the four injury categories used by the NTSB investigators: fatal, serious, minor, and none (Table 3). Significance values of (p<.05) show a relationship between the training flight accidents and non-training flight events. The odds ratio values listed in the table depict the odds a training flight would result in an accident compared to a non-training flight. Values <1 represent events less likely to occur while values >1 represent events more likely to occur. A pilot on a training flight that has an abnormal runway contact is .747 times less likely to be seriously injured, 1.268 times more likely to have only minor injuries and 2.169 times more likely to have no injuries than a pilot on a non-training flight that has an abnormal runway contact. The only group to show a significant relationship was injuries = “none” (p=.000). The Cramer’s V value of .122 was one of the strongest found in this study, although this is a very small strength. Table 3 Abnormal Runway Contact (ARC) - Injuries ARC df Chi- p n n Non- Odds Cramer’s V Cramer’s V Square Training training Ratio Value Sig. Fatal 1 - - 0 9 - .014 .224 Serious 1 0.460 .498 6 49 .747 .008 .498 Minor 1 0.762 .383 16 77 1.268 .010 .383 None 1 112.224 .000 215 605 2.169 .122 .000 Note. This group had a value less than 5. Published by Scholarly Commons, 2023 11 Journal of Aviation/Aerospace Education & Research, Vol. 32, No. 3 [2023], Art. 1 Next, damage to the aircraft was compared between training and non-training accidents (Table 4). The NTSB categories of aircraft damage, destroyed, substantial, minor, and none, were used for this comparison. Pilots conducting training flights were 1.744 times more likely to destroy the aircraft, and 1.998 times more likely to substantially damage the aircraft in an abnormal runway contact accident than pilots who were not on a training flight. Accidents with minor damage were close to equal with training flights 1.017 more likely to have minor damage to the aircraft. The only group to have a statistically significant association was the substantial damage group (p=.000) and a Cramer’s V of .116. Table 4 Abnormal Runway Contact (ARC) - Damage ARC df Chi- p N N Non- Odds Cramer’s Cramer’s Square Training training Ratio V Value V Sig. Destroyed 1 - - 2 7 1.744 .008 .482 Substantial 1 100.554 .000 234 715 1.998 .116 .000 Minor 1 - - 1 6 1.017 .000 .987 None 1 - - 0 15 - .018 .117 Note. This group had a value less than 5. Loss of control in flight accidents were the leading cause of all general aviation accidents for non-training flights and the second for training flights in this study (Table 5). Pilots conducting training flights were less likely to die (.515 times) or have serious injuries (.674 times) in a loss of control accident than pilots not on a training flight. Minor injuries were close to equally likely (1.104 times) for training flights vs non-training flights. Pilots on a training flight who had a loss of control accident were 2.013 times more likely to walk away from the https://commons.erau.edu/jaaer/vol32/iss3/1 DOI: 10.58940/2329-258X.1954 12 Walach: Safety in Flight Training - NTSB Data event with no injuries compared to pilots on non-training flights. The fatal group had a significant association (p=.000) as did the no injuries group (p=.000). Table 5 Loss of Control In-Flight (LOC-I) - Injuries LOC-I df Chi- p N N Non- Odds Cramer’s Cramer’s Square Training training Ratio V Value V Sig. Fatal 1 21.004 .000 46 545 .515 .053 .000 Serious 1 2.762 .097 19 172 .674 .019 .097 Minor 1 0.310 .578 36 199 1.104 .006 .578 None 1 36.656 .000 91 276 2.013 .070 .000 Loss of control in-flight data on aircraft damage shows most (83% training and 72% non- training) loss of control in-flight accidents substantially damage the aircraft (Table 6). Training flights are slightly more likely (1.132 times) to have substantial damage than non-training flights, however training flights are .575 times less likely to destroy the aircraft than non-training flights. There were no loss-of-control in-flight accidents that had no damage and only 1 training flight had minor damage. There was a significant association between the “destroyed” aircraft groups (p=.002) and a very small Cramer’s V of .035. Published by Scholarly Commons, 2023 13 Journal of Aviation/Aerospace Education & Research, Vol. 32, No. 3 [2023], Art. 1 Table 6 Loss of Control In-Flight (LOC-I) - Damage LOC-I df Chi- p N N Non- Odds Cramer’s Cramer’s Square Training training Ratio V Value V Sig. Destroyed 1 9.338 .002 31 329 .575 .035 .002 Substantial 1 2.395 .122 160 863 1.132 .018 .122 Minor 1 - - 1 0 - .029 .013 None 1 x x 0 0 - - - Note. This group had a value less than 5. System Component Failure Powerplant (SCF-PP) Pilots of training flights were about half as likely (.620) to be killed or sustain serious injuries (.461) compared to non-training pilots (Table 7). Minor injuries were still less likely (.717) and no injuries (.887) than non-training flights. A statistically significant association was found between the serious injury groups (p=.002) and a weak Cramer’s V of 0.36. Table 7 System Component Failure Powerplant (SCF-PP) - Injuries SCF-PP df Chi- p N N Non- Odds Cramer’s Cramer’s Square Training training Ratio V Value V Sig. Fatal 1 2.802 .094 13 128 .620 .019 .094 Serious 1 9.687 .002 16 212 .461 .036 .002 Minor 1 3.663 .056 35 298 .717 .022 .056 None 1 1.057 .304 77 530 .887 .012 .304 https://commons.erau.edu/jaaer/vol32/iss3/1 DOI: 10.58940/2329-258X.1954 14 Walach: Safety in Flight Training - NTSB Data Aircraft damage was almost always (91.9%) substantial when the powerplant had a system failure (Table 8). Like the injury statistics, pilots of training flights are about half as likely (.578) to destroy the aircraft and slightly less likely (.752) to substantially damage the aircraft. However, it should be noted that almost all (99.9%) power-plant failures, regardless of the type of flight either destroyed or substantially damaged the aircraft. A statistically significant association was found between the substantial damage group (p=.001) and a weak Cramer’s V of .034. Table 8 System Component Failure Powerplant (SCF-PP) - Damage SCF-PP df Chi- p N N Non- Odds Cramer’s Cramer’s Square Training training Ratio V Value V Sig. Destroyed 1 2.564 .109 9 95 .578 .018 .109 Substantial 1 11.489 .001 132 1072 .752 .034 .001 Minor 1 - - 0 1 - .005 .686 None 1 X X 0 0 - - - Note. This group had a value less than 5. Fuel Pilots who were involved in accidents on training flights with fuel related issues were less likely (.426) to result in fatalities than non-training pilots (Table 9). The chances (.407) of serious injuries in a fuel related accident were also less likely for pilots on training flights compared to those pilots on non-training flights. Minor injuries (.686) and no injuries (.722) were also less likely for training flights when a fuel-related accident occurred. The serious injury group had a significant association (p=.043) and a weak Cramer’s V of .023. Published by Scholarly Commons, 2023 15 Journal of Aviation/Aerospace Education & Research, Vol. 32, No. 3 [2023], Art. 1 Table 9 Fuel Related Accidents - Injuries Fuel df Chi- p N N Non- Odds Cramer’s Cramer’s Square Training training Ratio V Value V Sig. Fatal 1 - - 3 43 .426 .017 .139 Serious 1 4.095 .043 5 75 .407 .023 .043 Minor 1 1.311 .252 10 89 .686 .013 .252 None 1 1.957 .162 20 169 .722 .016 .162 Note. This group had a value less than 5. Fuel-related accidents resulted in substantial damage most (96.13%) of the time with a few (n=16) aircraft destroyed (Table 10). No fuel-related events resulted in no or minor damage. Training flights were about half as likely to have a fuel related event that resulted in substantial damage while none of the training flights that had a fuel related event were destroyed. A statistically significant association (p=.008) was found between the substantial damage groups with a weak Cramer’s V of .031. Table 10 Fuel Related Accidents - Damage Fuel df Chi- p N N Non- Odds Cramer’s Cramer’s Square Training training Ratio V Value V Sig. Destroyed 1 - - 0 16 - .019 .105 Substantial 1 7.127 .008 38 360 .644 .031 .008 Minor 1 X X X X - - - None 1 X X X X - - - Note. This group had a value less than 5. https://commons.erau.edu/jaaer/vol32/iss3/1 DOI: 10.58940/2329-258X.1954 16 Walach: Safety in Flight Training - NTSB Data Discussion In answer to question one, “How do flight-training (student) accidents differ from non- flight-training (non-student) accidents?” The primary factors leading to accidents in flight training are the same top five events that lead to accidents for non-training flights. These events are loss of control in flight, loss of control on the ground, abnormal runway contact, fuel related issues, and system component failure power plant. The order of these accidents is not the same between the two groups, however. Training flights have abnormal runway contact accidents as their top event type, while loss of control in flight is the top accident for non-training flights. Loss of control in-flight is the second most common accident for training flights. Fuel-related issues were ranked five for both training and non-training flights. In answer to Q2, “How are the risk factors that lead to training accidents associated with the risk factors that lead to non-training accidents?” data was analyzed using a Chi-square test of independence. The groups that did show association also showed the highest difference in odds ratio, with the odds favoring the training pilot in all instances except damage to the aircraft in an abnormal runway contact event. Abnormal runway contact was by far more dangerous for non- training pilots but more damaging to the aircraft if flown by a training pilot. In answer to question 3 “How do the odds of having a specific type of training accident differ from the odds of having a specific type of non-training accidents?” an odds ratio was used. To add more context to these findings, an odd ratio test was performed to indicate which group was more likely to experience a particular event. Each of the top five accidents will be discussed based on the Chi-square and odd ration results. Published by Scholarly Commons, 2023 17 Journal of Aviation/Aerospace Education & Research, Vol. 32, No. 3 [2023], Art. 1 Abnormal Runway Contact (ARC) While abnormal runway contact was the top cause of accidents for training flights, and only ranked fourth for non-training flights, these events were far more dangerous for non- training pilots. Of the 743 accidents for non-training flights, 14 pilots died because of abnormal runway contact while none of the 237 training flights had a fatality for ARC. Fatal accidents X2 (1, N = 9) = 1.476, p = .224 show no relationship between training and non-training flights. Serious injuries for ARC, X2 (1, N = 55) = .0.460, p = .498 are also more dangerous for non- training flights with training pilots 0.747 times less likely to sustain serious injuries. Most training flights have two pilots (a student and an instructor) onboard, and the supervision of the instructor may provide some amount of safety. It is also unknown how often an instructor pilot looks to a student pilot as a pilot or a resource in an emergency. This could be a possible avenue for future research. Abnormal runway accidents were also analyzed by damage to aircraft. These results were a little more surprising because while training flights were less fatal and had lower injuries, the aircraft were damaged to a higher degree. Students who had an ARC were 1.744 times more likely to destroy the aircraft and 1.998 times more likely to substantially damage the aircraft. No students in the sample had an ARC with no damage while 15 non-student pilots had an ARC with no damage. Loss of Control in Flight Loss of control in flight has been a known issue for general aviation for some time now (NTSB, 2000); however, the data analysis from this study uncovered some interesting findings. First, the odds of a loss of control event leading to a fatality was half as likely for students (.515 times), as were serious injuries (.674). Training flights were much more likely to have minor https://commons.erau.edu/jaaer/vol32/iss3/1 DOI: 10.58940/2329-258X.1954 18 Walach: Safety in Flight Training - NTSB Data (1.104) or no injuries (2.013) than non-training flights. While these findings are good news for students, they further highlight the need for research into how to prevent loss of control accidents in general aviation. The data very strongly suggests a training flight decreases the chances of a loss of control event leading to fatal or serious injuries. Like ARC events, loss-of-control in-flight, while less dangerous to the pilots of training flights, the aircraft do not share the same fate. All aircraft involved in accidents in the five years examined in this study were either destroyed (n=31 training, n=329 non-training) or sustained substantial damage (n=160 training, n=863 non-training) except for 1 training flight that sustained only minor damage. These results were very surprising considering the difference in injuries between the training and non-training groups. The chi-square results for destroyed aircraft X2 (1, N = 360) = 9.338, p = .002 show these two groups are closely associated. So, if students are destroying aircraft at near equal rates to their non-training counterparts, why are they walking away from these crashes with far less serious injuries? A loss-of-control in-flight event will almost always destroy or substantially damage an aircraft and will often take the life of the occupant. This area of aviation accidents needs substantial attention as little change has been seen in these types of accidents despite awareness and efforts by the general aviation community. System Component Failure of the Power-plant Based on the analysis of data in this study, system component failure of the power-plant has unfortunately often resulted in serious injuries or fatalities with substantial damage or destroyed aircraft. While not as deadly as loss-of-control events, power-plant failures are still injuring or killing many pilots. In every category of injury, students conducting training flights Published by Scholarly Commons, 2023 19 Journal of Aviation/Aerospace Education & Research, Vol. 32, No. 3 [2023], Art. 1 were less likely to sustain injuries than their non-student peers. Students were about half (.620 times) as likely to be killed, and about half (.461 times) as likely to sustain serious injuries. In most aircraft accidents, an engine failure should not be a life ending event; it just means the aircraft needs to land with little or no ability to go-around. The data seems to suggest engine failure in flight is less of an issue in a training situation than a non-training flight. Pilots not in a training environment are likely to be flying solo, or flying with non-pilots that may be more of a hindrance than a help if not familiar with the airplane or flight operations. Another possibility is flight instructors practice emergency procedures regularly as they instruct their students, and the procedures may be well-rehearsed. Pilots in a non-training environment may not have practiced engine out landings or other emergency procedures except during flight training or during their annual reviews. System component powerplant failures resulted in aircraft that were destroyed or substantially damaged in all accidents except one. Part of this high damage rate may be partially contributed to the fact that the event is caused by a damaged powerplant; therefore, all flights with a power-plant event are already damaged. Secondly, most power-plant events are likely to result in an off-airport landing, or a landing that is abnormal. Training flights are often conducted in and around the airport. An engine failure in the traffic pattern has a high probability of a close to normal landing, which is far less likely to cause damage than an engine failure enroute that will result in an off-airport landing. Fuel Related Fuel related safety accidents, like the other accidents presented, were generally less dangerous for students than they were for non-students. In most instances a fuel related issue in a training environment was half as likely (.426) to result in a fatality or serious injury (.407) than https://commons.erau.edu/jaaer/vol32/iss3/1 DOI: 10.58940/2329-258X.1954 20 Walach: Safety in Flight Training - NTSB Data for a non-training flight. Perhaps the addition of a more highly trained pilot in the aircraft is the reason for this higher safety factor; however, these results did not control for solo vs. non-solo flights. It would be interesting if further studies examined just fuel related flights and compared solo to non-solo and solo training to solo non-training accidents in an attempt to control for the two-pilot scenario that may be a factor in reduced risk. Damage to aircraft in fuel related accidents is like power-plant failure events as almost all aircraft are substantially damaged while 16 aircraft were destroyed. No training flights had destroyed aircraft because of fuel issues, and the number of training flights that were substantially damaged were related to the number on non-training aircraft substantially destroyed. As with power-plant failures, fuel issues are likely to result in a forced landing, possibly off-airport. Again, training flights have a higher likelihood of being in the traffic pattern or close to the airport while non-training flights are less likely to be near the airport. Closeness to an airport may account for some of the group differences. While these data show no minor, or no-damage events related to fuel, it could be that such events were not reported or investigated. If an aircraft had a fuel issue and made a safe landing, there would be no reason to report the event to the NTSB. Such events may be reported to NASA ASRS (Aviation Safety Reporting System). An analysis of ASRS data may be helpful to add some context to these findings; however, ASRS is voluntary self-reported data. Conclusion The results of this study show there may be less risk in a training situation than in non- training situations. It should be noted, however, that a dual-pilot scenario is almost always present on a training flight. In the non-training accidents examined in this study, it was unknown Published by Scholarly Commons, 2023 21 Journal of Aviation/Aerospace Education & Research, Vol. 32, No. 3 [2023], Art. 1 if the pilot in command was the only pilot onboard. An area of further study might focus on non- training accidents and separating the events into dual-pilot vs solo pilot events. Pilots who exit training seem to be at a higher risk of injury in loss of control, abnormal runway contact, and power-plant failures. It could be that emergency procedures that are practiced so often in training are less likely to be practiced after training. The need for more scenario-based emergency procedures in flight reviews may help. Development of a better model for annual flight reviews that more closely models what is happening with the Part 121carriers may help turn the tide of general aviation accidents. The Advanced Qualification Program (AQP) has been instituted by Part 121 and Part 135 carriers to review and provide recurrent training for their pilots (Cusick et al., 2017). Rather than just a simple checklist of maneuvers, the airlines select the skills they have identified to be problems and make these into scenarios that pilots must train for in the flight simulator during annual reviews. To function more smoothly in the existing general aviation structure, the AQP can be incorporated in the annual flight review and can be completed in the aircraft, in-flight, rather than in a flight simulator. AQP will be most effective if the pilots conduct an AQP flight review in the aircraft they fly, or at least, the same type. While flight instructors would need to be trained in how to conduct the review, the process is minimally invasive and does not require significant up-front costs. Loss of control in flight continues to be of major concern because, of all the types of accidents, loss of control in flight is by far the most deadly and unforgiving. This is an area that needs continued research and vigilance of the general aviation community to improve. A positive note to end on is that this study shows that flight training seems to have a higher degree of safety when compared to general aviation flights conducted after flight training ends. Perhaps finding https://commons.erau.edu/jaaer/vol32/iss3/1 DOI: 10.58940/2329-258X.1954 22 Walach: Safety in Flight Training - NTSB Data ways to continue to consistently practice emergencies after training is the best way to stay safe and reduce risk in general aviation. Published by Scholarly Commons, 2023 23 Journal of Aviation/Aerospace Education & Research, Vol. 32, No. 3 [2023], Art. 1 References Air Safety Institute. (2014). Accidents during flight instruction. Aircraft Owner and Pilots Association. https://download.aopa.org/asf/InstructionalAccidentReportFINAL.pdf Aviation Week Network Staff. (2020). Demand for pilots plunges, but retirements will drive recovery. Aviation Week Network. https://aviationweek.com/business-aviation/demand- pilots-plunges-retirements-will-drive-recovery Boyd, D., & Dittmer, P. (2016). Accident rates, phase of operations, and injury severity for solo students in pursuit of private pilot certification (1994-2013). Journal of Aviation Technology and Engineering, 6(1), 44-52. https://doi.org/10.7771/2159-6670.1139 Cusick, S. K., Cortes, A. I., Rodrigues, C. C. (2017) Commercial aviation safety. McGraw Hill Education. de Voogt, A., Kalagher, H., Santiago, B., Lang, J. W. B. (2022). Go-around accidents and general aviation safety. Journal of Safety Research. 82, 323-328. https://doi.org/10.1016/j.jsr.2022.06.008 Dizikes, P. (2020). Study: Commercial air travel is safer than ever. MIT News. Massachusetts Institute of Technology. https://news.mit.edu/2020/study-commercial-flights-safer-ever- 0124 Houston, S., Walton, R., & Conway, B. (2012). Analysis of general aviation instructional loss of control safety events. Journal of Aviation/Aerospace Education & Research, 22(1), 35- 49. https://doi.org/10.15394/jaaer.2012.1402 Ison, D. (2014). Correlates of continued visual flight rules (VFR) into instrument meteorological conditions (IMC) general aviation accidents. Journal of Aviation/Aerospace Education & Research, 24(1), 1-26. https://doi.org/10.15394/jaaer.2014.1628 https://commons.erau.edu/jaaer/vol32/iss3/1 DOI: 10.58940/2329-258X.1954 24 Walach: Safety in Flight Training - NTSB Data Kalagher, H., de Voogt, A., & Boulter, C. (2021). Situational awareness and general aviation accidents: An analysis of 94 US accident reports. Aviation Psychology and Applied Human Factors, 11(2), 112–117. https://doi.org/10.1027/2192-0923/a000207 Lee, S., Bates, P., Murray, P., & Martin, W. (2017). Training flight accidents: An explorative analysis of influencing factors and accident severity. Aviation Psychology and Applied Human Factors, 7(2), 107–113. https://doi.org/10.1027/2192-0923/a000121 Majumdar, N., Marais, K., & Rao, A. (2021). Analysis of general aviation fixed-wing aircraft accidents involving inflight loss of control using a state-based approach. Aviation (Vilnius, Lithuania), 25(4), 283–294. https://doi.org/10.3846/aviation.2021.15837 National Transportation Safety Board. (2014-2020). Aviation accident database & synopses. https://www.ntsb.gov/Pages/AviationQuery.aspx Randel, E., Le Couteulx, G., & Bahl, J. (2008). Loss of control on landing: We need to change our mindset: A service-learning project for flight safety. Journal of Aviation/Aerospace Education & Research, 17(3) 13-23. https://doi.org/10.58940/2329-258X.1452 Uitdewilligen, S., & de Voogt, A. (2009). Aircraft accidents with student pilots flying solo: Analysis of 390 cases. Aviation, Space, and Environmental Medicine, 80(9), 803–806. https://doi.org/10.3357/ASEM.2510.2009 Wilson, D. R., & Sloan, T. A. (2003). VRF flight into IMC: Reducing the hazard. Journal of Aviation/Aerospace Education & Research, 13(1) 29-41. https://doi.org/10.15394/jaaer.2003.1567 Published by Scholarly Commons, 2023 25