Skip Ribbon Commands
Skip to main content
Safety Recommendation Details

Safety Recommendation A-05-027
Details
Synopsis: The National Transportation Safety Board has found that, over the last 20 years, about two-thirds of all general aviation (GA) accidents that occur in instrument meteorological conditions (IMC) are fatal--a rate much higher than the overall fatality rate for GA accidents. Accordingly, the Safety Board undertook the safety study, Risk Factors Associated with Weather-Related General Aviation Accidents, to better understand the risk factors associated with accidents that occur in weather conditions characterized by IMC or poor visibility ("weather-related accidents").
Recommendation: TO THE FEDERAL AVIATION ADMINISTRATION: Develop a means to identify pilots whose overall performance history indicates that they are at future risk of accident involvement, and develop a program to reduce risk for those pilots.
Original recommendation transmittal letter: PDF
Overall Status: Closed - Unacceptable Action
Mode: Aviation
Location: United States
Is Reiterated: No
Is Hazmat: No
Is NPRM: No
Accident #: DCA05SS001
Accident Reports:
Report #: SS-05-01
Accident Date:
Issue Date: 10/12/2005
Date Closed: 5/13/2019
Addressee(s) and Addressee Status: FAA (Closed - Unacceptable Action)
Keyword(s): Training and Education

Safety Recommendation History
From: NTSB
To: FAA
Date: 5/13/2019
Response: We note that, in March 2012, you published Effects of Training School Type and Examiner Type on General Aviation Flight Safety (DOT/FAA/AM-13/4), which considered the effects that flight training type (Title 14 Code of Federal Regulations [CFR] Parts 61 or 141) and examining authority type (designated pilot examiner or FAA inspector) might have on a GA pilot’s subsequent flight safety record. Although your study concluded that these factors do not affect a GA pilot’s subsequent aviation safety record, you did recommend the following to improve future data analysis: · Develop a common identifier for pilots in our respective databases. · Make database improvements to streamline data comparison between our organizations. · Develop procedures for collecting total pilot flight hours at more frequent and standardized intervals. Our study found that pilot history of accident or incident involvement is associated with a higher risk of being involved in a future weather-related GA accident. Our study also pointed out that it would be possible to use existing records to develop the recommended program. We note that you analyzed accident causal factors to compare the type of flight training (14 CFR Part 61 or 141) and examining authority (designated pilot examiner or FAA inspector) to the pilot’s accident/incident and violation history, although our study did not indicate a preference for either Part 61 or 141 pilot training, nor did it discuss ideal pilot examiner characteristics. We encouraged you to develop the data improvements discussed in your 2012 study because we believed they would allow our organizations to streamline data comparison in the future. Pending completion of these actions, which would culminate in a program to reduce the chance that pilots with risky performance histories would be involved in future accidents, Safety Recommendation A-05-27 remained classified “Open—Acceptable Response.” We note that you considered the data improvements discussed in your 2012 report, but determined that they are neither feasible nor justified at this time. We also note that, although you plan to look for ways to work with the GA community to better understand pilots’ overall decision making processes and prevent future performance hazards, you consider your actions in response to this recommendation complete. Because you have not developed the program described in this recommendation and do not plan to do so, Safety Recommendation A-05-27 is classified CLOSED--UNACCEPTABLE ACTION.

From: FAA
To: NTSB
Date: 10/3/2018
Response: -From Daniel K. Elwell, Acting Administrator: The Board and the Federal Aviation Administration (FAA) both realize the complexities of trying to predict future accident risk on the basis of pilots' past performance. In our previous response dated July 24, 2014, the FAA discussed our March 2012 report "Effects of Training School Type and Examiner Type on General Aviation Flight Safety" (DOT/FAAIAM-13/4) (enclosed). This update provides additional information, analysis, and actions taken in response to this report and this recommendation. The first step in conducting this study required identifying key factors or markers that reliably predict future risk. Additionally, there were numerous candidate risk factors and interactions bet ween factors, making comprehensive, in-depth statistical testing problematic. The FAA judged two risk predictors worthy of testing. These were (I) the type of school a pilot attended and (2) the type of evaluator administering the practical exam(s). These two factors were fom1ally tested and the results were published in the March 2012 report. The results of this study implied that training school type does not affect a U.S. general aviation pilot's subsequent aviation safety record. According to the study, the subsequent accident rate of general aviation pilots earning their private pilot certificate under part 61 were on par with part 141 graduates. In addition. the study found that pilots tested by designated pilot examiners appeared to be equivalent to those tested under part 141 school authority. and graduates tested by an aviation safety inspector showed a statistically lower accident rate. Moreover, through careful methodology and statistical tracking with part 61 and141 instrument rated pilots, we discovered that having an instrument rating was associated with a higher accident frequency than non-instrument rated pilots. The study described that the risk factors faced by instrument rated pilots are arguably considerably different than those faced by non-instrument rated pilots. Ultimately. the study did not uncover decisive relationships between school type, examiner type. or instrument-rating and subsequent accident rates. However, the study did yield recommendations that would present the opportunity for improved data analysis in the future. Internal Recommendations. One internal recommendation called for the Board and the FAA to collaborate on developing a common identifier for pilots in their respective databases. This common identifier would be similar to what the FAA calls a .. UniqueID .. designator for the Comprehensive Airman Information System (CAIS) and Document Imaging Workflow System (DIWS) databases. This would allow the information in the Board's and the FAA ·s pilot databases to be sorted and matched against each other, thus aiding in future studies of this type. This internal recommendation would require structural system changes as well as policy changes that would be difficult to implement. The FAA determined that implementing a new identifier between the Board and the FAA would result in a significant rewrite to the CAIS and DIWS systems that would demand resources and budget allowances for all data systems involved. The expense of such an undertaking would not be the best course of action given the results of the study. Furthermore, the FAA does not allow outside agencies to have direct access to internal FAA systems such as CAIS, and Privacy Act requirements apply to records that do not include flight hour data. ln addition. because of the difficulty in implementing the first internal recommendation. the FAA believes the creation of a user guide (the second internal recommendation) would not aid in any solution. A third internal recommendation called for the FAA to augment its night hour information collected from pilots (such as during their medical certification or at other interactions with the FAA) to a 12-month total flight hour system. This would potentially allow for analysis of the relationship between total flight hours and the accident rate. We note that on July 15, 2016. Congress passed the FAA Extension, Safety. and Security Act of 20 16. This legislation included relief from holding an FAA medical certificate for certain pilots. The reporting of night hours by pilots has never been a mandatory requirement on the FA.A. medical application. The FAA. believes that the voluntary reporting of flight hours is sufficient and any further mandate will not provide an added safety benefit. We note the voluntary reporting mechanisms: • FAA Form 8710-1. Airmen Certificate and/or Rating Application Supplemental Information and Instructions, remains an option for all pilots to complete after a flight review or an instrument proficiency check. • Advisory Circular (AC) 6 l-98C, Currency Requirements and Guidance for the Flight Review and Instrument Proficiency Check, was revised and published November 20, 2015. The AC includes the following guidance: ·'When a pilot satisfactorily completes a flight review or IPC [Instrument Proficiency Check], the applicant should provide, and the evaluating Certified Flight Instructor (CFI) should submit. a completed Form 8710-1 to the Airmen Certification Branch (AFS-760). The FAA does not require Form 87 10-1 for a pilot's flight review or IPC; however, the FAA strongly encourages all applicants and CFIs to follow this recommendation. An airman certificate application updates a pilot's FAA record. Pilots should ensure that their data is current because up-to-date records benefit everyone. For example, a pilot's total flight time and aeronautical experience determines insurance premiums. If a pi lot loses his or her logbook an FAA record is on file and available. Nevertheless, submitting Form 8710-1 for a flight review or IPC is optional."' • The FAA amended the Integrated Airman Certification and Rating Application (IACRA) to allow optional documentation of flight reviews and IPC. IACRA now has the option to select ''Flight Review" or "Instrument Proficiency Check" separately or allow for both to be selected. Furthem1ore, effective May 2017, the Civil Aviation Registry is able to provide information on flight hours reported by airmen in IACRA. Additional Actions. In an effort to prevent poor individual pilot performance from becoming a greater risk, the FAA notes the General Aviation Joint Steering Committee (GAJSC) is involved in determining and addressing the risks associated with accidents (including pilot performance) for the general aviation community. The FAA continually works with aviation community stakeholders to help improve standards, guidance, and test development practices for airman certification. In June 20 16, a joint FAA and industry-developed Airman Certification Standards (ACS) replaced the FAA Practical Test Standards (PTS) for the Remote and Private certificate as well as the instrument rating. The FAA and industry partners determined the need for a systematic approach that would provide clear standards for aeronautical knowledge, list specific behaviors for risk management and aeronautical decision making, consolidate overlapping tasks in the PTS, and connect knowledge/skill standards to specific FAA publications. Furthermore, the ACS adds task-specific knowledge and risk management elements to the previous PTS Area of Operations/Task. The ACS provides pilots with a single source for standards for both the knowledge and practical exams. The integrated format of the ACS clearly explains to applicants, instructors, and evaluators what an airman must "KNOW. CONSIDER, and DO" to pass the knowledge test and the practical test for an airman certificate or rating. The ACS ensures that the individual using it for training. teaching, or testing will be better prepared for certification purposes as well as safe operation. Additional information on the ACS is available at http://www.faa.gov/training. Because the FAA understands the importance of identifying pilots whose overall performance history indicates that they are at future risk of accident involvement, we plan to continue to look for ways to work with the GA community to better understand pilots overall decision making processes so we can prevent future performance hazards. Based on the actions we have taken internally and in partnership with industry, I believe the FAA has effectively addressed this safety recommendation and consider our actions complete.

From: NTSB
To: FAA
Date: 9/16/2014
Response: We note that, in response to this recommendation, in March 2012 you completed a report titled Effects of Training School Type and Examiner Type on General Aviation Flight Safety (DOT/FAA/AM-13/4). While the study did not find a clear correlation between training, examination, or instrument-rating and the accident rate, the study produced three recommendations for improving data analysis. One of these was for the NTSB and the FAA to collaborate on developing a common identifier for pilots in both of our respective databases. We further note that you are considering developing procedures for collecting total pilot flight hours at more frequent and standardized intervals, and that you plan to discuss with us options for database improvements that would allow for streamlined comparison of data held by each of our organizations. We encourage you to develop these improvements and are ready to work with you to do so. This recommendation is now 10 years old. We generally expect that the actions we recommend can be completed within 3 to 5 years after the recommendation is issued. Although the FAA has not yet developed methods to identify pilots whose performance history indicates an increased risk of an accident, the March 2012 study provided valuable information. We encourage you to act in a timely manner to develop the data improvements described in the study, and then to repeat your analysis. Pending completion of these actions, culminating in the development of a program to reduce the risk of involvement in future accidents for pilots whose overall performance history indicates this risk, Safety Recommendation A-05-27 remains classified OPEN—ACCEPTABLE RESPONSE.

From: FAA
To: NTSB
Date: 7/24/2014
Response: -From Michael P. Huerta, Administrator: In 2012, the Federal Aviation Administration (FAA) completed a report titled Effects of Training School Type and Examiner Type on General Aviation Flight Safety. The FAA used this report to begin developing a means to review a pilot's overall performance history and improve future efforts to analyze performance data. While the results of the analysis are not definitive, we learned lessons that will improve future efforts. This study used log-linear statistical analysis to see if factors relating to pilot training, testing, and instrument rating could help identify GA pilots who are at risk of an accident. We compared training, examination, and instrument rating to subsequent accident rates. The following is a summary of the resulting analysis: • The subsequent accident rates for pilots trained under part 61 versus part 141 were equal; • Pilots tested by an aviation safety inspector appear to be less likely to have an accident compared to those examined by a designate pilot examiner, or part 141 pilot school self-examining authority. However, due to a high data loss rate, we made this conclusion based on a sample of 22 pilots. Therefore, this conclusion may be unreliable; and • Once school, examiner, and flight hours were statistically controlled for, we found that pilots holding instrument ratings appear to have higher accident rates. This result may not be meaningful due to the complex nature of log-linear statistical models in cases, such as this, where preliminary models failed to show an effect only emerging in the final model. While the results of the study did not indicate decisive causal factors between training, examination, and instrument-rating in relation to the accident rate, the efforts of the study yielded three recommendations that would allow for improved data analysis in the future. These recommendations are: • The Board and FAA should collaborate on developing a common identifier for pilots in their respective databases. This common identifier would be similar to what the FAA calls a "UniqueID" designator for the Comprehensive Airman Information System (CAIS) and Document Imaging Workflow System (DIWS) databases. This would allow the Board and the FAA pilot information databases to communicate seamlessly with each other, and would aid in future studies of this type; • The FAA should develop a manual for the CAIS and DIWS pilot information databases. This manual would be similar to the Board's Data Dictionary user manual; and • The FAA should augment its flight hour information collected from pilots (such as during their medical certification or at other interactions with the FAA) to a 12-month total flight hour system. This would potentially allow for better analysis of the relationship between total flight hours and the accident rate. The implementation of these recommendations would create an improved pilot information-gathering approach that the FAA and the Board could use to identify future at-risk pilots. The FAA will examine the feasibility of developing a manual for its CAIS and DIWS databases, as well as methods to collect total pilot flight hours at more frequent and standardized intervals. The FAA is also in the process of opening a dialogue with the Board to explore options for database improvements that would allow for streamlined comparison of data held by each organization. A formal request from the Board to the FAA to share a common pilot identifier could greatly help facilitate this process. I will keep the Board informed of the FAA's progress on this safety recommendation and provide an updated response by June 30, 2015.

From: NTSB
To: FAA
Date: 1/2/2014
Response: The FAA’s only letter concerning Safety Recommendation A-05-27, dated March 6, 2006, stated that the FAA was conducting an in-depth analysis of accident causal factors, emphasizing not only knowledge and practical test performance, but also detailed training history and the ways in which all of these tie to the types of accidents/incidents being discussed. The goal of the analysis was to develop a comprehensive overview of the factors most commonly associated with at-risk pilots in order to develop appropriate intervention strategies. Our December 13, 2006, reply to the FAA’s letter stated that we would appreciate receiving updates on the details of these initiatives as they were developed, the timeframe contemplated for completing the list of factors associated with at-risk pilots, and appropriate intervention strategies. Because the information provided appeared to indicate that the FAA’s efforts would address the intent of the recommendation, Safety Recommendation A-05-27 was classified OPEN—ACCEPTABLE RESPONSE on December 13, 2006. We have not received any additional information regarding the status of the FAA’s analysis in the intervening years and ask what progress has been made to address the recommendation.

From: NTSB
To: FAA
Date: 12/13/2006
Response: The Safety Board notes the FAA’s position that through quality flight training, deficits identified in both knowledge and practical test performance can effectively be addressed. However, the FAA acknowledges that the Board’s study found a statistical relationship between test performance and accidents that is too compelling to be dismissed. The Safety Board notes that the FAA proposes conducting an in-depth analysis of accident causal factors with an emphasis not only on knowledge and practical test performance, but also on detailed training history and how all of these tie to the types of accidents/incidents being discussed. The goal is to develop a comprehensive overview of the factors most commonly associated with at risk pilots in order to develop appropriate intervention strategies. The Board would appreciate receiving updates on the details of these initiatives as they are developed, the timeframe contemplated for completing the list of factors associated with at risk pilots, and appropriate intervention strategies. The information provided thus far appears to indicate that these efforts will address the intent of the recommendation; accordingly, Safety Recommendation A-05-27 is classified OPEN -- ACCEPTABLE RESPONSE.

From: FAA
To: NTSB
Date: 3/6/2006
Response: Letter Mail Controlled 3/15/2006 2:44:33 PM MC# 2060133: - From Marion C. Blakey, Administrator: FAA Comment. The FAA believes that many critical factors influence the quality of pilot training and performance. These factors not only include knowledge and practical test performance, but also quality of flight training and individual commitment to excellence. The Board cites a statistical comparison regarding pilots with poor knowledge and practical test performance and a history of being involved in accidents or incidents. The Board asserts that these pilots are at three times greater risk for a weather-related accident. However, the FAA points out that through quality flight training, the deficits identified in knowledge test performance can be effectively addressed. Similarly, any deficiencies leading to a failed practical test should serve as a focal point for the pilot and his or her instructor in developing a directed training regimen. The initiatives mentioned in the previous recommendations could be incorporated into this type of training. Current training initiatives aside, the statistical information presented by the Board is too compelling to be dismissed. However, instead of targeting individual pilots, the FAA proposes a more indepth analysis of accident causal factors with an emphasis on not only knowledge and practical test performance, but also detailed training history and how these tie to the types of accidents/incidents now being discussed. Such a study may highlight other deficiencies within the current training paradigm. Again, instead of targeting individuals, systemic changes can be made that will benefit the entire aviation community. The FAA has an initiative to review the training history of pilots related to their performance and safety. This initiative relates the type of flight training (14 CFR Parts 61 or 141) and examining authority (designated pilot examiner or FAA inspector) to the pilot's accident/incident and violation history. The FAA proposes to expand this study to include the pilot's knowledge test and practical test results and involvement in weather-related accidents. This effort would be pursued in conjunction with ongoing efforts to review pilot training (e.g., 14 CFR Part 61 versus 14 CFR Parts 141/142 designated pilot examiner administering the practical test). The goal is to develop a comprehensive overview of the factors most commonly associated with "at risk" pilots in order to develop appropriate intervention strategies. Again, the FAA emphasizes that its efforts are aimed at systemic improvements and not at targeting individual pilots. I will keep the Board informed of the FAA's progress on this safety recommendation.