Skip Ribbon Commands
Skip to main content
Bookmark and Share this page

Speeches

Testimony Before the Committee on Commerce, Science, and Transportation United States Senate on Highly Automated Vehicles: Federal Perspectives on the Deployment of Safety Technology
Robert L. Sumwalt
United States Senate, Washington, DC
11/20/2019

Download the testimony

Good morning Chairman Wicker, Ranking Member Cantwell, and Members of the Committee. Thank you for inviting the National Transportation Safety Board (NTSB) to testify before you today.

The NTSB is an independent federal agency charged by Congress with investigating every civil aviation accident in the United States and significant accidents in other modes of transportation—highway, rail, marine, and pipeline. We determine the probable cause of the accidents we investigate, and we issue safety recommendations aimed at preventing future accidents. In addition, we conduct special transportation safety studies and special investigations and coordinate the resources of the federal government and other organizations to assist victims and their family members who have been impacted by major transportation disasters. The NTSB is not a regulatory agency—we do not promulgate operating standards, nor do we certificate organizations, individuals, or equipment. The goal of our work is to foster safety improvements, through safety recommendations, for the traveling public.

Motor vehicle crashes are a leading cause of death and injuries in the United States. In 2018, 36,560 people lost their lives in crashes on our nation’s highways.[1] The large majority of these tragedies can be directly linked to human error. Humans make mistakes and bad decisions, such as driving while they are impaired, distracted, or fatigued. Automated vehicle (AV) and collision avoidance technology have the potential to reduce the number of crashes, injuries, and fatalities significantly.

Today I will discuss some of the lessons learned from NTSB crash investigations and recommendations regarding the safe testing and deployment of highly automated vehicles. A focus of my testimony will be an overview of the findings and recommendations of our recently completed investigation of a developmental automated driving system (ADS) that collided with, and killed, a pedestrian in Tempe, Arizona, on March 18, 2018.

While there is often a desire to jump directly to the end of the technological spectrum— highly automated “self-driving” vehicles—it is imperative that regulators and policy makers do not ignore the risks associated with partial driving automation systems currently being operated on our highways. I will provide an overview of NTSB crash investigations involving Tesla model vehicles operating with partial automation and related recommendations addressing the safe deployment of automated control systems.

Automated Driving Systems

The use of AV controls and systems is accelerating rapidly in all modes of transportation. We have monitored AV development and have a long history of calling for systems to assist the operator in performing the driving task. One of the main sources of confusion in discussions about AVs is the language used in the industry, and by researchers and regulators, compared to that used by the general public. Industry, regulators, and academics frequently use the six-level SAE automation taxonomy as a reference point when discussing vehicle capabilities and operator responsibilities.[2] However, the SAE automation levels may not be easily relatable to the general public. At the same time, the terms used by vehicle manufacturers to market their partial driving automation systems (SAE level 2) such as ProPilot (Nissan), Pilot Assist (Volvo), and Autopilot (Tesla)—can add to public confusion about the degree of automation in the production-level vehicles now available. Although the general public frequently uses “self-driving vehicle” to describe currently available vehicles, it is an incorrect portrayal of the capabilities of vehicles on the roads in the United States today.

In describing highly automated vehicles (SAE levels 3 to 5), SAE recommends the term “automated driving system.” The defining characteristic of an ADS is that the system takes full control of all aspects of the driving task. Although a geographical area, environmental conditions, or a human occupant’s availability may limit the domain where an ADS is operational, the system is responsible for controlling the vehicle and avoiding hazards in that domain. We recently completed our investigation of a fatal crash in Tempe, Arizona, involving an ADS-equipped vehicle and made recommendations regarding the testing and deployment of these systems.

Tempe, Arizona, Crash Investigation

On March 18, 2018, at 9:58 p.m., an automated test vehicle, based on a modified 2017 Volvo XC90 sport utility vehicle (SUV), struck a pedestrian walking midblock across North Mill Avenue in Tempe, Arizona. The SUV was operated by the Advanced Technologies Group (ATG) of Uber Technologies, Inc., which had modified the vehicle with a proprietary developmental ADS. An operator occupied the driver’s seat of the SUV, which was being controlled by the ADS. As a result of the crash, the pedestrian sustained fatal injuries.

We determined that the probable cause of the crash was the failure of the vehicle operator to monitor the driving environment and the operation of the ADS because she was visually distracted throughout the trip by her personal cell phone. Contributing to the crash were the Uber ATG’s (1) inadequate safety risk-assessment procedures, (2) ineffective oversight of the vehicle operator, and (3) lack of adequate mechanisms for addressing the operator’s automation complacency—all a consequence of inadequate safety culture. Further factors contributing to the crash were (1) the impaired pedestrian’s crossing of North Mill Avenue outside a crosswalk, and (2) the Arizona Department of Transportation’s insufficient oversight of AV testing.

At the time of the crash, the Uber ATG had an inadequate safety culture, exhibited by inadequate safety risk-management procedures and safety policies, lack of oversight of vehicle operators, and lack of personnel with backgrounds in safety management systems. For example, we concluded that the Uber ATG’s deactivation of the Volvo forward collision warning and automatic emergency braking systems without replacing their full capabilities removed a layer of safety redundancy and increased the risks associated with testing ADSs on public roads.

Although the ATG has made safety improvements in organizational, operational, and technical areas, we remain concerned regarding the safety culture of the numerous other ADS developers who are conducting similar testing.

Furthermore, a manufacturer is not the only entity with a role in ensuring the safe testing of AVs on public roads. To establish a robust safety framework, it is necessary to involve federal agencies, which can establish and mandate ADS performance standards, and the states, which traditionally regulate drivers and vehicle operation on public roads. During our review of the role of federal and state oversight, we identified the need for improved safety risk-management requirements for testing ADS on public roads.

Federal Oversight

We see enormous potential in the ability of ADS to mitigate or prevent crashes on our roadways. A promise of the upcoming ADSs is that such systems will be safer than a human driver. Until that promise is realized, the testing of developmental ADS—with all its expected failures and limitations—requires appropriate safeguards when conducted on public roads. Unfortunately, there has been an absence of safety regulations and federal guidance regarding how to adequately evaluate an ADS, which has prompted some states to develop their own requirements for AV testing.

Although the National Highway Traffic Safety Administration (NHTSA) has published three iterations of AV guidance, it provides insufficient instructions on how ADS developers should accomplish the safety goals of the 12 ADS safety elements—for example, training vehicle operators, ensuring oversight, and evaluating whether an ADS has reached a level of safety functionality.[3] More limiting aspects of the policy pertain to (1) the absence of a NHTSA process for evaluating the adequacy of a safety self-assessment report, and (2) the lack of a mandatory submission requirement.

The shortcomings of the policy are exacerbated by the lack of assessment procedures and the difficulties in their development. For example, one of the 12 safety areas is “object and event detection and response,” pertaining to the capability of an ADS to detect, classify, and respond to objects and events in the environment. In this regard, we understand the difficulties in developing a “vision test” or standardized metric for assessing the perception of an ADS. In another of the 12 safety elements of its automated vehicle policy, human-machine interface, NHTSA addresses the need for monitoring driver engagement. NHTSA guidelines states, “entities are encouraged to consider whether it is reasonable and appropriate to incorporate driver engagement monitoring.” Because of the complexity of assessing all the relevant safety elements, to determine if sufficient safeguards exist for the testing and deployment of ADSs, a holistic assessment is needed, particularly when performance metrics may not exist.

The traditional division of oversight, in which NHTSA controls vehicle safety and the states monitor drivers, may not be easily applicable to developmental automated test vehicles. It might not be immediately apparent who controls the vehicle, or whether vehicle control and supervision is shared between the computer (the vehicle) and the human operator. A lack of appropriate policy from NHTSA and the states leaves the public vulnerable to potentially unsafe testing practices. To ensure that testing of AVs on public roads is conducted with minimal safety risk, meaningful action from both NHTSA and the states is critical.

If the process of submission of safety self-assessment reports were mandatory and included a process for the ongoing evaluation by NHTSA, it could serve as a criterion for judging whether a manufacturer’s approach to ADS development and testing met the minimal intent of the 12 ADS safety elements. NHTSA’s evaluation of a safety plan could also provide a minimum safeguard for the testing of developmental ADSs on public roads. Furthermore, assessment by NHTSA would provide important support to states when evaluating the appropriateness of a developer’s approach to the testing AVs.

As an outcome of the Tempe, Arizona, investigation, we recommended that NHTSA require entities who are testing or who intend to test a developmental ADS on public roads to submit a safety self-assessment report to the agency. We also recommended that NHTSA establish a process for evaluating the safety self-assessment report and determine whether the plans include appropriate safeguards for testing a developmental ADS on public roads, including adequate monitoring of vehicle operator engagement, if applicable.

State Oversight and Legislation

In the absence of federal ADS safety standards or specific ADS assessment protocols, many states have begun legislating requirements for AV testing. The development of state-based requirements could be attributed to the concerns of many states about the safety risk of introducing ADS-equipped vehicles on public roads. The requirements vary. Some states, such as Arizona, impose minimal restrictions. Other states have established requirements that include a more in-depth application and review process. In the Tempe crash investigation, we determined that Arizona’s lack of a safety-focused application-approval process for ADS testing at the time of the crash, and its inaction in developing such a process following the crash, demonstrate the state’s shortcomings in improving the safety of ADS testing and safeguarding the public.

Currently, 21 states lack regulations pertaining to ADS testing. Although 29 states have some type of ADS-related policy, the requirements for testing vary considerably. Furthermore, the existence of a regulation is not a sure indication of a comprehensive and safety-driven ADS testing policy. In fact, Arizona was one of the 29 states that had some form of regulation pertaining to ADS testing, but, as stated previously, the safety application approval process was lacking.

States that have no, or only minimal, requirements related to AV testing can improve the safety of such testing by implementing a thorough application and review process before granting testing permits. The American Association of Motor Vehicle Administrators (AAMVA) has developed numerous model programs for motor vehicle administration, law enforcement, and highway safety in general. In May 2018, AAMVA published Jurisdictional Guidelines for the Safe Testing and Deployment of Highly Automated Vehicles.Although the guidance contains elements of ADS testing, the AAMVA document lacked specific guidance for developers on how to accomplish the included recommendations. The guidance did include a very important element—the need for jurisdictions to identify a lead agency and establish an AV committee to develop strategies for addressing AV testing. However, the guidance does not include recommendations requiring ADS developers to submit a safety plan and for the state’s AV committee to review and approve such a plan.

Because states would benefit from adopting regulations that require a thorough review of ADS developers’ safety plans, including methods of risk management, we recommended that AAMVA encourage states to (1) require developers to submit an application for testing ADS-equipped vehicles that, at a minimum, details a plan to manage the risk associated with crashes and operator inattentiveness and establish countermeasures to prevent crashes or mitigate crash severity within the ADS testing parameters, and (2) establish a task group of experts to evaluate the application before granting a testing permit. Similar recommendations were also issued to the state of Arizona.

Partial Driving Automation System Safety

Although much attention and federal effort has been focused on highly automated SAE Level 3–5 vehicles, of equal and more immediate concern should be the current deployment of partial driving automation systems on our nation’s highways. Between May 2016 and March 2019, we investigated four crashes—three resulting in fatal injuries—involving Tesla model vehicles with Autopilot engaged.[4] When Autopilot is activated and multiple subsystems, like traffic aware cruise control (TACC) and Autosteer, are combined to provide both lateral and longitudinal vehicle motion control, the system is considered an SAE Level 2 partial driving automation system. These Level 2 systems are considered by NHTSA to be advanced driver assistance systems.

Following our investigation of the March 2016 fatal crash involving a Tesla Model S 70D in Williston, Florida, we issued several safety recommendations aimed at preventing similar crashes involving vehicles operating with partial driving automation systems.[5] A few important safety issues identified in the Williston crash investigation included (1) limiting the operational design domains for partial driving automation systems, (2) monitoring an AV driver’s level of engagement, and (3) the need for more robust event data recorders for AVs.

Operational Design Domain Restrictions

SAE J3016 discusses the need for manufacturers to accurately describe AV features and clearly define the level of driving automation and its capabilities, but also its operational design domain—the conditions in which the driving automation system is intended to operate. Examples of such conditions include roadway type, geographic location, clear roadway markings, weather conditions, speed range, lighting conditions, and other manufacturer-defined system performance criteria or constraints. Tesla, for example outlined many operating conditions and limitations based upon the Autopilot partial automation system design, such as that it is (1) designed for use on highways with a center divider, (2) designed for areas with no cross traffic and clear lane markings, (3) not for use on city streets or where traffic conditions are constantly changing, (4) not for use on winding roads with sharp curves, and (5) not for use in inclement weather conditions with poor visibility.

Despite communicating to owners and drivers these operating conditions and limitations, Tesla Autopilot firmware does not restrict the system’s use based on functional road classification. Essentially, the system can be used on any roads with adequate lane markings. This situation allows a driver to activate driving automation systems at locations and under circumstances for which their use is not appropriate or safe, such as roadways with cross traffic. The Tesla Model S in the Williston, Florida, crash collided with a tractor-trailer combination vehicle crossing an uncontrolled intersection on a nonlimited access highway. Partial AV operation on nonlimited access highways presents challenges with the detection of crossing vehicles, pedestrian and bicycle traffic, and traffic controls at intersections, such as red traffic lights. As a result, we concluded that, if AV control systems do not automatically restrict their own operation to those conditions for which they were designed and are appropriate, the risk of driver misuse remains. We recommended that Tesla and other manufacturers of Level 2 automation:

Incorporate system safeguards that limit the use of automated vehicle control systems to those conditions for which they were designed. (H-17-41)

Five automobile manufacturers responded to this recommendation with steps they were taking to mitigate operation under conditions for which they were designed. Tesla, however, advised us that operational design limits are not applicable to Level 2 driver assist systems, such as Autopilot, because the driver determines the acceptable operating environment.[6]

Tesla vehicles continue to be involved in crashes with Autopilot engaged in operating areas outside the intended roadway operational design domain. In March 2019, in Delray Beach, Florida, a fatal crash involving a 2019 Tesla Model 3 occurred under circumstances very similar to the Williston, Florida, crash.[7] The Delray Beach highway operating environment, like the cross-traffic conditions in Williston, was outside the Tesla Autopilot system’s operational design domain.

Today’s Level 2 partial driving automation systems can assess the vehicle’s location and current roadway type or classification, and determine whether the roadway is appropriate to the system’s operational design domain. Following the Williston crash, we made a recommendation to NHTSA to address this vital safety concern. We recommended that NHTSA:

Develop a method to verify that manufacturers of vehicles equipped with Level 2 vehicle automation systems incorporate system safeguards that limit the use of automated vehicle control systems to those conditions for which they were designed. (H-17-38)

In response to Safety Recommendation H-17-38, NHTSA wrote the following:

The agency has no current plans to develop a specific method to verify manufacturers of vehicles equipped with Level 2 systems incorporate safeguards limiting the use of automated vehicle control systems to those conditions for which they were designed. Instead, if NHTSA identifies a safety-related defect trend in design or performance of a system, or identifies through its research or otherwise, any incidents in which a system did not perform as designed, it would exercise its authority as appropriate.

The current status of this safety recommendation is “Open—Unacceptable Response.” We believe that NHTSA’s reactive, rather than proactive, safety position is misguided, and the agency should take immediate action to verify that manufacturers are incorporating operational domain design safeguards into their systems.

Monitoring an AV Driver’s Level of Engagement

Based on system design, in an SAE-defined Level 2 partial automation system, it is the driver’s responsibility to monitor the automation, maintain situational awareness of traffic conditions, understand the limitations of the automation, and be available to intervene and take over for the partial automation system at any time. In practice, however, drivers are poor at monitoring automation and do not perform well on tasks requiring passive vigilance. Research shows that drivers often become disengaged from the driving task, both for momentary and prolonged periods during automated phases of driving.

In the Williston, Florida, crash, we found that the driver was disengaged from supervising the Autopilot partial automation. Tesla assesses the driver’s level of engagement by monitoring driver interaction with the steering wheel through changes in steering wheel torque. In the Williston accident, when Autopilot was active prior to the crash, the system detected that the driver applied steering wheel torque only 2 percent of the time. Because Tesla uses steering wheel torque as a metric of driver engagement, the low percentage of driver applied torque in the Williston crash indicated a highly disengaged driver. This measure of driver engagement, however, is misleading. Because driving is a highly visual task, a driver’s touch or torque of the steering wheel may not accurately indicate that he or she is fully engaged with the driving task. Simply checking whether the driver has placed a hand on the steering wheel gives little indication of where the driver is focusing his or her attention.

Following our Williston, investigation, we concluded that the way the Tesla Autopilot system monitored and responded to the driver’s interaction with the steering wheel was not an effective method of ensuring driver engagement. As a result, we recommended that six manufacturers of vehicles equipped with Level 2 driving automation systems:

Develop applications to more effectively sense the driver’s level of engagement and alert the driver when engagement is lacking while automated vehicle control systems are in use. (H-17-42)

In response to Safety Recommendation H-17-42, five of the six manufacturers responded with actions they were taking to monitor a driver’s level of engagement. Tesla was the only manufacturer that did not officially respond. Because the operational design of partial driving automation systems requires an attentive driver as an integral system element, we will continue to advocate for manufacturers’ improved monitoring of driver’s level of engagement while supervising automation.

Event Data Recorders for Automated Vehicles

Title 49 CFR Part 563 sets forth requirements for data elements, data capture and format, data retrieval, and data crash survivability for event data recorders (EDRs) installed in light vehicles manufactured on or after September 1, 2012.[8] The regulation did not mandate the installation of EDRs in light vehicles; rather, if the vehicle manufacturer chose to install an EDR, the regulation defines the format and specifies the requirements for providing commercially available tools and the methods for retrieving data from the EDR in the event of a crash.

On December 13, 2012, NHTSA issued a notice of proposed rulemaking (NPRM) that proposed a new Federal Motor Vehicle Safety Standard (FMVSS) mandating that an EDR that meets 49 CFR Part 563 requirements be installed on most light vehicles. On February 8, 2019, NHTSA withdrew the NPRM because the agency determined that a mandate was not necessary. NHTSA’s internal analysis showed that, for model year 2017, 99.6 percent of new light vehicles sold were equipped with EDRs that met Part 563 requirements. NHTSA added that, given the near universal installation of EDRs in light vehicles, it no longer believed that the safety benefits of mandating EDRs justified the expenditure of limited agency resources.

In withdrawing the final rule, NHTSA said that it would continue its efforts to modernize and improve EDR regulations, including fulfilling the agency’s statutory mandate to promulgate regulations establishing an appropriate recording duration for EDR data to “provide accident investigators with vehicle-related information pertinent to crashes involving such motor vehicles.”[9] Because 49 CFR 563 data recording requirements codified more than a decade ago are very limited (only 15 data elements require reporting), NHTSA stated that it is actively investigating whether the agency should consider revising the data elements covered by Part 563 to account for advanced safety features.

In recent Tesla crash investigations, we were able to retrieve data from the EDR, but the EDR data recorded did not address the partial driving automation system’s activation or engagement. As a result, we used other proprietary manufacturer data to interpret the automation system’s functionality, but this type of data is not available on many vehicles operating with these systems today. Further, there are currently no commercially available tools for an independently retrieving and reviewing any non-EDR vehicle data, and other manufacturers of vehicles with driving automation systems control access to the postcrash proprietary information associated with their vehicles.

As more manufacturers deploy driving automation systems on their vehicles, to improve system safety, it will be necessary to develop detailed information about how the active safety systems performed during, and how drivers responded to, a crash sequence. Manufacturers, regulators, and crash investigators all need specific data in the event of a system malfunction or crash. Recorded data can be used to improve the automated systems and to understand situations that may not have been considered in the original designs. NTSB investigators need effective event data to conduct valid and productive investigations involving vehicles using AV control systems. Further, data are needed to distinguish between automated control actions and driver control actions.

Following the Williston crash, we made a recommendation to the US Department of Transportation (DOT) regarding the need to define data parameters necessary to understand AV control systems and two recommendations to NHTSA to define a standard reporting format and to require manufacturers equipped with driving automation systems to report incidents, crashes, and vehicle miles operated with the systems enabled.[10]

To the DOT:

Define the data parameters needed to understand the automated vehicle control systems involved in a crash. The parameters must reflect the vehicle’s control status and the frequency and duration of control actions to adequately characterize driver and vehicle performance before and during a crash. (H-17-37)

To NHTSA:

Use the data parameters defined by the U.S. Department of Transportation in response to Safety Recommendation H-17-37 as a benchmark for new vehicles equipped with automated vehicle control systems so that they capture data that reflect the vehicle’s control status and the frequency and duration of control actions needed to adequately characterize driver and vehicle performance before and during a crash; the captured data should be readily available to, at a minimum, National Transportation Safety Board investigators and National Highway Traffic Safety Administration regulators. (H-17-39)

Define a standard format for reporting automated vehicle control data and require manufacturers of vehicles equipped with automated vehicle control systems to report incidents, crashes, and vehicle miles operated with such systems enabled. (H-17-40)

In response to these recommendations, NHTSA has communicated with SAE International about developing industry standards, but explained the following:

Manufacturers are not currently required to enable vehicles to record data from usage of driving automation systems (SAE levels 1-2) or operation of such systems during crash triggered events. The ability for traditional vehicle manufacturers and other stakeholders to report on automated technology system use and its operation during incidents and crashes is highly dependent on each vehicle’s specific recording and downloading technology.

Additionally, NHTSA stated that it believes developing recording requirements is best accomplished through voluntary compliance until industry consensus on standard data elements can be established.[11]

It is unlikely that crash investigators and regulators will fully understand the causal factors in a crash without easily accessible data from driving automation systems; therefore, we will continue to advocate action on these safety recommendations.

Conclusion

Thank you again for the opportunity to be here today to discuss highly automated vehicles and some initial steps that can be taken by the DOT and states to advance the safe testing and deployment of automated driving systems. I will be happy to answer any questions.

 

 


 

  1. NHTSA Traffic Safety Facts, 2018, Fatal Motor Vehicle Crashes Overview, DOT Hs 812 826, October 2019.
  2. SAE International Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles, Recommended Practice J3016, June 2018.
  3. See NHTSA 2016 Federal Automated Vehicle Policy—Accelerating the Next Revolution in Roadway Safety; NHTSA 2017 Automated Driving System 2.0: A Vision for Safety; and NHTSA 2018 Preparing for the Future of Transportation: Automated Vehicles 3.0. The 12 safety elements described in ADS 2.0 are: system safety, operational design domain, object event detection and response, fallback (minimal risk condition), validation methods, human-machine interface, vehicle cybersecurity, crashworthiness, post-crash ADS behavior, data recording, consumer education and training, and federal/state/local laws.
  4. Investigations into two of the fatal crashes occurring in Delray Beach, Florida, and Mountain View, California, are ongoing, with final reports scheduled to be released in early 2020.
  5. Collision Between a Car Operating with Automated Vehicle Control Systems and a Tractor-Semitrailer Truck Near Williston, Florida, May 7, 2016, NTSB/HAR-17/02.
  6. Tesla provided this response during NTSB’s ongoing investigation of the Mountain View, CA crash.
  7. See Delray Beach Highway Preliminary Report (HWY19FH008)
  8. The EDR requirements apply to “light vehicles” required to have frontal airbags—those with a gross vehicle weight rating of 3,855 kilograms (8,500 pounds) or less and an unloaded vehicle weight of 2,495 kilograms (5,500 pounds) or less.
  9. See the Fixing America’s Surface Transportation (FAST) Act Public Law 114-94 (Dec. 4, 2015) section 24303.
  10. The current status of safety recommendation H-17-37 is “Open—Initial Response Received.” H-17-39 and -40 are both classified “Open—Acceptable Response.”
  11. NTSB experience with crashes involving different levels of driving automation shows that the amount and availability of recorded data varies widely among manufacturers.