ML21063A231

NPIC-2021-Paper-Final-737MAX-03-04-21bbb

ML21063A231
Person / Time
Issue date:	03/04/2021
From:	Ismael Garcia, David Rahn, Michael Waters NRC/NRR/DEX/EMIB
To:
Mike Waters 415-4039
Shared Package
ML21063A216	List: ML21063A231
References
Paper ID 34348
Download: ML21063A231 (12)

Text

Paper ID 34348 PRELIMINARY INSIGHTS ON DIGITAL INSTRUMENTATION AND CONTROL REGULATORY LESSONS FROM THE BOEING 737 MAX 8 CRASH EVENTS Michael Waters, David Rahn, and Ismael Garcia U.S. Nuclear Regulatory Commission Washington, D.C. 20555 michael.waters@nrc.gov; david.rahn@nrc.gov, ismael.garcia@nrc.gov ABSTRACT In 2017, the Boeing 737 MAX 8 was certified for operation as an amendment to the preceding 737 Next Generation airframe and controls systems. The MAX 8 incorporated several design changes including new and larger engines, aerodynamic improvements to the aircraft, and the addition of the Maneuvering Characteristics Augmentation System (MCAS) software onto the flight control computer. MCAS was designed to compensate for potential aerodynamic stall hazards associated with the increased pitch rate created by the placement of the new engines. Two MAX 8 aircraft crashed in 2018 and 2019 shortly after takeoff due to repeated activation of the MCAS and the resulting orientation of the aircraft, which was not overcome in time by the aircraft pilots.

Multiple U.S. and international authorities investigated engineering and programmatic factors that may have that contributed to the MCAS design and subsequent crash events. The Nuclear Regulatory Commission (NRC) staff is reviewing multiple investigative reports to identify potential generic regulatory issues related to the implementation of new digital technologies in nuclear power plants. This paper describes the preliminary insights of the NRC evaluation of the reports in key areas such as design and implementation specifications, hazard and risk assessment assumptions, and the regulatory processes for approval and oversight of amended designs. The paper will also explore potential NRC regulatory improvements and organizational considerations for providing a holistic approach in maintaining reasonable assurance of digital I&C safety through licensing processes, inspection oversight, and overall safety culture.

Key Words: Boeing 737 MAX 8, Defense-in-Depth, Human Factors Engineering, Maneuvering Characteristics Augmentation System, Software 1 INTRODUCTION The NRC licenses and regulates the Nation's civilian use of radioactive materials to provide reasonable assurance of adequate protection of public health and safety and to promote the common defense and security and to protect the environment. In support of this mission, the NRC is responsible for ensuring safety of new digital instrumentation and control (I&C) technologies that are being incorporated into existing and new nuclear power plants. Specifically, the NRC regulates digital I&C through licensing and certification approvals of designs for both existing and new reactors, topical report approvals, inspection oversight, and regulation and guidance development.

A series of failures in the development and implementation of the MCAS certified by Federal Aviation Administration (FAA) for an upgraded Boeing 737 design (MAX 8), led to two aircraft crashes in 2018 and 2019. Investigative reports by FAA, Department of Transportation, National Transportation Safety Board, and other entities on the MCAS design process and the associated FAA certification processes raised potential generic regulatory lessons for consideration by the NRC. The NRC instrumentation and controls staff systematically evaluated the findings and recommendations regarding design, development, and regulatory oversight issues documented in these authoritative reports. The NRC staff considered the applicability of the documented findings and recommendations to the NRC digital I&C regulatory program, while factoring the challenges associated with comparing designs and regulations between aircrafts and nuclear power plants. The NRC staff focused on (1) identifying any

Paper ID 34348 relevant, potential regulatory gaps in NRCs digital I&C licensing and inspection program, including associated processes and culture; and (2) identifying elements of the NRCs regulatory program and organizational capabilities that should be maintained or improved to support the continued safe use of digital I&C in nuclear plants. The objective of the NRC staffs review was solely to assess its own processes regarding the introduction of new digital technologies into existing nuclear plant architectures, and development of highly integrated systems for new reactors.

2 BOEING 737 MCAS DEVELOPMENT AND CERTIFICATION CONSIDERATIONS In 2011, facing a competitive threat from Airbuss new, more fuel efficient, single-aisle A320 aircraft, Boeing believed it did not have time to create and certify a new plane from scratch [1].

Instead, Boeing opted to modify its existing 737 NG aircraft to make it more fuel efficient. To help accomplish this, Boeing installed, larger, more fuel-efficient engines on this new 737 derivative airplane model dubbed the 737 MAX aircraft. Thus, the plane is a modified version of the previous design with larger engines to improve flight economy. The MCAS was one of many upgrades associated with the new Boeing 737 MAX 8 to address changes in aerodynamics. FAA began its review of an amendment type certificate application in 2012 and issued its approval in March of 2017.

Because the larger engines could not have adequate ground clearance, they were moved forward and higher up (over the lead edge of the wing) of the aircraft. As a result, the aerodynamics of the plane were changed for certain flight maneuvers, especially during high angle of attack (AoA)1. If pilots applied engine thrust while the airplane was pitched up at a high AoA, the airplane could pitch up even more and result in an aerodynamic stall. Corrective actions by the pilots would involve pushing the nose down to increase airflow across the wings and regain lift.

To help the aircraft compensate for those flight conditions, Boeing developed MCAS software to run on the Boeing 737 flight control computers as an additional function of the existing aircraft speed trim system during manual flight. A version of MCAS was originally provided in the U.S. Air Forces KC-46A Pegasus refueling tanker, which was a derivative of Boeings commercial 767. The software was redesigned for the 737 MAX 8 [1]. MCAS is an automated system designed to activate if and when the 737 MAX reached limited flight configurations involving high AoA. The system was designed to work in the background to counteract potential nose pitch from the larger engine upgrades by instructing the stabilizer to pitch the plane back down when the angle-of-attack sensor exceeded a threshold based on airspeed and altitude. Specifically, an uncommanded activation of MCAS would move the planes horizontal stabilizer to push the planes nose in a downward direction to reduce the AoA. The goal was to make the plane feel and handle exactly like previous Boeing 737 versions that pilots around the world were accustomed, in order to minimize new pilot training requirements.

Based in part on probability and qualitative consequence assumptions, Boeing did not rank the MCAS upgrade in the highest risk category among all the changes pursued in the Boeing 737 Max 8 certification request. Boeing performed a functional hazard assessment related to the software, including an uncommanded MCAS activation that continued until the pilot took action. While not its intent, the MCAS under failure conditions would have the effect of moving the planes nose down during manual flight if not counteracted by the pilot. Boeing pilots and engineers assumed that commercial pilots would recognize the effect of unintended MCAS activation as a runaway stabilizer, which is a scenario addressed in commercial pilot training. Boeing tested a single, unintended activation of MCAS and assumed multiple activations of MCAS to be no worse than a single activation [2].

The FAA deemed that the certification of the MCAS portion of the design could be delegated to Boeing using the self-certification process, under the FAAs Organization Designation Authorization (ODA) program. Boeing conducted specific design, implementation, integration (e.g., with other equipment and operators), and testing activities in a lifecycle development process for the MCAS. As a 1 Angle of attack is the angle between the oncoming air or relative wind and a reference line on the airplane or wing [1].

Paper ID 34348 result of flight testing, MCAS was later programmed to also counteract accidental low speed stalls, with more aggressive authority to increase the rate of down pitch based on AoA sensor information.

While the Boeing 737 MAX is equipped with two AoA sensors, the MCAS software used input from a single AoA sensor only. The decision to rely on a single AoA sensor appears to be based on assumptions that an uncommanded MCAS failure condition would not lead to severe consequences.

Furthermore, Boeing had intended for all MAX 8 aircraft to be equipped with an alert to pilots when two AoA sensors disagreed by more than 10 degrees for at least 10 seconds. After certification by the FAA, Boeing discovered that not all 737 MAX aircraft were equipped with this alert but determined that the cockpit alert was not necessary for the safe operation because there were no required procedures associated with the alert. Boeing intended to correct the problem for the entire fleet but did not have to submit a formal notification to the FAA oversight office because it was not deemed as an operational impact. FAA did not become aware of this issue until after the Lion Air crash [2].

Pilots received no flight simulation training on this feature and MCAS was not identified in the flight manual, because again it was assumed that a pilot could handle any MCAS-related errors in a manner similar to errors in the familiar automatic trim controls of the horizontal tail known as runaway trim. But early in the 737 MAX program, Boeing was aware that it could take some pilots 10 seconds or longer to respond to runaway stabilizer trim or uncommanded MCAS activation [1]. Furthermore, Boeing also knew that its own test pilot took more than 10 seconds to respond to an uncommanded MCAS activation in a fight simulator and found the condition catastrophic [1].

On October 29, 2018, Lion Air Flight 610 crashed into the Java Sea shortly after departing Soekarno-Hatt International Airport, Jakarta, tragically resulting in 189 fatalities. MCAS was determined to be a significant contributor to the accident. It activated 24 times during the flight after receiving faulty data from one of the aircrafts two AoA sensors. A few months later, on March 10, 2019, Ethiopian Air Flight 302 crashed shortly after departing Addis Ababa Bole International Airport, resulting in 157 fatalities. In depth information on the details of the accident events, technical design and human factors issues, and regulatory issues can be found in the reports listed in Section 4 below.

3 ATTRIBUTES OF NRC I&C LICENSING AND FAA CERTIFICATION APPROACHES FOR SAFETY-CRITICAL SOFTWARE The FAA is responsible for regulating aviation safety, which includes approving the design and manufacture of new aircraft and aviation products before they enter air commerce. Safety-critical equipment applicable to each certified aircraft must receive approval by the FAA, using a rigorous process demonstrating that the equipment design is appropriate for the equipments intended functions.

At its highest-level, FAAs guidance provides a general safety assessment process and includes the ability to apply gradations to development and test activities. Airworthiness regulations [3] for the aircraft specify that systems and associated components, considered separately and in relation to other systems, must be designed so that, among other things, the occurrence of any failure condition which would prevent the continued safe flight and landing of the airplane is extremely improbable. The FAA has published advisory circulars that recognize voluntary consensus standards for aircraft avionics equipment to address the permitting process for each airframe as a part of the overall aircraft certification process

[4]. These voluntary industry consensus standards and recommended practices are coordinated internationally.

The FAAs permitting approach, known as its design approval, uses a combination of mandatory requirements and voluntary guidance. The process consists of five phases [5], each with different levels of engagement between the applicant and FAA, to increase efficiency. To leverage its staff resources, the FAA is authorized by statute to delegate certification and other functions of the agency to qualified individuals, including through the Organization Designation Authorization (ODA) program. Therefore,

Paper ID 34348 one practice used by FAA is the appointment of designated engineering representatives2 as third-party verifiers for the aircraft. This representative may approve or recommend approval of technical data to the FAA in support of aircraft certification. The following is an example of the amount of effort involved to review a design change to one aircraft type:

Under FAAs [Organization Designation Authorization (ODA)] program, the Agencys Boeing Aviation Safety Oversight Office (BASOO) provides oversight of authorized functions granted to Boeing. The BASOO is comprised of 45 FAA employees who oversee Boeings ODA. The Boeing ODA unit includes approximately 1,500 Boeing-designated ODA representatives. FAAs oversight program is based on managing and supervising an organization, rather than overseeing individual designees. [6]

The ODA program was intended to allow the FAA to delegate certification of well-understood, non-critical, or low-risk designs so that the FAA can remain directly involved in review and approval of higher-risk items, such as safety-critical or new and novel designs [1]. Nonetheless, the FAA bears ultimate responsibility for ensuring new aircraft designs are safe and comply with airworthiness standards. Such a delegation practice is not limited to the FAA, as civil aviation authorities worldwide also have delegation programs to leverage the product-specific knowledge of manufacturers qualified employees to determine a products compliance with government regulations or requirements. The personnel involved in the ODA and BASOO were in addition to FAA staff that performed the review of the amended type certificate (ATC) application for the Boeing 737 MAX 8. The review and approval process for a design change would take approximately five years to complete such as the ATC for the Boeing 737 MAX 8.

NRC has generally licensed operating nuclear power plants under a two-step process described in Title 10 of the Code of Federal Regulations (10 CFR) under Part 50 [7]. This process requires both a construction permit and an operating license. In 1989 the NRC established alternative licensing processes in 10 CFR Part 52 [8] that included a combined license. This process combines a construction permit and an operating license at the same time with conditions for plant operation. A combined license under 10 CFR Part 52 authorizes construction of the facility and specifies the inspections, tests, and analyses that the applicant must perform, including those for digital I&C. It also specifies acceptance criteria that are necessary to provide reasonable assurance that the facility has been constructed and will be operated in agreement with the license and applicable regulations. Closure of acceptance criteria are documented by the licensee after the licensing decision. NRC independently inspects vendors and licensees, and confirms selected analyses and tests are addressed appropriately.

The U.S. operating fleet of reactors also continue to implement several types of digital upgrades (including replacement of analog I&C equipment) without NRC approval under the requirements of 10 CFR 50.59 [7]. For this process, licensees are required to evaluate the effects of digital upgrades on the licensing basis of the plant with respect to potential changes in the likelihood or consequence of existing malfunctions and accidents in the licensing basis. Generally, licensees can make such changes if there is not more than a minimal increase in likelihood or consequences, or new types of malfunctions and results.

NRC recently issued guidance specific for digital I&C upgrades in Regulatory Issue Summary (RIS) 2002-22, Supplement 1 [9]. The guidance in part focuses on addressing 10 CFR 50.59 criteria in consideration of the existing I&C architectures that form the licensing bases of the plant, and potential new failure modes of the digital I&C. NRC may elect to inspect selected digital upgrades for conformance to 10 CFR 50.59 and the quality assurance requirements contained in 10 CFR Part 50 Appendix B.

2 A designated engineering representative is an individual, appointed in accordance with 14 CFR 183.29, who holds an engineering degree or equivalent, possesses technical knowledge and experience, and meets the qualification requirements of Order 8100.8.

Paper ID 34348 Licensee have submitted license amendment requests for major digital upgrades to safety critical systems such as reactor protection systems and emergency core cooling systems. The NRC reviews these requests using a standard review plan, interim staff guidance, and regulatory guides. Past license amendment approvals have involved a review of the design, diversity and defense-in-depth human factors engineering (HFE), and outcomes of the software development, including validation and verification (V&V) activities up to factory acceptance testing. NRC recently updated Interim Staff Guidance (ISG) 06 [10] to support approval of digital modifications earlier in the design phase before the system is fully integrated and tested. This newer approach focuses strongly on high-level design specifications, diversity and defense-in-depth, and software development plans, rather than the traditional review of the results of digital I&C end products. These later-stage development activities may be subject to NRC license conditions as part of the decision. NRC also plan to independently inspect vendors and licensees after license approval under this approach.

In comparison, the general design principles, methods for development, and overall regulatory principles to ensure safe and reliable performance of digital equipment can be generally compared between NRC and FAA. The FAA approach of focusing its attention on the most critical certification area seems compatible with the NRC risk-informed approach of focusing resources on the safety-significant aspects of a digital design. However, the two agencies have different licensing and certification processes that could not be directly compared further for this assessment. Examples of these differences include: (1) The FAAs certification approach and specific standards for digital avionics are different from the NRC risk-informed and deterministic approaches and standards for digital I&C; (2) the specific control and safety functions, and associated failure risks, are fundamentally different between aircraft avionics and nuclear digital I&C; and, (3) the scale and operational experience is vastly greater for the fleet of operating aircraft than that for digital I&C in operating U.S. nuclear power plants.

4 EVALUATION OF KEY REGULATORY AND TECHNICAL THEMES The NRC I&C staff systematically evaluated the findings and recommendations regarding various MCAS design, development, and regulatory oversight issues from three primary reports:

Official Report of the Special Committee to Review the Federal Aviation Administration's Aircraft Certification Process; [11]

Joint Authorities Technical Review - Observations, Findings, and Recommendations; [12] and, National Transportation Safety Board Report - Assumptions Used in the Safety Assessment Process and the Effects of Multiple Alerts and Indications on Pilot Performance. [13]

The staff also reviewed selected aspects from other reports including:

The House Committee on Transportation & Infrastructure Final Report on the Design, Development & Certification of the Boeing 737 Max; [1]

Department of Transportation - Office of Inspector General - Timeline of Activities Leading to the Certification of the Boeing 737 MAX 8 Aircraft and Actions Taken After the October 2018 Lion Air Accident; [6]

U.S. Department of Transportation Office of Inspector General Report - Weaknesses in FAAs Certification and Delegation Processes Hindered Its Oversight of the 737 MAX 8. [2]

U.S. Senate Committee on Commerce, Science, & Transportation Investigation Report on Aviation Safety Oversight. [14] and; Final KNKT.18.10.35.04 Aircraft Accident Investigation Report; [15]

Based on reviews of the reports, the NRC identified two generic categories of findings that were relevant to NRCs regulatory purview for digital I&C: (1) Design and Implementation Issues; and (2)

Regulatory Oversight Issues. Within the category of Design and Implementation, the staff defined the following themes for consideration: (1) Design Specifications and Defense-In-Depth; (2) Operational

Paper ID 34348 Specifications; (3) Safety Assessment including Hazard Analysis and Risk Assessments; (4) Equipment Design and Implementation; (5) Performance Monitoring; and, (6) Production and Certification. For the generic category of Regulatory Oversight, the staff identified the following themes for evaluation: (1)

Certification and Licensing Standards; (2) Amended Certification Processes; (3) Coordination among Regulatory Standards and Certification Bodies; (4) Delegation of Certification and Post-Certification Design Change Processes; (5) Regulating Technical Innovation; (6) Personnel Capabilities of the Regulator; and (7) Safety Culture.

The staff evaluated each of the three primary reports in depth, and to the extent practical, aligned specific report findings to one or more of the themes in a matrixed approach. This effort was challenging given the purpose and focus areas of each of the three reports are significantly different, given the different responsibilities and charter of the respective organizations that investigated the crash events. The NRC staff also iteratively adjusted the specific themes to accommodate unique insights that individual reports exposed.

Sample regulatory/technical themes and insights identified as part of the preliminary NRC staff assessment include the following:

Safety Assessment (including hazard analysis and risk assessments) - One of the reports identified the need to understand what needs to go right (performance and design specifications),

what could go wrong (human and equipment failure modes), what can prevent things from going wrong (controls and barriers), and the combination of events and scenarios in which the human-equipment system must function [11].

Such a recommendation is applicable to NRC approaches to addressing safety evaluations. NRC regulatory guidance documents (e.g., Regulatory Guide (RG) 1.174 [16],

NUREG-0800 Chapter 18 and its Attachment A [17], NUREG-1764 [18], NUREG-1852 [19])

factor in the assessment of human performance issues when assessing plant safety margin for responses to identified events--particularly the considerations regarding systematic human error assessment. For example, hazard analysis techniques and probabilistic risk assessments factor in combinations of equipment failures and operator errors. The NRC guidance addresses criteria for licensing both new designs and changes to existing plant designs. During past digital I&C licensing reviews, technical experts in the disciplines of I&C, human factors, and safety systems coordinate on significant issues during key points of the licensing review. NRC vendor inspectors are sometimes involved in the review process on software development and QA-related issues.

Equipment Design and Implementation - One of the reports identified the need for a Safety Management System (SMS) help to ensure a holistic, proactive assessment of whether the combination of design, procedures, and training will support effective safety performance [11].

Such a SMS requirement for design and manufacturing organizations would help ensure a comprehensive, systematic approach to aviation safety from design to operation.

Such a recommendation would be applicable to NRC digital I&C regulatory processes.

There are regulations in place (e.g., Appendix B in 10 CFR Part 50 [7]) along with guidance (e.g.,

RG 1.152 [20]) for licensees to ensure a complete, systematic approach to safety from design to operations. For example, Criterion III (Design Control) of 10 CFR Part 50, Appendix B establishes quality assurance requirements for the design, manufacture, construction, and operation of SSCs. Furthermore, RG 1.152 provides guidance for complying with the NRC regulations for promoting high functional reliability, design quality, and a secure development and operational environment, for the use of digital computers in the safety systems of nuclear power plants.

Paper ID 34348 Amended Certifications Process (new designs on existing architectures) - One of the reports discusses that while FAA followed regulations and guidance for determining whether the Boeing design qualified for evaluation as an amended type certificate, there is opportunity for regulatory improvement in areas such as: (1) understanding and documenting any assumptions related to pilot expectations for the performance of the modified design and whether there is a need for supplemental pilot training; (2) review of the cumulative effects of multiple changes to existing certified aircraft designs; and, (3) providing of a holistic system operational risk assessment, and internal communication [11].

Such findings are applicable to the NRC approach to digital I&C licensing oversight. As previously discussed, there are regulations (e.g., 10 CFR 50.59 [7]) for making changes without NRC approval and guidance for assessing the impact of a proposed I&C change on the existing approved licensing basis for the plant. For license amendment requests, there is design guidance and human factor guidance for ensuring designs will achieve safety functions under assumed accident conditions, including an examination of diversity and defense-in-depth against potential software common cause failures.

Delegation of Certification - One of the reports recommends that FAA and industry to work together to address concerns about potential undue pressure on an ODA Unit, in order to maintain the independent decision-making structure of the ODA and ensure that the ODA fulfills its requirement to serve as a representative of the FAA Administrator [11].

While such a recommendation does not directly correlate to the NRC regulatory infrastructure, the NRC strives to maintains independent processes for licensing reviews of digital designs and independent NRC inspection processes of digital I&C implementation into nuclear facilities. The NRC also cultivates and maintains a robust safety culture for its own employee organization and throughout the oversight of its licensee and applicant organizations. This safety culture incorporates elements of enforcement to ensure the organizational focus is always on safety, and that no employee or contractor may be subject to discrimination or retaliatory actions if they raise questions regarding the achievement of safety. Additional information on the NRCs safety culture is discussed below.

Safety Culture - According to investigative reports, Boeing engaged in multiple efforts to downplay the role and potential safety implications of MCAS on the Boeing 737 MAX 8 and attempted to delete reference to MCAS from various Boeing documents [1]. For example, Boeing sought and received approval from the FAA to remove references to MCAS from the airplanes flight crew operations manual and training materials.

Such a safety culture is contrary to the NRCs safety culture. Specifically, the NRC defines nuclear safety culture as the core values and behaviors resulting from a collective commitment by leaders and individuals to emphasize safety over competing goals to ensure protection of people and the environment [21]. The implementation of a safety culture within the NRC involves a series of traits further defining a positive safety culture. These traits describe patterns of thinking, feeling, and behaving that emphasize safety, particularly in goal conflict situations, such as when safety goals conflict with production, schedule, or cost goals. Such positive safety culture fosters an environment where issues potentially impacting safety are promptly identified, fully evaluated, and promptly addressed and corrected commensurate with their significance. The NRC also applies the Principles of Good Regulation, which include independence where all available facts and opinions must be sought openly from licensees and other interested members of the public

Paper ID 34348

[22]. As such, final decisions must be based on objective, unbiased assessments of all information, and must be documented with reasons explicitly stated.

5 PRELIMINARY REGULATORY INSIGHTS FOR NRC The preliminary evaluation performed by the NRC staff resulted in a series of generic regulatory lessons for consideration. The following list highlights some of key preliminary insights on regulatory aspects or activities that should be maintained and/or further improved (but is not all inclusive):

Regulatory Oversight Issues Integration and communication during the digital design review, HFE review, and subsequent inspection oversight processes are critical to understand and evaluate a digital design during its life cycle. The NRC I&C and HFE staff review digital designs following a standard review plan and typically coordinate on common review areas during the license review. Separately, NRC vendor and regional inspectors may elect to inspect detailed design implementation, testing, and installation within specific inspection procedures during and after the regulatory review. The insights emphasize the need to holistically consider the potential evolution of digital designs and associated assumptions from conception to installation within the fields of I&C and HFE.

The NRC I&C and HFE technical disciplines should consider even closer communication during the license review, with a continued safety culture of challenging each others assumptions in respective review areas. The NRC digital I&C staff have also started to methodically engage NRC risk assessment specialists to provide insights on key review issues within an integrated review strategy under LIC-206 [23].

The NRC intends to programmatically define the communications, interactions, and hand-off of technical issues between licensing and inspection staff for large scale digital modifications, especially under the new licensing process in ISG-06 [10]. The NRC, for example, will continue to embed licensing technical staff in selected inspection activities after licensing, and invite vendor inspectors to participate more directly in portions of the licensing review. The I&C technical review staff intends to clearly document recommended inspection items at the end of a major digital licensing review, and directly communicate with Regional inspectors through the testing and site-installation of approved digital systems.

Inspection priorities for digital I&C modifications made under 10 CFR 50.59 without NRC prior approval should be strategic and risk informed. Most digital modifications to existing nuclear power plants have been and will continue to be performed under 10 CFR 50.59. The NRC is beginning a Smart Sample initiative for digital upgrades across the U.S. operating fleet to ensure inspection resources are focused on the most important upgrades based on risk-insights and practical experience.

NRC should continue its focus on risk significant digital systems, including those evolving technologies with highly integrated digital systems. Specifically, a safety-focused approach should continue to be followed during the staff licensing reviews to ensure the agency resources are focused on the safety-significant items. The NRC should also continue to apply its risk-informing principles based on compliance, defense-in-depth, safety margins, probabilistic risk assessment, and operational performance.

An effective and forthright safety culture remains paramount and allows the agency to effectively fulfill its core regulatory and oversight mission to support the continued safe use of digital I&C in nuclear plants. A positive safety culture in our regulation of digital I&C should be maintained and

Paper ID 34348 NRC staff and management should continue to be emphasize and demonstrate safety culture attributes and the NRC Principles of Good Regulation.

NRC organizational capabilities and knowledge management activities should be maintained to address long-term attrition of expert Agency staff in the digital I&C disciplines.

Sharing and considering information and insights with international and domestic regulators on digital technologies provides for a more robust safety program. NRC digital I&C staff should continue practices of periodic seminars and exchanges with other regulators on common digital I&C issues. The NRC should continue to participate in I&C domestic and international standard bodies.

Design and Implementation Issues A defense-in-depth approach continues to be an effective engineering means to account for uncertainties in digital equipment and human performance, in particular to account for the potential for unknown and unforeseen failure mechanisms or phenomena. The NRC should continue to emphasize the need for applying a defense-in-depth and diversity approaches for digital I&C and the overall NPP design. Such an approach includes: (1) performing analysis of a proposed digital I&C system design to demonstrate that vulnerabilities to common mode failures have been adequately addressed; (2) examining a digital I&C system to identify hazards that have the potential to cause harm (e.g., radiological consequences, loss of life, damage to the environment), and (3) implementing I&C functional requirements and means to eliminate, prevent, or control those hazards.

Systematic hazard analysis techniques may be important to address new digital technologies that are highly integrated in nature. NRC is researching and evaluating options for performing systematic hazard analysis for digital I&C systems. For example, the NRC is examining Annex D (Identification and control of hazards) of IEEE 7 4.3.2-2016 [24] to confirm it supports an adequate technical basis for endorsement as a new hazard analysis technique that can be employed by applicants and licensees.

Operating experience and data are important for justifying reliability claims provided for digital designs and to ensure such claims remain valid during operation. Digital I&C licensing and inspection efforts that already consider operating experience include but are not limited to: (1) RIS 2002-22, Supplement 1 [9]; (2) Branch Technical Position 7-19 [25]; and, (3) RG 1.233 [26].

However, the NRC staff could benefit from enhanced understanding of the available digital &C operating experience with guidance for crediting it in reliability determinations for new digital systems and associated limitations. It would also enhance the long-term focus of NRC regulatory reviews and inspection activities related to (1) safety-related digital upgrades under 10 CFR 50.59; (2) new/advanced reactor applications; and (3) operating reactor license amendment requests.

Implementing a system wide engineering approach to safety from design to operation, maintenance, and human factors is important for ensuring that an approved and delivered I&C design has the intended system functionality. NRC regulations and guidance include those associated with HFE-related considerations such as: (1) the organization of Human System Interfaces (HSIs) into workstations (including consoles and panels), (2) the arrangement of workstations and supporting equipment into facilities, such as a main control room, remote shutdown station, local control station, technical support center, and emergency operations facility; and, (3) the environmental conditions in which the HSIs are used, including temperature, humidity, ventilation, illumination, and noise. The NRC should continue to focus on HFE as a critical component of new digital designs. NRC staff should continue to emphasize integrated technical teams to track and resolve digital design and HFE technical issues for safety significant digital I&C reviews.

Paper ID 34348 6 CONCLUSIONS The tragic crashes were the result of a series of engineering, programmatic, safety culture failures, and shortcomings related to the MCAS design, implementation, and training. It was challenging for the NRC staff to make an in-depth technical comparison of the safety functions, failure consequences, defense-in-depth, and risks of an avionics system and aircraft to the digital controls system of a nuclear plant. The NRC staff did not identify significant gaps in our NRC regulatory infrastructure for digital I&C licensing and inspection. The NRC staff has preliminarily identified multiple aspects of the NRCs digital I&C regulatory program and organizational capabilities that should be maintained or further improved to ensure the continued safe use of evolving digital I&C technologies in nuclear plants. The NRC intends to complete and issue its final assessment in 2021. This paper is not intended to represent formal NRC policy or position on a regulatory matter.

7 ACKNOWLEDGMENTS We thank our NRC staff colleagues, Mrs. Jeanne Johnston, Messrs. Dinesh Taneja, Norbert Carte, and Sergiu Basturescu, who provided insight and expertise that greatly assisted the preliminary evaluation, although they may or may not necessarily agree with all of the preliminary insights/conclusions documented herein. We would like to show our gratitude to these NRC staff colleagues for sharing their pearls of wisdom with us during the course of the preliminary evaluation documented herein. We also thank our NRC staff colleague, Mr. Eric Benner, who kindly reviewed the paper.

8 REFERENCES

1. The House Committee on Transportation & Infrastructure Final Report on the Design, Development

& Certification of the Boeing 737 Max, https://transportation.house.gov/committee-activity/boeing-737-max-investigation (2020).

2. U.S. Department of Transportation Office of Inspector General Report - Weaknesses in FAAs Certification and Delegation Processes Hindered Its Oversight of the 737 MAX 8, https://www.oig.dot.gov/sites/default/files/FAA%20Certification%20of%20737%20MAX%20Boein g%20II%20Final%20Report%5E2-23-2021.pdf (2021).

3. 14 CFR § 25.1309 - Equipment, systems, and installations, https://www.govinfo.gov/app/details/CFR-1999-title14-vol1/CFR-1999-title14-vol1-sec25-1309 (1999).

4. 14 CFR Part 21 - Certification Procedures for Products and Articles, https://www.govinfo.gov/app/details/CFR-2011-title14-vol1/CFR-2011-title14-vol1-part21 (2011).

5. The FAA and Industry Guide to Product Certification, Third Edition, https://www.faa.gov/aircraft/air_cert/design_approvals/media/cpi_guide.pdf (2017).

6. Department of Transportation - Office of Inspector General - Timeline of Activities Leading to the Certification of the Boeing 737 MAX 8 Aircraft and Actions Taken After the October 2018 Lion Air Accident, https://www.oig.dot.gov/library-item/37940 (2020).

7. U.S. Nuclear Regulatory Commission, Title 10 of the Code of Federal Regulations Part 50, https://www.nrc.gov/reading-rm/doc-collections/cfr/part050/index.html (2020).

8. U.S. Nuclear Regulatory Commission (NRC), Title 10 of the Code of Federal Regulations Part 52, https://www.nrc.gov/reading-rm/doc-collections/cfr/part052/index.html (2020).

Paper ID 34348

9. U.S. NRC, Regulatory Issue Summary 2002-22, Supplement 1 - Clarification on Endorsement of Nuclear Energy Institute Guidance in Designing Digital Upgrades in Instrumentation and Control Systems, https://www.nrc.gov/docs/ML1814/ML18143B633.pdf (2018).

10. U.S. NRC, Digital Instrumentation and Control - Interim Staff Guidance Licensing Process, Revision 2, https://www.nrc.gov/docs/ML1826/ML18269A259.pdf (2018).

11. Official Report of the Special Committee to Review the Federal Aviation Administration's Aircraft Certification Process, https://www.transportation.gov/briefing-room/official-report-special-committee-review-federal-aviation-administrations-aircraft-0 (2020).

12. Joint Authorities Technical Review - Observations, Findings, and Recommendations, https://www.faa.gov/news/media/attachments/Final_JATR_Submittal_to_FAA_Oct_2019.pdf (2019).

13. National Transportation Safety Board Report - Assumptions Used in the Safety Assessment Process and the Effects of Multiple Alerts and Indications on Pilot Performance, https://www.commerce.senate.gov/services/files/D5F47FAD-8DC6-479D-A53E-B727E80603BC (2019).

14. U.S. Senate Committee on Commerce, Science, & Transportation Investigation Report on Aviation Safety Oversight, https://www.commerce.senate.gov/2020/12/wicker-releases-committee-s-faa-investigation-report (2020).

15. Final KNKT.18.10.35.04 Aircraft Accident Investigation Report, http://docs.house.gov/meetings/PW/PW00/20191030/110066/HHRG-116-PW00-20191030-SD002.pdf (2019).

16. U.S. NRC, Regulatory Guide 1.174 - An Approach for Using Probabilistic Risk Assessment in Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis, https://www.nrc.gov/docs/ML1731/ML17317A256.pdf (2018).

17. U.S. NRC, NUREG-0800 - Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition, Chapter 18 (Human Factors Engineering) https://www.nrc.gov/reading-rm/doc-collections/nuregs/staff/sr0800/ch18/index.html (2016).

18. U.S. NRC, NUREG-1764 - Guidance for the Review of Changes to Human Actions, https://www.nrc.gov/reading-rm/doc-collections/nuregs/staff/sr1764/ (2007).

19. U.S. NRC, NUREG-1852 - Demonstrating the Feasibility and Reliability of Operator Manual Actions in Response to Fire, https://www.nrc.gov/reading-rm/doc-collections/nuregs/staff/sr1852/

(2007).

20. U.S. NRC, Regulatory Guide 1.152 - Criteria for Use of Computers in Safety Systems of Nuclear Power Plants, https://www.nrc.gov/docs/ML1028/ML102870022.pdf (2011).

21. U.S. NRC, Safety Culture Policy Statement, Federal Register, Vol. 76, pp.34773 - 34778 (2011).

22. U.S. NRC, Principles of Good Regulation, https://www.nrc.gov/docs/ML1413/ML14135A076.pdf (2014).

Paper ID 34348

23. U.S. NRC, LIC -206 - Integrated Risk-Informed Decision-Making for Licensing Reviews, https://www.nrc.gov/docs/ML1903/ML19031C861.pdf (2019).

24. IEEE, IEEE 7 4.3.2-2016, IEEE Standard Criteria for Programmable Digital Devices in Safety Systems of Nuclear Power Generating Stations, https://standards.ieee.org/standard/7-4_3_2-2016.html (2016).

25. U.S. NRC, Branch Technical Position 7 Guidance for Evaluation of Diversity and Defense-In-Depth in Digital Computer-Based Instrumentation and Control Systems, https://www.nrc.gov/docs/ML2033/ML20339A647.pdf (2021).

26. U.S. NRC, Regulatory Guide 1.233 - Guidance for a Technology-Inclusive, Risk Informed, and Performance-Based Methodology to Inform the Licensing Basis and Content of Applications for Licenses, Certifications, and Approvals for Non-Light-Water Reactors, https://www.nrc.gov/docs/ML2009/ML20091L698.pdf (2020).