ML24023A622

Final Report - Safety Evaluation Committee (Sec)/Safety Assessment Committee (Sac)

ML24023A622
Person / Time
Site:	National Bureau of Standards Reactor
Issue date:	09/05/2023
From:	US Dept of Commerce, National Institute of Standards & Technology (NIST)
To:	Office of Nuclear Reactor Regulation
Shared Package
ML24023A617	List: ML24023A618 ML24023A619 ML24023A620 ML24023A621 ML24023A622 ML24023A623
References
Download: ML24023A622 (1)

Text

Final Report Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

September 5, 2023

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 2 of 33 Executive Summary On August 1, 2022, a Confirmatory Order Modifying License No. TR-5 was issued by the U.S. Nuclear Regulatory Commission (NRC). NIST has committed to conducting third-party, independent assessment of 6 different program areas at the NCNR as specified in section 2.b, Nuclear Program Assessment(s), of the Confirmatory Order. One of the identified areas is item 5, Safety Assessment Committee (SAC).

Another is item 6, Safety Evaluation Committee (SEC) independence and effective oversight.

The assessment team reviewed documents, including charters and meeting minutes, interviewed SEC and SAC personnel, and observed two SEC meetings. The assessment was conducted as both a compliance assessment and a performance assessment.

As defined in the Project Execution Plan the assessment team classified issues as Areas for Improvement (AFIs) or Areas in Need of Attention (ANAs) as per the following definitions.

- An AFI characterizes an identified performance, program, or process element that requires improvement to obtain the desired results in a consistent and effective manner. AFIs identified in the Assessment Report will be addressed by the Action Plan Submitted to the NRC.

- ANAs identify a performance, program, or process element that is sufficient to meet its basic intent; however, management attention is required to achieve full effectiveness and consistency.

The assessment determined that the SEC/SAC meet the regulatory requirements of the NCNR Technical Specifications.

Effectiveness of the SEC can be improved by addressing the areas noted below:

AFIs AFI -01: The assessment team identified an unauthorized change to the Safety Analysis Report (SAR), weaknesses in justifications in evaluations. A procedure defining how to perform an evaluation has not been developed, and there is a lack of training or qualifications required to perform evaluations.

AFI- 02: A process or procedure to define the audit process has not been developed. ANSI/ANS 15.8, Quality Assurance Program for Research Reactors, section 2.18 states Assessments shall be performed in accordance with written procedures or checklists. Examples:

No procedure that defines the conduct of audits and/or assessments Notraining/qualifications to perform the audits Audit reports categorized identified points of interest as Findings. Many of these do not fit the industry accepted definition(s) of a finding.

AFI-03: Neither NBSR-0002-DOC-02, QA Program for Modifications to the NBSR Reactor, or AR procedures, identified as being needed but not yet developed, address all of the requirements of ANSI/ANS 15.8.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 3 of 33 ANAs ANA-01: The appointment letters for SEC members do not provide any explanation or justification on how the new members meet the charter and TS requirements.

ANA-02: There is no procedure or charter formalizing the SAC process to include a record of selection requirements and Director approval.

ANA-03: The Procedure Audit Subcommittee (PAS) requires management attention to ensure procedure audits, as specified in the PAS charter, are performed. No audits had been completed since the subcommittee charter was approved in September 2022. Several are now in progress.

ANA-04: There is no training or qualifications specified for individuals performing the TS required audits.

ANA-05: There is no training or indoctrination for new members of the SEC or subcommittees regarding their roles and responsibilities as a SEC member.

Positive Observations:

The new Chief of Reactor Engineering displayed an in-depth knowledge of the 50.59 process. He is on the National Institute of Engineering (NIE) committee that is writing a standard on the 50.59 process as it is applies to research reactors.

The audits performed by the SEC Audits subcommittee clearly documented that the frequency requirements specified in the TS had been met. The audits were noted to be comprehensive and in-depth.

This report also contains recommendations, particularly in regard to the ANSI/ANS 15.8 QA program that is being developed. Specifying audits and assessments performed by trained/qualified personnel in accordance with defined processes will assist the NCNR project in identifying potential issues and taking corrective action prior to more severe consequences. Appendix A presents a crosswalk of what ANSI/ANS 15.8 specifies and existing NCNR processes and procedures and identifies the deltas that are not being met at NCNR.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 4 of 33 TABLE OF CONTENTS 1.0 Background/Introduction................................................................................................................ 5 2.0 General Assessment Methodology................................................................................................. 5 2.1 The assessment followed the guidelines below as defined in the Project Execution Plan:............ 5 3.0 Recommendations and Observations............................................................................................. 6 3.1 The Assessment Report l identified recommendations and observations using the definitions below:.............................................................................................................................................. 6 3.2 Overall Effectiveness....................................................................................................................... 6 4.0 Objective 1...................................................................................................................................... 6 4.1 Determine whether the NCNR staff processes and procedures are compliant with Technical Specification requirements for the SEC and are effectively implemented..................................... 6 4.2 SEC Subcommittees....................................................................................................................... 12 4.3 Other SEC Issues/Observations..................................................................................................... 15 5.0 Objective 2.................................................................................................................................... 17 5.1 Determine a plan for the benchmarking of other similar reactor designs that have been identified as having a laudable oversight program as specified in the approved SOW for this task.

17 6.0 Objective 3.................................................................................................................................... 18 6.1 Determine whether the NIST programs or procedures implement the guidance in ANSI/ANS -

15.8, Quality Assurance Program Requirements for Research Reactors, as specified in the approved SOW for this task.......................................................................................................... 18 Appendix A, ASME/ANS-15.8-1995 (R2013), QA Program Requirements for Research Reactors Checklist Appendix B, Personnel Interviewed Appendix C, Documents Reviewed Appendix D, Assessment Team Biographies

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 5 of 33

1.0 BACKGROUND

/INTRODUCTION On August 1, 2022, a Confirmatory Order Modifying License No. TR-5 was issued by the U.S. Nuclear Regulatory Commission (NRC). The issued Confirmatory Order documentsthe NIST Center for Neutron Research (NCNR) completed and planned actions, as well as the commitments made by the U.S.

Department of Commerces National Institute of Standards and Technology (NIST) to ensure the safe operation of the research reactor in the NCNR, which experienced an unplanned shutdown on Feb. 3, 2021.

NIST has committed to conducting third-party, independent assessment of 6 different program areas at the NCNR as specified in section 2.b, Nuclear Program Assessment(s), of the Confirmatory Order. One of the identified areas is item 5, Safety Assessment Committee (SAC). Another is item 6, Safety Evaluation Committee (SEC) independence and effective oversight.

A Program Execution Plan was developed and approved to define the objectives of the third-party independent assessment. This report documents the results of the assessment.

2.0 GENERAL ASSESSMENT METHODOLOGY 2.1 The assessment followed the guidelines below as defined in the Project Execution Plan:

The assessment evaluated the NCNR programs for compliance with the requirements of the Technical Specifications.

The assessment evaluated the various NCNR programs for compliance with ANSI/ANS-15.8, 2007, Quality Assurance Program Requirements for Research Reactors.

The assessment reviewed SEC and SAC meeting minutes, interviewed a sampling of committee/subcommittee members, and observed meetings.

The assessment evaluated SEC/SAC functions, scope of reviews, required expertise, required reports to identify weaknesses, and areas for improvement.

The assessment evaluated the SEC structure (subcommittees), organization (membership, expertise), processes and functions (activities) of the NCNR SEC as they apply to effective safety oversight of NBSR operation.

The assessment defines a benchmarking study of two analogous internal and external oversight committees at nuclear reactor facilities of similar size and public safety risk exposure. This will be coordinated with the assessment team addressing benchmarking.

The assessment assessed the SAC and SEC effectiveness, including corrective actions taken in these areas and in identifying issues that require management attention to avoid future safety issues.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 6 of 33 3.0 RECOMMENDATIONS AND OBSERVATIONS 3.1 The Assessment Report l identified recommendations and observations using the definitions below:

- Area of Strength - This term is used to characterize demonstrated performance in a program or process element within an area being assessed that is exceptionally effective in achieving its desired results. An Area of Strength is a program, process, or activity of such a high quality that it could serve as an example for other similar elements.

- Positive Observation - This term refers to a noteworthy observation that does not rise to the level where it would be considered as a strength.

- Area in Need of Attention (ANAs) - This term is used to identify a performance, program, or process element that is sufficient to meet its basic intent; however, management attention is required to achieve full effectiveness and consistency. ANAs are not normally identified or addressed in action plans submitted to the NRC but are brought to management attention for consideration and entry into the NCNR Corrective Action Program or alternative tracking process.

- Area For Improvement (AFIs) - This term is used to characterize an identified performance, program, or process element that requires improvement to obtain the desired results in a consistent and effective manner. AFIs identified in the Assessment Report will be addressed by the Action Plan Submitted to the NRC.

3.2 Overall Effectiveness The Project Execution Plan specified that the Assessment Report identify the overall effectiveness of each area evaluated using the following definitions:

NOTE: Contractor may propose alternative definitions with submission of the Assessment Plan, subject to approval by the governments technical representative.

A statement outlining the Objective Current Status - a more specific status of the objective referencing the maturity of programs, actions taken, and completeness of actions planned.

The Gap-a more specific description of the gaps in guidance, conduct and/or effectiveness of the objective elements.

Vulnerability - a characterization of the vulnerabilities that exist in the subject objective, based on the assessment in the areas of performance, sustainability, and inspection activities.

The Project Execution Plan identified three objectives.

4.0 OBJECTIVE 1 4.1 Determine whether the NCNR staff processes and procedures are compliant with Technical Specification requirements for the SEC and are effectively implemented.

The roles and responsibilities of the SEC are delineated in Technical Specifications (TS) Section 6.2.1, Composition and Qualification, Section 6.2.2; Safety Evaluation Committee Charter and Rules;

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 7 of 33 Section 6.2.3, SEC Review Function; and Section 6.2.4, SEC Audit Function. Each of these TS sections were reviewed by the assessment team. Results of that review are delineated below.

TS Subsection 6.2.1, Composition and Qualifications This section defines the membership as no less than four members selected on their ability to provide independent judgment and to collectively provide a broad spectrum of expertise in reactor technology and operation.

Review Results:

NBSR-0001-CH-08, NCNR Reactor Evaluation Committee (SEC) Charter, states that the NCNR Director shall appoint all members and alternates to the SEC. The assessment team reviewed the appointment memos signed by the NCNR Director, and verified they existed. The SEC charter and TS 6.2.1 state that members and alternates are selected on their ability to provide independent judgment and to collectively provide a broad spectrum of expertise The assessment team noted that the appointment letters do not provide any explanation or justification on how the new members meet the charter and TS requirements. The assessment team recommends that NCNR evaluate whether the appointment letters should be revised to include justifications for existing SEC members.

One ANA was identified.

ANA-01: The appointment letters for SEC members do not provide any explanation or justification on how the new members meet the charter and TS requirements. The assessment team verified that the designated positions were included in the membership.

The assessment team interviewed 19 SEC members, Subcommittee chairs, and subcommittee members. No issues with level of independence were identified.

TS Subsection 6.2.2, Safety Evaluation Committee Charter and Rules This section specifies that the SEC shall conduct its review function in accordance with a written charter and the charter shall be consistent with ANSI/ANS-15-1-2007. The TS specifies that the charter includes:

1. Meeting frequency

2. Voting rules

3. Quorums

4. Methods of submission and content of presentation to the committee

5. Use of subcommittees

6. Review, approval and dissemination of minutes Review Results:

NBSR-0001-CH-08, NCNR Safety Evaluation Committee (SEC) Charter, was reviewed and the assessment team verified that it included the 6 attributes listed in the TS. The approved TS were reviewed against ANSI/ANS 15.1-2007. No deficiencies were identified.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 8 of 33 NBSR-0004-CH-05, Charter for the NCNR Audit Subcommittee of the NCNR Safety Evaluation Committee, was verified to require the audits defined in the TS. (See TS Subsection 6.4 for further details)

The assessment team observed two SEC meetings that where conducted utilizing Teams. The assessment teams overall assessment of the meetings was that they were good status meetings but did not provide meaningful oversight. The assessment team noted that the meetings met all administrative requirements and contained status reports on a variety of topics. It was noted that the meetings were not interactive with few questions for the presenters. Review of other SEC meeting minutes indicated that more discussion was held during other meetings.

During interviews, several members commented that they felt that the lack of interaction was due to the meetings being held virtually on Teams. One interviewee stated that pre-pandemic, the SEC meetings were held in-person and were more interactive. This was later discussed with the SEC chair who agreed that he would consider having a live meeting now that pandemic restrictions have been relaxed.

TS Subsection 6.2.3, SEC Review Function This section defines the areas/topics that the SEC or designated subcommittees are responsible for.

They include:

1. Review proposed tests or experiments receive adequate reviews in accordance with 10 CFR 50.59.

2. Review the circumstances of events specified in section 6.7.2, Special Reports, of the TS.

3. Review proposed changes to the NBSR facility equipment or procedures when changes have safety significance. Provide recommendations to the NCNR Director.

4. On a biennial basis review its charter and recommend to the NCNR Director any changes necessary to ensure continued effectiveness of the charter.

Review Results:

Technical Specifications (TS) sections 6.2.3 (1) and (3) define the SEC responsibilities in regard to 10CFR 50.59. The assessment team reviewed a sampling of 50.59 evaluations as a part of the review of procedures and Engineering Change Notices (ECNs).

Safety Analysis Report (SAR) Chapter 12, Conduct of Operations, contains a one-line reference to AR 1.0, Conduct of Operations. The change to this chapter of the SAR was approved via ECN 885 processed in October 2018. AR 1.0 was subsequently revised on 5/31/2022. The 50.59 evaluation screen for the procedure change stated NO and did not provide any justification for this determination. Additionally, and more significantly, the revision to AR 1.0 constituted a change to the FSAR; a full evaluation should have been performed.

ECN 885, Revision to SAR Chapter 12 (Conduct of Operations), revised Chapter 12 to remove all verbiage in Chapter 12 and replaced with links to AR 1.0, Conduct of Operations and a link to the TS.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 9 of 33 Previously, Chapter 12 of the SAR contained the following section:

12.9 Quality Assurance An established program following the guidelines of ANS 15.8, Quality Assurance Program Requirements for Research Reactors is in the administrative rules and procedures for the NBSR.

The Safety Evaluation and Conclusion of ECN 885 stated; The change to Chapter 12 of the SAR is an administrative action. During the revision of the chapter and upon completion of the revision of the chapter, the changes were confirmed to have no effect upon the license base and requirements of the NBSR. All the questions in the 10CFR 50.59 Pre-Screening were marked No. The pre-screen notes state: A yes or no response must be fully supported by information presented in the Description Summary. The Summary states: All content of the chapter can be found elsewhere. Revision will replace that content with two reference and two hyperlinks.

This statement is not correct. Neither AR 1.0 nor the Technical Specifications make any reference to ANSI/ANS 15.8. Therefore, the 50.59 evaluation did not consider the removal of the commitment to ANSI/ANS 15.8 from the SAR. NRC approval of the deletion of ANSI/ANS 15.8 was not requested or obtained.

Review of NRC Inspection Reports also identified several cases where justifications on completed forms were deemed inadequate. Two follow-up inspections that include references to 50.59 identified in the schedule of NRC Supplemental Inspections. IP69008 and IP 92701 are scheduled to be performed in August 2023.

The new Chief of Reactor Engineering was interviewed with an emphasis on the 50.59 process. He displayed an in-depth knowledge of the process. He is on the Nuclear Energy Institute (NEI) committee that is writing a standard on the 50.59 process that is applicable to research reactors. (Positive Observation)

Interviews also indicated that there is no procedure on performing 50.59 evaluations beyond specifying that one be performed. Also, there is no training or qualifications required to perform the evaluation.

The TS and SEC charter only directs that the SEC chair review 50.59s that have safety significance or involve an amendment to the facility license. There is no oversight review to assess whether screens are being properly utilized. Discussions with the SEC chair indicated that a 50.59 subcommittee would be pursued.

The assessment team identified one Area for Improvement.

AFI -01: The assessment team identified an unauthorized change to the Safety Analysis Report (SAR),

weaknesses in justifications in evaluations. A procedure defining how to perform an evaluation has not been developed, and lack of training or qualifications required to perform evaluations.

The 2021 SAC report contained an Observation: The Safety Evaluation Committee (SEC) is not serving its intended function. The SAC report recommended that the SEC charter should be reviewed and revised to include membership outside the NCNR reporting chain. The SAC opined that the

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 10 of 33 independence of the SEC could be compromised by the reporting chain of the SEC members. The SEC charter was revised to add members outside the NCNR reporting chain. However, interviews with newly appointed members indicated that they had not received any training or indoctrination on the roles and responsibilities of the SEC. The 2022 SAC report commended the project for the changes to the charter and membership additions.

An interview with the SEC chair indicated that SEC identified actions had not been entered into the Corrective Action Program (CAP) but current efforts were in progress to enter them into CAP. The Deputy Director also stated that SAC issues and recommendations have been entered into CAP. The Problem Identification and Resolution assessment, Task 3, already identified that issues are not being entered into the CAP so the AFI is not repeated here.

Recommendation: Current efforts to enter issues and recommendations identified by the SEC and SAC into the CAP program for disposition, tracking and trending should be continued.

TS Subsection 6.2.4, SEC Audit Functions This section defines the responsibility of the SEC or designated subcommittee for the following audits:

1. Facility operations at a frequency of once per calendar year not to exceed fifteen months.

2. Results of actions taken to correct deficiencies that affect reactor safety at a frequency of once per calendar year not to exceed fifteen months.

3. Requalification program at a frequency of every other calendar year not to exceed thirty months.

4. NBSR Emergency Plan at a frequency of every other calendar year not to exceed thirty months.

Review Results:

The assessment team reviewed the reports of the past 4 audits performed by the SEC Audit Subcommittee. The audits clearly documented that the frequency requirements specified in the TS had been met. The audits were noted to be comprehensive and in-depth. (Positive Observation) Each of the audits contained findings and recommendations. An interview with the subcommittee chairman indicated he had an in-depth knowledge of the topics of the audit. The findings of the audits were not entered into the CAP. However, each of the audit reports contains a table with the status of open findings and an explanation for the findings that remain open. The audit for 2023 isnt scheduled until September so no report for 2023 was available for review.

The assessment team noted that there are no other audits performed by this subcommittee or by other organizations, See discussion of ANSI/ANS 15.8 below. Also, there is no training or qualifications required to perform these audits. NBSR-0001-CH-08, NCNR Reactor Safety Evaluation Committee (SEC)

Charter allows the SEC Chair to review and approve changes to specified procedures and report the reviews to the full SEC at the next meeting. The assessment team verified this was occurring.

One minor inconsistency was noted. Appendix B of the SEC Charter lists the Training and Requalification Subcommittee as being responsible for the Training and Qualification Program Annual Audit. This audit is performed as part of the Audit Subcommittee annual audit.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 11 of 33 The assessment team noted one Area for Improvement and one Area in Need of Attention:

AFI- 02: A process or procedure to define the audit process has not been developed. ANSI/ANS 15.8, Quality Assurance Program for Research Reactors, section 2.18 states Assessments shall be performed in accordance with written procedures or checklists. Examples:

No procedure that defines the conduct of audits and/or assessments Notraining/qualifications to perform the audits Audit reports categorized identified points of interest as Findings. Many of these do not fit the industry accepted definition(s) of a finding.

Overall, the assessment team concludes that based on the quality of the audits, the SEC Audit subcommittee is effectively meeting the TS requirements.

TS Subsection 6.2.5, Safety Assessment Committee The NCNR Safety Assessment Committee (SAC) was established to provide an independent review or audit of NCNR reactor operations. This audit is to ensure safety reviews and reactor operations are being performed in accordance with regulatory requirements and public safety is being maintained. The Technical Specifications specify that the SAC shall be composed of at least three senior technical personnel who collectively provide a broad spectrum of expertise for reactor technology. It is appointed by the Director, NCNR. At least two members shall pass on any report or recommendations of the committee. The committee shall review or audit the reactor operations and the performance of the SEC.

The SAC shall report in writing to the Director NIST.

Review Results:

The assessment team reviewed SAC qualifications for the current SAC members. There are currently four members; each meeting the expertise requirements specified in TS 6.2.5. TS 6.2.5 requires the SAC committee members to be appointed by the Director, NIST Center for Neutron Research. The current practice is for Chief, Reactor Operations and Engineering (CROE) to meet with the Director to determine who should be appointed to the SAC. Following selection and approval of proposed team members, the CROE communicates and coordinates with the committee. No formal appointment letters are generated. The assessment team reviewed emails which confirmed the team membership that included the Director, NCNR as an addressee. Annual reports were reviewed and found to meet the SAC performance requirements. The SAC meets annually and reviews/audits the NCNR reactor operations and the performance of the SEC.

The assessment team reviewed SAC annual reports. The reports were thorough and appear to have identified several of the issues later captured in the root cause analysis and confirmatory order following the February 2021 event. For example, the importance of a CAP program (2018, 2019 SAC reports) and concerns with safety culture were identified in multiple SAC reports. The reports included recommendations that were tracked on a spreadsheet but were not entered into a formal CAP. No cause analysis was performed and no trending.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 12 of 33 Interviews were conducted with current available SAC members. Each SAC member is appointed to a one-year tenure. Typically, one or two members are invited to remain a member for an additional year for continuity. There is no charter or procedure for SAC members. They use the TS as their direction for conduct of the annual review. During the 2022 NCNR SAC entrance meeting, it was reported that the NIST Office of General Council had determined that, since the SAC is not staffed completely with full-time federal employees, it may not make recommendations or render advice. With that guidance, the SAC can only present findings (not a defined term at NCNR) and comments resulting from any audit/review that it conducts. This has led to a situation where the Deputy Director and Chief, Reactor Operations and Engineer has to determine which comments (issues) to enter into the CAP program.

The SAC members were interviewed to obtain input on current methodology and areas for improvement. The members were familiar with the CATS spreadsheet currently used by NCNR to track SAC/SEC actions. However, during interviews the members stated that they reviewed previous SAC reports and attempted to determine the status of previous recommendations. Each member listed implementation of the new CAP system as a high priority for NCNR. Interview with the Deputy Director indicated that he was currently in the process of rolling issues in CATS into the CAP system (See TS 6.2.3 for ANA).

There is no procedure or charter formalizing the SAC process to include a record of selection requirements and Director approval.

Recommendations:

1. Consider extending the tenure of each member to allow continuity of oversight and rotating members changeout on an alternating basis.

2. Set up a process to provide informational updates/new information (SEC meetings/actions, NRC Information Notices, CAP status, etc.) to ensure SAC members remain current on both NCNR issues and performance as well as current industry issues.

==

Conclusion:==

The SAC is meeting Technical Specifications 6.5.2 requirements.

4.2 SEC Subcommittees As allowed by the Technical Specifications the NBSR partially fulfills its responsibilities through the use of subcommittees. The existing subcommittees Include:

Procedure Audit Subcommittee Audit Subcommittee Beam Experiment Subcommittee Irradiation Subcommittee Corrective Action Tracking Subcommittee Incident Response Review Subcommittee Post -Criticality Oversight Subcommittee Training and Qualification Subcommittee Corrective Action Tracking Subcommittee

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 13 of 33 The assessment team reviewed the charters and reports of the various subcommittees. The Audit subcommittee is discussed in TS Section 6.4 above.

Procedure Audit Subcommittee:

The Confirmatory Order listed four notices of violation regarding procedures that would have been issued. However, as a result of a successful alternative dispute resolution (ADR) mediation session the Confirmatory Order was added to the NCNR license. A new SEC subcommittee, the Procedures Audit Subcommittee (PAS), was established to improve the quality of procedures. The charter, NBSR-0011-CH-01, Charter for the NCNR Procedure Audit Subcommittee (PAS) of the NCNR Safety Evaluation Committee, was approved in September of 2022. The charter states: The primary focus of the PAS audit process is to evaluate: 1) procedures for clarity, adequate level of detail, and compliance with NBSR Administrative Rule (AR) 5.0 Procedure Use and Adherence and AR 5.1 Procedure Writers Guide (current versions, and references contained therein); 2) procedural adherence through observation of work; and 3) processes associated with modification of procedures. The charter specified steps to audit specified procedures including field observation of simulations of performing the procedures.

Interviews indicated that no audits or observations had been completed to date. The SEC recognized this issue and was in the process of naming a new subcommittee chair with sufficient available time to implement the necessary audits. Several audits are now in progress.

The assessment team identified one Area Needing Attention.

ANA-03: The Procedure Audit Subcommittee (PAS) requires management attention to ensure procedure audits, as specified in the PAS charter, are performed. No audits have been performed since the subcommittee charter was approved in September 2022.

Audits Subcommittee:

The NBSR Technical Specifications section 6.2.4, SEC Audit Function, specifies four audits to be performed at specified frequencies. These audits are performed by the SEC Audit Subcommittee (see TS Subsection 6.2.4, SEC Audit Functions for details). No other audits are defined in the Technical Specifications.

Beam Experiment Subcommittee:

NBSR-0003-CH-05, Charter for the NBSR Beam Experiment Subcommittee of the NCNR Safety Evaluation Committee, was dated September 2022. Responsibilities of the charter include:

1. Review and evaluate all new beam experiment proposals and changes to existing approved beam experiment proposals.

2. Communicate recommendations to the SEC Chairman.

3. Refer to the full SEC for action, all proposed activities that in the subcommittee's judgment may conflict with the NBSR license or technical specifications or may fall outside that allowed by 10 CFR 50.59, or where the decision of the subcommittee is not unanimous.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 14 of 33

4. Maintain records of all policies, procedures, and copies of approved beam experiment proposals.

5. Maintain records documenting evaluation and review of beam experiment proposals pursuant to the requirements detailed in the Technical Specifications of the NBSR.

6. Submit a written annual report to the SEC Chairman or as requested.

7. Periodically re-evaluate approved, active beam experiments for potential inconsistencies with the approved envelope of operation.

Interviews with the subcommittee chairman indicated that there was no formal training for the positions of subcommittee member or chairman, although there was informal training mainly consisting in discussions with an outgoing member or the subcommittee chairman. Currently there is an indoctrination meeting with the SEC chairman and reading the Writers Guide. It was noted that the requirements for working on a committee are on top of an existing position and that it is time consuming to produce documents required, such as an audit report. Appointment letters were noted to not include qualifications.

The assessment team verified that reports have been submitted to the SEC. Based on review of these reports and conversation with the Subcommittee Chairman, it is determined that the Charter responsibilities are being met.

Irradiation Subcommittee:

NBSR-0002-CH-05, Charter for the NBSR Irradiation Subcommittee of the NBSR Safety Evaluation Committee was originally formed in November 1970, and currently is finalizing revision 3 of its Charter.

Its responsibilities include:

1. Review (Diane see Toms comment) and evaluate all new irradiation requests/proposals and changes to existing approved irradiation requests/proposals.

2. Recommend approval of all new irradiation requests/proposals in accordance with the SEC Charter and Irradiation Request Guide Rev 0.

3. Refer to the full SEC for action, any proposed activity that in the subcommittees judgment may conflict with the NBSR license or technical specifications or may fall outside that allowed by 10 CFR 50.59, or where the decision of the subcommittee is not unanimous.

4. Conduct tests and measurements, if needed, in support of evaluation of irradiation requests/proposals. At least two members of the sub-committee must know the details and agree to the test measurements.

5. Maintain records of all test irradiation data, material certifications, policies, procedures, and copies of all approved irradiation requests/proposals. Irradiation request and proposal records shall include initials or signatures of subcommittee members, and other reviewing officials, signifying review and approval.

6. Maintain records documenting evaluation and review of irradiation requests/proposals pursuant to paragraph (d)(1) of 10 CFR 50.59.

7. Prepare written report to the SEC annually and as requested.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 15 of 33

8. As necessary, delegate the activities of items 4, 5, and 6 to qualified personnel who may or may not be members of the subcommittee.

9. Take action on specific issues as delegated by the SEC.

Interviews with the subcommittee members indicated that there is no training for the position beyond speaking with the current chairman. Issues will be brought up to the SEC when indicated.

It noted that Irradiation Subcommittee Annual reports are given during the spring SEC meetings. No irradiations have been done since the shutdown.

Post-Criticality Oversight Subcommittee:

NBSR-0014-CH-01, Charter for the NCNR Post-Criticality Oversight Subcommittee (PCOS) is in draft form.

This committee was previously the Reactor Restart Readiness Subcommittee. The draft charter of the PCOS indicates its role is to provide advice to the NCNR Director concerning plans to operate the NBSR at power levels beyond 1 MWth based on operational readiness of both the NBS Reactor and the reactor-operations staff.

Interviews with the subcommittee chairman indicated that there was no formal training for the position of subcommittee chairman or members. Subcommittee meets weekly. A subcommittee report was made during the latest SEC meeting. There was one issue that was brought up to the SEC, although not entered into the CAP program. A memo was sent from the subcommittee to the SEC recommending a step increase to the reactor power. The memo was well received, and the SEC gave unanimous consent to proceed.

When asked what improvements could be made, the chairman suggested that some audit practices from the IRSC (Ionizing Radiation Safety Committee) could be adopted. Their method of annual assessment procedure was offered as an example.

4.3 Other SEC Issues/Observations Training Two common themes regarding training were identified during interviews of new and existing SEC members.

1.0 Training or qualifications are not specified for individuals performing the TS required audits or other assessments directed by the SEC.

ANA-04: ANA-04: There is no training or qualifications specified for individuals performing the TS required audits.

2.0 Indoctrination or training was not provided to SEC members upon initial appointment to the SEC or SEC subcommittee. Members were unsure of what were their roles and responsibilities in supporting the oversight function of the SEC.

ANA-05: There is no training or indoctrination for new members of the SEC or subcommittees regarding their roles and responsibilities as a SEC member.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 16 of 33 SEC Chair The assessment team observed that the SEC chairman has numerous responsibilities. The team recommends that NCNRconsider making this a full-time position reporting directly to the Director, NCNR. This position could assist in coordinating the SAC functions.

The section below, although not specifically related to an Objective of this report, is included as areas the project could address to improve the effectiveness of the SEC. INPO 12-103, Performance Objectives and Criteria, OR.5, Independent Oversight, provides recommendations for objectives that could be utilized while IEN 2023-03 provides further topics for potential audits or assessments. The SEC could then require the results of these audits or assessments be presented to the SEC, now in an oversight role.

NRC Information Notice (IN), 2023-03, Recent Human Performance Issues at Nonpower Production and Utilization Facilities.

IN 2023-03 was issued June 13, 2023, and describes numerous test reactors events that have occurred and have impacted the test reactor community. Event number 2 was a description of the NIST/NCNR fuel melt event.

The collective NRC identified issues described on page 6, Discussion section include:

Failure of licensees to provide adequate procedures.

o Failure of operators to properly follow those procedures.

o Underestimated or not considered risk.

o Events attributed to weaknesses in nuclear safety culture.

o Operators take actions when the consequences of those actions are uncertain or acting outside the scope of the relevant procedures.

o Human performance played a key role in all of the referenced events.

o Operating experience shows that errors often occur before, during, and immediately following evolutions that are less familiar to the operations staff because they areperformed infrequently.

o Some events show that individuals became complacent and did not recognize and planfor the possibility of mistakes, latent issues, and inherent risk.

o The events related to surveillance, such as calibrations and inspections, could have been prevented by fostering a positive nuclear safety culture.

In a Lessons Learned program, the process would ensure that each NRC issue identified above would be evaluated to determine if corrective actions were required to ensure a similar event could not occur. If no action is required (recent actions have already been taken) then the evaluation of no action required is documented.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 17 of 33 Overall Objective Results:

Objective - Determine whether the NCNR staff processes and procedures are compliant with Technical Specification requirements for the SEC and are effectively implemented.

Current Status-The NCNR processes and procedures meet the Technical Specification requirements. AFIs, ANAs and recommendations identify areas that may improve the effectiveness of the SEC.

The Gap - the assessment team identified areas that require management attention including:

o 10 CFR 50.59 inadequacies.

o Lack of an approved/implemented QA program.

o Procedure Audit subcommittee implementation Vulnerability -

o The 50.59 process is required to determine whether the project is required to obtain NRC approval prior to implementation of a change. Program implementation errors could lead to NRC violations.

o Not having an approved/implemented QA program does not meet regulatory requirements.

o Procedures were listed as 4 of the 7 violations that would have been issued if the resolution process had not resulted in the Confirmatory Order. Continued problems with procedures could lead to further NRC violations.

5.0 OBJECTIVE 2 5.1 Determine a plan for the benchmarking of other similar reactor designs that have been identified as having a laudable oversight program as specified in the approved SOW for this task.

Review Results:

The assessment team reviewed the technical specifications for oversight and audit functions for all non-power reactors. Most contained requirements very similar to NCNR. In interviews with current and past SAC members, the team requested the members to identify reactors/programs for NCNR to benchmark. The University of Missouri (MURR) was identified most frequently. Additionally, the HIFR facility in Oak Ridge was recommended. The assessment team recommends that NCNR ask INPO for their recommendation for a lauded independent oversight program in one of their top ranked members for benchmarking once NCNR becomes a member.

==

Conclusion:==

Overall Objective Results:

Objective - Determine a plan for the benchmarking of other similar reactor designs that have been identified as having a laudable oversight program as specified in the approved SOW for this task.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 18 of 33 Current Status - The current process at NCNR is consistent with the majority of oversight programs atsimilar research and test reactors. The assessment identified recommendations to perform benchmarking at MIT, University of Missouri, and HIFR as having good processes.

Additionally, the assessment team recommends once membership in INPO is attained, that they be requested to identify a couple of high performing plants with good oversight programs.

The Gap - The current SAC/SEC processes at NCNR have been strengthened since the fuel melt event. However, actions identified in SEC reports have not been entered into the CAP system.

Benchmarking the SAC/SEC processes and how they are entered into corrective action systems at high performing plants should identify improvements.

Vulnerability - Some of the issues identified in the root cause analysis had been previously identified in oversight reports without resolution.

6.0 OBJECTIVE 3 6.1 Determine whether the NIST programs or procedures implement the guidance in ANSI/ANS -15.8, Quality Assurance Program Requirements for Research Reactors, as specified in the approved SOW for this task.

The assessment team performed a gap analysis comparing ANSI/ANS-15.8 to existing NBSR processes and/or procedures. The analysis is Appendix A to this report.

The assessment team notes that the lack of a QA program was identified an AFI in the Nuclear Safety Culture assessment report.

Notes on the objective:

1. The NRC issued Revision 0 of Regulatory Guide (RG) 2.5, QA Program Requirements for Research and Test Reactors, in October 1977. The RG endorsed ANSI/ANS-15.8-1976. The RG noted that ANSI/ANS-15.8-1995 provide an acceptable method or complying with the program requirements of 10 CFR 50.34.

2. ANSI/ANS-15.1, Development of Technical Specifications for Research Reactors, references, in itsBasic Requirements and Practices forward, a number of guidance documents in the following supplementary American National Standards developed for research reactors and lists ANSI/ANS-15.8. So, the assessment team agrees that ANSI/ANS-15.8 is the standard to be used for the NCNRQA Program document (referred to as QA Manual).

3. NBSR-0002-DOC-02, QA Program for Modifications to the NBSR Reactor, was approved 8/28/2007.

The PI&R assessment identified that this procedure was active but had not been implemented because it was written to 10 CFR 50, Appendix B and ASME NQA-1-2004, Basic Requirements and Practices.

4. The analysis compared the ANSI/ANS-15.1 requirements to the processes specified in NBSR-0002-DOC-02 and existing engineering and operations procedures

5. This review did not review the implementation documents. The analysis only notes if one exists that could implement a given ANSI/ANS-15.1 requirement.

6.. The current version of the QA standard is ANSI/ANS-15.8-1995 (R2018), although R2013 was used for the comparison. These versions would either be the same or have minor differences.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 19 of 33

7. Many of the procedures reviewed were in draft form or simply identified as needed.

8. It was unclear to the team how many procedures fit together in satisfying ANSI/ANS-15.8. There are most likely documents not noted in the table which satisfy some of the gaps.

==

Conclusions:==

1. Neither NBSR-0002-DOC-02 or AR procedures, identified as being needed, address all of the requirements of ANSI/ANS 15.8.

2. There were many documents which have been identified as being needed that are either in draft form or are indicated as waiting to be written.

The gap analysis in Appendix A shows that there are a number of requirements for 15.8 that are not being met.

AFI-03 Neither NBSR-0002-DOC-02, QA Program for Modifications to the NBSR Reactor, or AR procedures, identified as being needed but not yet developed, address all of the requirements of ANSI/ANS 15.8.

Overall Objective Results Objective - Determine whether the NIST programs or procedures implement the guidance in ANSI/ANS -15.8, Quality Assurance Program Requirements for Research Reactors, as specified in the approved SOW for this task.

Current Status - The NCNR project currently does not have a QA program document that meets ANSI/ANS -15.8. Some of the requirements of 15.8 are implemented by existing engineering procedures (see Appendix A). NCNR has identified the requirement to develop a QA program that meets ANSI/ANS -15.8. No NRC approved alternate method of meeting QA program requirements were identified.

The Gap - The gap analysis, Appendix A, shows that there are QA program requirements that are not being met.

Vulnerability-Not meeting regulatory requirements could lead to NRC violations.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 20 of 33 Appendix A ASME/ANS-15.8-1995 (R2013), QA Program Requirements for Research Reactors Checklist Following facility construction and commissioning, the focus of the quality program shifts to establishing those controls that ensure proper and reliable facility operation. All of the program provisions established during the design and construction phase remain in place but will change in level of implementationappropriate to support facility operations. Each portion of Section 2 would be implemented only as necessary. The operating phase license or permit imposes additional requirements related to the conduct of operations. These additional program requirements are defined in Section 3 of this standard.

(From 1.2 Application)

ASME/ANS-15.8-1995 (R2013)

Requirements NBSR-0002-DOC-02 Section NCNR Implementing Documents Comments 2 Design, Construction, and Modifications-of a research reactor facility 2.1 Organization The organizational structure and assignment of responsibilities shall be defined and documented such that: (a) quality is achieved and maintained by those who have been assigned responsibility for performing work; and (b) quality achievementis verified by persons not directly performing the work.

2.2, Organization 2.2.1, QA Program Applicability - Resources, (a) and (b)

Could not locate associated document.

Persons responsible for ensuring that appropriate controls have been established, and for verifying that activities have been correctly performed, need sufficient authority, access to work areas, and freedom to: (a) identify problems ;(b) initiate, recommend, or provide corrective action; and (c) ensure corrective action implementation.

2.2.1, QA Program Applicability - Resources, (c)

Could not locate associated document.

2.2 Quality Assurance Program A documented quality assurance program shall be established in accordance with the requirements of this standard.

1.0 Introduction States facility-meets 10CFR50, App. B and NQA-1, NOT ANSI 15.8 The program shall identify the items and activities to which it applies and the extent of program application for each item and activity.

1.1 Application Applies to NBSR Reactor modifications, calibration, or repair related to reactor safety identified in Limiting Cond. for Operations Sect 3.0 of NBSR Tech Specs.

The program shall provide for the appropriate and necessary indoctrination and training of personnel.

2.1.1 Quality Prog Responsibility AR 2 series covers reactor operator training.

Need to reference training documents for facility and equipment modifications.

2.3 Design Control The responsible design organization shall prescribe, develop, document, and preserve the design of the structures, systems, and components of the research reactor facility.

2.4.1 Configuration Management (in Design Control section) references Eng. Change Notice (ECN NBSR-0003-DOC-06, Guidelines for Completing Engineering Changes 2.3.1, Design Requirements Applicable design inputs such as design bases, performance requirements, regulatory 2.4.2.2, Design Requirements Could not locate associated document.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 21 of 33 ASME/ANS-15.8-1995 (R2013)

Requirements NBSR-0002-DOC-02 Section NCNR Implementing Documents Comments requirements, codes, and standards, shall be identified and documented.

2.3.2, Design Process Design interfaces shall be identified and controlled, and the design efforts shall be coordinated among the participating organizations.

2.4.2.3, Design Process NCNR-1000-DOC-00, Engineering Change Control for NBSR Reactor Operations and Engineering, NCNR NCNR document stated it defines a top level process. Need to note the associated procedures.

The applicability of standardized or previously proven designs, with respect meeting pertinent design inputs, shall be verified.

Did not find any formal documentation for this requirement.

Deviations from the established and documented design inputs, including the reasons for the changes, shall be documented and controlled.

2.4.1 Configuration Management AR 6.1, NBSR Configuration Management Plan Identified as needed, but document is not written yet.

The final design shall: (a) be relatable to the design input by documentation in sufficient detail to permit design traceability and verification; and (b) identify assemblies and/or components that are part of the item being designed. When a computer design program is used to develop portions of the facility design or to analyze a design for acceptability, that program shall be fully documented, validated, and controlled to ensure the correctness of its output.

Did not find any formal documentation for this requirement.

When a design program must be developed, the program shall be controlled to assure that it is fully documented and validated. Where changes to previously valid computer programs are made, documented revalidation shall be required for the change. Verification of design-unique computer programs shall include appropriate benchmark testing.

Did not find any formal documentation for this requirement.

2.3.3, Design Verification Independent design reviews shall be used to verify the adequacy of designby one or more of the following: (a)the performance of design reviews, (b) the use ofalternate calculations, (c) the performance of qualification tests, or (d) comparison to similar proven systems.

Could not locate associated document.

The responsible design organization shall identify and document the particular design verification method or methods used.

Could not locate associated document.

Design verification shall be performed by competent individuals or groups other than those who performed the design, hut who may be from the same organization. In all cases the design verification shall be completed prior to reliance upon the component, system, structure, or computer program to perform its function in operations.

Could not locate associated document.

The need for or the use of qualification tests shall be defined in a formal test plan same organization. In all cases the design verification shall be completed prior to reliance uponthe component, system, structure, or computer program to perform its function in operations.

Did not find any formal documentation for this requirement.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 22 of 33 ASME/ANS-15.8-1995 (R2013)

Requirements NBSR-0002-DOC-02 Section NCNR Implementing Documents Comments The need for or the use of qualification tests shall be defined in a formal test plan that shall include appropriate acceptance criteria and shall demonstrate the adequacy of performance under conditions that simulate the most adverse design conditions. Test results shall be documented and evaluated by the responsible design organizationto assure that test requirements have been met.

Did not find any formal documentation for this requirement.

2.3.4, Design Documents and Records Design documents and records, which provide evidence that the design and design verification processes were performed, shall be collected, stored, and maintained for the life of the safety-related unit.

2.6.1, Design Documents and Records AR 6.3.1, NBSR Document Control and Records Management Appears to be met, but document is not written yet.

2.3.5, Commercial Grade Items The use of commercial grade equipment in safety-related applications shall be reviewed to assure that this equipment can adequately perform its intended function.

2.7.1, Commercial Grade Items NBSR-0003-DOC-06 Could not find instructions for CMTRs.

When a commercial grade item, prior to its installation, is modified or selected by special inspection and/or testing to requirements that are more restrictive than the supplier's published product description, the component part shall be represented as different from the commercialgrade item in a manner traceable to a documented definition of the difference.

Could not locate associated documents.

2.3.6, Change Control Modifications to facilitystructures, systems, components, or computer codes shall be based on a defined "as-exists" design.

AR 6.1.1 Engineering Change Management Appears to be met, but document is not written yet.

Changes to verified designs shall be documented, justified, and subject to design control measures commensurate with those applied to the original design.

2.4.1, Config. Management 2.4.2.1, Design Control NBSR-0003-DOC-06 NBSR document notes that ECN process addresses these requirements. The ECN process appears to apply to both Operations and Engineering.

These measures shall include assurance that the design analysesfor the structure, system, component, or computer code are still valid.

2.4.1, Config. Management NBSR-0003-DOC-06 Where a significant design change is necessary because of an incorrect design, the design process and verification procedure should be reviewed and modified as necessary.

2.4.1, Config. Management NBSR-0003-DOC-06 2.4 Procurement Document Control Procurement documents shall contain sufficient technical and quality requirements to ensure that the items or services satisfy the needs the purchaser.

2.5.1, Procurement Document Control AR 6.3.3, NBSR Procurement Control Identified as needed, but document is not written yet.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 23 of 33 ASME/ANS-15.8-1995 (R2013)

Requirements NBSR-0002-DOC-02 Section NCNR Implementing Documents Comments 2.5 Procedures, Instructions, and Drawings Activities affecting quality shall be performed in accordance with documented instructions, procedures, or drawings appropriate to the circumstances. These documents shall include or reference appropriate quantitative or qualitative acceptance criteria for determining that activities have been satisfactorily accomplished.

2.9.2, Procedures, Instructions and Dwgs.

AR 5.0, Procedure Use and Adherence Does not include reference to acceptance criteria.

2.6 Document Control The preparation, issue, and change of documents which specify requirements that affect quality, or prescribe activities affecting quality, shall be controlled to assure that correct documents are used.

2.6 Document Control NBSR-0003-DOC-06 Guidelines for Engineering Changes The ECN process appears to apply to both Operations and Engineering.

The document control system shall be documented and providefor: {a) identification of documents to be controlled and their specified distribution; {b) identification of assignment of responsibility for preparing, reviewing, approving, and issuing documents;and (c) review of documents for adequacy, completeness, and correctness prior to approval and issuance.

2.3.1 Document Control NBSR-0001-DOC-03, Document Control Plan Does not appear to apply to operations documents.

2.7 Control of Purchased Items and Services The procurement of items and services shall be controlled to ensure appropriate procurement planning, source evaluation and selection, evaluation of objective evidence of quality furnished by the supplier, source inspection, audit, and examine addition of items or services for acceptance upon delivery or completion.

2.7.2, Control of Purchased Items and Services Could not locate associated document.

2.7.1, Supplier Selection The selection of suppliers shall be based on evaluation of their capability to provide items or services in accordance with the requirements of the procurement documents.

2.7.3, Supplier Selection Could not locate associated document.

2.7.2, Work Control The purchaser shall establish measures to control the supplier's performance as appropriate.

2.7.3.1, Work Control NBSR-0015-DOC-00 Work Package Development 2.7.3, Verification Activities The supplier shall be responsible for the quality of his product and shall verify and provide evidence of that quality. The purchaser shall consider independently verifying the quality of a supplier's product.

2.7.3.2, Verification Activities AR 7.1, Conduct of System Review Teams Procedure does not fully implement requirement.

2.7.4, Item or Service Acceptance Thepurchaser shall establish a system to provideassurances that purchased items and services conform to procurement specifications.

2.7.3.3, Items or Service Acceptance Could not locate associated document.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 24 of 33 ASME/ANS-15.8-1995 (R2013)

Requirements NBSR-0002-DOC-02 Section NCNR Implementing Documents Comments 2.8 Identification and Control of Items When specified by codes, standards, or specifications that include specific identification or traceability requirements, the item identification and control process shall be capable of providing identification and traceability control.

2.8, Identification and Control of Items Could not locate associated document.

2.9 Control of Special Processes Special processes include any in which the results are highly dependent on the control of the process or the skill of the personnel. Special processes shall be controlled by instructions, procedures, drawings checklists, travelers, or other appropriate means.

2.9, Control of Special Processes Many documents concerning special processes already exist and are used.

Need to identify associated document.

Ensure that process control documents for special processes include or reference:

(1) procedure, personnel and equipment qualification requirements.

(2) conditions necessary to accomplish the process -proper equipment, controlled parameters of the process, specified environment, and calibration requirements.

(3) acceptance criteria.

2.9, Control of Special Processes Check AR 5.3, 5.4 Appears to be met, but documents not written yet.

2.10 Inspections Inspections to verify conformance of an item or activity to requirements shall be planned, documented, and performed.

2.10, Inspections Check AR 6.3.5, NBSR QA Assessments Identified as needed, but document is not written yet.

Each person who verifies conformance of work activities forpurposes of acceptance shall be qualified to perform the assigned inspection task.

2.10, Inspections Identified as needed, but document is not written yet.

The need for formal training shall be determined. Records of inspection personnel's qualification shall be established and maintained by the employer.

2.10, Inspections Identified as needed, but document is not written yet.

2.11 Test Control Formal testing shall be required to verify conformance of designated structures, systems, or components to specified requirements, and demonstrate satisfactory performance for service, or to collect data in support of design or fabrication.

2.11, Test Control Could not locate associated document.

2.12 Control of Measuring and Test Equipment Tools, gauges, instruments, and other M&TE used for activities affecting quality shall be controlled. Records shall be maintained.

2.13, Control of Measuring and Test Equipment NBSR-0005-DOC-00 Calibration Program

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 25 of 33 ASME/ANS-15.8-1995 (R2013)

Requirements NBSR-0002-DOC-02 Section NCNR Implementing Documents Comments 2.13 Handling, Storage and Shipping Handling, storage, and shipping of items shall be in accordance with work and inspection instructions.

2.5.2, Handling, Storage and Shipping NBSR-0004-DOC-05, NIST Packaging and Shipping QA Program for 10CFR71 -

Transport of Radioactive Materials.

AR 6.2 NIST Package and Shipping QA Program AR 6.2.x NIST Package and Shipping Admin.

Procedures Identified as needed, but some documents not written yet.

2.14 Inspection, Test and Operating Status The status of inspection and test activities shall be identified on the items or in documents traceable to the items.

2.12, Inspection, Test and Operating Status Could not locate associated document.

2.15 Control of Nonconforming Items and Services Items that do not conform to requirements shall be controlled to prevent inadvertent installation or use.

2.14, Nonconforming Materials and Parts AR 6.3.4, NBSR Control of Nonconforming Items and Services Identified as needed, but documents not completed or not written yet.

2.16 Corrective Actions Conditions adverse to quality shall be identified promptly and corrected as soon as practical.

2.15, Corrective Action AR 7.2, Corrective Action Program (CAP) 2.17 Quality Records A records system orsystems shall be established at the earliestpracticable time.

Note: AR 7.1 references System Record in Section

8.

The system or systems shall be defined, implemented, and enforced in accordance with written procedures, instructions, or other documentation.

2.17, QA Records NBSR-0001-DOC-02, NBSR Document Control Plan (-

03 not signed)

AR 6.3.1, NBSR Document Control and Records Management Appears to be met, but documents not completed or not written yet.

2.18 Assessments The owner/operator, or its representative, or both, shall conduct periodic assessments of quality-affecting activities during design, construction, or modification. Assessment results shall be documented.

2.18, Audits AR 6.3.5, NBSR QA Assessments Identified as needed, but document is not written yet.

2.19 Experimental Equipment The qualityassurance program shall provide controls over thedesign, fabrication, installation, and modificationof experimental equipment to the extent thatthese impact safety related items.

2.1.1 Quality Program 1.1 Application Could not locate associated procedure.

3 Facility Operations Many of the program requirements are satisfied by existingdocumentation, or by procedures and activitiesrequired by other standards and requirements ofthe chartering or licensing agency. Some requirementsof the quality assurance program foroperations may also be found in other documents, such asTechnical

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 26 of 33 ASME/ANS-15.8-1995 (R2013)

Requirements NBSR-0002-DOC-02 Section NCNR Implementing Documents Comments Specifications. Such requirementsdo not need to be duplicated in the qualityassurance program.

3.1 Organization Management shall provide sufficient resources in personnel and materials toconduct operations. Planning should anticipateneeds as appropriate for any task. The organizationalstructure shall be defined as required bythe Technical Specifications.

2.2.1 QA Program Applicability - Resources AR 1.0, NBSR Conduct of Operations, section 4, Organization (note that the Tech Specs does not indicate a Crew Chief) 3.2 Quality Assurance Program Managementshall establish a quality assurance program byimplementing a policy for the conduct of operations.

2.2.2 QA Program Applicability - Facility Could not locate associated procedure 3.3 Performance Monitoring Managementshall monitor facility performance relative to thegoals that will be used as performance indicators.

2.1.4 Performance Monitoring AR 7.1.1 System Health Summary Template Management shall document periodical observations of operations and identify any deficiencies. Management should assess deficiencies to ensure the execution of corrective actions that will prevent recurrence.

If appropriate, trend analysis should be performed to indicate where improvements or lessons learned could be implemented.

Violations of operating practices should be addressed and documented as appropriate.

AR 7.1, Conduct of System Review Teams references Trouble Tickets in Sections 8.2.5, 8.2.6 and System Health Summary in Section 4.4, 5.1.2, 5.1.3 AR 7.1.1, System Health Summary Template and Instructions AR 7.1.2, System Walkdown Guidance 3.4 Operator Experience Management shall document the methods for maintaining operator experience. (Training addressed in ANSI/ANS-15.4-1988)

AR 2.1, 2.2,.2.2.0, 2.2.1, 2.2.2, 2.2.3 Note the requirement in NBSR-0002-DOC-02 3.5 Operating Conditions Preoperations checklists shall be used to determine or verify required preoperational conditions and readiness to operate.

OI 1.1.0, Reactor Startup Checklist Note the requirement in NBSR-0002-DOC-02 Operating equipment shall be periodically monitored to detect abnormal conditions or adverse trends. Operating conditions should be documented in an operations logbook or other record. The operator should notify the appropriate level of management of any abnormal situations.

AR 1.3, Operations logbook Recommend noting the use of Operating Instructions in Manual. (Manual and associated ARs, OIs, TSPs, etc.)

3.6 Operational Authority Management shallestablish the method for conducting operations and the responsibility for each shift.

AR 1.0, Operate the NBSR, section 8 shift crew role and responsibilities Section 9 shift routines and practices Seem to be met through the procedures. Note the requirement in NBSR-0002-DOC-02.

3.7 Control Area Only persons specifically authorized or certified to operate the reactor shall operate control area equipment. Trainees may operate equipment only when they are directly supervised by certified operators.

2.9.1, Operation of Control Area Equipment Could not locate an associated procedure.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 27 of 33 ASME/ANS-15.8-1995 (R2013)

Requirements NBSR-0002-DOC-02 Section NCNR Implementing Documents Comments 3.8 Ancillary Duties Operators shall not be assigned ancillary duties to be performed during operations to the extent that these duties could interfere with the ability to monitor facility parameters and maintain control of the reactor.

2.1.6, Ancillary Duties Could not locate an associated procedure.

3.9 Emergency Communications Operatorsshall be able to contact the appropriate level ofmanagement rapidly.

NBSR Emergency Plan Docket No. 50-184 Emergency Instructions Met through the procedures. May not be written yet. Note the requirement in the QA Manual.

3.10 Configuration Control Equipment shall be identified that requires configuration control. Management is responsible for establishing and maintaining proper configuration and should authorize any changes to safety-related items. All configuration changes to safety-related items should be documented.

2.4 Design Control section 2.4.1, Configuration Management AR 6.1.1 Engineering Change Management AR 6.1 NBSR Configuration Management Plan 3.11 Lockouts and Tagouts Locks and tagsshall be placed on equipment when, for safety or other special administrative reasons, controls must be established.

AR 7.1, Conduct of System Review Teams, Section 8.2.3, Lock Out Tag Out AR 11, Lock Out/Tag Out NIST S 7101-56, Control of Hazardous Energy (Lockout/Tagout)

Met through the procedures.

Note the requirement in NBSR-0002-DOC-02.

3.12 Test and Inspection Tests shall be performed following system maintenance, design changes, or inspection that involves dismantlement of components or systems. The results of the test should be documented and retained.

2.16.2, Test and Inspection TSPs (Tech Spec Procedures), for example:

TSP 4.1.1(1) Operability Test of Reactor Safety System Channels TSP 4.3.1(2) Test Following Modifications to the Reactor Coolant System TSP 4.4(3) Test to be Performed when any Addition, Modification, or Maintenance has been made to Confinement Technical Specification Procedures (TSP) cover this requirement.

3.13 Operating Procedures Operating procedures shall provide appropriate direction. Operating procedures shall be written, reviewed, approved by appropriate management, controlled, and monitored to ensure that the content is technically correct, and the wording and format are clear and concise.

AR 5.0 Procedure Use and Adherence OIs (Operating Instructions) 3.14 Operator Aid Postings Any posted information that aids operators in performing their duties should be current and correct. Management should review operator aids to determine that they are necessary and correct before approving their posting. Postings There are postings. Include simple requirements in QA Manual.

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 28 of 33 ASME/ANS-15.8-1995 (R2013)

Requirements NBSR-0002-DOC-02 Section NCNR Implementing Documents Comments should be checked periodically for continued applicability.

3.15 Equipment Labeling Equipment shall be labeled to help facility personnel positively identify equipment they operate and maintain. Labels should be permanent, securely attached, readable, and have appropriate information.

See NBSR-0005-DOC-00, Calibration Program for Measuring and Test Equipment, Labeling section.

Does not indicate labels as noted in ANS 15.8 Existing facilities shall not be required to prepare quality assurance documentation for the as-built facility. However, all available as-built records should be collected and stored in accordance with the provision of 2.17. All replacements, modifications, and changes to safety-related items shall meet the applicable quality assurance requirements of this standard. The replacement, modification, or change to the facility shall meet or exceed the requirements of the original system or component.

The replacement, modification, or change shall be documented and maintained to establish the current configuration of safety related items at the facility. (from4. Applicability to Existing Facilities)

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 29 of 33 Appendix B Personnel Interviewed Position Director, NCNR Chief Rx Ops & Engr, Deputy Director Chief Reactor Operations Chief Reactor Engineering Chief, Aging Reactor Management Chief, Health Physics Safety Officer Engineering Manager SEC Chairman SEC Vice-Chair SEC Outside Members (4)

SEC Subcommittee (S/C) Chairs Beam Experiment S/C Irradiation S/C Audit S/C Procedures Audit S/C Corrective Actions Tracking S/C Incident Response Review S/C Post Criticality Oversight S/C Training and Qualification S/C SAC Members (4)

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 30 of 33 Appendix C Documents Reviewed Document No.

Rev/Date Title/Description NBSR-0001-CH-08 3

NCNR Reactor Safety Evaluation Committee (SEC) Charter NBSR-0002-CH-05 3

Charter for the NBSR Irradiation Subcommittee of the NBSR Safety Evaluation Committee NBSR-0009-CH-01 10/30/21 Charter for the NCNR Safety Evaluation Committee Corrective Action Tracking Subcommittee NBSR-0004-CH-05 9/26/22 Charter for the NCNR Audit Subcommittee of the NCNR Safety Evaluation Committee NBSR-0003-CH-05 9/29/22 Charter for the NCNR Beam Experiment Subcommittee of the NBSR Safety Evaluation Committee ANSI/ANS-15.1-2007 The Development of Technical Specifications for Research Reactors Technical Specifications Sept 25,2022 Reactor Operations Status Report to the Safety Evaluation Committee (SEC) (Meeting #408)

June 5, 2022 Personnel Dose Summary for the NCNR for Calendar Year (CY) 2021 Reactor Operations Status Report to the Safety Evaluation Committee (SEC) (Meeting #408)

Sept 12, 2019 2019 Reactor Audit in Accordance with TS 6.2.4(1-4)

Sept 10,2020 2020 Reactor Audit in Accordance with TS 6.2.4(1-4)

Sept 30, 2021 2021 Reactor Audit in Accordance with TS 6.2.4(1-4)

Sept 16,2022 2022 Reactor Audit in Accordance with TS 6.2.4(1-4)

NCNR 2021 SAC Report NCNR 2022 Safety Assessment Committee (SAC) Report March 2, 2022 SEC Minutes, Meeting No. 392 (Special - IRRS)

June 23, 2021 SEC Minutes, Meeting No. 396 SEC Minutes, Meeting No. 408, 9/26/22 NBSR-0010-CH-01 1

Charter for the NCNR SEC Reactor Restart Readiness Subcommittee NBSR-0014-CH-01 A

Charter for the NCNR Post-Criticality Oversight Subcommittee NBSR EMERGENCY PLAN Dec. 2008 ANS-15.8-1995, R2013 Quality Assurance Program Requirements for Research Reactors NBSR-0001-DOC-02,

-03 02, 03 draft NBSR Document Control Plan NBSR-0002-DOC-02 Quality Assurance Program

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 31 of 33 Document No.

Rev/Date Title/Description NBSR-0003-DOC-06 Guidelines for Engineering Changes NBSR-0004-DOC-05 NIST Packaging and Shipping QA Program for 10CFR71 -

Transport of Radioactive Materials.

NBSR-0005-DOC-00 Calibration Program for Measuring and Test Equipment NCNR-1000-DOC-00 Engineering Change Control for NBSR Reactor Operations and Engineering, NCNR NBSR Engineering Governance Hierarchy 2

AR 6.0 NBSR Engineering Manual Visual AR 1.0 Operate the NBSR Procedures Visual AR 1.0 2 Draft Operate the NBSR AR 1.3 Operations logbook AR 5.0 2 Draft Procedure Use and Adherence AR 5.2 1 Draft Document Routing Policy AR 5.3 1

Creating and Routing a Procedure in the EDMS for Review and Approval AR 5.4 1 Draft Observation Program and Checklist AR 7.1 1

Conduct of System Review Teams AR 7.1.1 1

System Health Summary Template and Instructions AR 7.1.2 1

System Walkdown Guidance AR 7.2 2

Corrective Action Program (CAP)

AR 11 E

Lock Out/Tag Out OI 1.1.0 4 Draft Reactor Startup Checklist NIST S 7101-56 5

Control of Hazardous Energy (Lockout/Tagout)

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 32 of 33 Appendix D Assessment Team Biographies Terrence Overlid (Team Lead)

Over forty years of experience in the operation, design, and maintenance of nuclear power plants including twenty years of progressive management experience. Fifteen years of experience in waste management at DOE decommissioning and decontamination site. Strong background in formality/conduct of operations including Senior Reactor Operator license. Proven ability to plan, lead and document all types of independent assessments, event investigations, and readiness assessments for new nuclear activities and implementation of new safety analyses. Served as the Team leader for CH2M WG Idahos Management Self-Assessment (MSA) for Sodium Distillation Project; MSA for Engineering, Maintenance and Nuclear Safety for Idaho Waste Treatment Unit (IWTU) project (extended, on-going review that lasted almost 1 year); Contractor Readiness Assessment (CRA) for Relocation of TN-BRP and TN-REG Casks to CPP-2707; CRA for Tank Closure Washing and Sampling Activities at the INTEC Tank Farm Facility; CRA for Removal of Resin from VES-FT-131 & VES-FT-132; CRA for CPP-666 Water Treatment System Resin Replacement Activities.

Managed successful preparation for Implementation Validation Review and Operations Readiness Reviews for Tank T-1A project. Project had failed previous reviews. Task included mentoring of operations supervision and facility management. Manager for the compliant implementation of TSRs.

Position requires extensive experience in the preparation, implementation and daily use of TSRs.

Responsible for Issues Management for Radioactive Waste Management Project (RWMC), Idaho Cleanup Project (RWMC), including Operations Assurance, Price Anderson, 10CFR 851, Occurrence Reporting and corrective action program implementation. University of California, Berkeley, Reactor Operator/Senior Reactor Operator for the university test reactor.

Steven Crowe Mr. Crowe has 45 years of experience in the operation and licensing of nuclear facilities, safe management of nuclear materials, decommissioning and demolition of nuclear facilities, and disposal of hazardous chemicals. He is a recognized subject matter expert (SME) in operational readiness and Conduct of Operations (CONOPS) and has mentored the recovery of troubled facilities prior to Operational Readiness Review (ORR) resumption. His relevant experience includes leading multiple root cause investigations and serving as a qualified team leader/advisor for numerous U.S. Department of Energy (DOE) ORRs and other assessments, including the Waste Isolation Pilot Plant (WIPP) restart. He is trained in MORT, Kepner-Tregoe, PII, and INPO Human Performance Evaluation/Root Cause Methodology. He provided specialty nuclear engineering, operations support, and consulting for the Depleted Uranium Hexafluoride (DUF6) facilities startup at the Portsmouth Gaseous Diffusion Plant (PORTS) and the Paducah Gaseous Diffusion Plant (PGDP). He served as a member of the Nuclear Safety Basis Review Board that reviewed and approved for submittal over 20 Documented Safety Analysis (DSA) documents for Bechtel Jacobs Company LLC (BJC). At the Rocky Flats Environmental Technology Site (RFETS), he managed all safety, health, and quality aspects for nuclear material-handling and rule and order compliance for the quality assurance (QA), nuclear safety, fire protection, waste management, and construction programs. He also managed the Westinghouse Government Services Quality Assurance Manual (QAM) Review and Gap Analysis. Mr. Crowe retired from TVA in 1995. While at TVA he served in a variety of positions including being a charter member of the nuclear oversight group, a shift outage manager, Human Performance Enhancement System coordinator, and the

Safety Evaluation Committee (SEC)/Safety Assessment Committee (SAC)

Final Report Page 33 of 33 coordinator of the off-site (Nuclear Safety Review Board) activities. Mr. Crowe graduated from the United States Naval Academy in 1977 with Bachelor of Science (with merit) in Operations Analysis.

Mike Fecht Mr. Fecht has forty-nine years of nuclear experience. He has held positions as Commercial Reactor and Senior Reactor Operator and Instructor, instructing a Limited Senior Reactor Operator class (refueling SRO with 100% passrate); Lessons Learned Manager (TVA), Nuclear Assurance and Licensing Manager (Sequoyah Nuclear Plant), TVA Corporate QA Director and was the TVA corrective action program troubleshooting lead in which capacity he led corporate teams at troubled nuclear sites to assist in recovery. He participated in TVA corrective acon program audits and provided overall oversight of the TVA Corporate and three nuclear power plants. In DOE, he the was a Senior Advisor to the Yucca Mountain DOE Quality Assurance Director. During this assignment he guided DOE corrective action program (regulated by the NRC) from ineffective to effective in 16 months. Within the DOE complex, he has conducted multiple Operational Readiness Reviews/ Readiness Assessments and performed root cause analyzes at both commercial nuclear and DOE sites. He has mentored troubled facilities at TVA and DOE facilities and performed the root cause analysis on the 2018 Idaho National Drum event where four re-packed drums ejected their lids and radioactive contents within a clean area in the ARP-V repackaging tent. In February 2020 he assumed the lead as the TFE acting project manager for the WIPP Records Management program. In 2022, Michael was a team member on the four-month Operational and Safety Culture Assessment of the US DOE Waste Isolation Pilot Plant outside Carlsbad NM. He also supported a DOE TRU waste processing plant recovery from multiple programmatic failures to being capable to complete the DOE contract activities and turnover operations to a new DOE contractor. He is trained in MORT, Kepner-Tregoe, PII, and INPO Human Performance Evaluation/Root Cause Methodology. He established the initial course of HPES root cause training for TVA and has conducted Corrective Action management training for TVA and DOE sites Diane Mlynarczyk Ms. Mlynarczyk is an engineer with over 35 years of experience primarily in nuclear safety analysis, quality assurance, and license renewal application review, probabilistic risk assessment (PRA) and nuclear fuel fabrication surveillance. She is responsible for the Nuclear QA Program and Code QA development and implementation at ISLs Energy & Space Division. She has over 25 years of experience as quality assurance (QA) specialist and supervisor for licensee and NRC support contracts ensuring all facets of the corporate program meet American National Standards Institute (ANSI), NQA, and organizational QA requirements. Ms. Mlynarczyk has also supported nuclear safety analysis and utility consulting activities. She has performed QA reviews on transient analyses and computer software programs. She prepared draft RELAP5-3D Commercial Grade Dedication and Compliance Matrix Report for NuScale Power. She assisted INL in updating their quality assurance program plan for the high performance workplace, which meets DOE O414.1C, 10CFR830 and ASME NQA-1-2000 requirements.Ms. Mlynarczyk assisted in the planning and performance of fuel fabrication review programs for fuel for both the Pressurized Water Reactor and the Boiling Water Reactor to ensure that the nuclear fuel being fabricated is adequate to meet operating objectives.

Assessment Team Biographies