ML24023A621
| ML24023A621 | |
| Person / Time | |
|---|---|
| Site: | National Bureau of Standards Reactor (TR-005) |
| Issue date: | 12/06/2023 |
| From: | Cavanaugh G US Dept of Commerce, National Institute of Standards & Technology (NIST) |
| To: | Office of Nuclear Reactor Regulation |
| Shared Package | |
| ML24023A617 | List: |
| References | |
| Download: ML24023A621 (1) | |
Text
1 TASK-03 PROBLEM IDENTIFICATION AND RESOLUTION (PI&R)
PERFORMANCE AREA ASSESSMENT REPORT FINAL December 6, 2023 CONDUCTED - MAY 8 - September 30, 2023 BY THE TASK 03 PI&R TEAM (GARY R. CAVANAUGH, TEAM LEAD)
2 TABLE OF CONTENTS TABLE OF CONTENTS....2 EXECUTIVE
SUMMARY
...3
1.0 ASSESSMENT
SCOPE AND PERFORMANCE OBJECTIVES. 6
2.0 BACKGROUND
...7 2.1 NCNR OVERVIEW 2.2 PI&R ASSESSMENT TEAM MEMBERS
2.3 ASSESSMENT
PROCESS AND CRITERIA 2.4 WORK PERFORMANCE OBSERVED
3.0 ASSESSMENT
RESULTS...15 Objective 1 Objective 2 Objective 3 Objective 4 Objective 5 Objective 6 Objective 7 Objective 8 Objective 9 Objective 10 4.0 CONCLUSIOD RECOMMENDATIONS........................35
4.1 CONCLUSION
S 4.2 RECOMMENDATIONS 5.0 Related Documents......40
3 Executive Summary ASSESSMENT SCOPE On February 3, 2021, the National Institute of Standards and Technology (NIST), Center for Neutron Research (NCNR) reactor experienced a damaged fuel element which caused high radiation detection from the confinement exhaust stack. A Nuclear Regulatory Commission (NRC) special inspection was conducted which documented seven apparent violations. NCNR participated in an ADR (Alternative Dispute Resolution) mediation session, the output of which was a legally binding Confirmatory Order (CO). The CO corrective actions and enhancements are meant to preclude recurrence of the event. Specific areas were identified for NCNR to implement corrective actions and enhancements which included implementation of a Problem Identification and Resolution (PI&R) Program, including the Corrective Action Program (CAP).
Specifically, NCNR agreed in the CO to hire third-party, independent nuclear consultants to conduct independent third-party assessments in six nuclear programs. This report documents the health of the Problem Identification and Resolution programs. It is noted that multiple corrective actions specified in the CO have already been developed and are in some stage of implementation. These include the Observation Program, System Review Teams, and CAP.
There is also a requirement to maintain these programs to track, trend, and correct failures and deficiencies to prevent recurrence.
For the purposes of this summary (as well as the body of the report), NCNR is the location of and the operator for the National Bureau of Standards Test Reactor (NBSR). The facility is located on the NIST campus in Gaithersburg, Maryland. In some cases, the facility will be noted as NCNR. In other cases, it will be referred to as NBSR. So as to not confuse a reader, these terms may appear to be used interchangeably.
This PI&R Assessment was conducted at the NCNR in 2023. This assessment is being conducted using guidance from the NRC Inspection Procedure (IP) 95003, Supplemental Inspection For Repetitive Degraded Cornerstones, Multiple Degraded Cornerstones, Multiple Yellow Inputs Or One Red Input. This assessment was intended to determine the current status and health of the PI&R process at NCNR.
The PI&R Self-Assessment Team conducted activities to collect information and to evaluate NCNRs ability to complete activities using NRC Inspection Procedure (IP) 71152, Problem Identification and Resolution. Since no specific PI&R guidance exists for the Research and Test Reactor (RTR) industry, the relied on IP 71152, which is the existing NRC standard for the commercial nuclear power industry. It also relied on ANS-15.8-1995, R2013, Quality Assurance Program Requirements for Research Reactors. This standard is the quality assurance program requirements for the RTR industry.
The team conducted an on-site visit in June 2023 and September 2023. In June 2023, some aspects of CAP were being conducted, but understanding of the process was low and interactions/meetings were informal. By September 2023, the mechanics of CAP had made good progress, but still did not meet the standards and expectations of IP 71152 and other industry guidance.
4 It should be understood that PI&R, as defined in IP 71152, is a relatively new concept to NCNR.
One of the programs is functional but not operating at a mature level. Other parts of PI&R are either under development or are planned to be instituted. In both cases, the team was to assess and provide recommendations (corrective actions) that will ensure that NCNR can meet the requirements for expansion of the PI&R program specifically called out in the CO.
ASSESSMENT RESULTS The Assessment Team evaluated 10 Performance Objective identified in the Project Execution Plan. Details of the assessment are noted in the full attached report.
The Team identified 1 Area in Need of Attention (ANA) and 8 Areas for Improvement (AFI) that included the following:
AFI-01: In addition to the Generic Communication Program, the Benchmarking, Cause Analysis, Trending, Operating Experience, and Metrics (Performance Indicators, are PI&R programs and have insufficient guidance in procedures/programs at NCNR to drive full implementation.
AFI-02: CAP procedures are incomplete and do not meet expectations noted in DOE-G414.1-5, Corrective Action Program Guide, or INPO - Principles for Effective Self-Assessment and Corrective Action Program.
AFI-03: While recent performance has improved since June 2023, CAP has not been well rolled out to all facility personnel and has not been consistently reinforced by management. Much work remains for full implementation.
AFI-04: A Quality Assurance plan or program must be revised to achieve full compliance with ANS-15.8.
AFI-05: NCNR does not have a single source entry for PI&R reporting. The consequences of multiple distributed safety, trouble and work request platforms affect the ability to ensure CAQ/SCAQ are corrected, the ability to trend performance, and as indicated by interviews - is confusing for staff as to which platform to use.
AFI-06: The facility is lacking Operability and Reportability guidance/procedures.
AFI-07: The facility inappropriately changed Chapter 12 of the SAR (NCNR license basis).
5 AFI While the facility can be operated safely, there are insufficient staff at NCNR to complete the added duties of implementing and maintaining PI&R, as defined in IP 71152.
ANA-01: The facility does not have Work Management Procedures/Processes that provide clarity and completeness to implement work at NCNR.
RECOMMENDATIONS The recommendations are summarized below. The details of the recommenda;ons are in Sec;on 4.2 of this document.
The team is including the following (non-detailed) 11 recommendations:
- 1. Through the NCNR procedure approval process, review/approve the Benchmarking Planning Program provided by the PIR Assessment team.
- 2. Using the new Benchmarking Program, write a benchmarking plan to develop and implement a Work Management Program, a Generic Communication Program, and the following PI&R programs:
- a. Self-Assessment Program
- b. Cause Analysis (Simple, Apparent, and Root)
- c. Trending Program
- d. Operating Experience Program
- e. CAP (at a minimum) Metrics /Performance Indicators Program
- 3. In addition to CAP and Benchmarking, develop training for each of the PI&R Programs in recommendation 2 above. To ensure that high quality materials are achieved, the training should be developed using the Systematic Approach to Training/Analyze, Design, Develop, Implement, and Evaluate (SAT/ADDIE) model for instructional design.
- 4. Revise the NCNR CAP procedures by consolidating them into either one procedure or a policy, guidance, procedure hierarchy. It is recommended that the revision include elements described in INPO 14-004 - Conduct of Performance Improvement, DOE G 414.1-5 3-2 Corrective Action Program Guide, and NEI 16-07 (Rev. A) - Improving the Effectiveness of Issue Resolution to Enhance Safety and Efficiency.
- 5. Resolve entry requirements into CAP at NBSR. Ensure that Conditions Adverse to Quality (CAQ) and Significant CAQs (SCAQ) are either directly entered into CAP or evaluated and documented into CAP in a timely manner.
- 6. Establish CAP into a software platform designed to manage CAP that will help NCNR manage the process.
- 7. Restore the NBSR Chapter 12 of the SAR (Licensing Basis). The revision will include re-establishment of ANS 15.8 as the governing guidance.
- 8. Revise the NBSR Quality Assurance Program to align with the ANSI/ANS 15.8 requirements. This includes the establishment of a Quality Assurance Plan/Program.
- 9. Develop Operability & Reportability Guidance.
- 10. Institute a NBSR Work Management Program.
- 11. Conduct an assessment of the various NCNR systems which could have CAQ/SCAQ.
Those programs include but are not limited to Trouble-Ticket [TT], NIST Maintenance Slips could be hidden. These include but are not limited to: Corporate Observations, IRIS, Reactor Aging Management, Engineering Change Request [ECR], CATS, WIRS,
6 N-ticket (Sales Force), Corrective Action Request [CAR], Engineering Non-Conformance Reports, System Review Teams, and Control Room Turnover Logs. Determine whether CAQ and SCAQ exist in those systems (if they do, ensure they are entered into CAP).
- 12. Determine interim external resources needed to accomplish the above actions. Acquire the resources in a timely manner to begin the work.
7
1.0 ASSESSMENT
SCOPE Scope The assessment scope was to provide an Independent and Comprehensive Assessment of NCNRs existing PI&R program. The assessment was performed in accordance with the requirements of the US NRC Confirmatory Order (CO) dated August 1, 2022, that modified NCNR License No. TR-5. This independent assessment identified areas for improvement, corrective actions, and other program improvement opportunities. The assessment was used to gauge the rigor, self-criticality, and overall quality of NCNRs internal self-assessment capabilities in this performance area.
The team assessed and provided recommendation (corrective actions) that will ensure that NCNR can meet the requirements for expansion of the PI&R program as discussed in the Confirmatory Order.
Since no specific PI&R guidance exists for the RTR industry, the PI&R Team relied on IP 71152, which is the existing NRC standard for the commercial nuclear power industry. In addition to US NRC IP 71152, this PI&R program assessment also references ANS-15.8-1995, R2013 (4240215) criteria. ANSI 15.8 provides quality assurance program requirements for research reactors.
The assessment window for this effort extends from February 1, 2011, to July 31, 2023. This window was established by predating the event (February 3, 2021) noted in the US NRC Confirmatory Action Order dated August 1, 2022, by 10 years.
The scope of this assessment is anyone and anything that touches the Problem Identification &
Resolution (PI&R) at the National Institute of Standards and Technology (NIST) Center for Neutron Research (NCNR). This assessment was conducted per the Nuclear Regulatory Commission (NRC) Confirmatory Order (CO) dated August 1, 2022. In the CO, the NRC noted that an assessment of PI&R was to be conducted.
The Team was aware that PI&R, as defined in IP 71152, is relatively new to NCNR and that some of the programs were functional but may not be operating at a mature level. Other parts of PI&R are not yet functional. In both cases, the Team assessed and provided recommendations (corrective actions) to ensure that NCNR can meet the requirements for expansion of the PI&R program.
8
2.0 BACKGROUND
2.1 NCNR OVERVIEW The National Institute of Standards and Technology (NIST) Center for Neutron Research (NCNR) owns and operates a heavy water-moderated nuclear test reactor and associated neutron beam research facility. The NCNRs reactor is licensed by the Nuclear Regulatory Commission (NRC) (USNRC License TR-5) under the name National Bureau of Standards Reactor (NBSR).
On February 3, 2021, the reactor exceeded its nuclear fuel design temperature limit that resulted in melted fuel. The Chief of NCNR Reactor Operations and Engineering notified the NRC of an alert concerning elevated radiation levels at the NBSR. Pursuant to the event notification received from NCNR staff on February 3, 2021, the NRC initiated a special inspection at the NBSR.
The NRC staff issued an interim special inspection report on April 14, 2021, to provide an initial assessment of their understanding of the event sequence, consequences, and the NCNRs response to the event.
On October 1, 2021, NCNR submitted a restart request to the NRC. This restart request documented the NCNR-identified root causes and corrective actions to preclude recurrence of the February 3, 2021, event.
On March 16, 2022, the NRC released a final report of its initial conclusion from its Special Inspection Team (SIT) report. The NRCs final report confirmed and expanded on many aspects of NISTs analysis of the incident, pointing to deficiencies in policies, procedures, training, and Nuclear Safety Culture as contributing to the incident.
The SIT report included: (a) two apparent violations related to exceeding the fuel cladding temperature safety limit; (b) three apparent violations related to inadequate fuel handling, startup, and emergency response procedures; and (c) two apparent violations related to inadequate fuel handling within the vessel and inadequate modifications that invalidated operators ability to meet a TS requirement. In addition to the apparent violations, the SIT also identified weaknesses in the licensees root cause analysis as well as the proposed corrective actions.
Following the issuance of the SIT report, the regulatory staff and NIST engaged in an Alternative Dispute Resolution (ADR) process and agreed to a series of corrective actions that were documented in an NRC Confirmatory Order (CO). The goal of the ADR was to allow the NRC and NIST to reach agreement on actions necessary to restore the facility to its design basis conditions. The NRC staff and NIST held three ADR sessions between May 10 and June 2, 2022.
On August 1, 2022, the NRC issued a Confirmatory Order Modifying License No. TR-5. The Confirmatory Order documented action completed and planned by NCNR as well as the commitments made by NIST to enable the safe operation of the NBSR.
NCNR engaged Advanced Technologies and Laboratories International, Inc. (ATL, Inc.) to support recovery from this event and to enhance facility safety. This task-based contract covers
9 a wide array of activities in six task areas. Task 3 is the conduct of an independent Third-Party PI&R Performance Assessment (Assessment) of NCNR.
When NCNR completes specific activities associated with February 2021 event, the regulator is expected to complete the Significant Issue Follow-up actions in the inspection table, IP 92701, Follow-up, IP 92702, "Follow-Up on Traditional Enforcement Actions Including Violations, Deviations, Confirmatory Action Letters, and Orders, IP 92703, "Follow-up of Confirmatory Action Letters or Orders", and the PI&R portions referenced within those documents.
In review of the IP guidance, NCNR should be ready for the following:
The onsite inspection, if necessary, is to both determine the adequacy of the licensee actions to correct the deficiencies associated with the traditional enforcement action, and to examine whether the licensee evaluations included a review of items from their internal self-assessment activities when assessing the repetitive and generic nature of an NRC action and the effectiveness of the related licensee programs. Where an item is identified as repetitive in nature, the licensee should have conducted an in-depth analysis of the effectiveness of their management control systems.
This analysis entails the determination of the causal factors (i.e., root cause(s)) of deficient controls and their potential generic implications. Note that this is the most important part of this inspection procedure. The OE reviewer will identify any necessary on-site verification of selected aspects of a licensee's commitments that are part of the ADR Confirmatory Order related to discrimination that shall be performed to ensure the adequacy of implementation. Confirm that the licensee has instituted appropriate corrective and preventive measures. Refer to IPs 71152, 95001, and 88005, and applicable inspection procedures for the program being reviewed (such as IPs listed in IMC 2600 Appendix B for fuel cycle facilities) for further guidance on the adequacy of the licensees investigation of the causal factors (i.e. root cause evaluations) and corrective action reviews".
2.2 PI&R ASSESSMENT TEAM MEMBERS The PIR Assessment Team was comprised of the following members:
Gary Cavanaugh Team Lead Colleen Amoruso Analyst Michael Fecht Assessor Steven Crowe Assessor Joseph Drago Assessor Mike Love Assessor Terry Overlid Assessor Mike Quinn Assessor John Ettien Oversight The team member biographies are in Attachment K in the Task 3 PI&R Assessment Checklist Details document.
10
2.3 ASSESSMENT
PROCESS AND CRITERIA 2.3.1 METHODOLOGY This assessment plan contains the same elements identified within the US NRC inspection contained in IP 71152. Although not a specific element in the IP 71152 inspection guidance, Performance Attributes refer to the disposition of operability issues. Therefore, disposition of operability issues was included and reviewed as part of this assessment plan.
The assessment conducted data reviews and/or interviews with facility personnel. The plan incorporates a series of checklists (Attachments A through J in the related document) that was used in the evaluation and validation of required elements for each of the identified objectives.
The checklists were completed in sufficient detail such that an independent reader could easily understand the basis for the comments.
It was not intended that this assessment review every document generated in the assessment period but conducted using a representative sampling of records. However, when weaknesses were identified in meeting an objective, the sample population was expanded to establish how broad the extent of condition was for the identified weakness or performance deficiency.
To conduct this inspection, it was necessary to contact several individuals at NCNR. A list of those individuals can be found in Attachment N of the Task 3 PI&R Assessment Checklist Details document.
PHASE 1: DOCUMENT REVIEW USING PERFORMANCE OBJECTIVES AND CHECKLISTS Using Checklists in the Task 3 - PI&R Assessment Report Checklist Details, document the results of the PI&R Assessment.
The 10 Objectives:
- 1.
Identification of Equipment, Human Performance, and Program Issues Review
- 2.
Issue Classification & Screening Review
- 3.
Operability Determinations / Functionality Evaluation Review
- 4.
Technical Specification Reportability Review
- 5.
Causal Analysis Review
- 6.
Extent of Condition Review
- 7.
Corrective Actions Commensurate with Significance Review
- 8.
Implementation of Effectiveness Reviews
- 9.
Trending Program Review
- 10. CAP Administrative Process Review Objective 1 -Identification of Equipment, Human Performance, and Program Issues Review Using checklists attached to this Plan, validate that equipment, human performance, and program issues are being identified at NCNR at an appropriate threshold and entered into the facility corrective action program (CAP) or its predecessor system or program.
11 This checklist also provides an assessment of interfacing facility programs to ensure these programs have the ability to reliably identify issues. These programs include:
Self-Assessment Benchmarking Simple Cause Analysis Apparent and Root Cause Analysis Trending Operating Experience Quality Assurance Audits Work Management External Assessments Use of Metrics Anonymous condition reporting generation Portions of this assessment checklist will be performed utilizing other assessment activities and the results incorporated into the summary report to meet the scope of this objective.
Objective 2 - Issue Classification & Screening Review Using the attached checklist, validate if NCNR is appropriately classifying issues entered into the CAP. This verification should ensure that NCNR appropriately follows screening procedures and that the NCNR screening process is consistent with NRC expectations.
Objective 3 - Operability Determinations & Functionality Evaluation Review Using the attached checklist, validate whether NCNR is appropriately performing Operability Determinations and Functionality Evaluations within the facility procedural requirements.
Objective 4 - Reportability Review Using the attached checklist, provide verification that NCNR is evaluating and appropriately conducting disposition for reportability issues in accordance with regulatory requirements. This objective will review facility practices for performing Technical Specification reporting requirements are provided in Section 6.7 Objective 5 - Quality of Causal Analysis Review Using the attached checklist, provide verification that NCNR is performing lower-level cause evaluation(s) at an acceptable level to appropriately identify causal factors, when required, to effectively implement corrective actions. This review and validation should be limited to those issues classified as upper tier, lower tier Apparent Cause Evaluations (ACEs), and a sampling of CAP level 3 evaluations. The review will also encompass the extent of cause review required by IP 71152 and facility procedures. The final assessment report for this assessment should
12 summarize those specific findings when providing an overall status of evaluation quality at NCNR.
Objective 6 - Extent of Condition Review Using the attached checklist, validate that the extent of condition is appropriately identified in a sampling of causal evaluations, when required. The extent of condition review should identify the population of items or issues with the same undesired condition as the item or issue that was the original subject of the identified condition.
Objective 7 - Corrective Actions Commensurate with Significance Review Using the checklist, validate that corrective actions commensurate with the significance of the issue have been identified and implemented by NCNR.
Objective 8 - Implementation of Effectiveness Reviews Using the attached checklist, validate through a sampling of completed effectiveness reviews that management oversight is being exercised over the facilitys implementation of self-assessment activities. The intent of this objective is to validate the facilitys effectiveness in monitoring the CAP.
Objective 9 - Trending Program Review Using the attached checklist, validate that NCNR is reviewing items entered into the CAP with the intent of identifying trends of conditions such as repetitive, long-term, or latent equipment failures or cross-cutting components. Personnel are alert in identifying adverse performance trends and the risk of significant or repetitive equipment failures.
Objective 10 - CAP Administrative Process Review Using the attached checklist, validate that appropriate administrative controls exist within the CAP process. This review should include an assessment of management oversight and engagement, performance indicator reviews, training and qualification requirements and record retention requirements to support adequate program structure for effective implementation.
PHASE 2: ONSITE DATA GATHERING After approval of the Project Execution Plan at the completion of Phase 1, the PI&R Assessment Team participated in a Pre-Assessment meeting. The meeting was to confirm a common understanding between NCNR and the PI&R Assessment Team. In that meeting, clarification was provided, as necessary. We also worked out logistical challenges in advance of the On-Site Assessment as needed.
13 During this phase, the On-Site Assessment will be completed per the approved Project Execution Plan developed during Phase 1.
During the onsite portion of the assessment, items that could not be completed remotely were addressed. These included:
Interviews of Senior Leadership Interview of Management Staff Interview of Support Staff Observations of daily and periodic meeting associated with PI&R Review of any necessary items that were not initially included.
The PI&R Assessment Team worked with other assessment team members to ensure that the team minimized the perturbance to the facility that resulted from this assessment effort.
PHASE 3: ASSESSMENT Phase 3A: Analysis Scope: Convert the data gathered in Phases 1 and 2 into actionable information for use in understanding and managing the Problem Identification & Resolution at NCNR.
The assessment activities performed by the PI&R Team will use all means necessary to evaluate the various processes in current and prior use at NCNR. The Team will cross-validate the results and issues, to the extent they exist. The specific methods will be selected by the Team and explained as appropriate, during and shortly following Phase 2 (Onsite Data Collection).
Task Lead assign Team members based upon experience/expertise in the areas being evaluated.
Team members determine the specific evaluation activities they will perform in consultation with the Team.
The Team will use various means to evaluate the PI&R, providing cross-validation of the results and identification of issues, to the extent they exist.
Team members will consider the inspection elements described in NRC IP 95003, as necessary, to assist in their review of materials and interviews.
Areas recently evaluated, assessed, inspected, or were the subject of causal analysis, the Team concludes that the results of those previous efforts are contemporary and may rely upon those results as part of its evaluation.
The Team also may perform independent evaluation activities to ensure that general conclusions are supported by the results of its own evaluation.
The Team meets collectively during the assessment to review information, develop conclusions, and to categorize the results in accordance with Section 5.0 of this Plan.
14 Phase 3B: Reporting Scope: Periodic and close-out communications between the Team and NCNR related to the PI&R Assessment.
The goal of communications between the Team and NCNR is the creation of a no surprises environment in which:
Team conclusions are not prematurely communicated before the Team understands and validates the conclusion(s), and Issues of interest identified in the course of the assessment are communicated to NCNR with a timeliness consistent with issue significance, as judged by the Team.
Phase 4: In-Process Communications Meetings The Task Lead will conduct daily Out-Brief Meetings of each days activities with NCNR Leadership and technical points of contact. These briefings will take place during Phase 2, On-Site Data Gathering.
The Task Lead will conduct an exit meeting with NCNR Leadership and technical points of contact to present a verbal summary of the Assessment, as well as a written summary of preliminary Assessment conclusions and results. This exit meeting will take place on the last day of Phase 2.
Phase 5: DRAFT AND FINAL Assessment Report The Team will provide a Draft Assessment Report to NCNR for review and comment fourteen calendar days following Completion of Phase 2 (On-site Data Gathering).
Upon receipt of NCNR comments, the Team will disposition those comments, factor in additional information discovered during the NCNR review, and produce a Final Assessment Report 14 days after receipt of NCNR comments.
The Assessment Report will identify observations and conclusions using the definitions in Section 5.0, Definitions and Abbreviations, item [7] (Assessment Observation Definitions). Of this Plan.
Identified issues of non-compliance (Findings) will include specific references to the applicable law, regulation, requirement, and/or standard, to provide to NCNR a clear characterization of any POs, ANIs, and/or AFIs the assessment identifies.
2.3.2 Assessment Criteria:
[1]
Area of Strength - This term is used to characterize demonstrated performance in a program or process element within an area being assessed that is
15 exceptionally effective in achieving its desired results. An Area of Strength is a program, process, or activity of such a high quality that it could serve as an example for other similar elements.
[2]
Positive Observation (PO) - This term refers to a noteworthy observation that does not rise to the level where it would be considered as a strength.
[3]
Area in Need of Attention (ANAs) - This term is used to identify a performance, program or process element that is sufficient to meet its basic intent. However, management attention is required to achieve full effectiveness and consistency. ANAs are not normally identified or addressed in action plans submitted to the U.S. Nuclear Regulatory Commission (NRC) but are brought to management attention for consideration and possible entry into the NCNR Corrective Action Program or alternative tracking process.
[4]
Area For Improvement (AFIs) - This term is used to characterize an identified performance, program, or process element that requires improvement to obtain the desired results in a consistent and effective manner. AFIs identified in the Assessment Report will be addressed by the Action Plan Submitted to the NRC.
[5]
Unable to Determine Effectiveness (due to lack of program maturity)
- In some cases, programs have been developed very recently but do not have enough run-time to determine whether they are fully effective.
2.4 WORK PERFORMANCE OBSERVED Plan of the Day Meeting 1 (June 2023): Personnel discussed ongoing facility activities.
Discussed issues that had occurred. Questions on CAP Items were discussed. The formality of the meeting needs to be improved.
Plan of the Day Meeting 2 (June 2023): Very informal and not much input from others in the room. Brief question on if a CAP Item had been issued and personnel did not know.
Main Control Room (June 2023): One SRO in training was at the main control room panel. The SRO seemed busy, did not acknowledge the visitors and appeared to be unfriendly. The Training Manager, Active SRO, was very knowledgeable and friendly, and answered numerous questions. Housekeeping on the control room floor level appeared to be very cluttered. Fire hazards could be an issue that a fire protection engineer should evaluate.
Two Plant Tours for Badging (June 2023): These tours were not as comprehensive as previous team members had experienced.
Weekly CAP Meeting (September 2023): This meeting was similar to a CAP screening meeting.
The meeting was intended to cover approximately 30 open CAPs. Less than 10 were actually moved through to active CAP. Follow up coaching was provided that some members in the meeting were in the details vs. setting level/ownership. Overall, the meeting was a good step forward in that no such meeting was occurring in June 2023.
16
3.0 ASSESSMENT
RESULTS In the related Task 3 - PI&R Assessment Report Checklist Details document, you will find the results of the 10 objectives investigations in Attachment A through J.
Objective 1 Identification of Equipment, Human Performance, and Program Issues Review In Objective 1, there are 11 areas that were assessed. In the corresponding Checklist A of the Task 3 - PI&R Assessment Report Checklist Details document there are 24 separate points that were assessed. One of those 11 areas is Quality of Causal Analysis Review. This area was deemed of such importance that its review was conducted separately under Objective 5 in this report. The other 10 areas were conducted under the 24 points in this Objective (1).
Using the checklist, validate that equipment, human performance, and program issues are being identified at NCNR at an appropriate threshold and entered into the facility corrective action program (CAP) or its predecessor systems or programs.
A checklist was used to provide an assessment of interfacing facility programs to ensure these programs have the ability to reliably identify issues. These programs include:
Self-Assessment Benchmarking Simple-apparent-root Cause Analysis Apparent and Root Cause Analysis Trending Operating Experience Quality Assurance Audits Work Management External Assessments Use of Metrics Anonymous condition reports Principal Conclusions
- 1. With the exception of SEC/SAC, the facility has not received any external assessments in the past 10 years.
- 2. The facility has completed some apparent/root cause analysis. This item is the subject of another assessment (Task 2). As a result, there will be no further comment from the team in this objective.
- 3. The facility inappropriately removed a commitment to ANS 15.8, Quality Assurance Requirements for Research Reactors, from Chapter 12 of the SAR.
17
- 4. The facility does not have procedures for, nor do they complete, benchmarking, simple-apparent-root cause analysis, trending, operating experience, quality assurance audits, metrics, or anonymous CAP report generation. While anonymous CAPs are possible through use of the facility suggestion boxes, it is not widely understood that this could be used in that manner. While some self-assessments have been conducted, it was without benefit of procedural/programmatic guidance.
- 5. The facility does have work management process/procedures. However, they do not implement the process at NCNR.
- 6. These items are more completely discussed in greater detail in the Task 3 - PI&R Assessment Report Checklist Details document.
Current Status The facility has expended significant effort to achieve a robust Problem Identification &
Resolution Program. The facility having no other PI&R examples in the Research and Test Reactor industry has hampered the progress.
Gap 1.1 The facility still has numerous avenues for NCNR staff to report problems and deficiencies.
These systems include, but are not limited to Trouble-Ticket (TT), NIST Maintenance Slips, Corporate Observations, IRIS, Reactor Aging Management, Engineering Change Request (ECR), CATS, WIRS, N-ticket (Sales Force), Corrective Action Request (CAR), Engineering Non-Conformance Reports, System Review Teams, and Control Room Turnover Logs, etc.
Some previously reported items in these systems have not been corrected in many years and appear invisible to facility management. While some items have since been input to CAP, many have not.
The Vulnerability Because the facility still has a high number of avenues for NCNR staff to report problems and deficiencies, the staff will continue to struggle with where to report items. This will continue to put the facility at risk of CAQ and SCAQ items residing outside of CAP. Furthermore, those items may not be resolved in a timely manner and issues in those systems may be invisible to NCNR. Unless an assessment of the numerous systems where problems and deficiencies remain is conducted, CAQ/SCAQ may be hidden from management attention. This may set the facility up for future regulatory and equipment problems.
Gap 1.2 Facility management does not consistently encourage that problems and deficiencies should be input to CAP. Management could write all problems and deficiencies into CAP. Then use CAP as a pointer to other systems as necessary.
The Vulnerability
18 By continuing to use the aforementioned systems rather than CAP, the facility will continue to struggle with a lack of problems and deficiencies being resolved in a timely manner.
Failure to use one system (CAP) will set the facility up for future failures.
Gap 1.3 In interviews, the team learned that the threshold for CAP reporting is too high. Two individuals in management noted that they would not write a CAP if they found equipment out of position (and there were no consequences).
The Vulnerability When the facility has occurrences of low-level problems and deficiencies, they should be placed into CAP. As the issues become numerous, a trend is established. The facility remains vulnerable to issues not being addressed at the trend level, and problems and deficiencies possibly manifesting through an event or other unintended consequence.
Gap 1.4 CAP timely reporting and adequacy of problem description do not meet expectations noted in DOE-G414.1-5 or INPO 14-004. In interviews, many individuals knew nothing of timely reporting. When problem descriptions in CAP were reviewed, many were blank or looked more like a placeholder than a problem description.
The Vulnerability When issues are not reported in a timely manner, the facility is vulnerable to small problems becoming larger problems, or unnecessary regulatory interaction.
If problem descriptions are not clear, facility management cannot adequately assess the problem or recommend adequate action items to resolve issues.
Gap 1.5 The concept of anonymous entry into CAP is not widely known at the facility. Suggestion boxes at the facility could be used for this purpose, but it is not widely known.
The Vulnerability With little to no ability for anyone in the facility to generate anonymous CAPs, the risk of some unreported problems and deficiencies exists. Some individuals may not be fully confident that there will be no management retaliation for input of issues into CAP reporting.
19 Gap 1.6 Other than SAC/SEC audits, Quality Assurance Audits/Assessments are not being routinely conducted at the facility.
The Vulnerability With no Quality Assurance audits/assessment, the facility has little regular independent oversight. As a result, the facility remains vulnerable to finding problems and deficiencies by chance or with failures of equipment/human performance/programs.
Gap 1.7 The facility does not have procedures for, nor do they complete benchmarking, simple cause analysis, trending, operating experience, quality assurance audits (other than SAC/SEC),
metrics, or anonymous CAP report generation. While some self-assessments have been conducted, it was without benefit of procedural/programmatic guidance.
The Vulnerability Failure to have current PI&R programs/processes will severely impede NCNR from safe and consistent operation. The failure to have these PI&R programs has left the facility vulnerable to experience problems and deficiencies associated with equipment, human performance, and programs.
Gap 1.8 The facility does have a work management process at NCNR. However, a rigid Work Management process is not implemented. Work is completed on an as-needed basis and directed by Operations Management.
The Vulnerability Lacking use of a solid and recognized work management process leaves the facility open to unexpected equipment failures and unnecessary cycling of systems and the reactor.
This also represents a challenge, in that configuration management depends on a strong work management process. Since many facility problems are in the non-CAP systems, this allows little possibility for management oversight. Equipment/Processes could be incorrect.
Gap 1.9 Contrary to ANS 15.8, Conditions adverse to quality (CAQ) and significant issues adverse to quality (SCAQ) are not clearly defined in NCNR procedures and processes. Staff at NCNR, for the most part, are unaware of these commonly used terms. Thus, issues are not consistently being placed into CAP.
20 The Vulnerability If facility procedures, including training, are not updated to include CAQ/SCAQ as part of the vernacular at NCNR, the facility will continue to struggle. Struggles will manifest with equipment/human performance and program issues. Also, it may be difficult to communicate with the regulator and others in the RTR industry (as the RTR industry matures).
Gap 1.10 NCNR has not been independently assessed in the past 10 years.
The Vulnerability Lacking independent assessment from outside subject matter experts (e.g., IAEA, INPO) leaves the facility myopic and insular. Over time, the facility could believe that they are completing work in a high quality manner. Individuals with experience from many facilities could bring fresh perspective to NCNR.
Gap 1.11 In 2016, the facility stopped reporting 10CFR50.59 screenings. In 2018, the facility inappropriately removed ANS 15.8, Quality Assurance Program Requirements for Research Reactors, from Chapter 12 of the SAR. This change was inappropriately completed using a 10CFR50.59 screening process. This item was identified by both the PI&R Assessment team and the NCNR Quality Assurance Manager through an assessment. While the PI&R Team understand that this item is being corrected, it was significant none the less.
The Vulnerability Because the facility stopped reporting 10CFR50.59 screenings, the regulator was unaware that the only guideline/oversight document for NCNR was removed from the SAR. While procedural requirement remained and the unit was operated safely, this left the facility with no formal regulatory framework to operate and maintain the unit.
Gap 1.12 The Team conducted a search of CAP and found no entries discussing an association with Safety Related or Important to Safety equipment-emergency planning-degradation failure items. With the significant problems the facility has undergone while attempting to restart, it seems unlikely that one or more of these terms would be absent from CAP. The team has seen no evidence that an assessment of these terms was undertaken by NCNR.
The Vulnerability Failure to document Safety Relate and Important to Safety items prior to restarting will be a red flag to the regulator.
21 Gap 1.13 The facility does not have a formal process regarding Generic Communication from the regulator or operating experience from other facilities.
The Vulnerability One individual at the facility is responsible for Generic Communications. Should this one individual miss something, it could be detrimental to the facility.
Objective 1 Associated ANAs and AFIs:
ANA-01 AFI-01 AFI-02 AFI-03 AFI-04 AFI-05 AFI-07 Objective 2 - Issue Classification & Screening Review Using the checklist, validate if NCNR is appropriately classifying issues entered into CAP. This verification should ensure that NCNR appropriately follows screening procedures and that the NCNR screening process is consistent with NRC expectations.
Principal Conclusions
Current Status As noted in the associated Task 3 - PI&R Assessment Report Checklist Details document (Attachment B), no procedural guidance was in place to direct CAP issue screening. While the facility did practice issue classification and screening in June 2023, it lacked formality and consistency. By September 2023, CAP meetings (screening) are weekly. The meeting was an improvement from June 2023, but spent much of the time discussing unnecessary details.
Further, time management left many of the CAP items not being screened. An extensive review of NCNR CAP procedures was conducted. Results of that review can be located in Attachment M of the Task 3 PI&R Assessment Report Checklist Details document.
22 Gap 2.1 The Facility process for screening and classification of issues is not consistent with industry practices or regulatory expectations. Facility procedures do not discuss or require issue screening or classification. While screening and classification meetings occur, they lack formality and consistency.
The Vulnerability Lacking procedural guidance and consistent management expectation, screening and classification of issues may let the facility fall behind a needed pace of improvement.
Gap 2.2 Conditions adverse to quality (CAQ) and significant issues adverse to quality (SCAQ) are not clearly defined in NCNR procedures and processes. Staff at NCNR, for the most part, are unaware of these commonly used terms. Thus, issues are not consistently being placed into CAP. As a result, no screening of these issues occurs.
The Vulnerability If facility procedures, including training, are not updated to include CAQ/SCAQ as part of the vernacular at NCNR, the facility will continue to find it difficult to achieve success. Struggles will manifest with equipment/human performance and program issues. Also, it may be difficult to communicate with the regulator and others in the RTR industry (as the RTR industry matures in this area). Consistent screening of these issues is imperative to the success of the facility.
Gap 2.3 Since consistent and formal screening do not occur as a procedural requirement, the informal process also lacks the determination of the need for remedial or immediate actions. These actions are typical for SCAQ items to address the condition in the interim until final corrective actions are identified and completed.
The Vulnerability Immediate and remedial corrective actions are reserved for the most risk significant and important to safety items. Lacking immediate and remedial corrective action development leaves the facility open to further degradation and increased regulatory interaction.
Gap 2.4 In reviewing SCAQ corrective action, the team notes that SCAQ is not defined. In review of the cause analysis for the February 2021 cause analysis, corrective actions were not included in the report. Newer Level 3 cause analyses do not have corrective actions assigned to the problems and cannot be evaluated for appropriate actions to correct the problem, root cause, and contributing causes. Also, contributing causes were not identified in the analysis report. Thus, corrective actions to address the contributing causes were not identified.
23 The Vulnerability Not correcting and preventing recurrence of Level 3 items leaves the facility open to recurrence of significant conditions adverse to quality.
Gap 2.5 In interviews, individuals at most levels believe that there are not enough staff to complete normal duties, much less increased duties that come along with PI&R as discussed in IP 71152.
The Vulnerability Without sufficient staff, the facility will continue to fall behind. Without dedicated staff for PI&R, it will only exacerbate the condition of reduced staff. As a result, the facility becomes more vulnerable to the occurrence of failures or unintended consequences.
Objective 2 Associated ANAs and AFIs:
AFI-01 AFI-02 AFI-03 AFI-08 Objective 3 Results - Operability & Functionality Review Using the checklist, validate whether NCNR is appropriately performing Operability Determinations and Functionality Evaluations within the station procedural requirements.
Principal Conclusions
- 1. NCNR does not have a process for performing Operability Determinations (OD) and Functional Evaluations
Current Status:
As noted in the associated Task 3 - PI&R Assessment Report Checklist Details document (Attachment C), documenting operability against TS and ANS 15.8 is new for NCNR.
Determining operability and reviewing open items prior to each operating cycle, including an aggregate effect, is a new concept for NCNR. No procedural guidance exists for operability/functionality.
24 Gap 3.1:
A limited number of interviews with personnel responsible for making ODs (Senior Reactor Operators) indicated a wide spectrum of understanding the concept of Operability.
The Vulnerability:
Lacking guidance on an Operability Determinations, the facility will be inconsistent and may make an incorrect call.
Gap 3.2:
The definitions of operable in the TS and ANS 15.8 are different, as indicated below:
Technical Specification definition of Operable:
The condition is a system or component when it is capable of performing its intended function, as determined by testing or indication.
The ANS-15.1-2007, definition of Operable:
Operable means a component or system is capable of performing its intended function.
The Vulnerability:
The TS is more conservative, thus safe operation of the facility is ensured. The definition of operable needs to be consistent for the benefit of the facility and the regulator.
Gap 3.3 In NCNR Assessment 2023-MA-06, PIR Process, it was noted that the Operability procedure was still in draft.
The Vulnerability:
Lacking guidance on an Operability Determinations, the facility will be inconsistent and may make an incorrect call.
Objective 3 Associated ANAs and AFIs:
AFI-06 Objective 4 Results - Reportability Review Using the checklist, provide verification that NCNR is evaluating and appropriately conducting disposition for reportability issues in accordance with regulatory requirements, the NCNR TS, and ANS 15.8. requirements.
25 Principal Conclusions
- 1. NCNR does not have a procedure driven process for the reportability determination and the verbal notifications to the NRC and other agencies. Reporting responsibilities are not documented.
- 2. NRC considered the causal analysis for the February 2021 fuel melt event to be incomplete.
Current Status:
As noted in the associated Task 3 - PI&R Assessment Report Checklist Details document (Attachment D), there is no process or procedure for reports to NRC that are required in the Technical Specifications. Verbal notifications to the regulator were completed within time requirements for three reports for Limiting Conditions for Operability (LCO) violations. Two of the three written reports were deemed adequate by the regulator. The third report addressing the fuel melt event was deemed to be an incomplete causal analysis, specifically due to the absence of addressing safety culture issues.
Gap 4.1:
NCNR does not have a procedure that describes the process for a reportability determination and notifications to the regulator or other agencies.
The Vulnerability:
Lacking a detailed reportability determination process documented in a procedure and the notifications - both verbal and in written, to the regulator and other agencies, the facility may not meet timeliness and quality requirements.
Gap 4.2:
NCNR does not have a robust causal analysis process for significant and consequential events or conditions.
The Vulnerability:
The absence of a robust causal analysis process increases the probability of significant and consequential events occurring.
Objective 4 Associated ANAs and AFIs:
AFI-01 AFI-02 AFI-06
26 Objective 5 Results - Quality of Causal Analysis Review Using the checklist, provide verification that NCNR is performing lower-level cause evaluation(s) at an acceptable level to appropriately identify causal factors, when required, to effectively implement corrective actions. This review and validation should be limited to those issues classified as upper tier, lower tier Apparent Cause Evaluations (ACEs), and a sampling of CAP level 3 evaluations.
Principal Conclusions
- 3. The facility procedures misclassified the significance of events (e.g., a dropped fuel element was a Level 0 (lowest level).
- 4. The fuel melt event was not included in the AR-20 CAP or the Trouble Ticket (TT) process.
- 5. Trouble Tickets do not have a significance classification. They are assigned a priority of critical, high, medium, or low. Open and Closed TTs did not have cause/discovery statements.
- 6. Repetitive scrams in 2020 did not prompt a causal analysis or evaluation of corrective actions from previous events.
Current Status:
As noted in the associated Task 3 - PI&R Assessment Report Checklist Details document (Attachment E), the NCNR corrective action procedures, AR 7.2-series, have been deployed since April 2023. However, these procedures do not provide sufficient rigor to adequately screen, determine significance, direct appropriately robust causal analysis processes, develop effective (i.e., SMARTER: Specific, Measurable, Accountable, Reasonable, Timely, Effective, Reviewed) corrective actions, guidance on performing Extent of Cause/Extent of Condition analysis for an operating nuclear RTR. There appears to be a misunderstanding about cause analysis. All cause analysis (regardless of simple-apparent-root) is referred to as Root-Cause.
Gap 5.1:
In June 2023, the screening process is assigned to one person, and that person could determine when a review committee is warranted. In September 2023, the committee was increased to all of the chiefs over various individuals at the facility and the Quality Assurance manager. The process has not had sufficient run-time to establish successful remediation.
The Vulnerability:
Using a screening committee rather than relying on a single person to determine significance provides management with a more robust barrier to classify events and conditions that may be precursors to more consequential events and conditions. Employing a screening committee to
27 review CAP entries is a best practice employed at all commercial nuclear facilities. However, lacking solid procedural guidance leaves the facility vulnerable to misclassified CAPs.
Gap 5.2 All cause analysis, by most people at all levels, is referred to as Root Cause Analysis. In actuality, there is no cause analysis (Level 0 CAP), simple cause analysis (Level 1 CAP),
apparent cause analysis (Level 2 CAP), and root cause analysis (Level 3 CAP).
The Vulnerability Incorrect use of cause analysis nomenclature confuses NCNR staff and anyone that hears/reads the term root cause. This will also be confusing to the regulator and anyone assisting as a consultant.
Gap 5.3:
ANS 15.8 references and defines CAQs and SCAQs. The facility needs to formally incorporate that standard in their quality assurance program.
The Vulnerability:
Categorization of CAQ and SCAQ events and conditions aids the facility in determining the level of attention (significance), necessary resources, and priority to identify the causal factors, investigate whether extent of condition is immediately warranted, and implementing the proper level of corrective actions. Proper classification results in a safer facility and more cost efficient operation.
Gap 5.4:
The facility has not implemented a trending program.
The Vulnerability:
Repetitive events are costly to the facility. A Trending Program can identify the trend prior to becoming detrimental to the facility. Some repetitive events (trends) may be indicative of unsafe operation (i.e., unreliable, or deficient equipment, inadequate procedures, lack of training, or weaknesses in using human performance). If trends are not identified, repetitive events are more likely.
Objective 5 Associated ANAs and AFIs:
AFI-01 AFI-02
28 Objective 6 Results - Extent of Condition Review Using the checklist, validate that the extent of condition is appropriately identified in a sampling of causal evaluations, when required. The extent of condition review should identify the population of items or issues with the same undesired condition as the item or issue that was the original subject of the identified condition. (Reference Attachment F)
Principal Conclusions
- 1. Neither an Extent of Condition nor an Extent of Cause was addressed in the February 2, 2021, event root cause report. The NCNR commissioned external and independent review reports did not address either Extent of Condition or Extent of Cause.
- 2. Neither Extent of Condition nor an Extent of Cause analysis is addressed in the NCNR procedures.
Current Status:
As noted in the associated Task 3 - PI&R Assessment Report Checklist Details document (Attachment F), the CAP procedures were recently deployed in April 2023. The Extent of Condition and Extent of Cause analysis is not included in the NCNR procedures.
Gap 6.1:
The nuclear industry defines these terms as follows:
The extent-of-condition is defined as the degree to which the actual condition may exist in other plant equipment, processes, or human performance.
The extent-of-cause is defined as the degree that the root causes actually exist across disciplines or departments for different programmatic activities, human performance, or different types of equipment.
The Vulnerability:
The Extent of Cause and Extent of Condition reviews are important to assure management that the causes of the undesirable consequences and the similar situations/conditions are evaluated to preclude the likelihood of similar events. Failure to complete those reviews increases the likelihood of repetitive events.
Objective 6 Associated ANAs and AFIs:
AFI-01 AFI-02
29 Objective 7 Results - Corrective Actions Commensurate with Significance Review Using the checklist, validate that corrective actions are commensurate with the significance of the issue and have been identified/implemented by NCNR.
Principal Conclusions
- 2. The facility has not closed corrective actions in a timely manner.
- 3. No further analysis of significant repetitive issues (e.g., shim system problems, multiple scrams without causal analysis) are examples of ineffective CAP elements.
- 4. CAP procedures do not identify a due-date extension process.
Current Status:
As noted in the associated Task 3 - PI&R Assessment Report Checklist Details document (Attachment G), CAP became active in late March 2023 and is in its infancy. Issues are not being resolved in a timely manner. In late September 2023, the following CAPs entered into the system had this status:
30 Gap 7.1:
The integrity of the due dates for actions in CAP is not maintained. Realistic due dates and infrequent due date extensions are the hallmarks of an effectively managed CAP. Prioritization of corrective actions is required to reduce the risk of the facility of experiencing a repeat event due to delay in completing the corrective action in a timely manner.
The Vulnerability:
The inability of the facility to prioritize and maintain the integrity of CAP increases the risk to the safety and efficiency of the facility to achieve its mission.
Gap 7.2:
CAP is in the early stages of implementation. Facility risk of repeat events of the same or more consequence has not been mitigated by the current program (not completing extent of condition/cause items).
The Vulnerability:
When corrective actions to prevent recurrence are not closed in a timely manner the probability of a repeat event is increased.
Gap 7.3:
Although the program is in its infancy, additional management attention enforcing the expectation that all CAQ are entered into CAP. Prioritization of resources and holding personnel accountable to complete corrective actions on schedule is warranted.
The Vulnerability:
An ineffective CAP places the facility at operational risk and is more costly than a well-run program. Issues not entered into CAP leave the facility vulnerable to repeat problems.
Gap 7.4:
Other than SAC/SEC, NCNR QA does not have a Quality Assurance audit program.
The Vulnerability:
Without a robust audit program, management does not have an independent means that requirements are being met in areas such as programs, equipment performance, and personnel competence. Failing to have independent audits increases the likelihood of finding problems and deficiencies either by chance or an unwanted event.
31 Objective 7 Associated ANAs and AFIs:
AFI-01 AFI-02 AFI-03 AFI-04 Objective 8 Results - Implementation of Effectiveness Reviews Using the checklist, validate through a sampling of completed effectiveness reviews that management oversight is being exercised over the stations implementation of self-assessment activities. The intent of this objective is to validate the stations effectiveness in monitoring CAP.
Principal Conclusions
- 1. Approved NCNR procedures do not require effectiveness reviews. No effectiveness reviews have been performed.
Current Status:
As noted in the associated Task 3 - PI&R Assessment Report Checklist Details document (Attachment H), the approved procedures that establish the elements of CAP do not require effectiveness review.
Gap 8.1:
The purpose of the effectiveness review is to verify that the intended or expected results were achieved after implementation of corrective actions, specifically, addressing the root cause(s), and confirm/verify that new problems or unintended consequences were not introduced by implementation of the corrective actions. An effectiveness review should verify that actions were taken, and that the actions had the desired effect. An effectiveness review provides a high level of confidence that the completed corrective actions to prevent the recurrence of the same or similar event have been effective.
The Vulnerability:
The lack of an effectiveness review is a missed opportunity for management to identify ineffective corrective actions that have been completed and closed out. Repeat events are an indication that the corrective action program is not effective in preventing recurrence. NCNR has had several repeat events.
Objective 8 Associated ANAs and AFIs:
AFI-02 AFI-03
32 Objective 9 Results - Trending Program Review Using the checklist, validate that NCNR is reviewing items entered into CAP with the intent of identifying trends of conditions such as repetitive, long-term, or latent equipment failures.
Personnel are alert in identifying adverse performance trends and the risk of significant or repetitive equipment failures. (Reference Attachment I)
Principal Conclusions
- 1. The facility does not trend CAP items or Trouble Tickets. Repeat events of both equipment and human performance issues are observed.
- 2. Ineffective corrective actions are evident as repeat events have occurred.
Current Status:
As noted in the associated Task 3 - PI&R Assessment Report Checklist Details document (Attachment I), the Trouble Tickets is the primary process the facility uses to record events and conditions that are off-normal. The facility was asked to provide any evidence of trending activities. None was provided.
Gap 9.1:
Trending is an essential process in an effective corrective action management program.
Examples of repeat events of similar type and common themes include:
- a. Fuel melt event with previous unlatched fuel events not identified as a performance issue. Task 2 team identified five other issues where fuel elements had become unlatched.
- b. Dropped Fuel elements identified previous issues that did not prevent the current dropped fuel event in December 2022. Assessment team identified several past dropped fuel elements (>5).
- c. Ventilation issues during recent startups have been identified in the past. The facility had not identified a trend of the ventilation system leaks.
- d. Recently numerous shim issues with thirty-one shim issues with the site not identifying the performance trend.
- e. Repetitive reactor scrams not identified by the site as an adverse trend.
- f. Repetitive technical specification action entries.
- g. Of the 90 CAP Items provided to the PI&R team in June 2023, over sixty identify procedure issues (Lack of Current Cap Trending)
33 The Vulnerability:
Without a formal trending program, personnel rely on their corporate memory to identify adverse trends, or repeat events, that over time reduce the safety and reliability of the facilitys operation.
Increased operational costs are also an undesirable consequence of not effectively evaluating events. Opportunities to improve procedures, provide supplemental or remedial training, reduce human performance errors, reduce the probability of significant equipment damage or unavailability, and strengthen safety culture are missed. ANS 15.8 Section 2.16, Corrective Actions, states: In the case of a SCAQ, the cause of the condition shall be investigated, and corrective action taken to preclude recurrence.
Objective 9 Associated ANAs and AFIs:
AFI-01 Objective 10 Results - CAP Administrative Process Review Using the checklist, validate that appropriate administrative controls exist within CAP process.
This review should include an assessment of management oversight and engagement, performance indicator reviews, training and qualification requirements and record retention requirements to support adequate program structure for effective implementation.
Principal Conclusions
- 1. Assignment of significance determined typically by one person in June 2023. The screening committee meetings were not regularly or frequently held.
- 2. Assignment of significance determined by committee in September 2023. The committee was comprised of Chiefs at the facility and led by the QA Manager. The discussion at the meeting degraded into cause and corrective action details. Not enough time was allowed to complete review of all CAPs for the week.
- 3. CAP has not established metrics (performance indicators) or a trend program.
Performance goals and action thresholds have not been established.
- 4. Record keeping has not been established.
Current Status:
As noted in the associated Task 3 - PI&R Assessment Report Checklist Details document (Attachment J), the CAP process is in its infancy. The process roll-out was in April 2023.
Currently all CAPs are in a SharePoint database. The data is reported in an Excel file format with the following fields: CAP ID, CAP Item Title, CAP Level, CAP Item Description, Immediate Remedial Actions, Reviewer Comments, CAP Leader, Status, Created Date, Proposed Closure Date, and Modified Date. The team requested and did not receive copies of records for TS compliance audits and documentation of SEC assignment letters.
34 Gap 10.1:
The CAP procedure AR 7.2 does not address the timeliness of processing the issues. Best practice is for the Shift Supervisor or Chief to evaluate the significance and reportability when received, followed by the screening committee evaluation for Level (0 through 3) no later than the next regular working day.
The Vulnerability:
Reliance on a committee that has no procedural guidance may result in incorrect determination of Level. In some cases, this may result in a missed opportunity for assigning a formal causal analysis or identifying an adverse trend.
Gap 10.2:
There are no procedures which identify the CAP metrics or performance indicators to be tracked.
The Vulnerability:
Best practice is to establish a set of metrics and performance indicators to monitor the health of the program, communicate to management the effectiveness of the program, and assign resources to areas which need to be strengthened. Without performance monitoring, goals, and thresholds for action, management does not have the data to ensure safe, efficient, and effective operation of the facility.
Gap 10.3:
There are no procedures which establish a trend program.
The Vulnerability:
Well-operated facilities document issues during all modes of operation. Ensuring that personnel promptly document issues through CAP is the first step in a well-structured program. Collating and evaluating the data is essential to identifying adverse trends promptly and initiating effective corrective actions.
Gap 10.4:
There is no Quality Assurance Plan reference standard in the TS or Safety Analysis Report.
The Vulnerability:
The facility does not have a national consensus quality assurance standard to establish and maintain a safe and efficiently operated nuclear facility.
35 Gap 10.5:
A best practice is to use a CAP database management system, not a SharePoint database and Excel used for reporting. There is no provision to attach documents to a CAP Item in the current format. The facility is not compliant with the TS or ANS 15.8 requirements for record keeping.
The Vulnerability:
A SharePoint database and reporting with Excel are not an efficient manner to manage this type of data. Failure to use a platform designed/developed to operate as a CAP leaves the facility at risk of an ineffective corrective action and/or use of a work-around. A CAP platform may actually reduce workload.
Gap 10.6 In review of several Operation Procedures, Maintenance Procedures, and Work Package Development Procedures, CAP was mentioned only once. Failure to coalesce CAP with all major programs/procedures at NCNR will make it difficult for existing and new staff to understand how CAP is part of the fabric for the facility.
The Vulnerability With CAP being the initial PI&R Program rolled out, it must be integrated with most major programs at NCNR. Intermingled Objective 10 Associated ANAs and AFIs:
AFI-01 AFI-02 AFI-04
36
4.0 CONCLUSION
S AND RECOMMENDATIONS
4.1 CONCLUSION
S, ANAs, AND AFIs The assessment team observed multiple barriers that face NCNR in their efforts for understanding and implementing programs where the facility staff have little experience.
The assessment team completed an assessment of the existing PI&R program at NCNR. The assessment was conducted in accordance with the requirements of the NRC August 1, 2022, Confirmatory Order (CO) Modifying the NCNR License No. TR-5.
The assessment team concluded that the PI&R program has not been sufficiently developed to meet the NRC IP 71152 PI&R guidance. Ten performance objectives were defined in the Project Execution Plan (PEP) to help guide the team through the assessment. The Project Execution Plan can be located in Attachment L of the associated Task 3 PI&R Assessment Checklist Details document.
Document reviews, CAP Items, Trouble Ticket information (and many other problem reporting systems at NCNR), and interviews conducted all support the assessment conclusions. The team has provided 1 Area in Need of Attention and 8 Areas for Improvement. These AFIs along with supporting recommendations, will provide sufficient direction to develop a healthy PI&R at NCNR.
NCNR did not expect that PI&R would be a healthy program since nothing similar existed before the August 2022 Confirmatory Order. A CAP, implemented in late March 2023, was noted to still be in its infancy during the assessment. By September 2023, the program had made good progress and was conducting regular CAP Meetings (screening). Following the September 2023 observation of the CAP Meeting, the assessment team provided additional feedback to assist the facility in further maturing the program.
The PI&R team identified that one of the most pressing issues hindering the facility from being successful is the lack of resources. These resources will support the present and future work to build a strong PI&R program. NCNR management is well aware that this need and by September 2023, incoming staffing is already planning 10 additional consultants to assist the facility.
There are eight processes which support PI&R implementation at NCNR. They include CAP, Benchmarking, Self-Assessment, Trending, Metrics, Cause Analysis, Operating Experience, and Safety Conscious Work Environment (SCWE). Of those, SCWE was assessed in Task 2.
Corrective actions were developed and monitored from that effort. Weaknesses in the other 7 items are the subject of the ANAs and AFIs below:
ANA-01: The facility does not have Work Management Procedures/Processes that provide clarity and completeness to implement work at NCNR.
While Work Management procedures/processes exist, the facility does not use them.
Trouble Tickets (TT) are used to drive work at the facility.
o Older TTs are 5-10 years old and deficiencies still exist.
37 NCNR staff admit that the process is disjointed.
AFI-01: In addition to lacking a Generic Communication Program, the following PI&R programs and have insufficient guidance in procedures/programs at NCNR to drive full implementation:
Self-Assessment.
Benchmarking.
Cause Analysis (Simple, Apparent, and Root).
Trending.
Operating Experience.
Metrics (Performance Indicators).
AFI-02: CAP procedures are incomplete and do not meet expectations noted in DOE-G414.1-5, Corrective Action Program Guide, or INPO - Principles for Effective Self-Assessment and Corrective Action Program. As a result, the following are (but not limited to) examples of the shortcomings:
Roles and responsibilities not defined.
Screening is not defined or discussed.
The timeliness of issue generation not defined or discussed.
Simple/Apparent/Root causes not defined or discussed.
CAQ and SCAQ are not defined or part of the vernacular at NCNR.
Trend/Cause codes not defined or discussed.
Some staff at all levels are unaware of management expectations regarding CAP.
Some CAQ and SCAQ items are not in CAP (some are in other various systems).
o Fuel melt o No NSC Actions o No Confirmatory Action items from the regulator CAP process not well integrated into major program/procedures at NCNR.
CAP is not effective in solving or preventing problems.
AFI-03: While recent performance has improved since June 2023, CAP has not been well rolled out to all facility personnel and has not been consistently reinforced by management. Much work remains for full implementation.
Training has been inconsistent and ineffective (not developed using SAT/ADDIE).
Management does not consistently support CAP.
Some individuals at all levels do not embrace CAP.
Many individuals are unsure of their CAP responsibilities.
Some individuals resist using CAP.
CAP is not effective in preventing recurrences and solving/preventing problems and deficiencies.
38 AFI-04: A Quality Assurance plan or program must be revised to achieve full compliance with ANS-15.8.
The facility is vulnerable to events occurring rather than being identified by regular audits/assessments at NCNR.
With no independent audits/assessments, the regulator has less confidence that issues are being identified before an event occurs.
Management had not received unbiased assessments of the operation of the facility with respect to safety, quality, and regulatory compliance.
AFI-05: NCNR does not have a single source entry for PI&R reporting. The consequences of multiple distributed safety, trouble and work request platforms affect the ability to ensure CAQ/SCAQ are corrected, the ability to trend performance, and as indicated by interviews - is confusing for staff as to which platform to use.
The facility has multiple systems/programs where staff can report problems and deficiencies.
NCNR PI-R Process Assessment (7/7/23) - 2023-MA-06 also noted that multiple input systems challenge users.
CAQ and SCAQ items are known to be in those multiple systems and not in CAP.
Multiple systems are confusing to the NCNR staff at all levels and is an inefficient use of resources.
The multiple systems are not effectively monitored by facility management and issues are not resolved in a timely manner.
Multiple systems make trend analysis more time consuming (inefficient) and difficult to perform.
AFI-06: The facility is lacking Operability and Reportability guidance/procedures.
Operability and reportability issues are not consistently managed at the facility.
NCNR 2023-MA PI-R Process Assessment noted in a finding that operability guidance is still in draft.
AFI-07: The facility inappropriately changed Chapter 12 of the SAR (NCNR license basis).
ECN 885 used a 10CFR 50.59 screening instead of a full 10CFR50.59 Evaluation.
Excluding screenings of 10CFR50.59s from the annual report in 2016 removed many of the changes to documents at NCNR. This did not allow the regulator to understand the implication of changes.
This change was noted to be administrative and clearly was not.
This change removed the governance for the RTR industry from SAR.
NCNR was left with no regulatory structure and the regulator was not cognizant of the change.
39 AFI-08: While the facility can be operated safely, there is insufficient staff at NCNR to complete the added duties of implementing and maintaining PI&R, as defined in IP 71152.
This feedback came from all levels throughout the organization in interviews.
4.2 RECOMMENDATIONS It is strongly encouraged that the following recommenda;ons be ins;tuted using the assistance of outside consultants with experience in the speci"ed areas.
- 1. The PI&R Assessment Team was tasked to develop a Benchmarking Program as part of the SOW. Through the NCNR procedure approval process, review/approve the Benchmarking Planning Program provided by the PIR Assessment team in the associated Task 3 - PI&R Assessment Report Checklist Details document (Attachment P).
- 2. Using the new Benchmarking Program, write a benchmarking plan to develop and implement a Work Management Program, a Generic Communication Program, and the following PI&R programs:
- a. Self-Assessment Program.
- b. Cause Analysis (Simple, Apparent, and Root).
- i. This will be part of CAP Procedures.
ii. TVA, Entergy, and INPO have publicly available cause analysis manuals.
- c. Trending Program.
- d. Operating Experience Program.
- e. CAP (at a minimum) Metrics /Performance Indicators Program.
- 3. In addition to CAP and Benchmarking, develop training for each of the PI&R Programs in recommendation 2 above. The training should be developed using the Systematic Approach to Training/ Analyze, Design, Develop, Implement, and Evaluate (SAT/ADDIE).
- 4. Revise the NCNR CAP procedures by consolidating them into either one procedure or a policy, guidance, procedure hierarchy. The revision shall include elements described in INPO 14-004 - Conduct of Performance Improvement, DOE G 414.1-5 3-2 Corrective Action Program Guide, and NEI 16-07 (Rev. A) - Improving the Effectiveness of Issue Resolution to Enhance Safety and Efficiency.
- 5. Resolve entry requirements into CAP at NCNR. Ensure that Conditions Adverse to Quality (CAQ) and Significant CAQs (SCAQ) are either directly entered into CAP or evaluated and documented into CAP in a timely manner.
- a. This could be accomplished by:
- i. Create a single entry system (CAP) for all deficiencies (Safety, Equipment, performance, etc.) and screen out to the NIST Specific Systems; or, ii. Provide clear procedural guidance on when to document a condition into a NCNR system other than CAP. This would require monitoring of the non-CAP systems for CAQs, SCAQs, and a methodology of Operability/Reportability Determination and documentation.
- b. These options or a combination of items need further analysis. Analysis includes:
40
- i. Determine ease and effectiveness to the user entry and retrieval of data, the ability to monitor all the other non-CAP systems for CAQ/SCAQ entry without a corresponding CAP entry.
ii. The likelihood of NIST in consolidating Safety Management Systems.
iii. Other work identification.
iv. Control systems to adequately support NCNR Operations.
- 6. Establish CAP into a software platform designed to manage CAP that will help NCNR manage the process. The process shall have the ability to (at a minimum) allow all users to enter issues into CAP, allow users to attach documents to the platform, allow users and management the ability to search issues/CAPs, be a platform that shows all corrective action/action items for each CAP issue, address operability/reportability issues, allow management to track and trend issues, can tie to other systems currently in place at NCNR, allows administrative personnel to apply trend/cause/special codes, allow management the ability to screen issues (Issue Level/Assignment), allow management the ability close issues, and allow management to review both open and closed items.
- 7. Restore the NBSR Chapter 12 of the SAR (Licensing Basis). The revision will include re-establishment of ANS 15.8 as the governing guidance.
- 8. Revise the NBSR Quality Assurance Program to align with the ANSI/ANS 15.8 requirements, which includes:
- a. Establishment of a QA Plan/Program
- b. Independent audits/assessments including SAC/SEC).
- c. QA Manager reporting to someone at NIST (in oversight of NCNR) and a dashed line to the facility senior director.
- 9. Develop Operability & Reportability Guidance, which includes:
- a. Defined roles/responsibilities.
- b. Is in accordance with regulatory requirements.
- c. Training (SAT/ADDIE).
- 10. Institute a NBSR Work Management Program. The program shall include:
- a. A program reflecting current industry terms and standards.
- b. Training (SAT/ADDIE).
- 11. Assess the following systems where CAQ/SCAQ items could be hidden - Systems include but are not limited to: Trouble-Ticket [TT], NIST Maintenance Slips, Corporate Observations, IRIS, Reactor Aging Management, Engineering Change Request [ECR],
CATS, WIRS, N-ticket (Sales Force), Corrective Action Request [CAR], Engineering Non-Conformance Reports, System Review Teams, and Control Room Turnover Logs.
- b. Determine if any of the systems could be closed to CAP.
- 12. Determine interim external resources needed to accomplish the above actions. Acquire the resources in a timely manner to begin the work.
In addition to the PI&R Assessment, the Task 3 team also assessed NCNRs performance toward completion of major milestones as identified in the CO. In review of major milestones in the CO, the team provides the following:
41 Event Date Task 03 PI&R Activity Award 05/08/2023 Draft PI&R Assessment Plan 06/02/2023 Final PI&R Assessment Plan 06/19/2023 Onsite Entrance Meeting: PI&R Assessment 06/20/2023 Onsite Assessment 06/20/2023 - 06/26/2023 Exit Briefing 06/26/2023 Collegial Review Meeting Conducted with NCNR 11/15/2023 Draft Assessment Report to NCNR for Comment 11/20/2023 Reconcile NCNR Comments 12/04/2023 - 12/07/2023 Final Assessment Report Delivered 12/08/2023 5.0 RELATED DOCUMENTS The following documents are associated with the PI&R assessment completed at NCNR:
- 1. Task 3 - PI&R Assessment Report Checklist Details