ML23290A222
| ML23290A222 | |
| Person / Time | |
|---|---|
| Issue date: | 11/20/2023 |
| From: | Kerri Kavanagh NRC/NRR/DRO/IQVB |
| To: | Russell Felts NRC/NRR/DRO |
| References | |
| Download: ML23290A222 (1) | |
Text
MEMORANDUM TO:
Russell Felts, Director Division of Reactor Oversight Office of Nuclear Reactor Regulation FROM:
Kerri A. Kavanagh, Chief Quality Assurance and Vendor Inspection Branch Division of Reactor Oversight Office of Nuclear Reactor Regulation
SUBJECT:
VENDOR INSPECTION PROGRAM ANNUAL SELF-ASSESSMENT REPORT FOR FISCAL YEAR 2023 The Vendor Inspection Program (VIP) verifies that reactor applicants and licensees are fulfilling their regulatory obligations with respect to providing effective oversight of the nuclear supply chain. It accomplishes this through limited scope, targeted inspections of vendor quality assurance programs, communicating relevant vendor information to stakeholders, and supporting allegation response activities. In addition, the VIP addresses interactions with nuclear consensus standards organizations, industry and external stakeholders, and international constituents.
The VIP also includes objectives and associated performance metrics to demonstrate that the overarching goals are being supported. The VIP performance metrics are assessed to ensure successful implementation and continuous improvement of the VIP. These performance metrics use objective measures and predetermined criteria to monitor the performance of the VIP as described in the Vendor Inspection Program Plan, Revision 29, dated September 2023 (Agencywide Documents Access and Management System Accession No. ML23263B066).
The performance metrics are based on input from various sources, including, but not limited to, inspection reports and vendor inspection staff comments. The vendor inspection staff collects data quarterly and uses pre-established success criteria to analyze the data. In most cases, success is defined as a steady or improving trend and achieving the goals of the program performance metrics. Performance metrics have been established for each of the following VIP objectives:
Enclosure:
Vendor Inspection Program Performance Metrics CONTACT: Ada Bowie, NRR/DRO/IQVB 301-287-9413 November 20, 2023 Signed by Kavanagh, Kerri on 11/20/23
R. Felts 2
VIP O-1: Verify that applicants and licensees are fulfilling their regulatory obligations with respect to providing effective oversight of the supply chain for operating reactors and reactor design and construction through a strategic sample of vendor inspections.
VIP O-2: Effectively communicate with internal and external stakeholders.
VIP O-3: Perform timely and adequate allegation follow up and closure.
VIP O-4: Ensure that vendor inspectors have the necessary knowledge and skills to successfully implement the VIP.
Each objective has a set of performance metrics associated with it in order to establish the overall success of the VIP.
The results of the vendor inspection staff analysis are enclosed. The vendor inspection staff found that for fiscal year 2023, the VIP met 10 out of 10 performance metrics by meeting the criteria defined in Appendix D, Vendor Inspection Program Performance Metrics, to the Vendor Inspection Program Plan.
R. Felts
SUBJECT:
VENDOR INSPECTION PROGRAM ANNUAL SELF-ASSESSMENT REPORT FOR FISCAL YEAR 2023 Dated: November 20, 2023 DISTRIBUTION NRR_DRO_IQVB RFelts PMcKenna AZoulis ADAMS Accession No.: ML
- via email NRR-106 OFFICE NRR/DRO/IQVB NRR/DRO/IQVB NRR/DRO/IQVB NAME MFitzgerald YLaw ABowie DATE 10/17/2023 10/23/2023 10/20/2023 OFFICE NRR/DRO/IQVB NRR/DRO/IQVB NAME YDiaz-Castillo KKavanagh DATE 10/23/2023 11/20/2023 OFFICIAL RECORY COPY
Enclosure VENDOR INSPECTION PROGRAM PERFORMANCE METRICS VIP-O-1A Accomplish DROs (Division of Reactor Oversight) Established Number of Inspections per Fiscal Year1 Definition:
Accomplish DROs established number of inspections per fiscal year (FY) to capture a reasonable perspective of industry performance (per NUREG-1100, Performance Budget/Congressional Budget Justification).
Criteria:
Expect DRO to perform the required number of inspections established at the beginning of the FY.
Goals:
Effective, Open Analysis:
DROs Operating Plan directed the vendor inspection staff to perform a minimum of 20 inspections during FY 2023. In FY 2021, the DRO metric was modified to include licensing audits, observations, and regional inspection support to the definition of inspections. As such, the vendor inspection staff completed a total of 22 inspections, audits, and observations. These include 18 vendor inspections, one audit, and three observations of Nuclear Procurement Issues Corporation (NUPIC) audits. Therefore, this represents a score of 100%.
Metric Criteria Met: Yes 1 This metric is counted based on the FY of the entrance date of the inspection. Therefore, other metrics associated with this inspection could be captured in a different FYs assessment.
Vendor Inspections Audits Observations FY 2023 Inspections VIP-O-1B:
Completion of Annual Assessment of the Number of Notices of Violations (NOV) and Notices of Nonconformance (NON)2 Definition:
Perform an annual assessment of NONs and NOVs to identify generic industry issues and take corrective actions as necessary. Corrective actions may include discussions at the vendor oversight workshop, issuance of generic communications, and other activities.
Criteria:
Expect industry attendance at vendor workshops and industry outreach meetings and through generic communications.
Goals:
Objective, Open, Risk-Informed Analysis:
The Nuclear Regulatory Commission (NRC) issued twelve nonconformances and four violations against eight vendors during FY 2023. The increase in the total number of NONs and NOVs issued from FY 2022 to FY 2023 are mainly in the areas of corrective actions, Part 21 and supplier oversight. Two vendor inspections account for four violations and three nonconformances. The Quality Assurance and Vendor Inspection Branch (IQVB) staff will focus on these areas during future inspections and stress the importance of adequately implementing corrective action and Part 21 programs to vendors. In the area of supplier oversight, the IQVB staff will focus on the areas of commercial grade dedication and supplier audits. Therefore, this represents a score of 100%.
Metric Criteria Met: Yes 2 This metric is counted based on the FY of when the inspection report is issued. Therefore, other metrics associated with this inspection could be captured in a different FYs assessment.
Criterion I 6.25%
Criterion II 6.25%
Criterion III 6.25%
Criterion IV 6.25%
Criterion VII 6.25%
Criterion VIII 6.25%
Criterion XVI 31.25%
Criterion XVIII 6.25%
Part 21 25.00%
VIP-O-2A Obtain Feedback from Vendors After the Vendor Inspection3 Definition:
The IQVB Branch Chief will solicit feedback from vendors on such aspects as the immediate inspection effort, vendor oversight, or NRC requirements and guidance.
Criteria:
Expect stable or increasingly positive perception over time.
Goals:
Effective, Open, Understandable Analysis:
This metric requires that the IQVB Branch Chief solicit feedback from the vendors management on such aspects as the immediate inspection effort, vendor oversight, or NRC requirements and guidance. In FY 2023, the IQVB Branch Chief solicited feedback from 13 vendors. Out of the 13, eight vendor responses were received. All responses, except one response, were positive.
This feedback was discussed with the vendor inspection staff. Feedback was received regarding the involvement of remote NRC staff members that led to the development of a desktop guide on the topic. The positive feedback received reflected the thoroughness of the inspections and the professionalism of the NRC staff. The discrepancy between the number of feedbacks sent to vendors and the number of inspections performed within a fiscal year is because when an inspection is performed at the end of a fiscal year, the feedback is sent and counted in the next fiscal year. Therefore, this represents a score of 100%.
Metric Criteria Met: Yes VIP-O-2B Notification of Inspection4 Definition:
Obtain data on the total number of inspections that were notified to the vendor within the timeliness goals stipulated in Section 10 of the Vendor Inspection Program (VIP) plan.
Criteria:
Expect ninety percent of inspections to be announced to the vendor within the VIP timeliness goals.
Goals:
Effective, Open, Predictable Analysis:
The metric for notifying the vendors of an NRC inspection is no less than 30 calendar days from the start date of the inspection unless the inspection is unannounced. In cases where the need for the inspection arises (i.e., allegation, testing schedule, and/or specific vendor activity) less than 35 days before the inspection start date, the Announcement Letter should be issued within five calendar days to the vendor of the need for the inspection. In cases where the need for the inspection arises less than five calendar days, the Announcement Letter should be issued prior to the inspection. Out of the 16 notifications of 3 This metric is counted based on the FY of the exit date of the inspection. Therefore, other metrics associated with this inspection could be captured in a different FYs assessment.
4 This metric is counted based on the FY of the first announcement of the inspection to the vendor if applicable. Therefore, other metrics associated with this inspection could be captured in a different FYs assessment.
inspection completed in FY 2023, 15 were within the VIP timeliness goal, which represents a score of 92.9%. One inspection was scheduled with less than 45 days notice, and thus the inspection announcement metric was unable to be met.
Although 18 total inspections were conducted in FY 2023, four announcement letters were captured in the FY 2022 metrics. Two Announcement Letters pertained to inspections will be conducted in FY 2024.
Metric Criteria Met: Yes VIP-O-2C Inspection Plans are Timely5 Definition:
Obtain data on the total number of inspection plans issued within the timeliness goals stipulated in Section 10 of the VIP plan.
Criteria:
Expect ninety percent of inspection plans to be issued within the VIP timeliness goals.
Goals:
Effective, Open, Predictable Analysis:
The metric for issuing inspection plans is no less than seven calendar days from the start date of the inspection. In cases where the need for the inspection arises (i.e., allegation, testing schedule, and/or specific vendor activity) less than seven calendar days before the inspection start date, the inspection plan should be issued prior to the inspection. Out of 18 inspection plans completed in FY 2023, 17 were completed within the VIP timeliness goal, which represents a score of 94.4%. Although a total of 18 inspections were conducted in FY 2023, no inspection plans were captured in the FY 2022 metrics. Two previously announced inspections were either cancelled or postponed, and two inspection plans pertained to inspections will be conducted in FY 2024.
Metric Criteria Met: Yes VIP-O-2D Inspection Reports are Timely6 Definition:
Obtain data on the total number of inspection reports issued within the timeliness goals stipulated in Section 10 of the VIP plan and Inspection Manual Chapter 0617, Vendor and Quality Assurance Implementation Inspection Reports, dated February 2020.
Criteria:
Expect ninety percent of inspection reports to be issued within the VIP timeliness goals.
Goals:
Effective, Open, Predictable 5 This metric is based on the FY of when the inspection was announced to the vendor. Therefore, other metrics associated with this inspection could be captured in a different FYs assessment.
6 This metric is based on the FY of when the inspection report was issued to the vendor. Therefore, other metrics associated with this inspection could be captured in a different FYs assessment.
Analysis:
The metric for issuing inspection reports is no later than 45 calendar days after the exit meeting, extended until the next business day if the 45 days end on a weekend or a Holiday. Out of the 16 inspection reports issued in FY 2023, 15 were completed within the VIP timeliness goal, which represents a score of 92.9%. One inspection report involved escalated enforcement actions and required additional time for issuance. Although 18 total inspections were conducted in FY 2023, it should be noted that two inspection reports were issued for inspections that were captured in the FY 2022 metrics and the NRC does not issue inspection reports for audits, or observations. In addition, two inspection reports will be captured in the FY 2024 metrics.
Metric Criteria Met: Yes VIP-O-2E Acknowledgment Letters Are Timely7 Definition:
Obtain data on the total number of acknowledgement letters issued within the timeliness goals stipulated in Section 10 of this plan.7 Criteria:
Expect ninety percent of acknowledgement letters to be issued within the VIP timeliness goals.
Goals:
Effective, Open, Predictable Analysis:
The metric for issuing acknowledgement letters is no later than 30 calendar days after the vendors last communication is entered into the Agencywide Documents Access and Management System (ADAMS). Out of the seven vendor NON and NOV responses received in FY 2023, all acknowledgement letters were issued within the VIP timeliness goal, which represents a score of 100%.
Metric Criteria Met: Yes VIP-O-2F Inspection Results Accepted by Stakeholders Definition:
Track the total number of NOVs and NONs contested by vendors.
Criteria:
Retract less than ten percent of NOVs and NONs because they are successfully contested by the stakeholders.
Goals:
Effective, Objective, Open, Predictable Analysis:
There were no contested NONs or NOVs by a stakeholder during FY 2023, which represents a score of 100%.
Metric Criteria Met: Yes VIP-O-3 Allegation Support 7 This metric is based on the date that the vendors correction actions are received by ADAMS and the date that the staff acknowledges receipt of the vendors corrective actions. Therefore, other metrics associated with this inspection could be captured in a different FYs assessment.
Definition:
Achieve the timely submittal of allegation response documents.
Criteria:
Conduct all support within the Allegation Program timeliness requirements.
Support includes, but is not limited to, providing input for Allegation Review Board (ARBs) materials, attending ARBs, providing input to requests for information, participating in phone calls with the Concerned Individual(s),
providing input for closure and response after closure letters, etc.
Goals:
Effective, Objective, Risk-Informed Analysis:
The vendor inspection staff supported 47 allegation actions during FY 2023. Of those allegations, one resulted in a reactive inspection based on the outcome of the ARB. All input provided to the Headquarters Allegation Team was submitted within the Allegation Programs timeliness requirements, which represents a score of 100%.
Metric Criteria Met: Yes VIP-O-4A Assessment of Trainee Qualifications Definition:
Branch Chiefs assess inspectors in training for progress in achieving qualifications at least quarterly.
Criteria:
Expect ninety percent of trainees to qualify in two years.
Goals:
Effective, Predictable, Understandable Analysis:
In FY 2023, one vendor inspector completed their qualifications within the two-year qualification timeline, which represents a score of 100%.
Metric Criteria Met: Yes VIP-O-4B Assessment of Inspector Proficiency Definition:
Maintain proficiency for all qualified inspectors.
Criteria:
Maintain annual proficiency for all qualified inspectors in accordance with the guidance set forth by the VIP for refresher and continuing training.
Goals:
Effective, Predictable, Understandable Analysis:
All qualified vendor inspectors met the annual proficiency requirements as stated in Section 12 of the VIP plan and Section C-8 of Appendix D1 to Inspection Manual Chapter 1245, Maintaining Qualifications, dated June 2020. Therefore, this represents a score of 100%.
Metric Criteria Met: Yes