ML17347B685
| ML17347B685 | |
| Person / Time | |
|---|---|
| Issue date: | 12/20/2017 |
| From: | Burke J, Todd Jackson, Kerri Kavanagh NRC/NRO/DCIP/QVIB3 |
| To: | Mcginty T Division of Construction Inspection and Operational Programs |
| Sakadales A | |
| References | |
| Download: ML17347B685 (10) | |
Text
December 20, 2017 MEMORANDUM TO: Timothy J. McGinty, Director Division of Construction Inspection and Operational Programs Office of New Reactors FROM: Terry W. Jackson, Chief /RA/
Quality Assurance Vendor Inspection Branch-1 Division of Construction Inspection and Operational Programs Office of New Reactors John P. Burke, Chief /RA/
Quality Assurance Vendor Inspection Branch-2 Division of Construction Inspection and Operational Programs Office of New Reactors Kerri A. Kavanagh, Chief /RA/
Quality Assurance Vendor Inspection Branch-3 Division of Construction Inspection and Operational Programs Office of New Reactors
SUBJECT:
VENDOR INSPECTION PROGRAM ANNUAL SELF-ASSESSMENT REPORT FOR FISCAL YEAR 2017 The Vendor Inspection Program (VIP) verifies that reactor applicants and licensees are fulfilling their regulatory obligations with respect to providing effective oversight of the supply chain. It accomplishes this through a number of activities, including: performing vendor inspections that will verify the effective implementation of the vendors quality assurance program, establishing a strategy for vendor identification and selection criteria, and ensuring vendor inspectors obtain necessary knowledge and skills to perform inspections. In addition, the VIP addresses interactions with nuclear consensus standards organizations, industry and external stakeholders, and international constituents.
The VIP also includes objectives and associated performance metrics to demonstrate that the overarching goals are being supported. The VIP performance metrics are assessed to ensure successful implementation and continuous improvement of the VIP. These performance metrics CONTACT: Antoinette Sakadales, NRO/DCIP 301-415-6441
T. McGinty 2 use objective measures and predetermined criteria to monitor the performance of the VIP as described in the Vendor Inspection Program Plan, Revision 17, dated September 2017 (Agencywide Documents Access and Management System Accession No. ML17220A144).
The performance metrics are based on input from various sources, including, but not limited to, inspection reports and vendor inspection staff comments. The vendor inspection staff collects data quarterly and uses pre-established success criteria to analyze the data. In most cases, success is defined as a steady or improving trend and achieving the goals of the program performance metrics. Performance metrics have been established for each of the following VIP objectives:
- VIP O-1: Verify that applicants and licensees are fulfilling their regulatory obligations with respect to providing effective oversight of the supply chain for operating reactors and reactor design and construction through a strategic sample of vendor inspections.
- VIP O-2: Effectively communicate with internal and external stakeholders.
- VIP O-3: Perform timely and adequate allegation follow up and closure.
- VIP O-4: Ensure that vendor inspectors have the necessary knowledge and skills to successfully implement the VIP.
Each objective has a set of performance metrics associated with it in order to establish the overall success of the VIP.
The results of the vendor inspection staff analysis are enclosed. The vendor inspection staff found that for fiscal year 2017, the VIP met 10 out of 11 performance metrics by meeting the criteria defined in Appendix D, Vendor Inspection Program Performance Metrics, to the Vendor Inspection Program Plan. The performance metric not met involved the timely release of acknowledgement letters, and it has been reviewed and evaluated by management and corrective actions were established. See Page 6 of the assessment which identifies the root cause and corrective actions.
Enclosure:
Vendor Inspection Program Performance Metrics
T. McGinty
SUBJECT:
VENDOR INSPECTION PROGRAM ANNUAL SELF-ASSESSMENT REPORT FOR FISCAL YEAR 2017 DATED December 20, 2017 DISTRIBUTION NRO_DCIP_QVIB-1 Distribution NRO_DCIP_QVIB-2 Distribution NRO_DCIP_QVIB-3 Distribution ADAMS Accession No.: ML17347B685 *via email NRO-002 OFFICE NRO/DCIP/QVIB-2 NRO/DCIP/QVIB-3 NRO/DCIP/QVIB-3 NRO/DCIP/QVIB-2 NAME YDiaz-Castillo ASakadales* KKavanagh* JBurke*
DATE 12/13/2017 12/13/2017 12/12/2017 12/12/2017 OFFICE NRO/DCIP/QVIB-1 NRO/DCIP NAME TJackson* TMcGinty DATE 12/12/2017 12/20/2017 OFFICIAL RECORY COPY
VENDOR INSPECTION PROGRAM PERFORMANCE METRICS VIP-O-1A Accomplish DCIPs (Division of Construction Inspection and Operational Programs) Established Number of Inspections per Fiscal Year1 Definition: Accomplish DCIPs established number of inspections per fiscal year to capture a reasonable perspective of industry performance (per NUREG-1100, Performance Budget/Congressional Budget Justification).
Criteria: Expect DCIP to perform the required number of inspections established at the beginning of the fiscal year (FY).
Goals: Effective, Open FY 2017 Inspections Region II Assists 2
NUPIC Observation 3
MDEP 1
AIA 2
OI Assissts 1
QA Implementation 2 Vendor Inspections 26 Analysis: DCIPs Operating Plan directed the vendor inspection staff to perform a minimum of 35 inspections during FY 2017. The vendor inspection staff completed a total of 37 inspections, including 26 vendor inspections, one Office of Investigation (OI) assist, two Region II assists, two aircraft impact assessment (AIA) inspections, three Nuclear Procurement Issues Committee (NUPIC) observations, two quality assurance (QA) implementation inspections, and one Multinational Design Evaluation Program (MDEP) inspection during FY 2017.
Metric Criteria Met: Yes 1
This metric is counted based on the FY of the entrance date of the inspection. Therefore, other metrics associated with this inspection could be captured in a different FYs assessment.
Enclosure
VIP-O-1B: Completion of Annual Assessment of the Number of Notices of Violations (NOV) and Notices of Nonconformance (NON)2 Definition: Perform an annual assessment of NONs and NOVs to identify generic industry issues and take corrective actions as necessary. Corrective actions may include discussions at the vendor workshop, issuance of generic communications, and other activities.
Criteria: Expect industry attendance at vendor workshops and industry outreach meetings and through generic communications.
Goals: Objective, Open, Risk-Informed Criterion X Criterion IV 2.13% Criterion XIII 2.13%
2.13% FY 2017 NOVs and NONs Criterion XII 2.13% Criterion I 2.13%
Criterion V 4.26%
Criterion III Criterion IX 25.53%
4.26%
50.150 4.26%
Part 21 6.38%
Criterion XVIII 6.38%
Criterion VII 21.28%
Criterion XVI 17.02%
Analysis: The NRC issued a total of 47 NOVs and NONs against applicants and vendors during FY 2017. This equates to approximately 8.47 findings per 1000 inspection hours. There was a significant increase in the amount of NOVs and NONs from FY 2016 to FY 2017, which can be attributed to the following reasons: (1) more inspections were performed; (2) one NOV and seven NONs were issued to one vendor (including a Criterion I finding); and (3) inspections were performed at 2
This metric is counted based on the FY of when the inspection report is issued. Therefore, other metrics associated with this inspection could be captured in a different FYs assessment.
vendors who had never been inspected by the NRC or have not been inspected by the NRC for a long period of time. There has been a significant increase of NONs in the area of Criterion VII, Control of Purchased Material, Equipment, and Services. The vendor inspection staff noted a lack of adequate review and verification in the control and oversight of sub-suppliers. NRC will address these findings at the next NUPIC vendor meeting and the NRC vendor workshop in June 2018. At these meetings, the NRC will discuss the importance of control of purchased material, equipment, and services and address the common areas that were deficient. In addition, the vendor inspection staff is developing a Regulatory Issue Summary to inform our stakeholders of the recent issues identified with the lack of adequate oversight of sub-suppliers.
Metric Criteria Met: Yes VIP-O-2A Obtain Feedback from Vendors During Vendor Inspections3 Definition: Branch chiefs and senior staff members will solicit feedback from vendors at the inspection site on such aspects as the immediate inspection effort, vendor oversight, or NRC requirements and guidance.
Criteria: Expect stable or increasingly positive perception over time.
Goals: Effective, Open, Understandable Analysis: This metric requires that the vendor branch chiefs and senior staff members, as applicable, solicit feedback from the vendor management on such aspects as the immediate guidance effort, vendor oversight, or NRC requirements and guidance. The NRC received positive feedback from the vendors on the thoroughness of the inspections and the professionalism of the staff.
Metric Criteria Met: Yes VIP-O-2B Notification of Inspection4 Definition: Obtain data on the total number of inspections that were notified to the vendor within the timeliness goals stipulated in Section 10 of this plan.
Criteria: Expect 90 percent of inspections to be announced to the vendor within the Vendor Inspection Program (VIP) timeliness goals.
Goals: Effective, Open, Predictable Analysis: The metric for notifying the vendors of an NRC inspection is no less than 30 calendar days from the start date of the inspection unless the inspection is unannounced. In cases where the need for the inspection arises (i.e., allegation, 3
This metric is counted based on the FY of the exit date of the inspection. Therefore, other metrics associated with this inspection could be captured in a different FYs assessment.
4 This metric is counted based on the FY of the first announcement of the inspection to the vendor if applicable. Therefore, other metrics associated with this inspection could be captured in a different FYs assessment.
testing schedule, and/or specific vendor activity) less than 35 days before the inspection start date, issue the Announcement Letter within 5 calendar days of being notified of the need for the inspection. In cases where the need for the inspection arises less than 5 calendar days, issue the Announcement Letter prior to inspection. Out of 29 notifications of inspection completed in FY 2017, the vendor inspection staff announced all within the VIP timeliness goal, which represents a score of 100%.
Metric Criteria Met: Yes VIP-O-2C Inspection Plans are Timely5 Definition: Obtain data on the total number of inspection plans issued within the timeliness goals stipulated in Section 10 of this plan.
Criteria: Expect 90 percent of inspection plans to be issued within the VIP timeliness goals.
Goals: Effective, Open, Predictable Analysis: The metric for issuing inspection plans is no less than seven calendar days from the start date of the inspection. In cases where the need for the inspection arises (i.e., allegation, testing schedule, and/or specific vendor activity) less than seven calendar days before the inspection start date, issue the inspection plan prior to the inspection. Out of 31 inspection plans completed in FY 2017, the vendor inspection staff completed 30 within the VIP timeliness goal, which represents a score of 96.77%.
Metric Criteria Met: Yes VIP-O-2D Inspection Reports are Timely6 Definition: Obtain data on the total number of inspection reports issued within the timeliness goals stipulated in Section 10 of this plan and Inspection Manual Chapter 0617, Vendor and Quality Assurance Implementation Inspection Reports, dated August 2017.
Criteria: Expect 90 percent of inspection reports to be issued within the VIP timeliness goals.
Goals: Effective, Open, Predictable Analysis: The metric for issuing inspection reports is no later than 45 calendar days after the exit meeting, extended until the next business day if the 45 days end on a weekend or Holiday. Out of 36 inspection reports completed in FY 2017, the 5
This metric is based on the FY of when the inspection was announced to the vendor. Therefore, other metrics associated with this inspection could be captured in a different FYs assessment.
6 This metric is based on the FY of when the inspection report was issued to the vendor. Therefore, other metrics associated with this inspection could be captured in a different FYs assessment.
vendor inspection staff completed 36 inspections reports within the VIP timeliness goals, which represents a score of 100%.
Metric Criteria Met: Yes VIP-O-2E Acknowledgment Letters Are Timely7 Definition: Obtain data on the total number of acknowledgement letters issued within the timeliness goals stipulated in Section 10 of this plan.7 Criteria: Expect 90 percent of acknowledgement letters to be issued within the VIP timeliness goals.
Goals: Effective, Open, Predictable Analysis: The metric for issuing acknowledgement letters is no later than 30 calendar days after the vendors last communication is entered into Agencywide Documents Access and Management System (ADAMS). Out of the 18 vendor responses received in FY 2017, the vendor inspection staff issued 16 acknowledgement letters within the VIP timeliness goals, which represents a score of 88.89 percent.
Metric Criteria Met: No Corrective Actions: The vendor inspection staff noted that the root cause for the late issuance of acknowledgement letters was due to the following reasons:
(1) In the first case, the vendors response letter was not submitted to ADAMS in a timely manner. Once the lead inspector was notified that the letter was in ADAMS, he needed time to review the corrective actions.
We have since updated the VIP Plan to allow the vendor inspectors 30 days from the date the response letter has been added into ADAMS to issue the acknowledgement letter.
(2) In the second case, the lead inspector assigned to this task took on another position. Once the inspection was reassigned, the new lead needed time to familiarize himself before accepting the corrective actions.
VIP-O-2F Inspection Results Accepted by Stakeholders Definition: Track the total number of NOVs and NONs contested by vendors.
Criteria: Retract less than 10 percent of NOVs and NONs because they are successfully contested by the stakeholders.
Goals: Effective, Objective, Open, Predictable 7
This metric is based on the date that the vendors correction actions are received by ADAMS and the date that the staff acknowledges receipt of the vendors corrective actions. Therefore, other metrics associated with this inspection could be captured in a different FYs assessment.
Analysis: There were four contested NONs by stakeholders during FY 2017. After the NRC conducted an independent review, none of the findings were retracted.
Therefore, no NONs or NOVs were retracted during FY 2017.
Metric Criteria Met: Yes VIP-O-3 Allegation Support Definition: Achieve the timely submittal of allegation response documents.
Criteria: Conduct all support within the Allegation program timeliness goals. Support includes, but is not limited to, providing input for Allegation Review Board materials, attending Allegation Review Boards (ARBs), providing input to Requests for Information, participating in phone calls with the Concerned Individual(s), providing input for closure and response after closure letters, etc.
Goals: Effective, Objective, Risk-Informed Analysis: The vendor inspection staff supported 62 allegation actions during FY 2017. Of those allegations, two resulted in reactive inspections based on the outcome of the ARB. All input provided to the Headquarters Allegation Team was submitted within the Allegation Programs timeliness requirements.
Metric Criteria Met: Yes VIP-O-4A Assessment of Trainee Qualifications Definition: Branch chiefs assess inspectors in training for progress in achieving qualifications at least quarterly.
Criteria: Expect 90 percent of trainees to qualify in two years.
Goals: Effective, Predictable, Understandable Analysis: In FY 2017, no vendor inspectors qualified. There were three trainees undergoing the qualification process at the end of FY 2017. They are all on track to qualify within the two-year qualification timeline.
Metric Criteria Met: Yes VIP-O-4B Assessment of Inspector Proficiency Definition: Maintain proficiency for all qualified inspectors.
Criteria: Maintain annual proficiency for all qualified inspectors in accordance with the guidance set forth by the VIP for refresher and continuing training.
Goals: Effective, Predictable, Understandable Analysis: All qualified vendor inspectors met the annual proficiency requirements as stated in Section 12 of the Vendor Inspection Program Plan, Revision 17, dated September 2017 and Section C-8 of Appendix D1 to Inspection Manual Chapter 1245, Maintaining Qualifications, dated December 2011.
Metric Criteria Met: Yes