ML21063A123
| ML21063A123 | |
| Person / Time | |
|---|---|
| Site: | 99902075 |
| Issue date: | 03/18/2021 |
| From: | O'Banion M Licensing Processes Branch |
| To: | Hashemian H Analysis & Measurement Services Corp |
| O'Banion M, NRR/DORL/LLLPB, 415-1233 | |
| Shared Package | |
| ML21063A121 | List: |
| References | |
| EPID L-2020-TOP-0037 | |
| Download: ML21063A123 (12) | |
Text
March 18, 2021 H.M. Hashemian, Ph.D.
President Analysis and Measurement Services Corporation 9119 Cross Park Drive Knoxville, TN 37923
SUBJECT:
REGULATORY AUDIT REPORT FOR JANUARY 26 TO FEBRUARY 24, 2021, AUDIT OF AMS-TR0720R1, ONLINE MONITORING TECHNOLOGY TO EXTEND CALIBRATION INTERVALS OF NUCLEAR PLANT PRESSURE TRANSMITTERS (EPID L-2020-TOP-0037)
Dear Dr. Hashemian:
By letter dated July 10, 2020 (Agencywide Documents Access and Management System (ADAMS) Package Accession No. ML20231A208), as supplemented by letter dated October 9, 2020 (ADAMS Package Accession No. ML20317A111), Analysis and Measurement Services Corporation (AMS) submitted a topical report (TR) request regarding online monitoring technology to extend calibration intervals of nuclear plant pressure transmitters.
The U.S. Nuclear Regulatory Commission (NRC) staff conducted a virtual audit on January 26 to February 24, 2021, to ensure understanding of the TR and to discuss questions enclosed to the audit plan dated January 15, 2021 (ADAMS Accession No. ML21014A337), as supplemented by audit plan dated January 28, 2021 (ADAMS Accession No. ML21027A367).
The purpose of this letter is to provide AMS with the results of the regulatory audit.
Documented in the enclosed report are the observations the NRC staff identified during the audit.
If you have any questions or require any additional information, please feel free to contact me at 301-415-1233 or via electronic mail at Margaret.OBanion@nrc.gov.
Sincerely,
/RA/
Margaret W. OBanion, Project Manager Licensing Processes Branch Division of Operating Reactor Licensing Office of Nuclear Reactor Regulation Docket No. 99902075
Enclosure:
Audit Report
Enclosure REGULATORY AUDIT REPORT FOR JANUARY 26 TO FEBRUARY 24, 2021 AUDIT OF ANALYSIS AND MEASUREMENT SERVICES CORPORATION AMS-TR-0720R1, ONLINE MONITORING TECHNOLOGY TO EXTEND CALIBRATION INTERVALS OF NUCLEAR PLANT PRESSURE TRANSMITTERS EPID NO. L-2020-TOP-0037 BACKGROUND:
By letter dated July 10, 2020 (Agencywide Documents Access and Management System (ADAMS) Package Accession No. ML20231A208), as supplemented by letter dated October 9, 2020 (ADAMS Package Accession No. ML20317A111), Analysis and Measurement Services Corporation (AMS) submitted a topical report (TR) regarding online monitoring (OLM) technology to justify and support extended calibration intervals of nuclear plant pressure and differential pressure transmitters. AMS requested formal review of the TR in accordance with the U.S. Nuclear Regulatory Commission (NRC) TR program for review and acceptance for referencing in regulatory actions. In support of its effort to evaluate this TR, the NRC staff conducted a virtual audit to increase efficiency in the review by gaining a better understanding of the report concepts and bases underlying its conclusions. The staff has prepared this audit report to document the results of the audit.
The scope of this audit was defined in the audit plan which was provided to AMS on January 15, 2021 (ADAMS Accession No. ML21014A337), as supplemented by audit plan dated January 28, 2021 (ADAMS Accession No. ML21027A367). In addition, the NRC staff enclosed a set of open items (questions) it intended to pursue during the audit in the audit plan.
REGULATORY AUDIT BASES:
The following regulations and regulatory guidance constitute the basis for the TR evaluation, and is the primary focus for this audit:
Title 10 of the Code of Federal Regulations (10 CFR) 50.36, Technical Specifications, requires that technical specifications (TS) include surveillance requirements (SR); where SR are requirements relating to test, calibration, or inspection to assure that the necessary quality of systems and components is maintained, that facility operation will be within safety limits, and that the limiting conditions for operation will be met. The AMS TR seeks to change how SR are implemented.
REGULATORY AUDIT ACTIVITIES:
The regulatory audit was conducted in accordance with the Office of Nuclear Reactor Regulation (NRR) procedure described in LIC-111, Regulatory Audits (ADAMS Accession No. ML19226A274).
AMS supported the audit by making appropriate AMS and consultant staff available for discussions who have detailed knowledge of AMS-TR-0720R1, and the principles and bases for the OLM processes discussed in the TR. All the underlying source documents referred to in the TR were made available to the NRC staff for review during the audit period. In addition, the AMS staff made several presentations to the staff to help address the open items and to provide clarification of information within the TR or its referenced documents. Since the volume of referenced documents was so large, the NRC staff chose to extend the audit period from an
2 initial two-day period to a three-week period, to provide the NRC staff ample opportunity to examine the underlying basis documents. A formal request to extend the audit period was made by letter dated January 28, 2021 (ADAMS Accession No. ML21027A334).
PRESENTATIONS BY AMS:
AMS provided the following presentations during the audit:
AMS OLM Fundamentals Review and discuss AMS responses to NRC pre-audit questions Review and discuss AMS responses to two NRC questions developed during the audit Review and discuss AMS plans for responding to draft NRC staff requests for additional information LIST OF AMS RESOURCE DOCUMENTS MADE AVAILABLE FOR THE NRC STAFFS AUDIT REVIEW:
AMS made the following documents available for the NRC staffs review by placing them on an AMS SharePoint portal throughout the duration of the audit period. (Note: These documents are shown using the document file names as they appeared on the AMS SharePoint Portal):
Agenda - NRC Audit of Understanding (1/26/21 - 1/27/21)
AMS Presentation for NRC Audit 1/26/2021 NRC Audit Questions and AMS Responses 1/25/2021 AMS-TR-07201R1 Sensor Performance and Reliability Maintenance of Process Instrumentation in Nuclear Power Plants Maintenance Optimization Through Data Analytics Reactor Sensor Surveillance Sensor Response Time Monitoring Equipment Health Research Reactors OLM at ATR DOE_ER84626_Volume1 DOE_ER84626_Volume2 OLM at Vogtle 1 VOG1905R0 OLM at Vogtle 2 VOG1906R0 OLM at Vogtle 1 VOG2005R0 TR-104965-R1 NRC SER EPRI-TR-103436-V1 Guidelines for AFAL TR-103335-R1 Guidelines for AFAL 3002002556 Rev 02 Instrument Drift Study Sizewell Investigation of Response Time - NP - 7243 Instrument Cal Monitoring TR-103436-V2 OLM Watts Bar Unit 1 Final Report OLM at Sizewell 1013486 Volume 1 OLM and Sizewell 1013486 Volume 2 OLM Requirements 1016725 TR-104965-R1 NRC SER Uncertainty Quantification Techniques
3 IAEA Advanced Surveillance NP-T-3.14 PWROG WDS1601R2 (Generic Transmitter Drift Study)
Transmitter Single Calibration Regression Methodology TSTF-425 Rev 3 ML090850627 NEI 04-10 ML071360456 PWROG-15057-P IEC 62385 IAEA NP-T-1.1 IAEA-TECDOC-1920 IAEA TECDOC-1830 Equipment Health Research Reactors OLM at ATR ISA67_04_01_2018 SZB_OLM_Acceptance_Criteria NII Letter 2005 Sizewell Sensor Calibration Extension ESR-503 NUREG 6895 Volume 1 NUREG 5903 NUREG-6343 NUREG-sr1475r1 NUREG 5560 NUREG 5383 NUREG 5851 WCAP-13632-P-A CEOG NPSD-1167-A ML010240308 BWROG SER on NEDC-32013P VC Summer LAR ML060400220 NUREG-0800 Table 7-1 ML16020A103 AUDIT ACTIVITIES PERFORMED BY THE NRC STAFF:
Meetings Held During the Audit:
Audit Opening Meeting held January 26-27, 2021 Audit Progress & Touch Base Meeting held February 17, 2021 Audit Closure Meeting held February 24, 2021 Note: Refer to list of audit participants at the end of this summary As shown above, the NRC staff held three meetings with the staff of AMS and its consultants.
The first meeting was held on January 26 and 27, 2021, and consisted of presentations made by the staff of AMS regarding OLM principles and to address the list of 14 pre-audit questions raised by the NRC staff in its audit plan dated January 15, 2021. Additional discussions were held regarding the following topics:
Accounting for and distinguishing between uncertainties in the portions of the instrument channels in the loop monitored by the OLM equipment and the drift performance of the transmitter
4 Descriptions regarding OLM techniques within the TR that do not appear to be sufficient for licensees to prepare a license amendment request (LAR) documenting the specific methods they will use to accomplish the required TS SR Recommended methods for accomplishing a staged approach for initiating extended OLM calibration intervals for redundant transmitters serving the same safety function Characterizing the types of devices that are recommended for inclusion within an OLM program and if it is it limited to only pressure and differential pressure transmitters Discussion as to why common mode drift is not considered a concern over long periods of time between successive manual calibrations, when the output of the transmitters are only compared against a weighted average of the transmitters serving the same process function, and not being calibrated against a traceable standard Reasonable approaches for flagging a transmitter for manual calibration using OLM techniques, and evaluation of transmitters that experience significant drift but were not flagged for manual calibration through OLM techniques Identification and discussion of the results of various drift studies that have been performed by entities other than AMS which exhibit results similar to the AMS-conducted studies identified in the text of the TR Identifying appropriate criteria for establishing the sampling frequency for a specific transmitter function, taking into account the dynamics of the process being monitored TR includes recommendations for using OLM techniques for the purposes of satisfying response time testing commitments, when the primary purpose is to flag transmitters needing calibration due to detected excessive drift After the first meeting, the NRC staff began a three-week review of documents on the AMS SharePoint site to review evidence supporting the claims made in the AMS TR.
The second meeting was held on February 17, 2021, after the NRC staff had time to review many of the documents placed on the AMS SharePoint site. The discussions during the second meeting concerned the following topics:
What should be the appropriate scope of devices and functions to be included within an OLM program Sizewell B OLM experience and reasonings why Sizewell team included the scope of certain devices Explanation and reasons for the lack of evidence that common mode drift exists Savings in personnel dose and resources and significant reduction of personnel errors in calibration in using OLM, versus ability to accurately compare a transmitters off-line performance against known calibration standards Types of failure modes that are detectable using OLM techniques, and failure modes that are not detectable Example criteria to be used to establish optimal sampling frequency OLM TR content statements made or implied regarding the crediting of OLM for meeting licensee commitments for response time testing Example language and content a licensee would need to include in any LAR or application proposing the implementation of OLM techniques which references the AMS TR Content of example TS-required SR table entry mark-ups when adopting OLM techniques for surveillance interval extensions Comparison of the transmitter calibrated span against the portion of the span that may be observed during start-up, operations, and shutdown, and the typical unmonitored
5 span portion that exists between the highest operating point and the safety setpoint that is to be actuated by the transmitter Following the second meeting, the NRC staff internally reviewed the responses received from the AMS staff regarding its initial list of pre-audit questions, and conducted further reviews of the referenced documents on the SharePoint portal. The staff then identified areas where additional information should be requested from AMS, and areas where the staffs initial pre-audit questions appear to have been resolved. Based on this evaluation, the staff prepared draft requests for additional information (RAIs) and prepared for an audit closeout meeting.
During this time, AMS prepared a document, dated February 21, 2021, which responded to two of the NRC staffs questions raised during the audit, and placed these among the documents available for audit on the AMS SharePoint site.
Finally, the staff held an audit closure meeting with AMS and its consultants on February 24, 2021. Topics discussed during the closeout meeting included:
The need for additional information to be docketed as part of the staffs evaluation of the TR A presentation made by AMS staff regarding its responses to two of the staffs questions raised during the February 17, 2021, meeting, as documented in the February 21, 2021, document placed on the SharePoint site The staffs plan for issuing RAIs, and the staffs desired date for receiving the AMS responses to those RAIs A discussion of the key take-aways the NRC staff had as a result of the audit (outlined below in the Summary of Key Audit Question and Discussion Topics)
A discussion by AMS regarding its understanding of what are the NRC staffs needs for completing the evaluation of the TR, and how AMS plans to respond to those needs
SUMMARY
OF KEY AUDIT QUESTIONS AND DISCUSSION TOPICS RAISED DURING THE AUDIT; PROPOSED PLAN FORWARD:
Generic Scope of the Topical Report Criteria for Transmitter Inclusion into an OLM program The NRC staff noted that the empirical data provided within the TR to support the technical basis for AMS OLM processes were based on a specific and limited set of pressure and differential pressure transmitters. It will therefore be challenging to accept the OLM methods on a more generic basis for all pressure transmitters of any make or model, or for other types of transmitters, without having data to support inclusion of each type of transmitter to a plant OLM program. While it may not be feasible to provide such data during the TR evaluation, it could be possible to accept a method that contains provisions for collecting transmitter data prior to inclusion in the plants OLM program. Alternatively, it may be possible to characterize the types of devices (e.g., transmitters implementing solid state transducers and sensors, with electronic transmission circuitry equipped with process and ambient compensation or stabilization features, etc.) that are the best candidates for inclusion within an OLM program. Neither the TR nor the references in the audit documents library currently describe such a process. Also, in a response to another audit question, AMS representatives noted that within the Sizewell B data, the setpoints associated with the safety transmitters were all within 10 percent (%) of the portion of the transmitter span experienced under startup, normal operations, and shutdown conditions.
This may also be a prerequisite for consideration of a particular group of transmitters to be
6 tracked under the OLM program. The NRC staff considered preparing a formal RAI to request information regarding how a licensee proposing to implement an OLM program would identify and incorporate specific make, model, and range of transmitter information into the scope of the program, or to otherwise characterize the specific types of devices that would be likely candidates for success within an OLM program, to justify inclusion in the plant OLM program advocated within the TR. Ultimately, the staff decided to limit its evaluation to the specific scope of transmitters as described and presented in the TR. (No RAI contemplated at this time.)
Common Mode Drift Concerns Under OLM, the signal output of each transmitter is continually compared against a form of averaging of the outputs of all the transmitters within the same functional service grouping.
Under traditional TS-related calibration surveillances, which are performed at least once per refueling outage, each transmitter within each safety-related service function is calibrated (benchmarked) individually against a known calibration standard that is traceable to National Institute of Standards and Technology (NIST) standards. It appears that exclusive application of the OLM methodology could miss the possibility that all four transmitters in the same functional service could be experiencing a common mode drift (i.e., drift of similar magnitude and direction due to a common mode effect). The NRC staff is looking for information in the TR that provides justification for continued operation of OLM-monitored transmitters over several years without performing a manual calibration against a traceable standard of at least one out of a set of redundant transmitters within each function at least once per refueling outage. The TR provides anecdotal evidence for the Sizewell B and McGuire plants in which the results of manual calibrations are compared against conclusions drawn during OLM sampling as to whether a manual calibration is warranted. Also, some of the referenced documents in the TR that have been placed in the audit documents library provide a catalogue of data comparing OLM results against subsequent manual calibration results. The data shows that in a small number of instances, non-conservative drift of a transmitter was discovered during a manual calibration that had not been detected with the OLM processes implemented. There is minimal analysis of the anecdotal evidence, however, either within the TR or within the referenced documents which support a technical basis for concluding that undetected common mode drift is not of concern when implementing OLM processes. The NRC staff will prepare a RAI requesting AMS to provide a summary of its analysis which justifies continued long-term operation of OLM transmitters without a periodic calibration against a known standard of at least one transmitter within a functional service grouping, at a maximum period of time that can reasonably be justified.
Pressure Transmitter Failure Mode Detection using OLM Methods (Detecting issues with linearity, hysteresis, non-dynamic processes)
In the response to audit question 5, AMS stated that setpoints were determined to be within 10 percent of the operating levels for all transmitters in the OLM program. This criterion could be used to substantiate the idea that OLM data collected over only a portion of the transmitter span experienced under startup, normal operations, and shutdown can support calibration accuracy assumptions at the safety actuation setpoint levels even if these levels are not achieved during normal plant operation.
The NRC staff understands this position and will take it under consideration as it evaluates use of the TR for acceptance. However, no such method for determining exercised span to setpoint level percentage or acceptance criteria for application of this information within an OLM program is included in the TR. The NRC staff will prepare a RAI requesting AMS to provide a description
7 of how an OLM program would establish a method for analyzing the OLM data in relation to the instrument safety setpoints. This analysis should include either criteria for determining if OLM data can be used to provide assurance of accuracy at the instrument setpoint level or a method of establishing such criteria.
OLM Techniques AMS stated that it is seeking approval for parity space and simple averaging techniques to estimate the process value and for noise analysis technique to identify sensing line blockages.
AMS further stated that it intended to be agnostic to algorithm/technique used to provide a process estimate and that other process estimation techniques will need to be identified in LAR.
The NRC staff acknowledged this position and agreed that only process estimation techniques provided within the AMS TR will be considered for approval. Any other technique, not described in the TR, to be employed by a licensee will be subject to further evaluation during an associated LAR review. (No RAI is contemplated at this time.)
Referenced OLM EPRI Report Discussed in the TR-Appendix A In response to this audit question, AMS stated that this TR is an alternative methodology to EPRIs methodology in Appendix A. AMS stated during the audit that it will remove Appendix A but incorporate important parts of that methodology into the report.
The NRC staff understands that the AMS TR methodology does not rely on the EPRI methodologies that are referenced. The NRC safety evaluation will therefore be based on the AMS TR contents alone. (No RAI is contemplated at this time.)
Subsequent LAR for OLM Implementation Expectations In response to this audit question, AMS agreed that clarity on expectations is needed regarding what information is to be included within a LAR that references the TR. Furthermore, AMS stated that it had provided implementation methodology in Section 11 to directly address this issue.
The NRC responded that the information in Section 11 is not clear enough. The NRC staff are looking for AMS to provide the specific methods, criteria, and associated justifications (e.g., TR Page 81, Step 6, Specify data collection duration and sampling rate). This step does not describe any particular methods or criteria for data collection or for determining whether the data collected is adequate for the intended purpose (e.g., there are no criteria for the sample rate or duration of sampling period).
AMS stated that plants sample at unique times so AMS kept the TR more generic but stated it would look into these steps to determine if specific criteria/limits can be included. A RAI will be prepared by the staff to request AMS to provide criteria for a licensee to follow when addressing the steps identified in Section 11 of the TR so that there is consistency in the content of LARs referencing the TR.
8 TS Changes needed for OLM Implementation AMS stated that TS changes (in TR, Appendix C) were provided in response to an NRC request, as an example of changes that may be needed. They were not provided as TS changes to be approved by the NRC.
The NRC staff states that these TS mark-ups were beneficial because they stimulated discussion; however, there is still some ambiguity on how these example mark-up TSs are interpreted and implemented. Therefore, additional explanation is needed. Specifically, the OR proposed means the licensee can switch back and for the between the two FREQUENCIES in the final TS; however, there is no explanation of how this is expected to be implemented. The NRC staff plans to consider the information provided in the TS mark-up section while evaluating the topic, and identify the type of information a licensee would need to place into its plant-specific TSs and TS Bases in order to have appropriate TS surveillance information for its particular plant.
Noise Analysis Techniques In response to this audit question, AMS stated that this is addressed in TR Section 11.2 Item No. 3 and that collecting noise data at 2000 Hertz for 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> will cover the entire dynamic range of typical pressure sensing systems in nuclear power plants.
The NRC staff understands that criteria presented for a typical transmitter may or may not be acceptable for a specific transmitter. However, the dynamics of the process being measured also have a significant impact on the effectiveness of the OLM program to detect certain failure modes. Instead of relying on data that would be typically acceptable for most sensing systems, the NRC staff would like to understand specifically what data collection rates and what collection time periods would be needed to support the OLM program objectives.
The NRC staffs expectation is that the plant OLM program would include a method of evaluating the characteristics of the process being measured by a group of transmitters to determine the required minimum sample rate and minimum duration of data collection needed for OLM. The NRC staff will prepare a RAI requesting AMS to provide a description of how an OLM program would determine the minimum sample rate and data collection duration for a process group of pressure transmitters to be included in the OLM program.
Transmitter Span Verification using OLM In response to this audit question, AMS stated that the TR does not include acceptance criteria as to how much a transmitters range must be covered by OLM. In addition, AMS stated that the methodology assumes that nuclear grade pressure transmitters are predominantly linear and that this assumption is substantiated with Sizewell data.
This transmitter linearity justification does not explain how transmitters that use a low percentage of the overall span during operation can be verified to remain within calibration limits throughout the range. The NRC staff notes that smaller exercised ranges would result in greater uncertainties in the unexercised portions of a transmitter span. In extreme cases, such as containment pressure measurement, where the transmitter range is not exercised at all during operation, OLM techniques would provide no assurance that the transmitter calibration span is accurate. The OLM program should include criteria for exercised span as well as a statistical basis to support calibration assumptions in regions of span that are not exercised
9 during operation. An analysis of the uncertainties associated with unexercised portions of transmitter span will be needed to support any assumptions of calibration accuracy in these regions of transmitter operation. The NRC staff will prepare a RAI requesting AMS to provide a description of how an OLM program will address uncertainties associated with the unexercised portions of transmitter range. This discussion should explain how uncertainties are to be quantified and how assumptions of calibration accuracy in these regions of transmitter operation will be supported as part of an OLM program.
EXIT MEETING:
As described above, an exit meeting was held on February 24, 2021. During this meeting, the staff discussed its key observations. The staff noted that based on its review of the audit documents and its discussions with the AMS staff and its consultants, there appears to be a significant amount of pertinent data supporting the claim that OLM techniques as described in the TR can successfully predict when a transmitter needs manual calibration. Also, there appears to be value in its implementation, with very few instances where OLM did not successfully predict drift of a transmitter beyond its established flagging limits. Based on studies conducted by EPRI and others, there appears to be corroborating evidence that OLM techniques can be successful in predicting when a transmitter needs to be manually calibrated against known standards. The staff noted that it has identified a few areas where AMS responses are needed to support the staffs TR review. These will be pursued via a formal RAI.
At the exit meeting, the staff discussed the specific topical areas where additional information will be requested, and identified that to support its review, certain of the AMS documents that were contained on the SharePoint site will be requested for submittal on the docket.
AUDIT OBJECTIVES ACHIEVED:
As a result of the NRC staffs review of the documents on the SharePoint site, the staff was able to narrow down its list of open items to a manageable number of items. These items will be pursued via the RAI process. The staff issued its RAIs on March 5, 2021 (ADAMS Accession No. ML21067A674).
LIST OF AUDIT PARTICIPANTS AND OVERSIGHT PERSONNEL:
Name Affiliation Tile Areas of Responsibility Maggie OBanion NRC Project Manager NRC Project Manager Norbert Carte NRC Sr. Electronics Engineer Technical Reviewer and Audit Leader Richard Stattel NRC Sr. Electronics Engineer Technical Peer Reviewer David Rahn NRC Sr. Electronics Engineer Lead Technical Reviewer Joseph Ashcraft NRC Electronics Engineer Project Technical Consultant Tarico Sweat NRC Reactor Systems Engineer TS Reviewer Hash Hashemian AMS President AMS Administrative & Organizational Facilitation
10 Greg Morton AMS Chief Technical Officer Technical support Brent Shumaker AMS Senior Engineering Manager Technical support Ryan OHagan AMS Senior Engineer &
Marketing Manager Technical support Tyler Gavin AMS Software Engineer Technical support Kate Davy AMS Assistant Manager, Operations AMS Management support Mark Burzynski NuClear Day AMS Consultant Licensing Consultant David Lillis Consultant AMS Consultant Technical Consultant
Audit Summary Package: ML21063A121 Audit Plan Letter: ML21014A010, ML21027A334 Audit Questions: ML21014A034 Audit Report: ML21063A123 *concurred via email OFFICE NRR/DORL/LLPB/PM NRR/DORL/LLPB/LA*
NRR/DEX/ELTB/BC*
NAME MOBanion DHarrison JJohnston (TDinesh for)
DATE 3/15/21 3/11/21 3/15/21 OFFICE NRR/DORL/LLPB/BC*
NRR/DORL/LLPB/PM NAME DMorey MOBanion DATE 3/16/21 3/18/21