ML20091H104
| ML20091H104 | |
| Person / Time | |
|---|---|
| Issue date: | 03/31/2020 |
| From: | Office of Nuclear Material Safety and Safeguards |
| To: | |
| Kim Lukes | |
| References | |
| Download: ML20091H104 (21) | |
Text
NATIONAL MATERIALS PROGRAM PILOT PROJECT 3 ON OPERATING EXPERIENCE EVALUATION EXECUTIVE
SUMMARY
The original intent of Pilot Project 3, as provided in SECY-02-0074, National Materials Program: Pilot Projects, was to evaluate a collective set of Agreement State and U.S. Nuclear Regulatory Commission (NRC) licensee events for possible generic implications and additional regulatory action. The charter and pilot mission were updated to address the broader issues related to operating experience evaluation. The pilot was designed to examine current programs and to develop strategies and tools to make the programs more scrutable, predictable, and transparent.
The working group sought broad stakeholder input in conducting the pilot. The working group conducted interviews and surveys with regulatory personnel to assess end-user decisions, evaluated findings and insights from incident or working group reports, and scrutinized selected areas of regulatory oversight, to examine the use of operating experience information in integrated decision-making processes. The working group made presentations to, and held discussions with, stakeholders, in three public meetings and a telephone conference call that was open to the public.
Overall, the working group found that both NRC and Agreement States do many things well, but that the sharing of results and insights could be improved substantially. NRC and Agreement States collect and use much of the same information. The working group found no major gaps, in the categories of event reporting, in providing the needed information for incident response and/or prompt regulatory action. A major challenge was the common use of terminology and methods for disseminating operating experience information. To improve the sharing of results and insights between the Agreement States and NRC, the working group identified four groups of recommendations for consideration in enhancing the materials operating experience program: communications, procedures, clearinghouse, and process. Communication recommendations focus on methods to improve communications of information and activities with stakeholders and the public and propose to make generic communications more explicit in referencing reports for precursor and prior-similar incidents. Procedural enhancements include recommendations to adopt the objectives and attributes of the reactor operating experience program, update procedures to reflect current practices for screening of events and generic assessment reviews, and provide guidance on for the use of risk analysis tools and insights.
Clearinghouse recommendations are directed toward developing a central focal point for communicating and coordinating operating experience information, using of electronic media to increase participation and effectiveness, and providing timely public access so that licensees that implement safety measures can benefit from lessons learned. Process enhancements are related to incorporating operating experience activities in NRC and Agreement State planning and budgeting assumptions, focusing more on decision-oriented activities, trending of root causes in application-and device-specific areas, and performing detailed risk analysis.
Although the original Alliance concept has substantial merit for developing selected work products, the working group believes that a diversity of approaches can be applied to the National Materials Program. The working group recommends that the operating experience program focus more on decision-oriented activities and suggests that working groups and surveys be conducted on a more selective basis. Operating experience activities can serve as the basis for shared planning assumptions and long-term budget planning.
This pilot project was conducted entirely through the use of electronic media, with no team travel. It demonstrated the ability of working groups to be conducted cost-effectively, through conference calls and without undue burden to the sponsoring organizations.
NATIONAL MATERIALS PROGRAM PILOT PROJECT 3 ON OPERATING EXPERIENCE EVALUATION FINAL PROJECT REPORT
1.0 PURPOSE
The purpose of the Operating Experience Evaluation Pilot was to optimize the common use of operating experience information from licensed facilities, and trending in integrated U.S. Nuclear Regulatory Commission (NRC) and Agreement State review, assessment, and decision-making processes. The pilot was designed to examine current programs and to develop strategies and tools to make the programs more scrutable, predictable, and transparent. The proposed outcome would be a set of recommendations that would lead to a revised process that could produce consistent analyses and results when implemented by NRC and the Agreement States.
2.0 BACKGROUND
The original intent of Pilot Project 3, as provided in SECY-02-0074, National Materials Program: Pilot Projects, was to evaluate a collective set of Agreement State and NRC licensee events for possible generic implications and additional regulatory action. During the early planning stages, preliminary work to develop a charter focused on lessons-learned from event evaluation, using the Office of Nuclear Material Safety and Safeguards (NMSS) Policy and Procedures Letter (P&PL) 1-57, Generic Assessment Process.
In late 2002, challenges were recognized in the reactor operating experience program related to materials degradation of the reactor vessel head at the Davis-Besse Nuclear Power Plant. In January 2003, the NRC Executive Director for Operations forwarded, to the Commission, the results of the Senior Management Review Team (RT) report, evaluating the Davis-Besse Lessons Learned Task Force (LLTF) Report. The RT report included an Assessment Plan for examining NMSS programs relative to the LLTF Report.
When this pilot was formed in early 2003, the working group updated the scope of the charter to focus on the common use of operating experience information. For the purpose of this pilot, the working group considered operating experience information to include: (1) domestic and foreign event data; (2) major team inspections and special studies leading to generic reviews and/or generic communications; (3) industry-wide analyses of performance and trends; (4) insights and metrics amenable to risk-informed decision-making; and (5) performance indicators and associated thresholds for increased regulatory attention.
3.0 DISCUSSION
3.1 Scope of Work The Work Product Plan was designed to evaluate regulatory processes and methods to address the following questions:
1.
How can operating experience information be better communicated between NRC and Agreement States?
2.
How can operating experience information and trending optimize NRC and Agreement State resource utilization?
3.
How can risk insights be better integrated into regulatory decision-making?
The pilot pursued a sampling approach to addressing these questions. The working group
Pilot Project 3 Final Report on Operating Experience Page 3 conducted interviews and surveys with regulatory personnel to assess end-user decisions (i.e.,
inspectors, reviewers, managers); evaluated findings and insights from incident or working group reports [Augmented Inspection Teams (AITs); Incident Investigation Teams (IITs); Special Inspection Teams (SITs); etc.]; and examined selected areas of regulatory oversight [e.g.,
portable gauges and intravascular brachytherapy (IVB)]; to examine the use of operating experience information in integrated decision-making processes.
The pilot sought broad stakeholder input in conducting the pilot. The working group made presentations at meetings of the Conference of Radiation Control Program Directors (CRCPD) in May 2003; Organization of Agreement States (OAS) in October 2003; and Advisory Committee on the Medical Uses of Isotopes, in June and November 2003. The working group held a telephone conference call that was open to the public, in October 2003, to discuss analysis of operating experience information performed by the University of Texas for the States of Texas and Maine. The working group also participated in deliberations of the NMSS Operating Experience Committee, which considered the impacts of the independent evaluation of the Reactor Operating Experience Task Force Report, resulting from the Davis-Besse LLTF.
Preliminary insights provided in the progress report for the National Materials Program (NMP) pilot projects were discussed with stakeholders during a public meeting in March 2004.
3.2 Communication
Early feedback indicated that a major challenge to the effective use of operating experience information was the common understanding of the term operating experience information.
This challenge was recurring during all phases of the pilot. NRC and Agreement State responses reflected good understanding of their respective processes for event evaluation, reporting, generic issues, licensing, and inspection. Individually, major gaps were not apparent in these program elements. Feedback from survey results and meetings with stakeholders confirmed, however, that more work is needed to establish mutual understanding of these individual processes as elements of an integrated program of assessment and feedback.
The working group found that both NRC and Agreement States do many things well, but that the sharing of results and insights could be improved substantially. At present, there is no single location that NRC or Agreement State can go to examine the diversity of operating experience information that is available. The Office of Nuclear Material Safety and Safeguards (NMSS) and the Office of State and Tribal Programs (STP) maintain separate internet websites. Each website provides valuable information but could be better linked to provide equal access from either website to an operating experience information center or central clearinghouse, as envisioned in the final report of the National Materials Program Working Group (SECY-01-0112).
The working group finds substantial merit in the current efforts to communicate issues between NRC, Agreement States, licensees, key stakeholders, and the public. Of particular importance are initiatives such as the Agreement State Letters; NMSS and Nuclear Materials Event Database (NMED) Newsletters; monthly NRC/OAS telephone conference calls; and the network-based e-mail communications tool RADRAP. The challenge arises in sharing information consistently across organizational boundaries. Working group survey results indicated that States normally find out about emergent issues through a diversity of means, including Agreement State Letters, meetings, and people-networking contacts. NRC and Agreement States function, to a large extent, autonomously of each other.
In its review of the Reactor Operating Experience Task Force Report, the working group found that NRC and Agreement States perform the necessary functions and process elements of operating experience evaluation. The working group also noted two items that, if adopted,
Pilot Project 3 Final Report on Operating Experience Page 4 could add substantial value to improving communication of operating experience information in the NMP. The objectives and attributes of an effective operating experience evaluation program (Enclosure 1), described in the task force report, are not unique to reactor applications and could be adopted easily in NMSS and STP procedures (e.g., NMSS P&PL 1-55 and 1-57 and STP SA-300) to promote the common understanding of operating experience evaluation across the diverse range of regulated activities. The working group recommends that NMSS and STP adopt the objectives and attributes of an effective operating experience evaluation program in their procedures.
The working group also finds substantial merit in the Task Force recommendation for the formation of an operating experience information clearinghouse. A clearinghouse could provide a more consistent approach to input and feedback processes among NRC and Agreement State programs. A clearinghouse would also provide a central location for licensees to access information that could impact and/or benefit their programs. The working group acknowledges that many elements of such a clearinghouse already exist in various locations within NRC and Agreement State programs and that such a clearinghouse can be accomplished in a phased manner, beginning with the linking of existing systems and databases. A centralized clearinghouse could consolidate information and reduce duplication of effort. The working group recommends that NRC and the Agreement States explore development of a central clearinghouse to serve as a focal point for communicating and coordinating the wide range of operating experience information.
Some of the following elements could serve as the foundation for an initial clearinghouse:
Event Notices Preliminary Notifications Generic Communications Team Inspection Reports NMED Database and Quarterly Reports Industry Operating Trends Equipment Failures and Reliability Data Sealed Source and Device Registry (SSDR) Issues Technical Assistance Request (TAR) Results Special Studies and Risk Insights Many of the above noted sources of operational data are discrete in that blending of databases is not easily done. As such, trending and special studies may require searches using multiple programs. The working group acknowledges that a substantial population of Agreement State legacy documents exist that may not be in electronic form amenable to merging into NRC databases. Any clearinghouse that is developed should be forward-looking and be built upon existing sources of data. Routine operating experience information from the States should be incorporated as autonomous work products without NRC peer review, except as it provides information necessary to complete the NMED events database. NRC review or screening processes for State submitted special studies should be limited in scope not to exceed that of NUREG contractor reports.
Fundamental to the success of any clearinghouse is the commitment of a critical group of people to handle inputs and feedback, to make judgments about the priority of information and the need to make certain documents conspicuous, and to determine whether communication plans are warranted to support the rollout of major activities and initiatives. The working group recommends the increased use of communications plans for the wider dissemination of information and activities to key stakeholders and the public.
Pilot Project 3 Final Report on Operating Experience Page 5
3.3 Participation
This pilot project was conducted entirely through the use of electronic media. No team meetings involving travel were held. This pilot demonstrated the ability of working groups to be conducted cost-effectively through biweekly conference calls, and without undue burden to the sponsoring organizations during periods of absence. The evaluation phase of the pilot benefitted substantially from the opportunity to engage in discussions of the NMSS Operating Experience Committee, which included briefings by the NMSS regional coordination staff, representatives of the NMSS Risk Task Group, and the Chairperson of the Reactor Operating Experience Task Force. The working group also conducted a publicly noticed telephone conference call, to discuss analysis of operating experience information, performed by the University of Texas, for the Agreement States of Texas and Maine. The working group notes that the monthly OAS/NRC telephone conference call further demonstrates the effectiveness of electronic media in achieving the objectives of the NMP. The working group does not believe that the quality of its review would have been substantially enhanced by travel for meetings, but believes that much more can be accomplished by the expanded use of telephone and video conferencing. The working group recommends the expanded use of electronic media as a means to increase participation and to promote effectiveness, efficiency, and openness in the NMP.
Although efficiencies were found in the use of electronic media, the working group notes that substantial challenges exist for participation in NRC and Agreement States pilot projects. Pilots and working group activities are often conducted with existing resources, in most cases, as an added activity to the existing work of candidate participants. As such, the ability of the working group to achieve progress and meet schedules is highly dependent on the staffing and budgeting environment of the sponsoring organization. In general, no work is given up, to support pilot and working group activities. Professional development activities also compete, with travel, in support of pilot activities, even if NRC funding can be arranged via the NMP.
Discussion during the March 2004 stakeholder meeting indicated that the budget process for supporting the NMP is a challenge. The NRC budget process normally forecasts budgets 3 to 5 years forward, with provision for continuous adding and shedding of activities, to address emergent issues. Agreement State budgets require legislative approval, which makes agreements and commitments difficult to achieve in support of the NMP, since NRCs budget contains significantly more detail than State budgets for the radiation programs, and the States and NRCs budget timetables are not synchronized. During the stakeholder meeting in April 2004, a suggestion was made that more can be done to share and communicate openly about planning assumptions and to agree on priorities. The working group agrees and believes more open involvement is needed for the budgeting and allocation of resources, particularly where emergent safety issues alter agreed plans. The working group recommends that formal methods be developed to better incorporate NMP operating experience activities in NRC and Agreement State planning and assumptions.
There were not many candidates that volunteered to join the working group. This may have been because the concept of operating experience evaluation was not understood, as was suggested by OAS during development of the charter. The working group accepts that criticism, but believes that a larger concern exists in relying primarily on working groups to achieve success for the NMP. Working groups and surveys appear to have, in some ways, outworn their welcome with candidate participants and respondents, in part, because of the relatively long time commitment and administrative burden of developing charters, work product plans, progress and final reports. For example, a working group on portable gauges continued to be chartered and to meet for 3 years. Most of the pilots conducted surveys of NRC and Agreement States during this NMP pilot initiative. Agreement State representatives objected to the number
Pilot Project 3 Final Report on Operating Experience Page 6 of surveys conducted by the pilots and other entities, during the OAS meeting in October 2003.
In April 2004, OAS solicited feedback, via RADRAP, from its membership and CRCPD on the difficulties in finding people to fill volunteer positions, including those associated with the NMP pilots.
Another issue that arose during the conduct of this pilot was the Agreement States desire to have more voice in NRC activities. During its Annual Meeting in October 2003, the OAS approved a motion to evaluate the feasibility of the Agreement States forming a Commission-level advisory committee. Although the working group acknowledges that OAS and CRCPD do conduct annual briefings of the Commission, it is apparent that the Agreement States would like to be more engaged in NRC regulatory decision-making processes. The working group recommends that the NMP approach to operating experience evaluation focus more on decision-oriented activities and that working groups and surveys be pursued on a more selected basis.
Some examples of decision-oriented activities that could offer the potential for relationship-building activities include:
NRC and Agreement State budget planning NRC/Agreement State Roundtable Annual OAS Meeting NMSS/Regional Counterpart meetings Agency Action Review Meeting NRC and Agreement State team inspections TAR analysis with State input Outreach initiatives/activities 3.4 Integrated Decision-Making The working group conducted surveys and interviews of NRC and Agreement State inspectors, reviewers, and managers to better understand how the different regulatory bodies collect, review, analyze, and disseminate concerns and lessons-learned. The initial survey was modified to also examine the pilot test cases for portable gauges and intravascular brachytherapy (IVB). The survey was directed toward assessing end-user needs for regulatory decision-making.
3.4.1 Information Needs Survey responses indicated that NRC and Agreement States collect and use much of the same information. Both evaluate daily event reports as the primary means of assessing emergent issues, indication of change, and the need for prompt regulatory action. Daily reports are screened primarily for incidents involving radiation exposures, contamination, loss of control, malfunctions, and misadministrations. Some responses indicated that inspection, enforcement, allegations and complaints, financial information, and business-plan information are also evaluated and trended. In general, there were no major gaps in the categories of event-reporting in providing the needed information for incident response and/or prompt regulatory action. The intravascular brachytherapy test case does recommend additional action related to the availability of information for failures and malfunctions, as discussed later in this report.
The working group reviewed the Final Report of the NRC/State Working Group on Event Reporting, dated May 17, 2001. That report offered a number of recommendations related to the evaluation and use of operating experience information. Tasks and recommendations of that Final Report included a comparison of NRC Strategic Plan and NRC reporting
Pilot Project 3 Final Report on Operating Experience Page 7 requirements, evaluation of licensee guidance and Agreement State guidance, and issues related to NMED reporting, generic issues, and software systems. The Final Report contained a number of worthy recommendations and initiatives. NMSS developed a draft action plan to assess those recommendations and develop proposed courses of action. The terrorist attacks of September 11, 2001, resulted in a substantial increase in the NMSS workload and a reallocation of resources to the newly formed Office of Nuclear Security and Incident Response.
Other operational challenges, including the aforementioned Davis-Besse LLTF Report, affected the prioritization of work through the Planning, Budgeting, and Performance Management.
Because no imminent safety issues were apparent in the Final Report, most of the recommendations were deemed to be of low priority and the NMSS action plan was not finalized. Subsequent government-wide initiatives to develop a National Response Plan (NRP) and National Incident Management System (NIMS) created additional resource challenges and presented issues on event reporting that compete with the recommendations of the Final Report.
In parallel with this pilot, a team led by STP conducted a self-assessment of the event reporting process, with particular emphasis on the NMED. On March 29, 2004, the self-assessment team issued its final report on Event Reporting. In that report, the team made a recommendation that NMP Pilot 3 on operating experience evaluation...conduct an evaluation of the material reporting requirement inconsistencies and schedules, based on risk significance, for all reporting requirements including those identified above and in the 2001 Event Working Group Report, and...any information deemed essential to NRC needs should be reflected in revised regulations and revised guidelines. The working group considered these recommendations and determined that no apparent safety issues are being missed in current reporting requirements. Although there is merit in making the reporting requirements more consistent, the lack of a safety argument makes it difficult to justify the work that would need to be given up to support the recommendations of the Self-Assessment Team. The working group agrees with the low priority given to the 2001 Final Report and believes that the follow-up recommendation of the Self-Assessment Team does present new information sufficient to justify the diversion of resources to support a broad population of rulemakings that may not be well-supported through backfit analyses. The working group suggests that a voluntary approach [i.e., possibly via a Regulatory Information Summary (RIS)] be pursued, to fill any gaps, in information needed for use in NMED, that is not already covered by statute.
NMED serves as the only events database broadly available for use by NRC and Agreement States. NMED is dependent, however, on the quality and completeness of event reporting.
NMED includes only Atomic Energy Act (AEA) materials and, therefore, does not cover all activities regulated by Agreement States, such as machine-radiation incidents (e.g., x-ray and accelerators). NMED can and should evolve to better facilitate trending, operational histories, precursor events, and the development of performance indicators. NMED has valuable search capability and NRC issues an NMED Quarterly Report and Newsletter. The working group considers NMED and associated work products to be integral parts of an operating experience information clearinghouse. The working group recommends that the information clearinghouse be made publicly available so that licensees that implement safety measures can benefit from lessons learned and so that stakeholders can have better access to, and more fully participate in, the regulatory process.
3.4.2 Regulatory Decision-Making NRC and Agreement State decision-making appears to follow a similar process, which involves initial screening of events for safety significance; decisions regarding the need for prompt regulatory action, including inspection and enforcement; review of incidents/events for broader implications and possible generic communications; trending and periodic reporting of results;
Pilot Project 3 Final Report on Operating Experience Page 8 and feedback for possible changes in regulations and associated guidance. Surveys indicated that the decisions are made on the basis of health and safety, protection of the environment, and compliance with regulatory and license requirements. Surveys also indicated that a major outcome of these deliberations is usually a reexamination of current and planned resources to determine what, if any, reprogramming of resources is required. Reprogramming of resources is usually done on the basis of consequences (e.g., dose) and the potential for generic implications. Both NRC and Agreement State programs conduct an ongoing add/shed process, to address emerging issues of concern.
Although surveys and interviews indicated that inspection reports provide insights critical to the integrated decision-making process, few respondents expressed the desire to use additional staff resources to perform comprehensive reviews of materials inspection reports for the purpose of evaluating performance and identifying trends. The number of licensees and inspection reports would be too numerous and labor-intensive for the limited insights that might be derived. Many States indicated that they review inspection reports on a quarterly basis.
Respondents indicated that NRC and/or Agreement State team inspections provide more in-depth analysis and valuable insights, but should be made more conspicuously available. The working group agrees and suggests that major team inspections be made available via the clearinghouse, if adopted.
Working group review of the event evaluation process described in NMSS and STP procedures for the routine review of events and the methods for evaluating incidents for generic implications indicated that these programs are generally effective. The working group noted that NMSS had adopted the recommendation of the 2001 Final Report of the NRC/State Working Group on Event Reporting, in P&PL 1-57, NMSS Generic Assessment Process, that a single manager be responsible for weekly event evaluation. That change was made, in part, to alleviate the resources being used in support of the weekly Generic Assessment Panel (GAP). Efficiencies were derived from that change without having any apparent adverse impact on the effectiveness of generic assessment reviews. The working group notes, however, that conforming changes were not made in P&PL 1-55, Procedures for Coordination of Regional Events and Enforcement Activities, and STP Procedure SA-300, Reporting Material Events, concerning conduct of the GAP review. NMSS recently initiated a revision to P&PL 1-57 to provide for a periodic self-assessment of the results of its event review for generic implications and to assess the effectiveness of generic communications. The working group views this as a positive initiative, but acknowledges that it may be too soon to determine the cost-benefit of the change.
The working group recommends that conforming changes be made to all affected NMSS and STP procedures, to reflect current practice concerning supervisory review, abatement of GAP, and adoption of the proposed self-assessment process.
The working group examined a sample of generic communications to consider their effectiveness in communicating issues and to evaluate regulatory follow-up to confirm the adequacy of licensee response. Unlike reactors, the use of NRC Bulletins and Generic Letters is rare in nuclear materials. Most materials-related generic communications are information notices (INs) and RIS that do not require written responses from licensees. Likewise, INs and RIS do not require follow-up by NRC or Agreement States. As such, it was difficult to assess the effectiveness of generic communications follow-up, except to the extent that recurring events resulted in the issuance of additional generic communications (e.g., INs for portable gauges). The NMSS Licensee Newsletter was found to be an effective tool for advising NRC, Agreement States, and licensees on recent events, significant enforcement actions, generic communications issued, and proposed and final rules. In general, INs and RIS were found to adequately address root causes and lessons learned from prior incidents. One notable observation, however, was that materials INs and RIS lack a degree of specificity, with regard to prior operating experience events, that is normally found in generic communications for
Pilot Project 3 Final Report on Operating Experience Page 9 reactors. The working group does not object to withholding the names of licensees and locations of materials facilities, on the basis of materials security for radioactive materials and quantities considered to be of high risk. However, more information (e.g., inspection report number and/or Agencywide Documents Access and Management System (ADAMS) accession number, manufacturer and model number of the equipment involved) can be provided for the majority of incidents, so that users can better evaluate the applicability to their activities and practices. The working group recommends that generic communications for materials be more explicit in referencing incident reports for precursor and prior-similar incidents.
The working group also examined NRC and Agreement State processes for evaluating risk in the evaluation of operating experience information. The working group acknowledges that progress toward a risk-informed regulatory environment is a work-in-progress. NRC initiatives to develop safety goals, risk analysis methods, and guidance for use in assessing operating experience information are ongoing, so that it may be too early to judge the effectiveness of many of these initiatives. Some initiatives, such as safety goals, appear to be fairly long-term efforts. Other initiatives are in various stages of analysis and/or completion. There is a need for industry guidance on methods to request regulatory action on the basis of risk. Examples of initiatives that would benefit from the development of implementing guidance, as they become available, include:
Safety goals for materials and waste; NUREG-6642; Draft Guidelines for Risk-Informed Decision Making in NMSS (under development by Brookhaven National Laboratory); and International Atomic Energy Agency Code of Conduct.
NMSS has conducted limited training on certain initiatives such as NUREG-6642, Risk Analysis and Evaluation of Regulatory Options for Nuclear Byproduct Material System. The working group found that few Agreement States were aware of this initiative and that few NRC personnel were using this analysis tool in evaluating incidents/events for risk significance. The working group recommends that NMSS and STP procedures be upgraded to provide guidance for use of risk analysis tools and insights.
Survey feedback from inspectors, reviewers, and managers indicated the desire to better understand risk insights and system vulnerabilities of significant materials events and incidents.
Respondents acknowledged a lack of time and capability to do such analyses. The working group believes that the Office of Nuclear Regulatory Research could play a larger role in the development of risk methods, as part of the current initiative to develop a more robust materials research program. Risk analysis work products should be tailored to the needs of the end user (i.e., inspector and license reviewer) and not be overly complex or voluminous.
Survey feedback indicated that Agreement States desire to have open access NRC TAR analyses and results. Another issue was raised, during the March 2004 stakeholder meeting, concerning the process available to Agreement States for requesting that NRC consider an issue for possible generic communications. Working group review of this matter indicated that NRC is in the process of developing a list of Questions and Answers and issue summaries, to address TARs related to the new 10 CFR Part 35, on the medical use of byproduct material, and providing access to the requested document and regulatory response (both publicly available), for other TAR requests. Some progress has also been noted in solicitation of Agreement State participation in a recent TAR panel. Because of the Agreement States broad experience and the likely introduction of new modalities and other radioactive materials uses under Agreement State jurisdiction, the working group encourages that more be done to make this standard practice. The working group encourages NRC to make certain technical analyses
Pilot Project 3 Final Report on Operating Experience Page 10 are available via Branch Technical Positions or other methods.
3.4.3 Test Cases In its survey, the working group requested explicitly that respondents identify three things that can be measured by NRC and Agreement States. Respondents indicated the following items as possible performance indicators: (1) average and maximum radiation exposures; (2) number of lost and stolen sources; and (3) violations, including severity level. Survey results indicated that a lot of effort is given to evaluating and certifying that event data are complete. The working group recommends that more effort be given to data evaluation processes and feedback, to include more trending of root causes in application and device-specific areas.
Two test cases were selected to evaluate the past NRC and Agreement State experiences with regard to operating experience and to solicit input on cumulative data and strategies that can be used to make the programs more predictable and transparent and produce consistent analyses when evaluated by NRC or Agreement States. Portable gauges were selected by the working group, since they represented a large and stable population of users; few changes had occurred over the years, with regard to the regulatory environment; and with extensive operating experience. One of the consistent operating issues with portable gauges is the number of lost or stolen devices. IVB was also selected, since it represented a medical modality whose use has grown substantially over the last few years. With the large number of new users and new devices, concerns exist that the regulations and assessment tools may not have kept up with the issues being realized from operating experience.
3.4.3.1 Portable Gauges Surveys and interviews indicated a variety of insights regarding the operational experience with portable gauges. Inadequate worker training appeared to be a common theme in incidents involving lost, stolen, and damaged gauges. Of particular concern was the amount of authority that device user assistants were given, with little or no training. Feedback indicated that device user assistants were the most likely to receive inadvertent exposures. Another issue was the fact that, historically, gauges were sometimes not well-accounted for, when in service, and that accountability lapses became more problematic when gauges were placed in long-term storage.
Another challenge cited was the lack of permanency of radioactive materials labels on gauges found in the public domain and, in particular, at metal recycling scrap yards.
The working group held a telephone conference call, that was open to the public, to discuss analysis of operating experience information, performed by the University of Texas for the States of Texas and Maine. The University of Texas studies considered the outcomes of inspection in aggregate, along with compliance. The early benefits of the studies were regulatory insights related to the causes of certain reportable events (e.g., 85 percent of portable gauges were lost or stolen during transport or temporary storage). Eventual benefits included an enhanced understanding of industry best-practices that could be communicated to licensees as a means of lessons-learned for improving safety and compliance. The results of these studies were incorporated into regulatory oversight programs and presented as an outreach initiative, by the State of Texas, to its licensees. The researcher published several articles in industry trade journals.
In response to the above noted concerns, a number of regulatory initiatives have been taken.
Some survey respondents indicated the need to reconsider the sufficiency of general licensing and suggested that radioactive materials be considered under specific licensing or as exempt quantities. Other respondents suggested that an explicit set of performance indicators be developed to focus explicitly on the root causes for lost and stolen devises. The State of Ohio
Pilot Project 3 Final Report on Operating Experience Page 11 reported that increased regulatory controls, related to transport and storage of devices for temporary job sites, have been effective in reducing the incidents involving lost and stolen gauges. Ohio also reported that the greatest impact/improvement has come from the increased attention on the security of radioactive materials. NRC, Agreement States, and licensees have, in partnership with government and international efforts, pursued a number of initiatives, particularly with regard to high-risk sources. The working group notes that more can be done to communicate progress on these activities and suggests that the proposed clearinghouse be used as a means of disseminating guidance consistently.
3.4.3.2 IVB Surveys and interviews indicate that more effort is needed to have a database designed to trend and facilitate the analyses of operational histories and precursor events related to specific materials modalities, to mitigate prospective events, including IVB. NMED serves as a record of NRC and Agreement State incidents that require reporting, according to NRC and Agreement State regulations. NMED contains information about the factual circumstances regarding events, and their causes and corrective actions. It does not, however, serve as a risk analysis tool for the systematic analysis of precursor events.
During the course of this pilot, NRC issued IN 2003-09, Source Positioning Errors and System Malfunctions During Administration of Intravascular Brachytherapy. That IN describes new equipment designs for which the users did not fully appreciate the differences in new methods, anomalous performance of the device, and a complete system failure for which emergency procedure implementation was required. The working group believes there is a need for more information on the equipment malfunctions and failures, and on human performance, to develop risk insights of benefit for use in regulatory programs. This information can be obtained through a number of ways, including inspection of the manufacturers and associated service records; inspection of the licensees; voluntary licensee reporting; and reporting of equipment service data to the NRC Sealed Source and Device Registry (SSDR) comparable to current reporting requirements of the U.S. Food and Drug Administration (FDA) for medical devices. Survey responses indicate that the FDA system has limitations in being a useful tool for trending and risk analysis.
Since 10 CFR Part 35 was amended in 2002 to reference the SSDRs, there is now a need to periodically review the SSDRs, to ensure that changes in the manufacturers procedures and quality assurance requirements are updated to meet current regulatory expectations and operating experience. Since SSDRs do not have an expiration date as found with the specific licenses issued to manufacturers to possess licensed materials, there is no incentive by the manufacturer or the regulatory agency incentive to periodically update the SSDRs.
Because a majority of IVB misadministrations are related to a single medical device, a more detailed risk analysis (e.g., using NUREG-6642, the NRC and Brookhaven National Laboratory (BNL) guidelines, or simplified probabilistic risk assessment) could be performed to provide the basis for recommended enhancements that might be made to the operational experience program for the analysis of accident precursors. Insights on design and operating vulnerabilities gained could provide the basis for changes to licensing and inspection and provide the foundation for a more robust materials program. A more complete evaluation may be necessary to identify sources of NRC and Agreement State operational data (e.g., reliability, failure, and human performance data) necessary to perform a realistic risk analysis of the operating system.
The working group recommends that a detailed risk analysis be performed on a single device, based on appropriate operational data collected from NRC and Agreement States, to provide the basis for recommendations to enhance licensing and inspection
Pilot Project 3 Final Report on Operating Experience Page 12 programs.
3.4.4 Major Incident Inspections and Reports The working group examined a number of special inspections and reports to consider how operating experience insights were used in regulatory programs. In particular, the working group evaluated reports from IITs, AITs, and SITs, to determine the effectiveness of generic communications, regulatory follow-up to emergent issues and trends, and enhancements made as a result of lessons learned. Special inspections and studies examined by the working group include:
Leakage of an Irradiator Source -- the June 1988 Georgia Radiation Sterilizers, Inc.,
Incident, report dated February 1990; Emergency Response to a Highway Accident in Springfield, Massachusetts, on December 16, 1991";
Loss of an Iridium-192 Source and Therapy Misadministration at Indiana Regional Cancer Center, Indiana, Pennsylvania, on November 16, 1992";
Ingestion of Phosphorus-32 at Massachusetts Institute of Technology (MIT),
Cambridge, Massachusetts, Identified on August 19, 1995";
Source Disconnects Resulting from Radiography Drive Cable Failures, Final Report, June 1998; Extremity Exposures in Excess of Regulatory Limits, Mallinckrodt, Inc., Maryland Heights, Missouri, report dated September 5, 2000; Loss of Control of a Well Logging Source Resulting in Radiation Exposures to Members of the Public, Havre, Montana, May 21, 2002; Overexposures to Members of the Public at St. Joseph Mercy Health System, Ann Arbor, Michigan, July 1-7, 2002; Degradation of the Davis-Besse Nuclear Power Station Reactor Pressure Vessel Head Lessons Learned Report, Final Report, September 30, 2002; Reactor Operating Experience Task Force Report, dated November 26, 2003.
Each of the above-noted incidents received substantial regulatory attention. Event reports and preliminary notifications were issued as a means of communicating emergent issues. All the above-noted incidents had major team inspections and most had, in addition to the inspection, a NUREG report issued, which provided insights and lessons-learned.
The working group found that the use of generic communications was generally effective in addressing emerging issues and incidents of significance. For incidents involving the loss of Iridium-192 and therapy misadministration at Indiana Regional Cancer Center in Pennsylvania, two NRC Bulletins were issued requesting actions and written responses. Information derived through those Bulletins indicated both operational weaknesses and device vulnerabilities.
The working group noted that the notification of licensees regarding precursor events could, in some cases, be improved. NRC issued an IN for the 1995 incident involving the ingestion of Phosphorus-32 (P-32) incident at MIT. From 1978 to 1995, there were eight precursor events, involving malevolent acts, that resulted in the contamination, ingestion, and/or willful exposure of personnel. Licensees were not informed of the eight precursor events before the incident at MIT. Likewise, a similar P-32 ingestion incident at the National Institutes of Health (NIH) was not communicated to licensees until 4 months after the event was reported. The NIH incident involved a malevolent act and resulted in an investigation by the NRC Office of Investigations.
The working group acknowledges that much industry-wide effort is ongoing to address the security of sources and potential for malevolent acts. This was an older event and much has been accomplished since the terrorist acts of September 11, 2001.
Pilot Project 3 Final Report on Operating Experience Page 13 The 1988 incident involving leakage of an irradiator source indicated problems with the technical analysis used to support conversion from Cobalt-60 to Cesium-137 irradiator cells.
Issues raised within the manufacturers organization were not made apparent during NRCs licensing review. Continuous monitoring systems were not required to detect the spread of contamination in the irradiator pool. Also, financial assurance issues were identified related to recovery from the contamination incident. The regulations in 10 CFR Part 36 were revised to address explicitly the licensing, design and construction, and radiation safety, for irradiators.
Issues regarding financial assurance of facilities having large quantities of sources are again under regulatory review, as a result of the bankruptcy and abandonment of the facility and sources at an irradiator facility in Pennsylvania.
Several incidents, including Mallinckrodt and Davis-Besse, resulted in the implementation of major actions and/or assessment plans for which implementation required resource commitments over a period of years. The Mallinckrodt incident resulted in a Phase II effort to reassess certain aspects of the regulatory oversight program. Of particular importance was the proposed revision to NRC Inspection Manual Chapter (IMC) 2800, Materials Inspection Program. Revision to IMC 2800 was also the subject of NMP Pilot Project 5. The Davis-Besse incident led to a major reassessment of the reactor operating experience program and had a substantial influence on the materials safety program including the focus of this pilot. Specific actions in the NMSS Assessment Plan are addressed throughout this report.
The 2002 well-logging incident in Havre, Montana, identified a number of issues that require continuing efforts. A number of follow-up activities and lessons-learned resulted from this event. Extensive evaluations were conducted to better understand the numerous well-logging-handling tool designs, operating practices, and dose estimate methods and results. Even though an IN was issued concerning this event in 2003, two subsequent events occurred in 2004. The working group recognizes that an important part of operating experience is enforcement. The 2002 incident resulted in a substantial civil penalty. The precursor events cited in the 2003 information notice, the 2002 event, and 2004 incidents all involved the same licensee. Regulatory action to address the 2004 incidents is on-going in the associated Agreement States. Development of lessons learned are also on-going.
The working group found that operating experience insights get incorporated in a diversity of regulatory initiatives. The 1991 highway accident in Springfield, Massachusetts, involving a truck fire that engulfed a shipment of new reactor fuel served, in part, as the basis for a subsequent emergency response tabletop exercise. Development of the National Response Plan (NRP) and National Incident Management System (NIMS) will influence the direction of the National Materials Program and the use of operating experience information and feedback from exercises. Likewise, international activities for the security and control of sources, including the International Atomic Energy Agency, Code of Conduct, are continuing and will affect thresholds for regulatory action and possible future reporting criteria.
The working group noted that stakeholder interest and participation are key elements of an effective operating experience program. The 2002 overexposure incident involving family members of a terminal radiation therapy patient at St. Joseph Mercy Health System indicated the importance of having technical regulatory capability in the dose reconstruction, but also highlighted the importance of communicating effectively with diverse stakeholders.
In general, the working group found that the regulatory response was effective in addressing emerging issues of concern. Major incidents do receive a high level of regulatory attention and often shape the future of regulatory programs. In that context, the working group views the use of operating experience information as an iterative process or work-in-progress. Feedback from incident investigation and stakeholder involvement do shape the lessons learned and future
Pilot Project 3 Final Report on Operating Experience Page 14 direction. The use of risk-informed tools and methods is not yet well-integrated into the overall process for analyzing major incidents. The working group believes that continued integration of risk into implementing programs will benefit the screening of incidents and resource allocation.
3.5 RESOURCES
The working group included equal representation by NRC and Agreement State representatives. The working group had Co-Chairpersons from both NRC and Agreement States, one additional Agreement State representative, and one NRC representative from an NRC Regional Office materials-related program. Members of the pilot included: Marcia Howard, Co-Chair, State of Ohio; Michael Markley, Co-Chair, NRC, NMSS, Division of Industrial and Medical Nuclear Safety; Debbie Gilley, State of Florida; and Duncan White, NRC Region I.
This pilot was conducted entirely through the use of electronic media. No team meetings involving travel were held. This pilot demonstrated the ability of working groups to be conducted cost-effectively through biweekly conference calls, without undue burden on NRC or Agreement State participants.
3.6
SUMMARY
OF RECOMMENDATIONS:
Overall, the working group found that both NRC and Agreement States do many things well but that the sharing of results and insights could be improved substantially. NRC and Agreement States collect and use much of the same information. The working group found no major gaps in the categories of event reporting in providing the needed information for incident response and/or prompt regulatory action. A major challenge was the common use of terminology and methods for disseminating operating experience information. To improve the sharing of results and insights between the Agreement States and NRC, the working group identified four groups of recommendations for consideration in enhancing the materials operating experience program: communications, procedures, clearinghouse, and process. Communication recommendations focus on methods to improve communications of information and activities with stakeholders and the public and propose to make generic communications more explicit in referencing reports for precursor and prior-similar incidents. Procedural enhancements include recommendations to adopt the objectives and attributes of the reactor operating experience program, update procedures to reflect current practices for screening of events and generic assessment reviews, and provide guidance on for the use of risk analysis tools and insights.
Clearinghouse recommendations are directed toward developing a central focal point for communicating and coordinating operating experience information, using of electronic media to increase participation and effectiveness, and providing timely public access so that licensees that implement safety measures can benefit from lessons learned. Process enhancements are related to incorporating operating experience activities in NRC and Agreement State planning and budgeting assumptions, focusing more on decision-oriented activities, trending of root causes in application-and device-specific areas, and performing detailed risk analysis.
3.6.1 Communications 1.
The working group recommends the increased use of communications plans to provide for the dissemination of information and activities to key stakeholders and the public.
(Section 3.2) 2.
The working group recommends that generic communications for materials be more explicit in referencing incident reports for precursor and prior-similar incidents. (Section 3.4.2)
Pilot Project 3 Final Report on Operating Experience Page 15 3.6.2 Procedures 3.
The working group recommends that NMSS and the Office of State and Tribal Programs adopt the objectives and attributes of an effective operating experience evaluation program in their procedures. (Section 3.2) 4.
The working group recommends that conforming changes be made to all affected NMSS and STP procedures to reflect current practice concerning supervisory review, abatement of GAP, and adoption of a the proposed self-assessment process. (Section 3.4.2) 5.
The working group recommends that NMSS and STP procedures be upgraded to provide guidance for use of risk analysis tools and insights. (Section 3.4.2) 3.6.3 Clearinghouse 6.
The working group recommends that NRC and the Agreement States explore development of a central clearinghouse to serve as a focal point for communicating and coordinating the wide range of operating experience information. (Section 3.2) 7.
The working group recommends the expanded use of electronic media as a means to increase participation and to promote effectiveness, efficiency, and openness in the National Materials Program. (Section 3.3) 8.
The working group recommends that the information clearinghouse be made publicly available so that licensees that implement safety measures can benefit from lessons learned and so that stakeholders can better access to and more fully participate in the regulatory process. (Section 3.4.1) 3.6.4 Process 9.
The working group recommends that formal methods be developed to better incorporate National Materials Program operating experience activities in NRC and Agreement State planning and assumptions. (Section 3.3) 10.
The working group recommends that the National Materials Program approach to operating experience evaluation focus more on decision-oriented activities and that working groups and surveys be pursued on a more selected basis. (Section 3.3) 11.
The working group recommends that more effort should be given to data evaluation processes and feedback to include more trending of root causes in application-and device-specific areas. (Section 3.4.3) 12.
The working group recommends that a detailed risk analysis be performed on a single device, based on appropriate operational data collected from NRC and Agreement States, to provide the basis for recommendations to enhance licensing and inspection programs. (Section 3.4.3.2)
Pilot Project 3 Final Report on Operating Experience Page 16 ENCLOSURE 1 OBJECTIVES AND ATTRIBUTES OF THE REACTOR OPERATING EXPERIENCE PROGRAM The task force recommended the following three objectives for the agencys reactor OE program.
OE is collected, evaluated, communicated, and applied to support the agency goal of ensuring safety.
This objective is the primary focus of the agencys reactor OE program. To accomplish this objective, the agency will have an effective, coordinated program to systematically collect and evaluate OE, identify and resolve safety issues in a timely manner, and apply lessons learned from OE to support the agency goal of ensuring safety. The agency will share OE information with the nuclear industry in a timely manner so the industry can ensure safety.
OE is used to improve the effectiveness, efficiency, and realism of NRC decisions.
Evaluations of OE provide fundamental information necessary to improve safety assessments and the realism of NRC decisions. Lessons learned from OE evaluations will be used to improve NRC regulatory programs, including licensing and inspection.
The public, Congress, and other external stakeholders are provided with accurate, timely, and balanced information regarding operational experience, including actual or potential hazards to health and safety.
Timely sharing of OE information with the public, Congress, and other external stakeholders will enhance their understanding of the performance of licensed plants.
To accomplish the objectives of a reactor OE program, the task force identified the following attributes it believes are necessary for the program to be effective:
1.
Clearly defined and communicated roles and responsibilities.
Management expectations are clearly articulated and communicated and organizational roles and responsibilities clearly defined. Organizational responsibilities include collection, screening, evaluation, corrective action, and followup activities.
Responsibilities for internal and external coordination and communications are also clearly defined, including the interfaces between the organizations reviewing OE and the inspection, licensing, and research organizations. A single point of contact is established to provide overall coordination for responsibilities distributed throughout the agency.
Pilot Project 3 Final Report on Operating Experience Page 17 2.
Efficient collection, storage, and retrieval of OE.
Sources of OE for collection, storage, and retrieval are identified. These sources include OE from industry and foreign sources, as well as agency-generated information. The sources of OE are sufficiently comprehensive and of sufficient quality to meet specific user needs and the collection and storage minimizes duplication by multiple organizations. Data systems provide user-friendly retrieval capabilities for a wide range of users.
3.
Effective screening of OE for followup evaluation.
OE is promptly screened for followup using appropriate criteria and thresholds to determine whether the OE is, or could be, risk significant; has, or could have, generic implications; or is, or could be, important from a public confidence perspective. Priority is assigned for evaluation commensurate with the overall significance of the OE.
4.
Timely communication of OE to stakeholders for information or evaluation.
OE is communicated to stakeholders in a timely manner for information or evaluation.
The communication clearly and concisely identifies the issue of concern and puts its significance in proper perspective.
5.
Timely and thorough evaluations of OE to identify trends, recurring events, or significant safety issues for appropriate followup actions.
Timely and thorough evaluations of OE will involve both short-term and long-term efforts to identify trends, recurring events, or significant safety issues. Timely short-term evaluations are necessary to promptly initiate regulatory actions aimed at resolving immediate safety issues and precluding or correcting similar conditions at other facilities.
Long-term evaluations to assess safety performance typically use a broader range of OE input, including reports on individual events and conditions, performance measures, and retrospective information. Long-term evaluations also identify trends and safety issues and their implications for NRC programs. Evaluations are sufficiently thorough to understand the event or condition, contributing factors, root causes, safety significance, and generic implications. Appropriate internal and external organizations are involved, as necessary, to ensure evaluations are complete and accurate.
6.
Timely decisions on implementation and appropriate followup resulting from the review of OE.
Timely decisions and actions are taken in response to short-term and long-term evaluations of OE. The decisions address the need for externally directed regulatory actions as well as appropriate changes to NRC programs. The OE program identifies activities or actions necessary to ensure timely implementation and followup in response to a regulatory determination. The OE program also assesses the effectiveness of regulatory and licensee actions taken in response to a lesson learned from the OE program.
7.
Periodic assessments of the OE program to determine its effectiveness and to identify needed improvements.
Periodic assessment of the OE program is conducted to determine how effective the
Pilot Project 3 Final Report on Operating Experience Page 18 agency has been in using OE to reduce the severity or recurrence rate of industry events. An effectiveness review provides feedback from stakeholders to agency management and recommends corrective actions to address identified deficiencies.
Pilot Project 3 Final Report on Operating Experience Page 19 ENCLOSURE 2 SECY-02-0074 SUCCESS MEASURES 1.
Provide insights into whether an informal coalition of State programs and NRC, as envisioned under the Alliance Option, is viable and can produce results meeting needs of both NRC and Agreement States.
This pilot demonstrated that a diversity of approaches can be applied to the National Materials Program (NMP). The working group believes that the operating experience evaluation should focus on decision-oriented activities and recommends that working groups and surveys be pursued on a more selected basis. The working group sees substantial benefit in making the operating experience evaluation an on-going and iterative process between NRC and Agreement States, rather than a product-driven activity, as provided in the initial Alliance concept.
2.
Provide insights that the Alliance Option has the potential to be a sustainable program structure for the NMP which will result in fewer NRC resources being needed for the development of products needed by NRC and the Agreement States.
This pilot demonstrated that electronic media can be used to effectively achieve initiatives and partnering between the Agreement States and NRC, without the undue burden and expense of travel. This pilot project was conducted entirely through the use of electronic media. No team meetings involving travel were held. This pilot demonstrated the ability of working groups to be conducted cost-effectively through biweekly conference calls and without undue burden to the sponsoring organizations during periods of absence. The working group does not believe that the quality of its review would have been substantially enhanced by travel for meetings and does believe that much more can be accomplished by the expanded use of telephone and video conferencing.
3.
Provide demonstration that the States can assume and carry out greater responsibility for the development and maintenance of products under the NMP.
This pilot demonstrated that the Agreement States can provide the diversity of valuable perspectives needed for an effective operating experience program. With about 80 percent of all materials licensees, the Agreement States are uniquely well-positioned to demonstrate leadership in decision-oriented activities and to contribute to a clearinghouse that can serve mutual regulatory needs.
4.
Provide greater assurance that individual State programs are willing and able to commit resources, and to produce products on a schedule that can be utilized by NRC and the Agreement States.
The working group notes that substantial challenges exist for participation in NRC and Agreement State pilot projects. Pilots and working group activities are often conducted with existing resources, in most cases, as an added activity to the existing work of candidate participants. Individual States will participate in working groups when the issue is important to them, but an adequate level of involvement can be achieved when the solicitation of working group activities and of membership is done mutually by NRC and the Agreement States. The ability of the working group to achieve progress and meet schedules is highly dependent on the staffing and budgeting environment of the sponsoring organization. In general, no work is given up to support pilot and working
Pilot Project 3 Final Report on Operating Experience Page 20 group activities. Professional development activities also compete, and travel, in support of pilot activities may not be well-supported, even if NRC funding can be arranged via the NMP. States indicated a desire to participate where the initiative addresses concerns that are of mutual interest to them, but it was noted that some States may not share the same issue or be able to support projects requiring long-term commitments.
5.
Provide insights into whether the NRC will be able in the future to realize resource savings and efficiency gains through shifting of work to States under the Alliance structure.
This pilot identified opportunities for effectiveness and efficiency through Agreement State and NRC partnering and noted that results can be achieved on an on-going basis, through decision-oriented activities. Equity issues associated with the budgeting process and licensee fees remain to be resolved, however, if an operating experience information clearinghouse is established at NRC.
6.
Provide demonstration that NRC can operate in a NMP framework and will be able to use products which may have been developed by a single State or group of States without the need for major change.
This pilot acknowledges that both NRC and Agreement States do many things well. The major challenge appears to be in communicating effectively across organizations.
Likewise, early partnering on planning assumptions will help to shape outcomes that meet mutual needs. However, the working group recognizes that variations will continue to exist between NRC and States and from State-to-State. For many years, NRC has strived to achieve consistency across four, and previously five regions. In extending regulatory authority to 33 Agreement States, a certain tolerance for variability needs to exist, above certain thresholds. On a voluntary basis, it is unlikely that either NRC or States will uniformly adopt some proposals on a voluntary basis. However, the National Materials Program Working Group final report indicated that NRCs senior management can take the lead in making a commitment to the NMP and the products that are contributed by the Agreement States.
7.
Provide demonstration that NRC is willing to share with the States the establishment of priorities for the NMP including rule and guidance work needed to support the materials and waste arenas.
The processes of developing planning assumptions, establishing priorities and strategies, and implementing budgets are all subject to adding and shedding processes realized through operating experience information. The reactor oversight process has undergone revision for more than 8 years and is still subject to emergent challenges from operating experience. This pilot recognizes that an effective operating experience program for materials and waste should be viewed as a work-in-progress striving continuously for improvement, openness, and flexibility.
8.
Provide insights to help understand the degree to which Agreement States are aligned with NRC Policy direction to use a risk-informed and performance-based regulatory approach.
The use of risk information has substantial benefit for the allocation of resources for both NRC and Agreement State programs. A challenge is that risk-informed work products need to be packaged to serve end-user needs. States do not have staffs trained
Pilot Project 3 Final Report on Operating Experience Page 21 explicitly in the use of risk analysis methods. NRC has developed some tools, expertise, and methods for integrating risk into regulatory decision-making. More work is needed to codify these actions into procedures for use in the operating experience program.