ML19275H068

Topical Report Transformation Working Group Recommendations

ML19275H068
Person / Time
Issue date:	10/23/2019
From:	Joseph Holonich NRC/NRR/DLP/PLPB
To:
Holonich J, NRR/DLP/PLPB, 415-7297
References
Download: ML19275H068 (14)

Text

Topical Report Transformation Working Group Recommendations Problem Statement: The timeliness of topical report (TR) reviews has been raised as a concern by industry.

Options Recommended for Adoption in the Near Term:

1. Simplified/Limited Safety Evaluation and Reduced Scope TR PROBLEM: The staff conducts an extensive review and prepares a detailed safety evaluation (SE) for all TR submittals regardless of the TR complexity.

SOLUTION: Look to establish a review process that allows for the preparation of a Simplified SE using a standard form (3-6 months) or Limited SE (6-12 months)

RECOMMENDATION: Adopt. Implementing a test case in the watchdog timer revision, WCAP-16097-P/WCAP-16097-NP, "Common Qualified Platform Topical Report". A second potential pilot is under consideration.

Revise LIC-500? Yes Next Steps: A. Complete test case and evaluate efficiency B. Revise LIC-500 C. Expand to other TRs as appropriate

a. Applying the questions found in the flowchart on the next page, vendors would identify a TR that could have a less complex review.

b. A simplified SE using the form in Appendix A of this file would be completed in the 3 month time frame while a limited SE would be completed in a 6-12-month time frame.

c. Some examples of topical reports which may use either simplified or limited SEs include:

• Minor updates to approved methods

• Use of an approved correlation in a different, but approved, computer code

• Errata or other corrections to approved topical reports that require NRC staff review

• Minor changes to approved topical reports

• Design changes that do not impact the safety of the original design PROS

• Could reduce review time and staff determination.

• Focus resources on risk significant topical reports.

• Allows applicants to submit smaller changes which get reviewed faster

• Allows for a topical report update process

• Allows for a distinction based in TR complexity, not all TRs need to take 2 years.

CONS

• Challenging to establish the reduced review scope.

• Challenging to project what an appropriate SE would be

• Challenging to limit staff hours/writings based on years of working with current approach.

Simplified and Limited Safety Evaluations Simplified SE A simplified safety evaluation is an SE in which the NRC staff are generally not making a new regulatory finding but confirming that a previous NRC finding is still applicable. These evaluations require minimal documentation (e.g., a few sentences) and generally confirm that the NRC staff has reviewed the submitted material and believes that the conclusions in the original SE still apply. The NRC staff would expect to be able to complete such a review in 3 - 6 months.

Limited SE A limited safety evaluation is an SE in which the NRC staff are making a new regulatory finding, but that finding is of very limited scope and requires minimal assessment. These evaluations require some documentation (e.g., a few pages) as the NRC staff is making a new finding.

However, it is expected that the review effort is minimal, especially compared with the scope of the standard topical report. The NRC staff would expect to be able to complete such a review in 6 - 9 months.

2. Modify the TR Review Process PROBLEM: Acceptance reviews may involve several iterations before a TR is found acceptable for review. In addition, requests for Additional Information (RAIs) are an extra step in the process and there are often multiple iterations of RAIs.

SOLUTION: Focus the acceptance reviews to be more rigorous and risk informed plus eliminate deny with opportunity to supplement. Identify issues via the draft SE and eliminate RAIs except in rare situations where open items in the draft SE were not resolved and can easily be addressed via an RAI.

RECOMMENDATION: Adopt LIC-500 Revision? Yes Next Steps: A. Identify and complete test case and evaluate efficiency B. Revise LIC-500 C. Expand to other TRs as appropriate

a. Under the current process, vendors submit a topical report with the goal of getting NRC staff acceptance for use for the topical report.

b. Often this goal to get the topical report accepted results in continuous engagement with the NRC staff to address a lack of necessary information by allowing a rejection of the topical report with the opportunity to supplement and with working with staff through RAIs, and at times multiple RAIs. This vendor goal also extends the time for the staff to issue its final SE and rarely, if ever, results in a TR being not accepted.

c. Make acceptance reviews more rigorous and binary o Develop criteria and maybe a checklist for acceptance reviews that would help staff ensure that a topical report was complete and high-quality enough to begin the technical review.

o Staff would conduct its acceptance reviews with a much tighter sieve on what would be a topical report

d. Eliminate the ability to reject a topical report with opportunity to supplement. Rather, staff would either find the topical report complete enough to begin its review or reject the report. The rejection would with an opportunity to supplement extends the acceptance review. Under the revision to the process, staff would either accept or reject the topical report. If a TR was rejected, the staff would ensure that the missing information would impact one of the five key principals of risk informed regulation. A TR would not be rejected if information was merely missing and did not impact one of these five key principles. This would eliminate an extended acceptance review process.

e. Second big change would be to prepare the draft SE documenting the areas where it could find the topical report acceptable and include open items where it could not make a conclusion or found the topical report unacceptable. The draft SE would be provided to the vendor. No RAIs would be issued. If the vendor response to the draft SE closes all the open items, then the staff would proceed to a final SE.

f. If the vendor did not acceptably address all the open items in the draft SE and the open items could be easily resolved with more information, a round of RAIs would be issued.

When a vendor responds to the RAIs, staff would complete its safety evaluation. If the RAI responses did not resolve the open items, staff would find the TR unacceptable for use.

g. If a vendor does not respond to the draft SE open items or to the RAIs in the time frame specified, the staff would inform the vendor that the review was terminated without a conclusion on the acceptability of the TR.

h. Third change is requiring the vendor to prepare the nonproprietary SE instead of the staff. This change would mean that once the staff addresses the comments on the draft SE, a proprietary version would be sent to the vendor. The staff would not prepare a nonproprietary version like is currently done. Rather, the vendor would prepare the nonproprietary version and include it in the NP-A version of the TR.

PROS

• Vendors need to provide a complete and high-quality TR and to provide needed information in a timely manner.

• This process eliminates the multiple rounds of back and forth that leads to extended schedules.

• The schedule is maintained CONS

• External pressures often push the NRC to continue to work with vendors until an acceptable TR is achieve.

• Interactions to reach an acceptable TR take time that extend the review schedule.

• Vendors might not achieve their goal of having a TR acceptable for use in licensing actions.

• If TR reviews are terminated, vendors may lodge different complaints about not getting NRC staff to do what is needed.

3. Tiger Team Review Approach PROBLEM: All reviews are conducted in the same manner through a standard process with the same steps.

SOLUTION: For TRs where staff has great familiarity, expand the use of Tiger Teams to focus the review within a distinct period at the vendor facility.

RECOMMENDATION: Adopt LIC-500 Revision? Yes Next Steps: A. Revise LIC-500 B. Expand to other TRs as appropriate

a. For TRs where the staff has a great familiarity, the NRC staff could conduct an extended audit and write the SE at the audit.

b. Multiple knowledgeable staff members would be able to quickly determine what is required, and document it.

c. This approach has been used on select TRs in the past.

d. Appendix B list the analysis questions that would be used to determine if a Tiger Team was a viable option for a TR.

PROS

• Draft SE would be completed by the end of the audit for understanding (very early in the review, within the first six months)

• Consistent with Option 2, if the vendor did not acceptably address all the open items in the draft SE and the open items could be easily resolved with more information, a round of RAIs would be issued. When a vendor responds to the RAIs, staff would complete its SE. If the RAI responses did not resolve the open items, staff would find the TR unacceptable for use.

• Utilizes experienced staffs knowledge to greatly shorten the review process.

CONS

• Not applicable to all TRs.

• Requires not only the right type of TR, but also availability of knowledgeable staff.

4. Submittal of Documentation on the Docket PROBLEM: Information received on telephone calls that help clarity docketed information is not obtained without the preparation of a follow-up RAI and formal response.

SOLUTION: Have vendors provide the information without the need for an RAI or document the information in the final SE without the need for additional information from the vendor.

RECOMMENDATION: Adopt LIC-500 Revision? Yes Next Steps: A. Incorporate into next LIC-500 revision

a. Following a clarification telephone call or meeting, vendors would provide the needed information without the need for a formal RAI. Staff would use the letter as its base for reaching a conclusion on the TR.

b. The SE not only provides the staff evaluation but is itself a source of documentation.

The documentation is of information discussed with the submitting organization which was not docketed. Rather, it was institutionalized in the SE. If the submitting organization did not provide comments on what the staff documented in the draft SE it would de facto be agreeing with it. In addition, the submitting organization would be required to reflect the change in the -A version of the TR.

PROs

• Potential to greatly reduce number of RAIs and time needed to generate the final version of the SE.

• This could save a lot of time, as a many RAIs are there for documentation purposes (i.e., we heard the answer, we liked the answer, we just need to see that answer on the docket) - this is about 70%-80% of my RAIs.

CONs

• Potential confusion in terms of information on the submittal provided in the SE, but not provided by the vendor.

• PM burden in tracking all forms of information submission for SE generation and -A confirmatory reviews.

5. Administration PROBLEM: Administrative processes are not time efficient and have no methodology in place for prioritization, performance monitoring or accountability.

SOLUTOIN: Identify ways to improve the totality of administrative process.

RECOMMENDATION: Adopt LIC-500 Revision? No Next Steps: A. Evaluate changes in the administrative processes for TRs.

B. Solicit management sponsorship C. Review time charges for TRs and compare against budget.

D. Look for ways to do better resource sharing among divisions in the administrative areas.

a. The administrative processes are not always timely in supporting SE issuance. Improve accountability for timely support of TR documents.

b. Administrative support may often add time to the issuance of TR documents

c. The budget does not accurately reflect the amount of FTE involved in TR reviews and TRs are given a lower priority.

d. Load sharing among divisions is needed to help more effectively manage workloads.

PROs

• Making budgets accurately reflect the TR effort will provide for needed FTE to get the reviews completed in a more-timely manner.

• Increasing TR priority will keep needed staff, both technical and administrative, focused on completely the reviews quicker.

CONs

• Administrative staff are currently over loaded so additional pressure may not help.

• Increasing TR priority may not be possible if some other work is not lowered in priority

• Accurately reflecting TR FTE in the budget may make that area a focus for cuts by the Commission or Congress.

Options NOT Recommended for Adoption:

1. Eliminate the -A Version Approval Process for Speedier Regulatory Approval RECOMMENDATION: Do not adopt. The -A version is minimal effort and occurs after all the detailed review work is complete. It also consolidates all the relevant information into one document for ease of referring.

a. Staff would complete draft SE and send to vendor for proprietary review

b. Staff would ask vendor to provide final topical report that is updated to reflect & integrate any information/commitments that were made through the RAI response. The final topical report would represent the design/methodology that is being approved by staff

c. The updated topical would not contain previous RAIs or the staff SE

d. Upon receipt of final topical report, staff would issue final approval with the final SE.

e. Final SE would include the record of NRC RAIs and vendor docket submittals.

PROS

• Resource savings due to reduced review effort of a resubmitted -A topical

• Time savings from the current -A process

• Consistent with most other licensing, certification, and topical actions in Agency CONS

• Vendors could change TRs after the fact.

• May lose a valuable time saving opportunity by having vendors submit the SE under oath and affirmation - may be able to say more in SE that has not been previously stated on the docket and the submittal of the -A version has the vendors submitting that information on the docket.

2. Dedicated Technical Review Teams

a. Reorganize and establish branches dedicated to special projects and topical reports.

PRO

• Establishes special teams dedicated to TR reviews

• Avoids loss of resources to competing priorities such as licensing and regional support CON

• TR workloads may not justify the commitment of resources to a special branch

3. Develop Change Process for Topical Reports that Avoids Formal Revision RECOMMENDATION: Nothing to adopt. Previously the Office of the General Counsel has said we cannot make a conclusion on a vendor change process.

Revise LIC-500? No but revise NRC TR Webpage

a. Vendors would use the process change TRs for administrative corrections or to add data without requiring a topical report revision.

b. This is in use by several vendors in some areas. For example, Westinghouse has a change process for digital I&C designs which includes thresholds for when changes need to be made to the TR and submitted to NRC. In other areas, staff has been providing flexibility for changes in safety evaluations.

c. Process would be defined in SE.

PROS

• Resource savings due to reduced number of revisions CONS

• Describing limits on when change process may be invoked.

Appendix A Office of Nuclear Reactor Regulation Simplified Topical Report Safety Evaluation Topical Report Information Review Information Report Number: Division/ Branch:

Title:

Project Manager:

EPID: Reviewers:

Docket No.:

Staff Hours: Technical Reviewers Total Project Manager Use of Simplified Safety Evaluation Is this the review of narrow scope? Yes No Is this a revision to an accepted Topical Report? Yes No Does the TR change maintain the original SE Yes No conclusions?

Is this a small change? Yes No If any of the above questions are answered no, a simplified safety evaluation cannot be used.

Applicable Review Guidance Used Description of Topical Report Content (1000 Word Maximum)

Office of Nuclear Reactor Regulation Simplified Topical Report Safety Evaluation Topical Report Information Review Information Report Number: Division/ Branch:

Title:

Project Manager:

EPID: Reviewers:

Docket No.:

Technical Evaluation (Limited to space provided)

Office of Nuclear Reactor Regulation Simplified Topical Report Safety Evaluation Topical Report Information Review Information Report Number: Division/ Branch:

Title:

Project Manager:

EPID: Reviewers:

Docket No.:

Conclusions Conditions and Plant-Specific Action Items Approval Printed Name Signature Date Technical Branch Chief PLPB Branch Chief Appendix B Tiger Team Guidelines Contents What is a Tiger Team? ............................................................................................................. When can a Tiger Team be more efficient? ............................................................................. When can a Tiger Team be less efficient? ............................................................................... When can a Tiger Team be used? ........................................................................................... When should a Tiger Team be used? ...................................................................................... Conclusion ............................................................................................................................... What is a Tiger Team?

A Tiger Team consists of a group of experienced reviewers who are focusing on a single application to greatly decrease the review time of that application. In general, it is best to think of a regular review as having a primary reviewer and a secondary reviewer for each technical subject. In such instances, the bulk of the review (e.g., 80-90%) will be performed by the main reviewer and the secondary reviews role (the remaining 10-20%) is limited to peer review.

The Tiger Team approach can be thought of as elevating the secondary reviewer to main reviewer status. Thus, in a Tiger Team, there will be two main reviewers for each technical subject instead of one.

When can a Tiger Team be more efficient?

A Tiger Team can be more efficient during portions of the review process which require the NRC staff to reach technical conclusions. Two experienced reviewers working as team can often reach technical conclusions much faster than each reviewer separately. For example, deciding what is needed in the Safety Evaluation, what is an Open Item, and if the RAI response fully addresses the open item can often be reached much faster by two reviewers working together than reviewers working alone. This increased efficiency has the potential to decrease the total review time by over 50%.

When can a Tiger Team be less efficient?

For many parts of the review, the Tiger Team is less efficient. For example, suppose a topical report is 50 pages. Further suppose it would take the main reviewer about 40 hours4.62963e-4 days <br />0.0111 hours <br />6.613757e-5 weeks <br />1.522e-5 months <br /> to understand the topical report. With two main reviewers, a total of 80 hours9.259259e-4 days <br />0.0222 hours <br />1.322751e-4 weeks <br />3.044e-5 months <br /> would be used to understand the topical report, twice the number of hours of a regular review.

When can a Tiger Team be used?

There are certain requirements which need to be fulfilled if a Tiger Team is to be used. Those requirements are:

1) Does the agency believe the topical report should be prioritized?

Those reviews who are participating on the Tiger Team will be unviable for other work during portions of the review. Unlikely other work which tends to take a few hours a day, much of the Tiger Team work will results in the reviewer being unviable for one or two weeks at time. Therefore, management must ensure that they wish to commit the staff resources to the Tiger Team.

2) Are two experienced technical reviewers per technical subject available?

Tiger Teams require the availability of at least two experienced reviewers per technical subject. Often those same reviewers are assigned numerous other tasks and it is possible other assigned work may prevent their availability. Such challenges can be resolved by managers re-assigning priorities and duties. However, for some review areas, the NRC may not have two experienced reviewers. It is common for the NRC to review new or unique topical reports. In such reviews, it seems unlikely that a Tiger Team approach would increase the efficiency of the review, as technical reviewers are often hesitant to reach quick technical conclusions on new topic areas.

3) Is the applicant willing to support an accelerate schedule?

While it is the dream of most applicants to have a very fast review, applicants need to ensure that they can support such an accelerated work schedule. Tiger Teams generally consistent of longer audits (e.g., a week or two instead of a few days), quicker formulation of RAIs (e.g., RAIs within three months instead of a year), and longer phone calls (e.g., 4 or 5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> instead of 30 or 40 minutes). Applicants will generally not mind committing to such schedules, but they need to be aware of them and plan for them.

When should a Tiger Team be used?

Just because a Tiger Team can be used does not mean it should be used. While a Tiger Team may dramatically reduce some portions of the review process, it does double the resource cost of other portions of the review process. The resource cost for portions of the review process where the NRC needs to read and understand information is doubled (e.g., reading and understanding the initial topical report, reading and understanding any supplemental information, reading and understanding RAI responses). This increase in resource cost can be greatly overcome by the NRC staff coming to much faster technical conclusions and documenting those conclusions in the SE result in completed reviews in 9-12 months instead of two years.

The following are examples of reviews where a Tiger Team should not be used:

1) A topical report on a new technical area.

For a new technical area, it is unlikely that the NRC staff will have the necessary experience to expedite the review process. It is even possible that the staff will require two main reviewers for each technical subject to complete the review within the normal review schedule.

2) An n-th of a kind, well understood, topical report.

Surprisingly, it is probably not a good idea to use a Tiger Team on a topical report that staff often reviews as well. A Tiger Team could be used here, but it seems like it would be overkill. For example, the NRC staff has reviewed over 30 critical heat flux and critical power topical reports and have recently published NUREG/KM-0013 which captures the process used to review such information. It would seem unnecessary to use a Tiger Team on a new critical heat flux topical report, and likely result in a longer and more resource intensive review than using the normal review process.

The following is the main example of the type of review where a Tiger Team should be used:

1) A topical report with technical subject familiar to the NRC staff with high agency priority.

Conclusion The Tiger Team approach has increased costs in reading and understanding material, but often those costs are offset by dramatically shorter review schedules and possibly less resources spent overall. Such an approach should not always be used, but its use, where appropriate, has the potential to great increase the effectiveness and efficiency of the NRC.