ML23256A092

From kanterella
Jump to navigation Jump to search
Operator Licensing Annual Public Meeting Slides 09-21-2023
ML23256A092
Person / Time
Issue date: 09/21/2023
From: Correll J
NRC/NRR/DRO/IOLB
To:
References
Download: ML23256A092 (1)
v • d • e


Text

Public Meeting with Industry Operator Licensing Representatives September 21, 2023

Safety Message Jeff Correll jeffrey.correll@nrc.gov

Introductions and Opening Remarks Maurin Scheetz maurin.scheetz@nrc.gov

Examination Scheduling Brian Tindell brian.tindell@nrc.gov

The examination scheduling process is working.

  • Insert text, insert text, insert text, insert text, insert text, insert text, insert text, insert text, insert text, insert text, insert text, insert text, insert text

Be flexible.

25 30 35 40 45 50 2022 2023 2024 2025 Total Examinations

Written Examination Outline Generator Brian Tindell brian.tindell@nrc.gov

The outline generator saves time and increases accuracy.

The outline generator saves time and increases accuracy.

  • The publicly-available web-based code for the outline generator is coming soon.

Operator Licensing Public Dashboard Jeff Correll jeffrey.correll@nrc.gov

We are creating an Operator Licensing Public Dashboard.

o The NRC public website is being updated fall 2023 with an interactive display of exam results that will update automatically as exams are completed.

o This will supersede the static PDF document (ML23082A071) that is updated annually.

o Details on the level of interaction available on the public website are still being finalized.

Operator Licensing Program Feedback Theresa Buchanan theresa.buchanan@nrc.gov

The Operator Licensing Program Feedback database provides a lot of useful information.

  • Answers to previously asked questions about operator licensing topics
  • Historical information located in the Archived Questions OLPF
  • If additional information needed, can contact us

- https://www.nrc.gov/reactors/operator-licensing/contact-us.html

  • Response back to the individual

- May add to the OLPF or next NUREG revision depending on the topic and question Contact us for questions not answered by the Operator Licensing Program Feedback database.

Advanced Reactors Jesse Seymour jesse.seymour@nrc.gov

Advanced Reactor work is steadily increasing.

  • Ongoing reviews of white papers, topical reports, and licensing submittals from advanced reactor designers such as X-energy, TerraPower, Westinghouse, Oklo, General Electric, HOLTEC, and NuScale
  • Stakeholder engagement on topics like flexible plant operations (e.g., grid load following, H2 production, etc.),

artificial intelligence, simulation technologies, advanced digital interfaces, and remote and autonomous operations

Ongoing Regulatory Activities Jesse Seymour jesse.seymour@nrc.gov

Three ongoing rulemaking activities involve operator licensing topics.

1. Part 53 proposed rulemaking (with the Commission)
  • New framework for staffing, personnel qualifications, training programs, operator licensing examinations, and human factors
  • Provisions for load following, online refueling, customized licensed operator staffing, engineering expertise, facilities administering own operator licensing examinations, and the tailoring of operator licensing examinations
2. Parts 50 and 52 Alignment of Licensing Processes and Lessons Learned from New Reactor Licensing Rulemaking (proposed rule with the Commission):

Clarifies how the plant-referenced simulator applies to plant that is under construction Allows use of alternatives for in-plant JPMs when plant is under construction Allows licensee to request waiver of examination and test requirements when applicant applies for a license for unit(s) of the same design Requires licensee to maintain an applicants KSAs when there is time gap between passing initial licensing examination and participation as a licensed operator in the licensed operator requalification program Includes draft regulatory guidance that will accompany proposed rule:

Draft NUREG-1021 Rev. 13 Draft RG-1.149 Rev. 5 Addresses ANSI/ANS-3.5-2018 Three ongoing rulemaking activities involve operator licensing topics.

3. Regulatory Improvements for Production and Utilization Facilities Transitioning to Decommissioning Rulemaking (final rulemaking in process):

Includes an alternative approach that avoids the need for separate CFH training program approval Specifies that neither the STA nor licensed operators are required at a decommissioning plant Clarifies the applicability of training rule program requirements to avoid requiring unneeded programs Three ongoing rulemaking activities involve operator licensing topics.

There are additional resources for operator licensing for new reactors.

Issued draft ARCAP guidance on organization and human-system considerations to support non-LWR reviews under Part 50 or 52 Available at https://www.regulations.gov/document/NRC-2022-0078-0004 New FAQ resource for new reactor facility applicants Available at https://www.nrc.gov/reactors/operator-licensing/licensing-process/faq-operator-licensing.html

NUREG-1021 Revision 12 Effectiveness Review Maurin Scheetz maurin.scheetz@nrc.gov

We continue to monitor the use of NUREG-1021 Rev 12.

Effectiveness review was initiated in May 2022 to do the following:

Monitor the use of new and revised instructions and guidance in NUREG-1021, Revision 12 Determine if additional actions are needed (i.e., training or clarifications)

Check for any unexpected outcomes of the changes Interim Report 39 initial licensing examinations and 430 applicants April 2022 - April 2023

The effectiveness review focuses on the major changes made in Revision 12.

Data collection for the review includes:

Number of generic fundamentals questions per exam Quality of generic fundamentals questions How many applicants took/did not take an NRC Generic Fundamentals Examination before their initial licensing exam Inspection findings SRO/RO performance on generic fundamentals questions Critical task quality Number of Significant and Critical Performance Deficiencies Time applicants spend as extra-person on shift Use of ACAD allowances for related science degrees Ease-of-use metrics such as hours recorded for exam development, administration and grading, and colloquial feedback Number of appeals Average pass rate

There is room for improvement in the writing of generic fundamentals questions.

Interim Observations Average 7 generic fundamentals (GF) questions per examination (range 6-9)

Generic fundamentals are sampled in Tier 2 and Tier 4 of the written examination Psychometric errors found:

o GF questions marked as new/modified but do not meet the criteria for a new or modified question (they were bank) o GF questions not modified to make them site-specific (Plant 1) o GF question with low level of difficulty (LOD 1) o More than one answer choice correct, cue in stem, two-part question error in first part makes second part

Average performance on the set GF questions is 88%.

Interim Observations (430 applicants, 39 exams) 0 50 100 150 200 250 10-20%

40-50%

50-60%

60-70%

70-80%

80-90%

90-100%

Number of Applicants Percent Correct Percent of GF Questions Answered Correctly

Most applicants took the NRC GFE before they took a Rev 12 Examination.

Interim Observations 377 of 430 applicants previously took the NRC GFE (88%)

86%

89%

70.0%

75.0%

80.0%

85.0%

90.0%

95.0%

100.0%

no yes Average performance on GF Questions Applicants did not take NRC GFE Applicants took NRC GFE

SRO applicants performed slightly better than RO applicants on the set of GF questions.

Interim Observations 10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

RO SRO-I SRO-U Average Performance on GF Questions

It is difficult to compare NRC GFE performance with performance on the set of GF questions.

50 Facility Type No. of Exams No. of Examinees Mean Score

(%)

BWR 101 5245 90.5 PWR 101 9357 91.3 Total 202 14602 91.0 Facility Type No. of Exams No. of Examinees Mean Score

(%)

Multiple 39 430 88.5 50 question examination set of 6-9 questions

We added more actions for tracking performance on GF questions.

Revised the effectiveness review plan to:

Increase the duration of the review to allow for more meaningful data collection from applicants who have NOT previously taken the NRC GFE Include questions that may indirectly test generic fundamentals topics Added specific thresholds for performance that will trigger the NRC staff to revisit the sample plan distribution

Critical tasks meet the Rev 12 CT Methodology.

Interim Observations (from sample of six operating tests):

One task met the CT criteria but was not marked as a CT in the scenario guide (isolate a faulted steam generator)

Alternate boundary conditions used in about 50% of CTs Two operating tests had alternate boundary conditions that appeared to be arbitrary (the reason for time-based conditions not documented on simulator guide)

Critical performance deficiencies are infrequent.

Interim Observations (from 39 initial licensing examinations and 430 applicants):

5 critical performance deficiencies 18 significant performance deficiencies Guidance in NUREG-1021 for identifying CPDs and SPDs is being applied properly and consistently

There were very few observations of the use of new allowances in ACAD 10-001 Rev 2.

Interim Observations (from detailed review of ~10% of applications over 1 year):

2 observations of an SRO-instant applicant with a related science degree Most facility licensee training programs have maintained the previous requirement for applicants to spend at least three months on shift as an extra person (EPOS)

No significant change was observed in pass/failure rates.

Interim Observations (from 39 initial licensing examinations and 430 applicants):

Average Pass and Failure Rates Rev 11 Exams Rev 12 Exams Pass Rate 96.5%

96.9%

Failure Rate 3.5%

3.1%

Written Test Pass Rate 95 - 98%

97.1%

Operating Test Pass Rate 98 - 99%

99.5%

7 requests for NRC staff administrative review during first year of Rev 12 initial licensing examinations

No significant change was observed in written exam average scores.

Interim Observations (from 39 initial licensing examinations and 430 applicants):

Overall Written Examination Averages Rev 11 Exams Rev 12 Exams RO Portion 84 - 92%

89.1%

SRO Portion 89 - 91%

90.0%

Overall Exam 90.1%

89.6%

Industry Topics

Public Comments

Closing Remarks

Retrieved from "https://kanterella.com/w/index.php?title=ML23256A092&oldid=3768333"