ML19273A904

2019 Public Meeting Presentation: Operating Test, KA Catalog, Examination Schedule

ML19273A904
Person / Time
Issue date:	10/15/2019
From:	Brian Tindell NRC/NRR/DIRS/IOLB
To:
Tindell B, NRR/DIRS, 301-415-2026
References
Download: ML19273A904 (13)

Text

Public Meeting with Industry Operator Licensing Representatives October 15, 2019

Agenda

• Introduction and Opening Remarks

• Operating Test

- Performance Deficiency

- Critical Tasks

- Relocating Technical Specification Testing

• Knowledge and Abilities Catalog

• Examination Schedule Efficiencies

• Topics from Industry

• Public Comment

• Adjourn 2

Operating Test

• Performance Deficiency

• Critical Tasks

• Relocating Technical Specification Testing 3

KA Catalog

• Importance Ratings for Section 5 (Components) &

Section 6 (Theory)

• Publication Schedule 4

Exam Schedule Efficiencies

• The NRC issues a yearly Regulatory Issue Summary (RIS) soliciting exam scheduling requests from facilities. Reponses are voluntary.

• The staff is proposing a RIS enclosure with regional availabilities on a monthly basis.

• A one-exam-per-month scheduling approach promotes efficiency and the effective utilization of resources

• The RIS will continue to request both primary and alternate dates for exams.

• Assignment of exam dates to facilities will be on a first-come-first-serve basis.

• Facilities requesting the same dates may be asked to coordinate and/or swap.

• Scheduling is regional-specific. However, a different Region may cover an exam outside of their region if a facilitys exam cannot be otherwise accommodated.

• The following slide contains an example of such a RIS enclosure 5

Revised Scheduling RIS Example 6

Topics from Industry 7

Public Comment 8

Performance Deficiencies during Simulator Scenario Operating Tests During administration of the simulator operating test, the NRC examination team observes an applicants performance in a licensed operator position. In accordance with 10 CFR 55.45, Operating Tests, the NRC uses these observations to determine if an applicant has demonstrated, [emphasis provided] an understanding of and the ability to perform the actions necessary to accomplish a representative sample from the 13 items listed in § 55.45(a).

In accordance with 10 CFR 55.33, Disposition of an Initial Application, by approving an initial application the NRC has determined the applicant has learned to operate a facility competently and safely, and additionally, in the case of a senior operatorthe applicant has learned to direct the licensed activities of licensed operators competently and safely.

Performance Deficiency Definition In the context of the simulator operating test, a performance deficiency is an observed action or inaction (including operational tasks, procedure/process implementation, communications, and administrative functions), or a statement of understanding or intent, which demonstrates a lack of ability or understanding as outlined by an established standard for operator performance (e.g., facility procedure, policy, learning objective, regulatory requirement, etc.).

Implementation Guidance All performance deficiencies shall be noted on the applicants Form ES-303, regardless of whether or not they affect the applicants simulator operating test score. In order to determine if the performance deficiency causes a point deduction, it is assessed using the rating factor competency descriptions established by NUREG-1021, Forms ES-303-3/4 and Appendix D, Section E. If the examiner concludes a No response exists after comparing the performance deficiency against the ES-303-3/4 rating factor questions and assessing the performance using the guidance below, the examiner is required to lower the applicants score in the corresponding rating factor(s) by the appropriate number of points.

When applying the performance deficiency definition above, the following considerations should be used when determining whether or not the performance deficiency affects the applicants simulator operating test score.

For a performance deficiency related to the applicants ability to operate (including operating controls, directing operations, and implementing procedures), a point deduction in the applicable rating factor(s) occurs if either of the following criteria is observed:

1. The applicants action or inaction fails to meet an expectation established by a standard for operator performance.

2. The applicants inaction or intent to perform an action, would have failed to meet an expectation established by a standard for operator performance but was corrected by another crew member.

For a performance deficiency related to the applicants understanding (including diagnosing plant conditions and understanding system operation), a point deduction occurs if any of the following criteria are observed:

1. The applicant exhibits a lack of understanding by providing an erroneous response to a follow-up question related to an observed potential performance deficiency during the scenario.

2. A statement (verbal or written) reveals a lack of understanding (related to required operator knowledge), that is uncorrected by the applicant, using his or her own knowledge, prior to taking improper action based on the misunderstanding or prior to the initiation of the next scenario event.

3. A delay in taking a required action reveals a lack of understanding, that is uncorrected by the applicant, using his or her own knowledge, resulting in action not being taken prior to complicating the crews response to the event.

Note: Typically, an unsatisfactory delay in taking required action should be assessed in Rating Factor 3.a as lack of ability to manipulate controls in a timely and accurate manner. However, if misunderstanding is the primary cause of the applicants delay, then Rating Factor 3.b can be considered. The NRC does not expect the applicant to exhibit an immediate and unwavering understanding of plant conditions at all times. The applicant will need a period of time to evaluate plant conditions and come to a state of understanding. The applicant may pause while operating the plant to establish, improve, or confirm this understanding. This behavior is consistent with industry expectations and standards for operator human performance techniques (e.g. STAR). The examiner must provide evidence of the applicants lack of understanding while operating plant controls in order to cite Rating Factor 3.b for a point deduction. This can be confirmed by any of the following:

• The applicant relies on another crew members knowledge to effectively assess and operate the plant.

• The applicant does not successfully exhibit understanding within the constraints required to take prompt and prudent action to avoid more complicated plant conditions.

• The applicant provides an erroneous response to follow-up questions.

For a performance deficiency related to the applicant's ability to communicate, a point deduction occurs if the following criteria is observed:

• Communication made by the applicant is inaccurate, incomplete, or not in accordance with the licensees established means for ensuring precise communications (e.g., three-part communication), the communication is not self-corrected, and the communication is needed to support effective plant operation.

Note: Temporary miscommunication that results simply from one applicant mishearing another, as long as it is corrected by licensees established means for ensuring precise communications, shall not result in a point deduction.

Clarifying Guidance

This guidance is intended to ensure greater consistency in identification and grading of performance deficiencies. In any instance where this guidance is in conflict with NUREG-1021, Revision 11, then the requirements of NUREG-1021 take precedence.

In accordance with NUREG-1021, ES-303 D.2.b, no point deduction is taken for the first performance deficiency assessed in the communications competency, and the minimum score for communications rating factors is a 1.

In accordance with NUREG-1021, ES-303 D.1.d, a single performance deficiency shall be cited in no more than two different rating factors.

For performance deficiencies related to technical specifications, follow the grading process detailed in ROI-17-13 (ML17213A397).

For a single performance deficiency to result in a three-point reduction within a single competency rating factor, it must result in the failure to meet a critical task as defined by NUREG-1021, Appendix D, Section D.

Proposed Critical Task Approach for Rev. 12 of NUREG-1021 Errors related to Critical Tasks (CTs) are referred to as Critical Errors. A Critical Error on the part of an applicant results in failure of the simulator operating test. Inherent in the evaluation of Critical Errors is the need to properly categorize Performance Deficiencies, as defined, including those instances where individual errors are corrected by other crew members. CTs that are initially incorporated into a scenario are referred to as preidentified CTs. The difficulty level and equitable administration of the operating test must be considered when assessing the appropriateness of the number of such preidentified CTs in a scenario or scenario set. ES-301 outlines the target number of preidentified CTs per scenario. Preidentified CTs are part of the scenario design and are included on the ES-D forms. In contrast, post-scenario CTs are created by the occurrence of unexpected applicant actions during a scenario. Both preidentified and post-scenario CTs are identified and designated using the same criteria (discussed below).

If a facility maintains a CT list derived from guidance provided by their vendor owners group, this list should be referenced as an aid in identifying CTs. Be aware that such CT lists have not been subjected to NRC review and may contain tasks that are not sufficiently discriminatory for the purposes of an NRC operating test. Furthermore, scenario CTs are dependent upon both specific equipment configurations and malfunctions, while owners group CTs are based upon specific accident sequences that may not match those of a given scenario.

In conjunction with facility CT lists (or in the absence of such a list), the following guidance shall be applied in the identification and designation of CTs:

• Where a success path exists, applicants must prevent significant safety challenges; this includes preventing conditions that warrant initiation of emergency depressurization (BWR specific), result in orange or red path critical safety functions (Westinghouse and AP-1000 specific), warrant transitioning to functional recovery guidelines (CE specific),

or adversely impact the implementation of those emergency operating procedure actions essential to the mitigative strategy for the event in question (B&W specific).

• Applicants must properly implement procedural actions for mitigating significant safety challenges when those actions directly lead to restoring safety functions.

• Applicants must properly implement procedural actions of emergency operating procedures when those actions are essential to an events overall mitigative strategy.

• When possible, applicants must avoid invalidating FSAR accident analysis assumptions.

• Applicants must avoid unnecessarily creating situations that would result in EAL entry or escalation on loss or potential loss of more than one fission product barrier per the facilitys EALs.

• Note: in applying this guidance, the specific equipment configurations, malfunctions, and accident sequences of a given scenario must be carefully considered to ensure that CTs are designated and bounded in a manner this is reasonable for evaluation purposes.

Additionally, CTs must also possess each of the following attributes:

• Initiating Cue: An initiating cue is an expected signal or notice (indication, alarm, communication, or procedure step) that designates when a CT should be performed.

The cue need not indicate that the action is a CT.

Proposed Critical Task Approach for Rev. 12 of NUREG-1021

• Performance Feedback: During the time span of a CT, performance feedback must be available to at least one member of the crew. This feedback provides the crew member with information about the effect of the crews actions or inaction related to or because of the CT. The crew must be able to oversee that its action had an impact or that its inaction is causing plant conditions to degrade.

• Measurable Performance Standard: The measurable performance standard for a CT consists of observable actions taken by at least one member of the crew. Consequently, the performance standard for a CT includes both expected action and boundary conditions that clearly identify at what point a CT must be accomplished; such conditions must be objective in nature. The ES-D forms should document the limits for each preidentified CT before the examination begins.

In establishing objective boundary conditions, the criteria used shall be developed based upon the guidance in the following list, which is presented in the preferred order of usage:

1. Thresholds where safety functions are severely challenged or lost;

2. Thresholds that result in changes to the mitigative strategy for an event, such as transitions to contingency procedures or functional recovery procedures;

3. Accident analysis assumptions, bounding conditions, and limits from the facilitys FSAR;

4. Technical Specification Safety Limits;

5. Exits or transitions from the procedure that first directs CT accomplishment;

6. The expiration of a reasonable period of time or exceeding of a parameter value as mutually agreed upon by the Chief Examiner and facility (note: for emergent CTs during major transients, judgement of the lead examiner present dictates scenario endpoint).

Significant Errors are more severe than errors that would only result in a single point deduction, but do not meet the criteria for a Critical Error. The identification of a Significant Error only occurs post-scenario because of an error made during the scenario by one or more applicants. Significant Errors result in larger Rating Factor point deductions (i.e. 2 points) than errors not meeting the criteria to be considered a Significant Error. Significant Errors consist of:

• Errors that either cause an automatic RPS/ESF actuation or that warrant or involve a manual RPS/ESF actuation that should have otherwise been avoidable had the applicant responded to the event as expected (note that subsequent ESF actuations that do not alter equipment alignments would not be considered Significant Errors);

• Errors constituting significant reactivity management events under the facilitys program;

• Errors that would result in EAL entry or escalation per the facilitys EALs.

Table 1: Error Grading Summary by Type Competency / Critical Error Significant Error Error (regular)

Rating Factor (go/no-go criteria) point deduction point deduction Rating Factors other than Failure of Simulator 2 points 1 point those in Communications Operating Test Communications Rating Failure of Simulator 1 point Refer to ES-303 Factors only Operating Test (minimum R.F. score of 1)