NRC Inspection Manual 0609/Appendix I
text
Issue Date: 01/10/19 1 0609 Appendix I
NRC INSPECTION MANUAL IOLB
INSPECTION MANUAL CHAPTER 0609 APPENDIX I
LICENSED OPERATOR REQUALIFICATION
SIGNIFICANCE DETERMINATION PROCESS
Effective Date: 01/01/2019
0609I-01 PURPOSE
The Licensed Operator Requalification Significance Determination Process (SDP) is used for
determining the risk significance of findings identified during the inspection of licensed operator
requalification activities and licensed operator performance.
0609I-02 BACKGROUND
This SDP was designed to assess the risk significance of findings associated with Inspection
Procedure 71111, Attachment 11 (IP 71111.11), “Licensed Operator Requalification Program
and Licensed Operator Performance” in the following areas: (1) requalification examination
results, (2) biennial requalification written examinations, (3) annual requalification operating
tests, (4) administration of an annual requalification operating test, (5) requalification
examination security, (6) remedial training and re-examinations, and (7) the control room
simulator.
With regard to conformance with operator license conditions, such as the medical fitness of
licensed operators and compliance with the regulations contained in Title 10 of the Code of
Federal Regulations (10 CFR) Section 55.53, it may be appropriate to use traditional
enforcement to disposition violations. Inspectors should refer to guidance in Inspection Manual
Chapter 0612, “Issue Screening,” and the Enforcement Manual.
0609I-03 GUIDANCE
Figure I.1, a flowchart contained on the following pages, presents a series of yes/no decision
blocks for assessing licensed operator requalification and licensed operator performance
findings. Following the flowchart, a description of each flowchart block is presented.
Issue Date: 01/10/19 2 0609 Appendix I
Figure I.1 – Licensed Operator Requalification SDP Flowchart
1
Licensed Operator
Requalification Finding
2
Related to
Requalification
Exam Results?
3
Failure rate
greater than
40%?
White Finding
NO Go to A
on page 3
YES
YES
NO Green
Finding
4
Related to
Biennial Requal.
Written Exam
Quality?
NO
NO
5
Were greater than
40% of the
reviewed written
examination
questions flawed?
YES
YES
White Finding
NO Green
Finding
Issue Date: 01/10/19 3 0609 Appendix I
Figure I.1 – Licensed Operator Requalification SDP Flowchart (continued)
A from
page 2
YES
Green Finding
NO
White
Finding
YES
6
Related to
Annual Requal.
Operating Test
Quality?
7
Were greater
than 40% of the
reviewed JPMs
flawed?
NO
NO
Go to B
on page 4
YES
Green
Finding
NO
9
Related to Licensee
Admin. of an Annual
Requal. Operating
Test?
8
Were greater than
40% of the
reviewed simulator
scenario events
flawed?
White
Finding
Finding
YES
NO
Issue Date: 01/10/19 4 0609 Appendix I
Figure I.1 – Licensed Operator Requalification SDP Flowchart (continued)
B from
page 3
YES
YES
NO
10
Related to
Requalification
Exam Security?
11
Was there an actual
effect on the equitable
and consistent
administration of any
examination required
by 10 CFR 55.59?
NO
NO
Go to C
on page
5
YES
Evaluate using
traditional enforcement
against 10 CFR 55.49.
and evaluate using IMC 0609 Appendix M.
Green
Finding
NO
12
Related to
Licensee
Remedial Training
and Re-exams?
Green
Finding
Issue Date: 01/10/19 5 0609 Appendix I
Figure I.1 – Licensed Operator Requalification SDP Flowchart (continued)
C from
page 4
YES
Reference appropriate
SDP to determine
significance of operator
performance issues
YES
Green
Finding
NO
13
Related to Simulator
Performance,
Testing,
Maintenance, or
Modification?
16
Re-evaluate the finding by
entering the SDP at block 1.
NO
14
Was a simulator
performance,
modeling, or
fidelity deficiency
identified?
(Simulator testing,
maintenance, or
modification deficiency)
15
Did deficient simulator
performance, modeling, or
fidelity negatively impact
operator performance in an
actual plant event where the
risk increase due to the
operator performance was
>10E-6 delta CDF or >10E-7
delta LERF?
YES
Green
Finding
NO
Issue Date: 01/10/19 6 0609 Appendix I
Flowchart Block Descriptions:
- 1 – The SDP starts after a single licensed operator requalification finding is identified from IP 71111.11 and screened through Manual Chapter 0612, Appendix B. Each specific finding must
be evaluated separately.
- 2 – This is the top-level entry block associated with licensed operator performance as
measured by the results of the requalification examinations required by 10 CFR 55.59(a)(2).
This block is answered “yes” or “no” based upon completing the specific guidance contained in
Section 03.03 of IP 71111.11 and upon completing the screening of inspection issues in
accordance with IMC 0612.
- 3 – Based upon the requalification examination results collected at the end of the testing cycle,
was the failure rate greater than 40%? This block will be answered “yes” if either:
(a) The individual examination failure rate is greater than 40% (IP 71111.11, Line 4 of Table
03.03-1), or
(b) The crew simulator scenario failure rate is greater than 40% (IP 71111.11, Line 7 of Table
03.03-1).
- 4 – This is the top-level entry block associated with the quality of biennial requalification written
examinations that are required by 10 CFR 55.59(a)(2). This block is answered “yes” or “no”
based upon completing the specific guidance contained in Section 03.04.a and Appendix B of IP 71111.11.
- 5 – Were greater than 40% of the reviewed written examination questions flawed? In
answering this question, the inspector will need to review the results from section 03.04.a and
Appendix B of IP 71111.11. If the answer to this block is “yes,” then a White finding results,
based upon a higher percentage of flawed written examination questions used on a
requalification examination required by 10 CFR 55.59(a)(2). If the answer to this block is “no,”
then a Green finding results, based upon a lower percentage of flawed questions or other
written examination deficiency.
- 6 – This is the top-level entry block associated with the quality of annual requalification
operating tests that are required by 10 CFR 55.59(a)(2). This block is answered “yes” or “no”
based upon completing the specific guidance contained in Section 03.04.b and Appendix C of
- 7 – Were greater than 40% of the reviewed job performance measures (JPMs) flawed? In
answering this question, the inspector will need to review the results from Section 03.04.b and
Appendix C of IP 71111.11. If the answer to this block is “yes,” then a White finding results,
based upon a higher percentage of flawed JPMs used on a requalification examination required
- 8 – Were greater than 40% of the reviewed simulator scenario events flawed? In answering
this question, the inspector will need to review the results from Section 03.04.b and Appendix C
of IP 71111.11. If the answer to this block is “yes,” then a White finding results, based upon a
higher percentage of flawed simulator scenario events used on a requalification examination
required by 10 CFR 55.59(a)(2). If the answer to this block is “no,” then a Green finding results,
Issue Date: 01/10/19 7 0609 Appendix I
based upon a lower percentage of flawed simulator scenario events and JPMs (checked in
block 7 above), or based upon some other operating test deficiency.
- 9 – This is the top-level entry block associated with the licensee’s administration of annual
requalification operating tests that are required by 10 CFR 55.59(a)(2). This block is answered
“yes” or “no” based upon completing the specific guidance contained in Section 03.04.c and
Appendix D of IP 71111.11.
- 10 – This is the top-level entry block associated with requalification examination security. This
block is answered “yes” or “no” based upon completing the specific guidance contained in
Section 03.04.d and Appendix E of IP 71111.11.
- 11 – Was there an actual effect on the equitable and consistent administration of any
examination required by 10 CFR 55.59? In these instances, a licensed operator has gained an
unfair advantage on an examination required by 10 CFR 55.59, and this condition was not
corrected prior to being authorized to resume licensed duties. These occurrences can be willful
or intentional (“cheating”) or unintentional. Examples of gaining an unfair advantage on an
examination include: (1) a licensed operator obtains unauthorized assistance during an
examination, such as by receiving assistance on a test item during an examination from an
unauthorized individual or by copying answers from another examinee; (2) a licensed operator
obtains specific knowledge of or is exposed to requalification examination content prior to taking
the requalification examination; (3) a licensed operator is used to validate requalification
examination test items during exam development, and is then subsequently administered a
requalification examination with any test items duplicated from those that the operator
previously validated. IMC 0609, Appendix M should be used to evaluate the significance of
these types of inspection findings. Note that the traditional enforcement process may also be
used for violations of 10 CFR 55.59 (e.g., in cases where the violation involves willfulness or
impacts the NRC’s ability to perform its regulatory function). Refer to IMC 0612 and the
Enforcement Manual for guidance on dispositioning traditional enforcement violations.
- 12 – This is the top-level entry block associated with remedial training and re-examinations,
which occurs whenever a licensed operator fails any portion of a requalification examination
required by 10 CFR 55.59(a)(2). This block is answered “yes” or “no” based upon completing
the specific guidance contained in Section 03.04.e and Appendix F of IP 71111.11.
- 13 – This is the top-level entry block associated with control room simulator performance,
maintenance, and testing, as specified in 10 CFR 55.46. This block is answered “yes” or “no”
based upon completing the specific guidance contained in Section 03.04.g and Appendix G of
- 14 – Was a simulator performance, modeling, or fidelity deficiency identified? This block is
used to differentiate between deficiencies associated with simulator performance (including
deficiencies with modeling or fidelity) and deficiencies associated with simulator testing,
maintenance, and modification. These issues are treated slightly differently in the SDP, due to
the potential for unrealistic operator training due to deficient simulator performance. If this block
is answered “no,” the deficiency is associated with simulator testing, maintenance, or
modification (as verified in the next block), and results in a Green finding. If this block is
answered “yes,” proceed to block 15.
- 15 – Did deficient simulator performance, modeling, or fidelity negatively impact operator
performance in the actual plant during a plant event? The concern with this block is that the
Issue Date: 01/10/19 8 0609 Appendix I
simulator provided un-realistic or negative training to licensed operators (due to deficiencies in
simulator performance, modeling, or fidelity), and that this un-realistic simulator training was the
primary cause of negatively impacted operator performance during an event. Reference
appropriate SDP guidance (At-Power, Shutdown, or others) to determine if the negative
operator performance resulted in a risk increase of greater than 10E-6 delta CDF or greater
than 10E-7 delta LERF. Qualitative SDP results may also be used to determine if the risk
increase is greater-than-Green. If the answer to this block is “yes,” then this results in a finding
with significance commensurate with the risk increase due to the negative operator
performance, based upon the appropriate SDP guidance. If the answer to this block is “no,”
then this results in a Green finding, since deficient simulator performance was still identified.
of top-level entry blocks, and block 16 should not occur unless all the entry blocks have been
answered “no.” If this is the case, re-evaluate the finding and enter the SDP at block 1, or
consult with the program office for guidance.
0609I-04 REFERENCES
1. IP 71111, Attachment 11, “Licensed Operator Requalification Program and Licensed
Operator Performance”
2. IMC 0612, Appendix B, “Issue Screening”
4. IMC 0609, Appendix M, “Significance Determination Process Using Qualitative Criteria”
Issue Date: 01/10/19 Att1-1 0609 Appendix I
ATTACHMENT 1 - Revision History – IMC 0609, Appendix I
Commitment
Tracking
Number
Accession
Number
Issue Date
Change Notice
Description of Change Description of Training
Required and
Completion Date
Comment Resolution
and Closed
Feedback Form
Accession Numbers
(Pre-Decisional, NonPublic Information)
N/A ML021060448
03/27/2002
CN 02-011
Revised the description of the flow chart
blocks to: 1) incorporate the first year’s
lessons learned, 2) reflect the change to 10 CFR 55.46 (Simulator Rule), and 3) align with
10 CFR 55.49 (integrity of examinations and
tests).
None N/A
N/A ML0524300990
8/22/2005
CN 05-023
Revised to match current revision to IP 71111.11 (Operator Requalification) and to fix
several flaws that have been identified and will
enhance the flowchart and matrix.
None N/A
N/A ML113270313
12/06/11
CN 11-040
Complete re-write of document. Arranged
flowchart to mirror inspection areas of revised
IP 71111.11, removed all minor finding blocks
(minor findings should be screened out prior to
reaching the SDP), and simplified examination
results logic.
Training held by
teleconference with
Regional examiners on
11/30/11
N/A ML18178A571x
01/10/19
CN 19-001
Reformatted and streamlined to reflect revision
to IMC0040. Added guidance to refer to IMC 0609 Appendix M in certain instances. Tied
White finding for simulators to the delta CDF
and delta LERF of the negative operator
performance. Added reference list.
None ML18177A421
Closed FF:
0609I-2232
0609I-2160
0609I-2309