ML12297A123

7 North Anna 2012-301 Final Administrative Documents

ML12297A123
Person / Time
Site:	North Anna
Issue date:	10/23/2012
From:	NRC/RGN-II
To:	Virginia Electric & Power Co (VEPCO)
References
50-338/12-301, 50-339/12-301
Download: ML12297A123 (32)

Text

ES-201 Examination Preparation Checklist Form ES-201-1 Facility:

Nvn 4 t4NY Date of Examination: s6l-r I2 Developed by: Written

- FacilityL NRC C II Operating

- Facility2 NRC Target Chief Date*

Task Description (Reference)

Examiners Initials

-180 1.

Examination administration date confirmed (C.1.a; C.2.a and b)

-120 2.

NRC examiners and facility contact assigned (C.1.d; C.2.e)

)fl3

-120 3.

Facility contact briefed on security and other requirements (C.2.c)

-120 4.

Corporate notification letter sent (C.2.d)

[-90]

[5.

Reference material due (C.1.e; C.3.c; Attachment 3)]

{-75}

6.

Integrated examination outline(s) due, including Forms ES-201-2, ES-201-3, ES-301-1, ES-301-2, ES-301-5, ES-D-ls, ES-401-1/2, ES-401-3, and ES-401-4, as applicable (C.1.e and f; C.3.d)

{-70}

(7.

Examination outline(s) reviewed by NRC and feedback provided to facility licensee (C.2.h; C.3.e)}

{-45}

8.

Proposed examinations (including written, walk-through JPMs, and scenarios, as applicable), supporting documentation (including Forms ES-301-3, ES-301-4, ES-301-5, ES-3D 1-6, and ES-401-6, and any Form ES-201-3 updates), and reference materials due (C.1.e, f, g and h; C.3.d)

-30 9.

Preliminary license applications (NRC Form 398s) due (C.1.l; C.2.g; ES-202)

-14 10.

Final license applications due and Form ES-201-4 prepared (C.1.l; C.2.i; ES-202)

-14 11.

Examination approved by NRC supervisor for facility licensee review (C.2.h; C.3.f)

-14 12.

Examinations reviewed with facility licensee (Cli; C.2.f and h; C.3.g)

-7 13.

Written examinations and operating tests approved by NRC supervisor (C.2.i; C.3.h)

-7 14.

Final applications reviewed; I or 2 (if >10) applications audited to confirm qualifications / eligibility; and examination approval and waiver letters sent (C.2.i; Attachment 5; ES-202, C.2.e; ES-204)

-7 15.

Proctoring/written exam administration guidelines reviewed with facility licensee (C.3.k)

-7 16.

Approved scenarios, job performance measures, and questions distributed to NRC examiners (C.3.i)

• Target dates are generally based on facility-prepared examinations and are keyed to the examination date identified in the corporate notification letter. They are for planning purposes and may be adjusted on a case-by case basis in coordination with the facility licensee.

[Applies only] (Does not apply} to examinations prepared by the NRC.

ES-201, Page 25 of 28

ES-201 Examination Outline Quality Checklist Form ES-201-2 Date of Examination Facility:

NoTj..,

4 dY4M /W Tff104 t /,o/, 7-Initials Item Task Description

a b*

1.

a.

Verify that the outline(s) fit(s) the appropriate model, in accordance with ES-401 j

w R

b.

Assess whether the outline was systematically and randomly prepared in accordance with I

Section D.1 of ES-401 and whether all KJA categories are appropriately sampled.

f?21 wi T

T c.

Assess whether the outline over-emphasizes any systems, evolutions, or generic topics.

IL8 E

N d.

Assess whether the justifications for deselected or rejected K/A statements are appropriate.

/tj $ f 2.

a.

Using Form ES-301-5, verify that the proposed scenario sets cover the required number of normal evolutions, instrument and component failures, technical specifications, y)43 S

and major transients.

M b.

Assess whether there are enough scenario sets (and spares) to test the projected number u

and mix of applicants In accordance with the expected crew composition and rotation schedule L

without compromising exam integrity, and ensure that each applicant can be tested using A

at least one new or significantly modified scenario, that no scenarios are duplicated

T from the applicants audit test(s), and that scenarios will not be repeated on subsequent days.

0 R

c.

To the extent possible, assess whether the outline(s) conform(s) with the qualitative 9

and quantitative criteria specified on Form ES-301-4 and described in Appendix D.

3.

a.

Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2:

(1) the outline(s) contain(s) the required number of control room and in-plant tasks W

distributed among the safety functions as specified on the form I

(2) task repetition from the last two NRC examinations is within the lmts specified on the form T

(3) no tasks are duplicated from the applicants audit test(s)

(4) the number of new or modified tasks meets or exceeds the minimums specified on the form (5) the number of afternate path, low-power, emergency, and RCA tasks meet the criteria on the form.

b.

Verify that the administrative outline meets the criteria specified on Form ES-301 -1:

(1) the tasks are distributed among the topics as specified on the form (2) at least one task is new or significantly modified

?

(3) no more than one task is repeated from the last two NRC licensing examinations c.

Determine if there are enough different outlines to test the projected number and mix

of applicants and ensure that no items are duplicated on subsequent days.

4.

a.

Assess whether plant-specific priorities (including PRA and IPE insights) are covered in the appropriate exam sections.

G E

b, Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate.

Jj N

E c.

Ensure that K/A importance ratings (except for plant-specific priorities) are at least 25.

-A4! 2 *-

R d.

Check for duplication and overlap among exam sections.

A L

e.

Check the entire exam for balance of coverage.

.iL f.

Assess whether the exam fits the appropriate job level (RO or SRO).

fl -

a. Author

/At/o4LL-Printe meSi q/t7/

9/?/

b. Facility Reviewer (*)

fi 13.

1-

c. NRC Chief Examiner (#)

tML

d. NRC Supervisor

£46.LLIkI.W 1

7

/sJtr Note:

1. Independent NRC reviewer initial items in Column c; chief examiner concurrence required.

Not applicable for NRC-prepared examination outlines

fVtç9_L Page.i. of 3

ES-201 Examination Security Agreement Form ES-201 -3 1.

Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of

/,1 as of the date I 0 I of my signature.

I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner.

I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensees procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee.

I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2.

Post-Examination To the best of my knowl 7

dgp, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of zFrom the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME 2.

s-,e //-

3. b&j-

4. IA.

ML.L. G4Ei-r

-rs

6. MAR14. c.

• Kth#ttfW E (CC)1 9

-L 9.

4 1o.j_..,.,.1

11. (pL tIia

12.

• 7 M5 C

l3.y.t) LMoSeIC 14.M4N.

thvó 15.

/

NOTES:

JOB TITLE I RESPONSIBILITY ui (fa,MI

-yc-4c/&,.

w, IA C,

, 7 viLscA4+o thQI.. 4 r

V L

______Iz-.

, 2

..cff2q )_

J-1-2.0- iZ.

5l(1L2 77-G SIGNATU DAT 1

E SIGNATUfl.g (2) i/I/k 4-?.

/

1

/Li t-

R 5M

t)ffoit?

f SL1PP&7 d9 7-DATE NOTE 5- (Ct_(Dt

&,f r?C.

Po

-J

.._-Ir.,-.-

tA-1/M

--1 L-SZ4 ff1

/7/: 1Z:

42 4.

.4D.iL 7/1 74,a

-v-ES-201, Page 27 of 28

ES-201 Examination Security Aqreement Form ES-201-3 1.

Pre-Examination

/

c,.

/7 2Dr I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of 1 as of the date t)82crz-of my signature.

I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner.

I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensees procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee.

I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2.

Post-Examination To the best of my knowIede, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of7-From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME 1.

L 2.

3.

CoJy,i...

P1

4. A 4

d C.

5.

I4-

6. iflat cX.r 7.

8.

9. fi. tAJe1ej 10.

ftm 11./V)lCI+4L. (4 12.flo4Lj 444orr) 13.

14.5A,t 4 1143J1(

15.

NOTES:

JOB TITLE / RESPONSIBILITY

,LL L/Rl4 0

r O3 NuT oiS Si L4-c/ 5tfiF14G-.--

cA-uJ

ç

• fS LUJ,1 c4PE6-dac co c-Ip JCi\\ I-...

SIGNATUR 1)

/4 &CMO4 DATE SIGNATURE (2)

DATE NOTE I

7-i&z

• 7/L1/l7.-

I

/2L7/12-q -z--

a VZ/ 1

,--.-.J--

ES-201, Page 27 of 28

ES-201 Examination Security Agreement Form ES-201-3 1.

Pre-Examination P

oc-j iz acknowledge that have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of 9191 as of the date of my signature.

I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner.

I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensees procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee.

I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2.

Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME i.trifl 2.

71t 3

4.

5. *Ip McniJJ

6. 4fg ;4

7. ot/-

frpi.AA

8.,8 WXL.iioL1e

9. Al4

10. p4tSrl.

11. %Jt4( \\).

12.

1 3 I 4.

15.

NOTES:

JOB TITLE / RESPONSIBILITY 1

SIGNATURE (1)

DATE 1fb)I UR(2) 0.t7k

V.°M tkT7 144/cdcZ4r-.

DATE NOTE qf 11,

1kL. 1J4fp.A.d L

fLo//Z_

2L4K N

(I N

N 7

Tf,L. i--

2.?//

1 zsIrL N

N N

ES-201, Page 27 of 28

ES-301 Administrative Topics Outline Form ES-301-1 Facility: North Anna Power Station Date of Examination:

9/10/2012 Examination Level:

Combined (See Below)

Operating Test Number:

Administrative Topic Type I

Describe activity to be performed (see Note)

Code*

M, R Determine Shutdown Margin (1-PT-10.1).

Conduct of Operations G2. 1.25 (ALL) (RO 3.9 I SRO 4.2)

M, R Calculate Reactor Coolant System Leak Rate (Hand Conduct of Operations G2. 1.20 Calculation) (1 -PT-52.2).

(ALL) (RO 4.6 I SRO 4.6)

M, R Perform Alternate Core Cooling Assessment Equipment Control G2.2.37 (1-G0P13.o & 1-GOP-13.1)

(ALL) (RO 3.6 I SRO 4.6)

M, R Select RWP Task; determine dressout & dosimetry Radiation Control G2.3.7 requirements, and calculate stay time.

(All) (RO 3.5! SRO 3.6)

M, R Classify an emergency event (EPIP-1.01)

Emergency Plan G2.4.41 (SRO ONLY) (RO 2.9 I SRO 4.6)

NOTE: All items (5 total) are required for SROs. RO applicants require only 4 items unless they are retaking only the administrative topics, when all 5 are required.

• Type Codes & Criteria:

(C)ontrol room, (S)imulator, or Class(R)oom (D)irect from bank (<3 for ROs; <4 for SROs & RO retakes)

(N)ew or (M)odified from bank (> 1)

(P)revious 2 exams (<1; randomly selected)

ES-301 Control Roomlln-Plant Systems Outline Form ES-301-2 Facility:

North Anna Power Station Date of Examination:

9/10/2012 Exam Level: RD SRO-l SRO-U Q Operating Test No.:

1 Control Room Systems@ (8 for RO); (7 for SRO-l); (2 or 3 for SRO-U, including 1 ESF)

System I JPM Title (KA)

Type Code*

Fn a)

Initiate Bleed & Feed in Response to Loss of Secondary Heat Sink. (1-FR-Hi)

A, E, M, L, S 4 (Pri)

E05-- EA1.1, (4.1 /4.0)/(ALL) b)

Align Service Water to Cool Charging Pumps with Gravity Flow. (0-AP-12)

A, E, N, L, S 4 (Sec) 076 A2.01, (3.5 / 3.7) / (ALL) c)

Align Quench Spray in response to a LOCA. (1-E-0)

A, EN, N, L, S 5

026

-- A2.04 (3.9 / 4.2) (ALL) d)

Defeat a failed loop AT/Tave channel in accordance with 1-MDP-55.74.

M, S 7

016A4.01, (2.9/2.8) (ALL) e)

Perform an RCS Depressurization. (i-E-3)

A, EN, N, L, S 3

038-- EA1.04 (4.3/4.1) (ALL) f)

Place Containment Purge in Service.

D, S 8

029 A4.01 (2.5 / 2.5) (RO ONLY) g)

Perform an RCS Dilution.

A, D, E, S 8

004 -- A2.25 (3.9 / 3.7) (ALL) h)

Start and Load an Emergency Diesel Generator.

D, EN, 5 6

062 A4.07_(3.1_/_3.1)_(ALL)

In-Plant Systems@ (3 for RO; 3 for SRO-l; 3 or 2 for SRO-U) i)

Locally reset trip switches on breakers in accordance with 1-AP-20, Attachment 2 N

6 062 K2.01, (3.3 / 3.4) (ALL)

[j Align Auxiliary Feedwater to Supply the HCV Header.

D, E, P 4 (Sec)

[

061 A2.04 (3.4 / 3.8) (ALL) k)

Emergency Borate the RCS Locally.

D, E, R 1

024 AA2.05 (3.3 / 3.9) (ALL)

All RD and SRO-l control room (and in-plant) systems must be different and serve different safety functions; all 5 SRO-U systems must serve different safety functions; in-plant systems and functions may overlap those tested in the control room.

• Type Codes Criteria for RD / SRO-l / SRO-U (A)lternate path 4-6 / 4-6 / 2-3 (C)ontrol room (D)irect from bank 9 /

8 /

4 (E)mergency or abnormal in-plant 1 /

1 /

1 (EN)gineered safety feature

- /

/

1 (control room system)

(L)ow-Power/Shutdown 1 / 1 / 1 (N)ew or (M)odified from bank including 1(A) 2 / 2 /

1 (P)revious 2 exams (similar topic) 3 /

3 I 2 (randomly selected)

(R)CA (S)imulator Note: The are NO SRO-U candidates for this exam.

ES-301, Rev. 9 Operating Test Quality Checklist Form ES-301-3 Facility:

Date of Examination:

Operating Thst Number:

Initials

1. General Criteria a

b*

c#

a.

The operating test conforms with the previously approved outline; changes are consistent with sampling requirements (e.g., 10 CFR 55.45, operational importance, safety function distribution).

/(_4(

b.

There is no day-to-day repetition between this and other operating tests to be administered 1

Q, during this examination.

c.

The operating test shall not duplicate items from the applicants audit test(s). (see Section D.1.a.)

Jt.? 5.L 4 d.

Overlap with the written examination and between different parts of the operating test is within

/13-acceptable limits.

IL,

e.

It appears that the operating test will differentiate between competent and less-than-competent applicants at the designated license level.

2. Walk-Through Criteria a.

Each JPM includes the following, as applicable:

initial conditions initiating cues references and tools, including associated procedures reasonable and validated time limits (average time allowed for completion) and specific designation if deemed to be time-critical by the facility licensee operationally important specific performance criteria that include:

detailed expected actions with exact criteria and nomenclature

system response and other examiner cues

statements describing important observations to be made by the applicant

criteria for successful completion of the task

identification of critical steps and their associated performance standards

restrictions on the sequence of steps, if applicable I

b.

Ensure that any changes from the previously approved systems and administrative walk-through outlines (Forms ES-301-1 and 2) have not caused the test to deviate from any of the acceptance criteria (e.g., item distribution, bank use, repetition from the last 2 NRC examinations) specified fl 44 on those forms and Form ES-201-2.

I IY

3. Simulator Criteria The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES-301-4 and a copy is attached.

Printed Name / Signature Date a.

Author

/

b.

Facility Reviewer(*)

c.

NRC Chief Examiner (#)

tA4RI l. t Ai a.4qz d.

NRC Supervisor l

NOTE:

The facility signature is not applicable for NRC-developed tests.

Independent NRC reviewer initial items in Column c; chief examiner concurrence required.

ES-301 Simulator Scenario Quality Checklist Form ES-301-4 Facility: North Anna Date of Exam: 9/10/2012 Scenario Numbers: 1 /2 / 3/4 Operating Test No.: 1 QUALITATIVE ATTRIBUTES Initials a

b*

c#

1.

The initial conditions are realistic, in that some equipment and/or instrumentation may be out of service, but it does not cue the operators into expected events.

2.

The scenarios consist mostly of related events.

Ifl 3.

Each event description consists of the point in the scenario when it is to be initiated the malfunction(s) that are entered to initiate the event the symptoms/cues that will be visible to the crew c

the expected operator actions (by shift position) the event termination point (if applicable) 4.

No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario

,j without a credible preceding incident such as a seismic event.

5.

The events are valid with regard to physics and thermodynamics.

,Øj 6.

Sequencing and timing of events is reasonable, and allows the examination team to obtain d

complete evaluation results commensurate with the scenario objectives.

7.

If time compression techniques are used, the scenario summary clearly so indicates.

Operators have sufficient time to carry out expected activities without undue time constraints.

,4 1(

Cues are given.

8.

The simulator modeling is not altered.

cg_ EL 4 9.

The scenarios have been validated. Pursuant to 10CFR55.46(d), any open simulator performance deficiencies or deviations from the referenced plant have been evaluated to ensure that IL-.

functional fidelity is maintained while running the planned scenarios.

10.

Every operator will be evaluated using at least one new or significantly modified scenario.

All rftl_

L_.

other scenarios have been altered in accordance with Section D.5 of ES-301.

11.

All individual operator competencies can be evaluated, as verified using Form ES-301-6 (submit the J4..

form along with the simulator scenarios).

12.

Each applicant will be significantly involved in the minimum number of transients and events specified on Form ES-301-5 (submit the form with the simulator scenarios).

13.

The level of difficulty is appropriate to support licensing decisions for each crew position.

4 it1t Target Quantitative Attributes (Per Scenario; See Section D.5.d)

Actual Attributes 1.

Total malfunctions (58) 6 9

8 8

2.

Malfunctions after EOP entry (12) 1 2

3 2

3.

Abnormal events (24) 3 6

4 5

QQ_

4.

Major transients (12) 2 1

1 1

5.

EOP5 entered/requiring substantive actions (12) 1 2

1 1

6.

EOP contingencies requiring substantive actions (02) 0 0

1 1

7.

Critical tasks (23) 32 6

1

ES-301 Transient and Event Checklist Form ES-301 -5 NOTE: I/C Events in bold happen before EOP entry Facility: North Anna Power Station Date of Exam: 9/10/2012 Operating Test No.:1 A

E Scenarios P

V P

E 1

2 3

4 T

M L

N I

I T

CREW CREW CREW CREW N

C POSITION POSITION POSITION POSITION T

A T

N y

S A

B S

A B

S A

B S

A B

T p

R T

0 R

T 0

R T

0 R

T o

L U

E 0

C P

0 C

P 0

C P

0 C

P R

IU RO RX 1

7a 2

4a 110 NOR 4a 1

1 7a 1

2 la 4a 1

1 1

I/C 2,4 3,6 2,4 3,5 3,5,9 4,6 1,3 2,4b, 4

4 2

6,9 9,10 8,9 6,7 67, 10 MAJ 5,7 5,7 8

8 7

7 5

5 2

2 1

TS 022 RX 4a 7a 2

4a 1

10 NOR 1

1 1

la 1

1 1

SRO-I I/C 2,3 2,3 3,4,5 1,2,3 4

4 2

4,6 4,5,6 6,8,9 4b, 9,10 6,7 MAJ 5,7 8

7 5

221 TS 2,4 1,6 3,6 1,3 0

2 2

SRO-I RX 1

7a 2

4a 1

1 0

As RO NOR 4a 1

1 la 1

1 1

I/C 2,4 2,4 3,5,9 1,3 4

4 2

6,9 6,7 10 MAJ 5,7 8

7 5

2 2

1 TS 022 Instructions:

Check the applicant level and enter the operating test number and Form ES-D-1 event numbers for each event type; TS are not applicable for RO applicants. ROs must serve in both the at-the-controls (ATC) and balance-of-plant (BOP) positions; Instant SROs must serve in both the SRO and the ATC positions, including at least two instrument or component (I/C) malfunctions and one major transient, in the ATC position.

If an Instant SRO additionally serves in the BOP position, one I/C malfunction can be credited toward the two I/C malfunctions required for the ATC position.

2.

Reactivity manipulations may be conducted under normal or controlled abnormal conditions (refer to Section D.5.d) but must be significant per Section C.2.a of Appendix D.

(*) Reactivity and normal evolutions may be replaced with additional instrument or component malfunctions on a 1-for-i basis.

3.

Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to the applicants competence count toward the minimum requirements specified for the applicants license level in the right-hand columns.

ES-301 Competencies Checklist Form ES-301-6 Instructions:

Check the applicants license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.

Facility: North Anna Date of Examination: 9/10/2012 Operating Test No.: 1 APPLICANTS RD RD RO/BOP SRO-I S

-I Competencies SCENAPJO SCENAPJO NAPJO SCENARiO 1

2341 234 1

234 Interpret/Diagnose 1.2.

1,2, 3,4 1,2 1.2.

1,2, 1,2, 1,2, 3,4 1,2 Events and Conditions 5:6, 3,4, 4.7 4,6, 7,9 5,6.

3,4, 5,6.

3,4.

5,6, 5,6, 7.8, 4a 7,8 5.6 5,6, 7,8, 4a, 7

7.8.

9 4b, 9

7 7,8 9

4b 9

5.6.

10 9

5.6 10 7

10 7

Comply With and 1.2.

1,2, 1,2 1,la 1.2, 1,2, 1.2 1,la 1.2, 1,2, 1,2 1.la Use Procedures (1) 3,4.

3,4, 3,4 2,3, 4,7 4,6, 3,5, 2,3 3,4, 3,4, 3,4 2,3.

,6

,6,

,6, 4,4a 7a,8 7,9 4 4a D,6 D,6,

,6.

4 4a 7

7a,8 7,8 5,6, 5,6 7

7a,8 7,8 5,6 9

9 7

7 9

9 7

10 10 Operate Control 1,2, 1,2, 1,2 1,2 1,2.

1,2, 1,2 1,la Boards 3.4, 3,4, 3,4 la,3 4,5.

4,6 3,5, 3.4a

/

5.6.

5,6, 5.6.

4a 7

7a.8 7.9 5.6.

7 7a,8 7,8 4b, 7

9 9

5,6 10 7

Communicate 1.2.

1.2, 1.2 1.la 1.2 1,2.

1.2 1.la 1.2, 1.2.

1.2 1.la and Interact 3,4, 3,4, 3,4 2.3.

4,5.

4,6 3,5.

3.4, 3.4, 3,4.

3.4 2.3 5,6.

5,6, 5 6 4 4a 6.7 7,7a 7 9 4a 5.6 5 6 5 6 4.4a 7

7,7a 7,8, 4b.

8 5,6 7

7,7a 7,8 4b 8,9 9

5,6 7

8,9 9

5,6 10 7

10 7

Demonstrate 1,2, 1,2, 1,2 1,la Supervisory Ability (3) 7 7.7a 7,8 4b, 8.9 9

5.6 10 7

Comply With and 2,4 1.6 3,6 1.3 Use Tech. Specs. (3)

Notes:

(1)

Includes Technical Specification compliance for an RO.

(2)

Optional for an S RD-U.

(3)

Only applicable to SROs.

ES-401 Record of Rejected K/As Form ES-401-4 Tier /

Group Randomly Selected K/A Reason for Rejection 1 / 2 RO 028AK3.04 028AK3.03 Could not write discriminating Q to the original KA.

1 / 2 RO 067AA2.10 067AA2.13 Could not write discriminating Q to the original KA.

1 / 2 RO 076AG2.2.3 076AG2.2.12 Could not write discriminating Q to the original KA.

2 / 1 RO 026K4.09 026K4.01 Could not write discriminating Q to the original KA.

2 / 2 RO 011A2.08 011A2.01 Could not write discriminating Q to the original KA.

2 / 2 RO 068K4.01 068G2.1.23 Could not write discriminating Q to the original KA.

1 / 2 SRO 060AG2.1.19 005AG2.1.19 Could not write discriminating Q to the original KA.

1 / 2 SRO 061AA2.05 061AA2.03 Could not write discriminating Q to the original KA.

2 / 1 SRO 063G2.4.9 010G2.4.20 Could not write discriminating Q to the original KA.

2 / 2 SRO 028G2.1.25 035G2.1.23 Could not write discriminating Q to the original KA.

2 / 2 SRO 071A2.01 001A2.12 Could not write discriminating Q to the original KA.

2 / 2 SRO 079A2.01 016A2.01 Could not write discriminating Q to the original KA.

3 SRO G2.2.3 G2.2.18 Could not write discriminating Q to the original KA.

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 North Anna 2012-301 Q#

KIM B

L L

Psychometric Flaws Content Flaws U

Comment Nt 0

0 Stem Cues T/F 1 Non

>1 Non Partial Mm Q

SRO E

Explanation N

K D

Focus Cred Cred K/A Only S

Dist Dist B/W

GENERAL COMMENT

S B= Bank / MModified / N=New / FFundamental Level (I.E. Memory) / H=Higher Cognitive Level (I.E. C/A)

For All BANK questions: swap the order of the answer choices so that applicants cannot rely on recall of the correct answer location.

U designations are preliminary, and subject to change after discussions with the licensee.

Of the questions designated as BANK, which ones were used on the previous four NRC exams at North Anna 9

How many questions test knowledge associated with (the exam feels heavy in the following areas) list the question numbers for the items below:

an inadvertent safety injection?

Increasing charging vs initiating SI?

Safety injection, either inadvertent or per design General comment: Do a word search and modif the questions that ask for what operators WILL do. Change the question statements to test what they are REQUIRED to do by procedure, etc. Who knows what an operator will do? We can only test on what an operator is required to do. This comment likely applies to several questions and rather than make the comment several times, I am just going to make a general comment.

RO EXAM 1

OO1AA2.04 B

H 2

X B

Discuss the possibility of changing reactor power remaining the same vs S

decreasing and the possibility of the trip setpt remaining the same vs decreasing.

Include in the discussion any possibility of there being two correct answers.

Median Select Hi used instead of First Stage P. OK MAB 2

002A4.08 N

H 2

5 No comments.

3 003A4.06 N

F 2

S No comments.

4 003K5.04 N

H 1-

?

S I am still having difficulty with the reactor trip part of the question. If my 2

understanding is correct, a tripped RCP will ALWAYS cause a reactor trip and when the reactor trips you ALWAYS go to E-O and when you go to E 0 you ALWAYS manually trip the reactor whether the rods have already fallen or not.

We can discuss this one, but there is a possibility that we may need to do something different with at least that piece of the question.

Q OK following discussions. MAB Page 1 of 17

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 North Anna 2012-301 Q#

KJA#

B L

L Pymetric Flaws_____

Content Flaws U

Comment M

0 0

Stem Cues T/F I Non

>1 Non Partial Mm Q=

SRO E

Explanation N

K D

Focus Cred Cred K/A Only S

Dst Dist B/W 5

004K2.07 N

F 2

S No comments.

6 004K5.15 N

H 2

S No comments.

7 005A2.03 N

H I -

?

The only pump providing cooling has tripped. The increase in water 2

S temperature alone would seem to lead the applicant to the conclusion that water level would rise. We can discuss the plausibility of remaining constant, but I have concems that there is no reasonable misconception to lead an applicant to the answer choices with remains constant.

Modifications made. OK MAB 8

005 K5.05 N

H 1-x U

Band C do not appear to be plausible due to the inter-relationship 2

5 between the first and second part of those answer choices. How could isolating letdown ever cause RCS pressure to go down? How could raising letdown ever cause RCS pressure to go up?

We can discuss, but this question may need to be modified/replaced.

Errors corrected. MAB 9

006A3.03 B

F 2

S No comments.

10 006A4.08 N

F 2

5 No comments.

11 007A3.0l N

H 2

7 F

Does North Anna have components that receive their control power from S

Vital AC buses? Provide some examples if possible. Ijust want to ensure that an applicant just cant assume that control power will be supplied by a DC source, therefore rendering control power to the MOV nonplausible.

Examples provided. OK MAB 12 OO8AAI.02 B

H 2

S No comments.

13 008G2.l.27 N

H 2

E Discuss how/why this question is not testing knowledge required only of S

the SRO license level. Applying the SRO-only guidance, would this question screen as SRO-only based on Tech Spec Bases knowledge? If the answer can be concluded based on Lesson Plan information for CC system purpose, then maybe the question will be OK.

Following discussion with exam writers, Ops Mgt, and Tmg Mgt, the question will be allowed on RO exam. MAB 14 009EK3.06 N

H 2

U How are accumulators injecting plausible? Accumulators may inject as the S

result of pressure going down, but accumulators will usually not cause/create a pressure/temperature trend.

Question stem should ask for the actions to take iaw ES-l.2.

Consider:

Use similar conditions in the stem.

Given the current conditions, which one of the following states the method used for depressurization iaw l-ES-l.2 AND whether or notl-ES-1.2 Page 2 of 17

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 North Anna 2012-301 Q#

KIA#

B L

L Pometrk Flaws_____

Content Flaws U

Comment M

0 0

Stem Cues T/F I Non

>1 Non Partial Mm Q=

SRO F

Explanation N

K I)

Focus Cred Cred K/A Only S

Dist Dist BIW allows the depressurization is to continue?

A. Aux Sprays, Depressurization can continue B. PORV; Depressurization can continue C. Aux Sprays, Depressurization cannot continue D. PORV, Depressurization cannot continue When looking at my suggestion, evaluate whether you think the KA is still being tested.

I think it can be justified because the reason the depressurization is stopped is because of pressurizer level (or inventory balance).

Suggestion incorporated. MAB 15 010K2.02 N

H 2

S No comments.

16 01 lA2,0l N

H 2

?

Is C potentially correct? Would the actions be correct, although maybe S

not solving the problem in its entirety?

Modifications made to address comment. MAB 17 0IIEA2.08 B

F 2

S Nocomments.

18 Ol2Al.0l N

H 2

?

Discuss in more detail the impacts of delta flux. What will delta flux be at S

30%? Will it trend in the positive direction? Will this change the trip setpt? Why is C not a correct answer?

Would it be possible to modif the answer choices to state whether delta flux AND Thot both impact the setpoint change VS. only Thot impacting the setpoint change. This may address my above concem. Discuss.

Changes were made to address comments, OK MAB 19 013K6.Ol M

H 2

x D does not appear to be plausible. Two channels failed high must cause S

an ESF actuation.

A may also not be plausible.

I am not sure I understand how an applicant could have a reasonable misconception that CDA would actuate, yet SI would not.

I would also suggest placing the plant conditions in the stem and only place plant responses in the answer choices.

Also, I would suggest limiting the information in the answer choices to that which makes the answer choices different.

For example:

Add to the stem that a subsequent failure of l-LM-P-100C fails high.

Maybe test CDA actuation with a different second part.

Answers:

A. CDA will actuate; Page 3 of 17

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 North Anna 2012-301 Q#

K/A#

B L

L Pmetric Flaws_____

Content Flaws U

Comment M

0 0

Stem Cues T/F I Non

>1 Non Partial Mm Q

SRO E

Explanation N

K D

Focus Cred Cred KJA Only S

Dist Dist B/W B. CDA will NOT actuate.

C. CDA will actuate.

D. CDA will NOT actuate.

Licensee provided satisfactory basis for answer choices and made modifications to address comments. OK MAE 20 Ol4Kl.02 N

H 2

?

E Is Tave really plausible?

I understand that the procedure states this as an S

indication when above POAH, but with critical data being taken, it is pretty obvious that they are below POAH. What would SR instruments be reading (On-scale)? Is there any possibility to use SR instead of Tave without having two correct answers?

Changed critical data to an IR value. OK MAE 21 0l7K6.01 B

F 1-

?

E I have concerns with the plausibility of5 vs 4 CETs.

I also think testing 2

S the difference between HL RTDs and CETs, although still easy, would be a little more operationally valid. Consider changing the first part of the answer choices to test the difference between the average of CETs vs average of HL Temps. Using an incorrect temperature indication could lead to incorrectly implementing a procedure step.

The question really needs to test the KA in a site-specific manner. The second part of the question is what is being used to test knowledge required by the KA. The second part, however, primarily tests GFE knowledge, which is not sit-specific.

The question was rated as E, but in reality significant modification/replacement may be needed to test the KA in a site-specific manner.

Modifications made to ensure site-specific knowledge is tested and plausibility corrected. OK MAB 22 022AK1.04 N

H 2

?

E Should the stem state what procedure is being performed? It states what S

actions are being performed; therefore, it follows that we should tell them what procedure is being used.

Operator actions should be tied directly to a procedure if possible. If we do not tie actions to a procedure, then the question is open to unintended consequences of another procedure providing conflicting guidance or providing options to do something contrary to the intended answer.

Where in the referenced procedure is 14590 ever placed in AUTO? In order for 14590 to be plausible, it needs to be placed in AUTO at some point.

How could D ever be wrong when compared to C? 0 states to adjust 1459G as required; therefore, if no adjustment is needed, then g

required would cover making no adjustments. The same argument could Page 4 of 17

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 North Anna 2012-301 Q#

K/A#

B L

L Pççnietric Fbws_____

Content Flaws U

Comment M

0 0

Stem Cues T/F 1 Non

>1 Non Partial Mm Q=

SRO E

Explanation N

K 0

Focus Cred Cred K/A Only S

01st Dist B/W be applied to comparing B to A.

A value for 14590 was provided as 38% for normal steady state conditions. This question is at 50% RTP. Is the value of 38% still a normal value for 50% RTP?

The answer choices should really be reduced to only what makes them four unique answer choices.

A.

PRZR level is below program; l-RC-LC-1459G adjustment is required prior to placing 1-CH-FCV-l 122 to AUTO.

B.

PRZR level is below program; l-RC-LC-1459G adjustment is NOT required prior to placing l-CH-FCV-l 122 to AUTO.

C.

PRZR level is above program; l-RC-LC-l459G adjustment is required prior to placing l-CH-FCV-l 122 to AUTO.

D.

PRZR level is above program; l-RC-LC-1459G adjustment is NOT required prior to placing t-CH-FCV-l 122 to AUTO.

Modifications made to address comments. OK MAB 23 02202.1.7 B

H 2

S No comments.

24 025AK1.04 B

H 2

The question states the status of I-RH-P-lB and 1-CC-P-IB, but it does S

not State the status of other pumps. Does this impact the question? For instance, if I-RH-P-lA was initially running (which the question forces an assumption here), would the answer be different?

Pump status added. MAB 25 026K4.0l B

F 2

Is there any potential confusion that could be introduced by not describing S

which setpoint caused the pump start?

Clarification added. MAB 26 027AK2.03 B

H 2

S No comments.

27 027K2.0I N

F 2

5 No comments.

28 028AK3.03 N

H 1-

This question states that RCS subcooling is based on CETs. Depending on 2

S what happens with Q21, this may need to be addressed. Is there a reason the question cannot just state the value of subcooling without stating how the value is derived?

This question is only testing GFE knowledge. A site-specific question needs to be constructed because this is the Site-Specific Written Exam.

This question was rated as an E, but significant work needs to be done to test site-specific knowledge.

Modification made to address comments. OK MAB Page 5 of 17

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 North Anna 2012-301 Q#

KIA#

B B

L Py.1etnc Fla.s_____

Ctient Flaws U

Comment M

0 0

Stem Cues T/F 1 Non

>1 Non Partial Mm Q=

SRO E

Explanation N

K D

Focus Cred Cred K/A Only S

Dist Dist BIW 29 029A3.01 N

F 2

S No comments.

30 029EK2.06 M

H 2

g Add a question statement prior to the fill in the blank. I.E. Based on above S

conditions, which one of the following correctly states (1) the status the SI reset, and (2) the status of SI automatic initiation.

Corrected. OK MAB 31 035A1.02 N

H 2

x U

The question is asking for a primary to secondary dP. How are A and S

B plausible? It appears that the plausibility justification has nothing to do with a primary to secondary pressure difference.

Would operators use RCS Temperature to perform this evaluation? Why would the operators not use SG pressure to perform the evaluation?

If a cooldown is in progress, would the SG temperature be lower than the RCS temperature? If so, how would you determine an accurate SG pressure based on RCS temperature?

Q modified to address plausibility. OK MAB 32 038EK1.02 B

F 1-x g

C is not plausible because PTS is not a huge concern when you already 2

S have a LOCA into the SG. If PTS was the concern then you would limit the cooldown prior to depressurizing. Distractor will need to be modified/replaced.

Comments addressed. OK MAB 33 039A2.0l N

H 2

g It appears that the only thing in the stem that indicates that the condenser is S

not available is the definition of a design basis LOCA. Is this correct? Is the RO license level required to know Safety Analysis information (SRP Chapter 15 Calcs) at this level? Is there an RO learning objective associated with this?

Tie answers to E-1 and the Background document.

SGTR does not appear to be very plausible. In general it is usually better to limit the answer choices to only what is needed to make those answer choices unique. Is it possible to test just the remove heat from containment part. I.E.

Which one of the following correctly states the method for depressurizing the SOs and correctly states part of the basis for the depressurization in accordance with l-E-1 and its Background document?

The depressurization will be performed using the (I)

AND the depressurization (2) performed to remove heat from containment.

A.

(1) Dumps; (2) is B.

(1) Dumps; (2) is NOT

C.

(1) PORVs; (2) is Page 6 of 17

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 North Anna 2012-301 Q#

KIA#

B L

L Pometric Flaws_____

Content Flaws U

Comment M

0 0

Stem Cues T/F I Non

>1 Non Partial Mm Q=

SRO E

Explanation N

K D

Focus Cred Cred K/A Only S

Dist Dist B/W D.

(1) PORVs, (2) is NOT Comments addressed. OK MAB 34 040AG2.4.20 B

F 1-x 120 gpm does not appear to be plausible. The applicant must first 2

S determine the wrong limit (340 gpm) as the initial misconception, then arbitrarily add 20 gpm to that and divide by 3 to come up with a value that is not referenced for AFW control. Let me know if I have a misunderstanding of the plausibility justification. If there is training simulator guides or some other documentation that shows that applicants are trained to maintain 120 gpm in certain circumstances, then the plausibility can be reconsidered.

Suggestion provided and incorporated during exam review. OK MAB 35 045K5.17 B

H 2

F The question appears to be testing only GFE knowledge. Site-specific S

knowledge is typically required, but this KA is really concerning a GFE topic.

Operational validity is subject too.

I can never recall a time when reactivity changes are made with the intent of controlling the value of MTC. Also, would rods ever be held constant from 25% to 65% power?

Is this a realistic (operationally valid) situation?

Lets discuss the following:

Which one of the following correctly states (1) how main Steam header pressure changes as load is raised from 25% to 65% at MOL, AND (2) if the value of MTC will be more or less negative as a result of matching Tave with Tref and maintaining AO on target?

A.

Lowers; More negative B.

etc Comments addressed. OK MAB 36 054AK3.04 N

H 2

E B Plausibility: Bleed and Feed is a more specific form of providing an S

alternate heat sink as compared to the choice provided in A. Therefore, it would always be safe to choose the more general response. Suggest changing the second part ofA and B to ask whether Feed and Bleed Criteria have been met or have NOT been met.

Suggest changing the second part of C to Enter H. I only after exiting E 0.

Suggest changing second part of D to Enter H.l only when directed by E-0.

We can discuss the possible changes.

I think they will add plausibility.

Comments addressed. OK MAB 37 055EA2.03 N

F 2

I need the licensee to help me understand the interlock defeat switch and S

the plausibility and correctness of the answer choices. Discuss.

OK after discussion with licensee, MAE 38 055K3.0l B

H 2

The notes and steps in the procedure state requirements for condenser Page 7 of 17

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 North Anna 2012-301 Q#

K/A#

B L

L Prnietric Flas Content Flaws U

Comment M

0 0

Stem Cues TIF I Non

>1 Non Partial Mm Q

SRO E

Explanation N

K D

Focus Cred Cred K/A Only S

Dist Dist BIW S

pressure and the question is discussing condenser vacuum. What indications are provided in the control room on the panels vacuum or pressure? The question needs to reflect the control room indications (maybe it already does?).

Walk me through the confusion of the procedure requirements for pressure.

Then help me apply those requirements as written to the conditions for vacuum in the stem and the answer choices. A vacuum of 4 inches does not even appear to be in the gray area. Ijust want to make sure we are not using trickery to create plausibility, etc. A good vacuum would approach 30 inches and according to the Stem it has decayed to 4 inches.

Comments incorporated. MAB 39 056A.A2.73 N

H 2

F Be specific in the question statement for the second part. The question S

only asks for the capacity as stated in the LCO. The question would be more precisely worded if it asked only for that capacity.

Corrected. MAB 40 057AG2. 1.31 N

H 2

F Suggest changing the second part of the question to eliminate the S

possibility of there being two correct answers. Apply the logical connector or to the second part of answer choice C. Rods actually can be moved in or out. They cannot be moved out, but the question statement is correct.

The easy fix is to just test whether outward rod motion either IS or IS NOT possible for all of the answer choices.

Corrected. MAB 41 058AK3.01 M

H 2

S No comments.

42 059Kl.02 N

I-I 2

5 No comments.

43 O6OAA1.0l N

Ti 2

It would probably be more correct to test which recorder would display an S

elevated reading based on the event. It may be a little picky, but testing what someone WILL do is technically not a sound way to ask questions because we see operators do the wrong thing on a frequent basis. See general comment at beginning of 401-9.

Comment incorporated. MAB 44 061 K6.0 1 N

F 2

F I would suggest that this is a memory level question rather than a higher S

cognitive thought process.

I think most applicants will answer this just by knowing the system flowpath and failure mode

- placing two knowledge items together to answer the question without any real analysis required.

Incorporated. MAB 45 062KI.02 N

F 2

The correctness of the answer and incorrectness of the distractors depends S

highly on the precision of 2 seconds and 7.5 seconds. The question statement must be very specific of exactly when that clock starts to measure the time of the bus stripping. Suggest tightening the stem to ensure correctness of the answer and incorrectness ofB. Discuss if necessary.

Incorporated. OK MAB Page 8 of 17

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 North Anna 2012-301 Q#

KIA#

B L

L PLometric_Flaws Content Flaws U

Comment M

0 0

Stem Cues TIF I Non

>1 Non Partial Mm Q=

SRO E

Explanation N

K D

Focus Cred Cred K/A Only S

Dist Dist B/W 46 063A4.0l N

H 2

S No comments.

47 064A1.03 B

H 2

S No comments.

48 064K4.0I B

H 2

x B

15 seconds does not appear to be plausible. Having read the plausibility S

justification, I am still struggling with the plausibility of 15 seconds.

Either I need to better understand the plausibility or the first part of the question will need to be modified.

12 sec now_used._OK_MAB 49 067AA2.l3 B

F 2

E Is the 72 hour8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> requirement RO knowledge? Is there an RO leaming S

objective associated with the question.

Modified based on concern. OK MA.B 50 068AK2.07 B

F 2

S Have the licensee recheck the question analysis. It may need to be revised.

The justification for the answer choices does not appear to match the actual answer choices.

OK MAB 51 068G2.l.23 N

F 2

B I would prefer the question and answers be worded a little differently to be S

more precise, and in doing so be more correct by ensuring plausibility of distractors. Resetting Phase A would not normally be the only thing that needs to be done to restore containment sump pumping capability. We can discuss, but the way the question is worded, the question may be open to not being accurate. It is usually a better question writing practice to test only what is different between answer choices to ensure a correct answer.

I.E. For the first part, the answer choices should be worded as follows and the question statement modified accordingly:

Which one of the following correctly (1) states whether the CLOSE pushbuttons for CNTMT sump pump discharge trip valves (1-DA-TV-1 OOA&B) must be pushed after resetting phase A in order to restore sump pumping capability, and (2) states the tanks to which the CNTMT sump pumps normally discharge?

A. (I) close pushbuttons are required to be pushed after phase A reset (2) Low Level Liquid Waste Tanks (LLLW Tanks)

B. (1) close pushbuttons are NOT required to be pushed after phase A reset (2) Low Level Liquid Waste Tanks (LLLW Tanks)

Etc.

Also, I suggested using the (1) / (2) format for this question because I have seen it used in other questions within this exam. It is not required, but I would suggest that you use this same convention just for consistency and readability to help your applicants. Problem not good to mix styles within an exam but do as you wish with this comment.

Q reworded to address issues. OK MAB 52 073K3.cJl B

F 2

S It is discouraged to write NOT questions, but it is not prohibited.

Therefore, as long as the licensee is comfortable presenting the question in this manner then it will be OK.

OK MAB Page 9 of 17

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 North Anna 2012-301 Q#

K/A#

B L

L Pkometric Flaws_____

Content Flaws U

Comment M

0 0

Stem Cues T/F 1 Non

>1 Non Partial Mm Q=

SRO E

Explanation N

K D

Focus Cred Cred K/A Only S

Dist Dist B/W 53 073K4.01 N

F 2

x N

I am not convinced that auto re-open is plausible for a rad monitor alarm.

I S

would need to see evidence that this actually does occur in similar circumstances or same circumstances in a different system. Otherwise, we may need to modif the second part of the answer choices.

OK_after_discussions_with_licensee._MAB 54 076Al.02 N

H 2

S Question is OK as long as the licensee ensures me that this procedure selection is RO-required knowledge.

OK after discussions with licensee. MAB 55 076AG2.2.12 B

F 2

S No comments.

56 076K3.Ol B

H 2

N What is the status of the other SW pumps? Should the question state the S

other SW pumps status?

I am not understanding the plausibility of decreasing.

I need to either understand the plausibility a little better or we may need to consider modification.

Discussions with licensee and modification resulted in sat Q. MAB 57 077AK1.Ot B

H 2

S No comments.

58 078K1.05 N

F 2

N I think the question needs to be re-worded a Little. Six is pretty heavily S

cued in the question statement. t think the question can still be asked without using six in the question statement.

I.E.

(SOV5),

any one of the SOVs will close 2-MS-TV-201B.

Reworded. OK MAB 59 OO1AA2.04 B

H 2

The question statement at least needs to be tied to Tech Specs. What if S

there was another procedure that had a more limiting action? Then there would be no correct answer.

What is meant by most limiting action? I think it can be assumed by looking at the answer choices, but why not just ask the question to solicit the answer? I.E. Based on the time of discovery, which one of the following correctly states (1) the maximum time in which the first Tech Spec directed action is required to be completed and (2) the LCO which contains this action?

Both comments addressed. OK MAB 60 G2.I.26 N

F 1-x CautionTagisnotplausible.

2 5

B is not really plausible. Why would it make any difference who made the request?

Consider the following:

A tagout has been hung for work on a system by Maintenance and the Electric Shop.

Maintenance has determined that they will need to manipulate one of the Page 10 of 17

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 North Anna 2012-301 Q#

KIA#

B L

L Pyçmetric Flaws_____

Content Flaws U

Comment M

0 0

Stem Cues T/F I Non

>1 Non Partial Mm Q=

SRO E

Explanation N

K D

Focus Cred Cred K/A Onv S

Dist Dist BIW components on the tagout and obtain an Operating Permit.

Which one of the following, in accordance with O-AA-200, Equipment Clearance, correctly states (1) which individual(s) is permitted to operate the component, and (2) whether the Operating Permit ownership is allowed to be transferred to another individual if the Operating Permit Holder notifies all work order holders prior to transferring ownership?

A.

(1) Any WO Holder signed on to the tagout can authorize manipulation of the component.

(2) The Operating Permit ownership is allowed be transferred Etc.

Q modified. OK MAB 61 G2.l.40 N

F 2

g NOT questions are discouraged but not prohibited. If licensee is OK with S

asking the question in this format, then it will not be an issue.

I am confused as to what both channels of excore NIs operable means.

Which excores Source Range? Power Range? Etc. How can a channel be operable if the detector is not operable?

I tried reading the Tech Specs to get clarification, but remained confused. Why does the question even state that both channels of excore NIS are operable? Discuss and modify question as needed.

Clarifications made. OK MAB 62 02.2.12 N

F 2

x

?

I Applicants are required to know 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> actions from memory we cannot S

provide them with half of the answer.

Your answer provides options that are less than an hour for the first performance when in fact they have a full hour. Same concern for the second part.

I do not see much plausibility in A and C. The only reason to ask this question is that at least the initial performance of the surveillance or subsequent performances have a grace period. A does not contain any grace period and is therefore not plausible. Also, I do not think it is plausible to have a grace period associated with the initial performance, but then not have a grace period for subsequent performances. Therefore, C is not plausible.

Are SR rules of usage RO knowledge for North Anna? Does North Anna have an RO leaming objective that supports this? We almost always accept these rules of usage as SRO-only subject matter unless the licensee can show how it is an ROjob function at their facility.

Assuming that SR rules of usage are RO knowledge at North Anna, the plausibility issue and math issue can be addressed

. Why dont you just write the question to test the usage rules for SR rather than trying to make it a math problem? I.E.

Page 11 of 17

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 North Anna 2012-301 Q#

K/A#

B L

I.

Pometric Flaws_____

Content Flaws U

Comment M

0 0

Stem Cues T/F I Non

>1 Non Partial Mm Q

SRO E

Explanation N

K D

Focus Cred Cred K/A Only S

Dist Dist 8/W Which one of the following correctly states the correct application of Tech Spec SR 3.0.3? (or if you want to save words, write it as a fill in the blank)

A.

If it is discovered that a Surveillance was not performed within its specified Frequency, then compliance with the requirement to declare the LCO not met may be delayed, from the time of discovery, up to 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> or up to the limit of the specified frequency, whichever is GREATER.

B.

If it is discovered that a Surveillance was not performed within its specified Frequency, then compliance with the requirement to declare the LCO not met may be delayed, from the time of discovery, up to 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> or up to the limit of the specified frequency, whichever is LESSER.

C.

If it is discovered that a Surveillance was not performed within its specified Frequency, then compliance with the requirement to declare the LCO not met may be delayed, from the time the missed surveillance was due, up to 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> or up to the limit of the specified frequency, whichever is GREATER.

D.

If it is discovered that a Surveillance was not performed within its specified Frequency, then compliance with the requirement to declare the LCO not met may be delayed, from the time the missed surveillance was due, up to 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> or up to the limit of the specified frequency, whichever is LESSER.

Q replaced to address comments. Original Q was E. OK MAB 63 G2.2.43 N

F 2

S No comments.

64 G2.3.l3 B

F 2

S No comments.

65 G2.3.14 N

F 2

S No comments.

66 G2.3.4 B

F 2

S No comments.

67 G2.4.23 M

F 2

x Why are the reasons being tested in the answer choices? Would there be S

four unique answer choices without the reasons?

Perhaps you could use P.1 as a distractor since it at least has a Red Path associated with it.

I would suggest using it to replace A.

Corrected. MAB 68 G2.4.29 B

F 2

S No comments.

69 G2.4.43 N

F 2

x N

I do not think issuance of a press release is plausible. Distractors A S

and C will need to be modified.

The question statement does not reflect the procedure requirement. The question states that a must be provided every 60 minutes, but the procedure states should. You may be able to fix this by mimicking the Page 12 of 17

ES-401, Rev. 9 Written Exam Review Worksheet North Anna 2012-301 Form ES-401-9 Q#

K/A#

B L

L Pymetric Flaws_____

Content Flaws U

Comment M

0 0

Stem Cues T/F 1 Non

>1 Non Partial Mm Q=

SRO E

Explanation N

K D

Focus Cred Cred K/A Only S

Dist Dist B/W word should in the question.

Consider the following to address the plausibility issues discussed above:

In the second part of the question test whether an update condition is met for a downgrade in the classification. You can do this without the press release stuff. Just test whether the downgrade IS or IS NOT one of the update criteria listed in the procedure.

Corrected. OK MAB 70 WEO4EA1.1 N

H 2

x Question does not test the KA. The question only tests how the starting S

circuit works.

Q changed to test KA. MAB 71 WEO5EK2.2 B

F 2

5 No comments.

72 WEO8EK1.3 N

H 2

S No comments.

73 WE1OEK2.2 M

H 2

S No comments.

74 WE1 I EG2.4.6 B

H 2

Discuss with the licensee how ECA-1. 1 Step 7.a is answered No. Other S

than the fact that ECA-l.1 is being performed, what would lead an applicant to understand that recirc spray sump level is less than 4 ft 10 inches? There may just be something that I am not understanding.

Small enhancement made to ensure accuracy. OK MAB 75 WE15EK3.1 N

F 2

S No comments.

EXAM I

I I

I]

I I

[IC W DO THE SIlO QUI STIO? IS MAP HROUG [1 THE S O GUI DANCE ON TI E N] tC OPERATOR LICENSING WEBPAGE?

Similar criteria must be used for questions on the RO exam for consistency. I.E. How are procedure actions tested on the RO exam different that the procedure actions tested on the SRO ex: rn? WI at ma ces certai i proced re action SRO v RO? F ow doe: the,RO guidance support that?

h I

I i

s I

i 76 OOIAA2.04 N

H 2

B Do we need to ask in the question for the procedure directed by B-A7?

I S

have a concern that someone will be able to argue that they in fact did use Ap-]. 1 because it then directed them to go to B-A7, which then directed them to go to AP-3. Discuss. I think we may need to tighten some wording, but otherwise the question should work OK.

Comment addressed. OK MAB 77 005AG2.1.19 N

F 2

x Discuss the SRO-only aspects of the question. Reactor Trip Criteria, S

which is an E-0 entry condition, is usually RO knowledge. I.E. an RO is required to know when he must trip the reactor.

One idea may be able to test, with similar conditions, whether the reactor IS required to be tripped, or IS NOT required to be tripped. Then add a second part of the question which tests whether the rod is OPERABLE or I1JOPERABLE. The Tech Spec Bases clearly states that misalignment and operability are defined differently and a rod can be OPERABLE even though it is misaligned. An additional item may need to be placed in the

stem which would provide a reason for the rod not moving, but would not Page 13 of 17

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 North Anna 2012-301 Q#

K/A#

B L

L Pgometric Flaws_____

Content Flaws U

Comment M

0 0

Stem Cues T/F 1 Non

>1 Non Partial Mm Q=

SRO E

Explanation N

K D

Focus Cred Cred K/A Only S

Dist Dist B/W provide any evidence that tripping would be a problem.

I think if we head in this direction, we can have a good Q at the appropriate license level.

Q modified to address comments. MAB 78 008AG2.4.6 M

H 2

x x

U I am assuming that there is no way for subcooling to be greater than 25F S

[75Fj. Is this true? Discuss.

I have concerns about the plausibility of ES-l.2. The LOCA just occurred and has not yet been dealt with in the EOPs.

I am struggling to see how Post-LOCA Cooldown makes sense prior to dealing with the LOCA in E-I.

Foldout Criteria are typically RO knowledge items. This may present a problem wit to the question not testing SRO-only knowledge.

Comments addressed. MAB 79 012A2.0I N

H 2

?

B Only include the necessary items in the answer choices that make them S

unique from each other. The reasons behind the actions appear to be unnecessary. Suggest the following:

A. Trip the reactor and enter 1-E-0.

B. Begin an orderly shutdown of the reactor.

C. Enter l-AP-4. 1, title, shutdown or trip not required D. Enter l-AP-4.2, title, shutdown or trip not required Compare this question to the RO question where you were testing SR rules of usage. How can that RO question be considered RO knowledge and LCO rules of usage (3.0.3) be considered SRO-only knowledge? The answer to this question may result in one of those two questions requiring modificationlreplacement.

Comments incorporated and overlap reviewed. OK MAB 80 025AA2.02 M

H 2

S No comments.

81 035G2. 1.23 N

H 2

?

There are questions on the RO exam that test procedure bases knowledge.

S Therefore, I must conclude that EOP bases knowledge is required of the RO position.

I would also think that an RO would be required to know feed restrictions to the SOs.

I rated the question initially as an E, but I am having trouble understanding how this question tests SRO-only knowledge.

Modified to ensure SRO. MAB 82 039A2.03 M

1-1 2

B SRO-only aspects associated with the timing of the procedure selection for S.

t do not see any supporting documentation for the requirement to perform prior to contacting Chemistry. If there is not a REQUIREMENT, then this part of the question may not work. We will not be able to allow a question based only on a subjective good practice

I.E. what an operatorjd do.

I have not made this comment on all questions, but we should word

question statements to ask for what actions are required, not what an Page 14 of 17

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 North Anna 2012-301 Q#

KIA#

B L

L P

1 ometric Flaws Content Flaws U

Comment M

0 0

Stem Cues T/F 1 Non

>1 Non Partial Mm Q=

SRO E

Explanation N

K D

Focus Cred Cred K/A Only S

Dist Dist BIW operator 5h2!d do.

I may have stated this once before, but a word search for the entire exam should be performed to ensure there are no potential issues.

Comments incorporated. MAB 83 056AG2.2.40 N

H 2

7 I do not like asking for the MOST LIMITING required action because it is S

not a defined term. Would D be more limiting than B because restoring an entire offsite circuit may be more limiting than restoring one EDG?

Because MOST LIMITING is not defined, when arguments like these exist, it damages the credibility of the distractors. If the MOST LIMITING only refers to shortest duration then the question needs to specifically state that and stay clear of the MOST liMITING term unless that is a defined station term.

ROs are required to know one hour or less Tech Specs from memory. This question appears to be RO knowledge. Systems knowledge combined with Tech Spec less than one hour knowledge will lead to the answer.

Q_sat_after_discussions_and_slight_modification._MAB 84 061AA2.03 B

H 2

5 Question appears to test a process monitor, but the KA requires an area monitor. Due to the difficulty of matching KA at the SRO level, the question is satisfactory.

I suppose we can acknowledge that even a process monitor is measuring radiation in the area of the process flow path.

OK MAB 85 062A2.l5 N

H 2

E The wording of the question will need to be modified to test requirements S

and stay away from the wording of what an operator bd do. Suggest testing what is allowed or not allowed iaw Tech Specs and Bases or some altemative wording.

OK MAB 86 063G2.4.9 N

H 2

x E

Again-, question needs to be worded to solicit the actions that are required 5

who knows what an operator WILL do?

I do not understand how this question is SRO-only knowledge. Power supply knowledge coupled with knowing whether or not SI is required seems to be all that is needed to answer the question. How is the knowledge different in this question than several of the RO questions where procedure actions were being tested.

We can replace the KA if needed. Q was rated as an E due to the difficulty of the KA. We can discuss merits of current question or replace KA if we cannot come to agreement.

KA Change required. MAB 87 065AA2.02 N

H 2

x U

I do not understand how this question is SRO-only knowledge. How is the S

knowledge different in this question than several of the RO questions where procedure actions were being tested.

Do PORVs receive their motive force from instrument air? Are the

PORVs OPERABLE when a backup motive force is lined-up to the Page 15 of 17

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 North Anna 2012-301 Q#

K/A#

B L

L Pmetric Flaws_____

Content Flaws U

Comment M

0 0

Stem Cues T/F 1 Non

>1 Non Partial Mm Q

SRO E

Explanation N

K D

Focus Cred Cred K/A Only S

Dist Dist B/W PORV? Testing OPERABILITY in this way may get you to an SRO question. I.E. the PORV may work as designed, but it may not be OPERABLE. ROs would be expected to know operationally how to handle a loss of air to PORV, but they would not be expected to know the OPERABLE status iaw Tech Specs.

Q replaced to address comments. MAB 88 069AG2.l.31 B

F 2

S No comments.

89 089A2.0l N

F 2

I do not understand how this question is SRO-only knowledge. How is the S

knowledge different in this question than several of the RO questions where procedure actions were being tested. Can an element of procedure selection be tested? Could you test whether or not OP23.2 is needed to complete the activities?

KA Change required. MAB 90 079A2.0I N

H 2

x JJ I do not understand how this question is SRO-only knowledge.

S Has seal injection been lost? If not, then seal cooling has not been lost and systems knowledge combined with AP-33.2 entry condition knowledge would lead an RO applicant to the answer.

KA Change required. MAB 91 103 N

H 2

I need help understanding why certain sizes of MSL leakage could not S

cause the conditions in the stem.

OK after discussion with licensee. MAB 92 02.1.15 13 F

2

?

It does not make sense to me that a Tech Spec can be made less restrictive S

without a license amendment orNOED. If the question is limited to the TRM and silent on Tech Specs, then plausibility will be better.

Who are FSRC members or altemate members? Will the Director Nuclear Station Safety & Licensing be a member or altemate member, which would lead to an argument of two correct answers.

The word OR always creates a logic problem. The way the question is worded, D is also correct. Only test the difference between answer choices. I.E. lAW OP-AA-lOO Standing Orders MAY! MAY NOT be issued to impose less restrictive TRM requirements.

Perform a word search for or to ensure that we do not miss this issue with other questions.

I know I have made this comment for at least one other question.

Slight modification and wording changes. OK MAB 93 02.2.18 N

F 2

The question is confusing. It asks for the latest time then lists four S

different activities.

I think this is going to cause a lot of confusion.

The question needs to be presented in a different way. Maybe ask for the activity listed in OP-3.4 which requires containment closure team assignments to be made? Maybe list the activities in a timeline and then

ask at which time the containment closure team assignments are required.

Page 16 of 17

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 North Anna 2012-301 Q#

KIA#

B L

L Pmetric Flaws_____

Content Flaws U

Comment M

0 0

Stem Cues T/F 1 Non

>1 Non Partial Mm Q=

SRO E

Explanation N

K B

Focus Cred Cred K/A Only S

Dist Dist B/W We can discuss, but Ijust want to ensure that the applicants are not confused people here in the office were confused when reading the Q.

I am willing to accept the KA match. These activities could be needed to facilitate maintenance.

Q clarified. OK MAB 94 G2.2.7 B

F 2

?

optional does not appear plausible.

S Ideas to correct plausibility:

Test specifically if a Shift Manager may assume the Oversight role of the Senior Ops Manager. OR Test responsibilities.

Change the evolution to a Category II, vs I, so that some credibility will be added to an optional Test Coordinator.

This question was rated as an E due to perceived easy fixes wrt plausibility.

Licensee provided further justification. OK MAB 95 02,3.15 B

H 2

E No comments.

96 G2.3.5 N

F 2

x U

Not SRO-only. This appears to be Rad Worker knowledge which would S

apply to ROs also.

Provided comments during exam review were incorporated. MAB 97 G2.4.25 N

F 2

E Test what Action Level is required to be declared.

S Discuss whether the question needs to state that Emergency Director judgment is not used.

Is North Anna management OK asking the E-Plan classification in a closed-book format?

Reference provided. OK MAB 98 G2.4.30 N

H 2

S Is North Anna management OK asking the E-Plan classification in a closed-book format?

Reference provided. OK MAB 99 WEO4EG2.4. t8 N

H 2

Does the question statement need to state where the operators are in E-0?

S I.E. Immediate actions have just been completed? It may not be needed, but Ijust want to ensure that forcing them to make an assumption as to where they are in the procedure could cause an alternate correct answer.

Clarified. OK MAB 100 WE05EA2. 1 N

H 2

E Test what is reguired not what an SS will do.

S Make the question statement ask for the information provided by the answer choices. Is the intent to ask for procedure transitions from ECA 2.1? If so, then ask the question to solicit that.

Modifications made to address comments. MAB Page 17 of 17

ES401 Written Examination Quality Checklist Form ES-401-6 Facility: NO&

71 Pow S yX041 Date of Exam:9ht/Iz Exam Level: RoI SRO Initial Item Description a

b*

c#

1.

Questions and answers are technically accurate and applicable to the facility.

2.

a.

NRC K/As are referenced for all questions.

j b.

Facility learning objectives are referenced as available.

3.

SRO questions are appropriate in accordance with Section D.2.d of ES-401 4.

The sampling process was random and systematic (If more than 4 RD or 2 SRO questions were repeated from the last 2 NRC licensing exams, consult the NRR CL program office).

5.

Question duplication from the license screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate:

the audit exam was systematically and randomly developed; or the audit exam was completed before the license exam was started; or the examinations were developed independently; or 7the licensee certifies that there is no duplication; or L/ Wi!

other (explain) from the bank, at least 10 percent new, and the rest I

6.

Bank use meets limits (no more than 75 percent Bank Modified I

New new or modified); enter the actual RO I SRO-only 27 7 5-1 1) 9z iif3 question_distribution(s)_at_right.

7.

Between 50 and 60 percent of the questions on the RO Memory j

C/A exam are written at the comprehension/ analysis level; I

the SRO exam may exceed 60 percent if the randomly 3 /

I Lj C)

/

1 7 selected K/As support the higher cognitive levels; enter the actual RD / SRO question distribution(s) at right.

I D

8.

References/handouts provided do not give away answers z_/ 3 or aid in the elimination of distractors.

9.

Question content conforms with specific K/A statements in the previously approved examination outline and is appropriate for the tier to which they are assigned; fl3 deviations are justified.

10.

Question psychometric quality and format meet the guidelines in ES Appendix B.

I 11.

The exam contains the required number of one-point, multiple choice items;

,4J.

?tL.

the total is correct and agrees with the value on the cover sheet.

Date Printed Name / Signature 1/171/?

a. Author ff,4WoALL &rr/i

b. Facility Reviewer (*)

c. NRC Chief Examiner (#)

1. r/

9/Mh7

d. NRC Regional Supeisor o/,cJ/17 Note:

• The facility reviewers initials/signature are not applicable for NRC-developed examinations.

1. independent NRC reviewer initial items in Column C; chief examiner concurrence required.

NORTH ANNA 2012-301 FINAL WRITTEN EXAM SAMPLE PLAN North Anna Final Written Exam Sample Plan is the combination of the Draft Sample Plan and Form ES-401-4.

ES-403 Written Examination Grading Form ES-403-1 Quality Checklist Facility: North Anna Power Station Date of Exam: 09/19/12 Exam Level: ROE1 SROL1 Initials Item Description a

b c

1.

Clean answer sheets copied before grading s

MQ 2.

Answer key changes and question deletions justified p3 and documented 3.

Applicants scores checked for addition errors (reviewers_spot_check>_25%_of_examinations) as_applicable,_+/-4%_on_the_SRO-only)_reviewed_in_detail 4.

Grading for all borderline cases (80 +/-2% overall and 70 or 80, 5.

All other failing examinations checked to ensure that grades are_justified 6.

Performance on missed questions checked for training deficiencies and wording problems; evaluate validity 4Ai of_questions_missed_by_half_or_more_of_the_applicants Printed Name/Signature Date 5M1

//

c 70

a. Grader gIz.,Elrc 1

b.

Facility Reviewer(*)

?/O/242/2 c.

NRC Chief Examiner (*) M 1

q A A

d.

NRC Supervisor (*)

(*)

The facility reviewers signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.

ES-403, Page 5 of 5