Text

Initial Exam 2008-301 Final Administrative Documents
	ML091480156
Person / Time
Site:	Farley
Issue date:	05/28/2009
From:	Richter C; - No Known Affiliation
To:	; NRC/RGN-II
References
	Download: ML091480156 (108)
	v • d • e

ES-201 Examination Preparation Checklist Form ES-201-1 Facility: ~/lJt- Date of Examination: /CJ /Oy Developed by: Written - Facility ifNRC D II Operating - Facility ~NRC D Target Chief Date* Task Description (Reference) Examiner's Initials

-180 1. Examination administration date confirmed (C.1.a; C.2.a and b) cItt rl/f$>

-120 2. NRC examiners and facility contact assigned (C.1.d; C.2.e) ~ IJ4S

-120 3. Facility contact briefed on security and other requirements (C.2.c) rkmg

-120 4. Corporate notification letter sent (C.2.d) ciI(;rtE

[-90] [5. Reference material due (C.1.e; C.3.c; Attachment 3)]

dK !YfP

{-75} 6. Integrated examination outline(s) due, including Forms ES-201-2, ES-201-3, ES-301-1, ES-301-2, ES-301-5, ES-D-1's, ES-401-1/2, ES-401-3, and ES-401-4, as applicable (C.1.e and f; C.3.d) c/lI( /1$

{-70} {7. Examination outline(s) reviewed by NRC and feedback provided to facility licensee (C.2.h; C.3.e)} eM f)1$

{-45} 8. Proposed examinations (including written, walk-through JPMs, and scenarios, as applicable), supporting documentation (including Forms ES-301-3, ES-301-4, ES-301-5, ES-301-6, and ES-401-6, and any Form ES-201-3 updates), and reference materials due (C.1.e, f, g and h; C.3.d)

Df!IcfY$

-30 9. Preliminary license applications (NRC Form 398's) due (C.1.1; C.2.g; ES-202) tJ/{m$

-14 10. Final license applications due and Form ES-201-4 prepared (C.1.1; C.2.i; ES-202) cfJI( J1,4b

.-~

-14 11. Examination approved by NRC supervisor for facility licensee review (C.2.h; C.3.f)

t{ IY1J3

-14 12. Examinations reviewed with facility licensee (C.1.j; C.2.f and h; C.3.g) rJI{ ;no

-7 13. Written examinations and operating tests approved by NRC supervisor (C.2.i; C.3.h) c6( [}iP

-7 14. Final applications reviewed; 1 or 2 (if >10) applications audited to confirm qualifications / eligibility; and examination approval and waiver letters sent (C.2.i; Attachment 5; ES-202, C.2.e; ES-204)

CIt{ (f$

-7 15. Proctoring/written exam administration guidelines reviewed with facility licensee (C.3.k)

(fI/ 11$

-7 16. Approved scenarios, job performance measures, and questions distributed to NRC examiners (C.3.i) r/{rrff;

Target dates are generally based on facility-prepared examinations and are keyed to the examination date identified in the corporate notification letter. They are for planning purposes and may be adjusted on a case-by-case basis in coordination with the facility licensee.

[Applies only] {Does not apply} to examinations prepared by the NRC.

ES-201, Page 25 of 28

ES-201-2 Examination Outline Quality Checklist Facility: FA2008-301 Date of Examination: Oct 27 - Nov 10, 2008 Initials Item Task Description a b*

1.

W R

a.

b.

Verify that the outline(s) fil(S) the appropriate model, in accordance with ES-401.

Assess whether the outline was systematically and randomly prepared in accordance with we... f8-o it il\~

I Section 0.1 of ES-401 and whether all KIA categories are appropriately sampled. ~ ~

c#

T

~

T c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics. fJ-

, f'-

E N d. Assess whether the justifications for deselected or rejected KIA statements are appropriate. ~ ~

2. a. Using Form ES-301-5, verify that the proposed scenario sets cover the required number II\\D S

of normal evolutions, instrument and component failures, technical specifications, and major transients.

~ fI..o cJf{

I M b. Assess whether there are enough scenario sets (and spares) to test the projected number ilYD U and mix of applicants in accordance with the expected crew composition and rotation schedule L without compromising exam integrity, and ensure that each applicant can be tested using at least one new or significantly modified scenario, that no scenarios are duplicated f¥-

A T from the applicants' audit test(s), and that scenarios will not be repeated on subsequent days. ~ ctIt

~

0

c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative R

and quantitative criteria specified on Form ES-301-4 and described in Appendix D. ~ P-

3. a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2: (it)

(1 ) the outline(s) contain(s) the required number of control room and in-plant tasks W distributed among the safety functions as specified on the form

/ (2) task repetition from the last two NRC examinations is within the limits specified on the form T (3) no tasks are duplicated from the applicants' audit test(s) ~

(4) the number of new or modified tasks meets or exceeds the minimums specified on the form ~

(5) the number of alternate path, low-power, emergency, and RCA tasks meet the criteria on the form. cJl

b. Verify that the administrative outline meets the criteria specified on Form ES-301-1: I~~

(1 ) the tasks are distributed among the topics as specified on the form (2) at least one task is new or significantly modified '4< IJ.4J 1cP(

(3) no more than one task is repeated from the last two NRC licensing examinations

c. Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days.

Cqi:.. ~ \~

4. a. Assess whether plant-specific priorities (including PRA and IPE insights) are covered in the appropriate exam sections. ~ ()..o ~~

G

b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate. ft/l.. ~

~~~

E

~

N

c. Ensure that KIA importance ratings (except for plant-specific priorities) are at least 2.5. ~

E R d. Check for duplication and overlap among exam sections. 1.1-A L e. Check the entire exam for balance of coverage. ~ ~

1~~Wf~

f. Assess whether the exam fits the appropriate job level (RO or SRO). fie:. ~ i~

t:.R...p:iQ!~me/Signature Date

a. Author C. Vince Richter / I" ~ 9 -P'?]

b. Facility Reviewer (*) Ga,ryOhmstedeL .t!... M~ 7i:Ji. (ohl,84'8 L!'

c. NRC Chief Examiner (#) \'A_A F,AT'SIIh'.kD~("""" Ie 61£.1,g"i!J1l,

d. NRC Supervisor jiiJI.MLi)l..:r. \!!'I!';;tl)JJ;,Aj,.ML_

lC,d .Lf.

'If? tIl j'1",h-,

! ',r, ~ ~

Lf .*'.

Note: # Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.

Not applicable for NRC-prepared examination outlines

ES-201 Examination Security Agreement Form ES-201-3

1. Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of 10

2 7. bf as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2. Post-Examination To the best of ~ k~VJ4edge, I diQ .tIot divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the we~kraj 0 J /I etdt. '1!Tc~m the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide perfo ance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC. , /-/ p - ?.P'l? fJ .

PRINTED NAME JOB TITLE I RESPONSIBiliTY DATE DATE NOTE

':f1"'Pd0J'<~'t:V),"kf'I*Q!..\ ~~LI'i""""'Y'_!<.,~""""'7"!J\k:f.~ . ~ ~'\~~ ~ ~ q*I).ol 1t.f"'~ ~

~=

A'r..,,~~ r~

t.("I'I~;i'l.J'.u".. wOf'" ...,c~.< ("6"-' .~~I"- ~~ohfJ-=-+--J'~'--,~--- {*/~*;;'jbliC;;W;; - u ~

.(Vi ,'-'",,, L fte./le $1 fYtl!lZ",fo v C-"fyc//,,,dir - :;y, d NIA

_ L/~~~CI JII: - . _... /d I

. .~

"""'.'t< .-

~~f- 01',

fl"'nl; oW.. b>!:

.s~s -of.<-t flc.",k DG;;:'~

PU.**..T_~ "VllAprL

==

-;~

l~

7-'-ca

?-/o-6'j

""7-1 S"-of'

'1-irK 7-t;s;.Ol' 1I1I'f/oR l1.:.1J~

-1l}..L~

~6-¥-

.JJ.=.!....'-/.- a;p ES-20 1, Page 27 of 28 10 -r -~

ES-201 Examination Security Agreement Form ES-201-3

1. Pre-Examination I acknowledge that! have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of )0 -~7-o$as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2. Post-Examination To the best of my knowled~e, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of~l~7,-'l~i. From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

itt PRINTED NAME JOB TiTLE / RESPONSIBILITY DATE SIGNATUR/~ ~2). DATE NV;E

1. oPs MiJWta:{Je.t' J ~ lI!t'f/!r:r, .

2. ~c.. 7.ifj§JL-

3. 7.1.4wl- ~Iot/ 7-Zt.*()3 If-fC{.oY_ _

4. Cb:A&l I~ 1~;"¥~~ .1.l::.J2::..-o~

5. SH . 8'+0-W.L.~

!'Il.-t r :r~ II-Ir~r

£0._ ~~

"'_~~f!LP- _ _ _ tMvt __

___ _ -!Mi~:.'l[-===r-_ _ _ _ _ _ ~~=- .!:..---!l~~~---.,.-_ _ Jl.:::.J..z:-O r 1o.....3 fJ, (J ( ~ -t/.::...Lff..-d"il

11. _...:::S'-":~-::-T_ _

12. ~~_~:k; fl~q~

13. _~I!.d"r /1 I~ :-7Sil;-

14. fUg ..D~_r,\.:..L..-'_

15. _ _ __

NOTES:

ES-201, Page 27 of 28 p-cb '7

ES-201 Examination Security Agreement Form ES-201-3

1. Pre-Examination I acknowledge that! have acquired specialized knowledge about the NRC licensing examinations scheduled for the week( s) of 10 - ;)..7"12 ~as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or

7 the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security have been compromised.

(jI Post-Examination To the best of my knowledj\e, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of~l~7,-'t~1. From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME JOB TITLE / RESPONSIBILITY SIGNATUR~ ~2) . DATE ~O;E

1. ()Ps MtJWl.cvzJe-yr ~//J ~ IIll'f!1!2 2 $S ~

3' PI4w l -'lJ'<<v-IJlov ~

4: Ca:n &1 iee:i\ ..1.l::.J2::-e~

5. 5 S$ -1l/LV!£:...

)'1\. -I, 5-'f'( '-1. l...-tJ Ii -J r--fJr iJ._ ""k1'0 ~~

_.j./I:J<~~ _ _ _ _ _._ _ _ ~_ __!:H1::l. __

j' v e -l:t -4 __ J.i:dl.:-E.L......

P,i)

- ( ~*z'"

: :: S~~to f -Il::..Jff..-d~

1/.'Zo*oY

~~~~.-:=..--~-:--~

~

- - if(~qz III~

,-"'¢1'~"""9 tf I' =- I~

.. --c:-S

, ...... \~\ * ~ "'l"~ 1\ 1l.(8.o~ tUg .D~a::.

ES-201, Page 27 of 28

~4 -"7

'~

ES-201 Examination Security Agreement Form ES-201-3

1. Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of 10-27-2008 as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised. '

2. Post-Examination To the best of my knrwledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the w'@e¥(s}~ Ii -I (J -2Dd From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME JOB TITLE / RESPONSIBILITY SIGNATURE (1) DATE SIGNATURE (2) DATE NOTE

1. c..tY~;~

-ti~Jtt;;;R~~";'Jt:~ ~.

'* 5.

~6.

"LIO'fJ'jf'(>

'<i_ ** '"

'I. (....

1.'1'\",,,,,,

0.2 > ,;:/..z'l5~r-LS'~ei'tV~~.g;

"'~''H_\'......\....~J<,r.>f **Il,,~~::-

C .c ~

7._______ _ ___

8._______ _ ___

9. _ _ _ _ _ _ _ _ _ __

10. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __

11. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __

12. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __

13._______ _ ___

14._______ _ ___

15.==-:::----clr--=-::-_ _ _ --:----;----,....---:-------:-:-- ----,,------::------,---- --:--- - - - - : : - - - - - : - 1 r - - - : : ; ;

NOTES: ~ ~7C' ~~ NP~ 1:>7' 4-l\ ~ c-re- ~ ~ ~l2-~ I' '/--y CA..-ffr~ ~ ~

c~~~ ~~~_

f Q tt-- 3~ _ 3-.---,

ES-301 Administrative Topics Outline Form ES-301-1 Facility: Farley Nuclear Plant Date of Examination: Oct 27, 2008 Examination Level: SRO + RO Operating Test Number: FA2008301 Administrative Topic (see T~e Describe activity to be performed Co e

Note)

Conduct of Operations R/N A1.1 Makeup to the RWST RO 006A1.02 RO-3.0 SRO-3.6 A1.1 Makeup to the RWST SOP-2.3 sect 4.2.3 and Conduct of Operations R/N determine the Tech specs that are applicable with the out of SRO spec reading given.

006A1.02 RO-3.0 SRO-3.6 Conduct of Operations A1.2. CRO-122 Perform A Quadrant Power Tilt Ratio SRO+RO R/D/P Calculation STP-7.0 015Al.04 RO-3.5 SRO-3.7 A2 Complete STP-9.0, DETERMINE RCS LEAK RATE, Appendix 1, and identify conditions that do not meet Equipment Control RIM acceptance criteria. CRO-035 RO 004 A2.22 RO-3.2 SRO-3.1 A2 Complete STP-9.0, DETERMINE RCS LEAK RATE, Appendix 1, and identify conditions that do not meet Equipment Control RIM acceptance criteria. Then apply Tech specs for the SRO conditions that exist. CRO-035 004A2.22 RO-3.2 SRO-3.1 Radiation Control RIM A3 Determine if a job assigned can be accomplished based on Admin limits and RWP requirements given.

SRO+RO G2.3.4 RO-3.2 SRO-3.7 Emergency Plan - RIM A4 A Site Area emergency exists. Plant and meteorological conditions have changed. Evaluate as the SROONLY ED whether a follow-up message or an upgrade to a GE is required and fill out applicable forms for notification. An upgrade to a GE will be required and PAR recommendations will be required lAW NMP-EP-109.

G2.4.44 SRO-4.4 Time critical JPM

Type Codes & Criteria: (C)ontrol room, (S)imulator, or Class(R)oom (D)irect from bank (~3 for ROs; ~ 4 for SROs & RO retakes) [1/1]

(N)ewor (M)odified from bank (~_ 1) [3/4]

JP2revious 2 exams (~ 1; randomly selected) [1/1]

Farley OCT 2008 NRC ADMIN exam outline Page 1 of 1

ES-301 Control Room/In-Plant Systems Outline Form ES-301-2 lity: Farley Nuclear Plant Date of Examination: October 27, 2008 Exam Level (both): RO SRO-I SRO-u Operating Test No.: FA2008301 Control Room Systems (8 for RO; 7 for SRO-i; 2 or 3 for SRO-U)

System / JPM Title Type Safety Code* Function

a. CRO-NEW Ramp Down To 540 MW at 4 MW/Min and S/ N / A 1 Perform Corrective Actions In Response To A Malfunction SRO-U Of The Rod Control System For A Continuous Rod Insertion.

001A2.11 RO-4.4 SRO-4.7

b. CRO- 066D (modified) Borate the RHR system to S/ M/ N L 2 prepare for RCS cooldown SRO-U 004A2.10 R03.9 SRO 4.2

c. CRO-333A (modified) Perform the Required Actions S/ M/ N L 3 for Cold Leg Recirculation starting at step 7 OF ESP- SRO-U 1.3.

011 EA1.11 RO-4.2 SRO-4.2

d. CRO-358B, Place a SGFP on service while in FRP- S / D/ L 4 H.1 W/E05 EA1.1 RO-4.1 SRO-4.0

e. CRO-406E Two Train Verification of ECCS equipment S/D/ A 6 064A4.06 RO- 3.9 SRO- 3.9

f. CRO- 328B Restore Instrument Air to Containment S/ N D/ P 8 065AA1.03 RO-2.9 SRO-3.1

g. CRO-NEW Respond to a FH5, SFP AREA RE25A OR S/ N / A 9 B HI RAD, alarm 060AA1.02 RO-2.9 RO-3.1

h. RO ONLY CRO- 343H Align R-11/12 to normal in S/ D /L 7 response to a spurious SI RO 073A4.02 RO-3.7 SRO-3.7 ONLY In-Plant Systems (3 for RO; 3 for SRO-i; 3 or 2 for SRO-U)

System / JPM Title Type Safety Code* Function

i. SO - 130 Make Up To SFP From the RWST D/R 8 033A1.01 RO-2.7 SRO-3.3 SRO-U

j. SO-351 A Start A 4075 KW Diesel Generator From D/E 6 the DLCP In Mode 4 SRO-U 064A4.01 RO-4.0 SRO-4.3

k. SO-090 Place 'A' BAT O/S & 'B' BAT O/R D/R 1or2 004K1.16 RO-3.3 SRO-3.5 Farley October 2008 NRC JPM exam Page lof2

ES-301 Control Room/In-Plant Systems Outline Form ES-301-2 All control room (and in-plant) systems must be different and serve different safety functions; in plant systems and functions may overlap those tested in the control room.

Type Codes Criteria for RO/ SRO-if SRO-U [ACTUAL]

(A)lternate path 4-6/4-6/2-3 [6/6/3]

(C)ontrol room (D)irect from bank ::::.9/::::.8/::::.4 [7/6/2]

(E)mergencyor abnormal in-plant :2: 1/:2: 1/:2: 1 [1 ]

(L)ow-Power / Shutdown :2: 1 /:2: 1 /:2: 1 [5/4/3]

(N)ew or (M)odified from bank including 1(A) :2:2/:2:2/:2: 1 [4/4/3]

(P)previous 2 exams ::::. 3 / ::::. 3 / ::::. 2 (randomly selected) [1/1/0]

(R)CA :2: 1/:2: 1/:2: 1 [2]

lS~mulator Farley October 2008 NRC JPM exam Page 2of2

ES-301-3 Operating Test Quality Checklist Facility: Farley Nuclear Plant Date of Examination: Oct 27 - Nov 10, 2008 OperatinQ Test Number: FA2008-301 Initials

1. General Criteria a b* c#

Ij.ft.?

The operating test conforms with the previously approved outline; changes are consistent with a.

sampling requirements (e.g., 10 CFR 55.45, operational importance, safety function distributionL-if(. /J-o cJII{

b. There is no day-to-day repetition between this and other operating tests to be administered during this examination. 4(l.. a..o 1~1

c. The operating test shall not duplicate items from the applicants' audit test(s). (see Section D.1.a.) LcIL l~ i~At'

d. Overlap with the written examination and between different parts of the operating test is within acceptable limits. ~ t)J6 ~

e. It appears that the operating test will differentiate between competent and less-than-competent applicants at the designated license level.

lIP- ~ ~

2. Walk-Through Criteria -- -- --

a. Each JPM includes the following, as applicable:

- initial conditions initiating cues

- references and tools, including associated procedures reasonable and validated time limits (average time allowed for completion) and specific designation if deemed to be time-critical by the facility licensee N~

operationally important specific performance criteria that include:

- detailed expected actions with exact criteria and nomenclature v:- (No

- system response and other examiner cues

- statements describing important observations to be made by the applicant

- criteria for successful completion of the task

- identification of critical steps and their associated performance standards rJI(

- restrictions on the sequence of steps, if applicable

b. Ensure that any changes from the previously approved systems and administrative walk-through outlines (Forms ES-301-1 and 2) have not caused the test to deviate from any of the acceptance

~ k .~~

criteria (e.g., item distribution, bank use, repetition from the last 2 NRC examinations) specified on those forms and Form ES-201-2. cM1

3. Simulator Criteria -- -- --

The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES-301-4 and a copy is attached. *- ~ ~

Printed Nam~nature Date

a. Author C. Vince Richter I kI!.. ;, '* Lf2- u..- ~

b. Facility Reviewer(*) Garv Ohmstede I A trt.~ l%t/ Ulo3

c. NRC Chief Examiner (#) ' ' ARt< A,13f-lTt:S Ilt2j(}7,>.::Jf:5fk.i .. lrlllfl. J~ j{)lIU /AC('fi.

d. NRC Supervisor P~JJU.ilJ,A ~r. \ViJ)~-"J!<J /f~,(i~t'nfJJk-*L~ f ,/~ .,/ ,

illl ,.lIe t,!l r

( 'j NOTE:

The facility signature is not applicable for NRC-developed tests.

Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.

ES-301-4 Simulator Scenario Quality Checklist II Date of Exam: Oct 27 Nov 10, 2008 Scenario Numbers: 2/3 0eerating Test No.: FA2008-301 QUALITATIVE ATTRIBUTES Initials a b* c#

1. The initial conditions are realistic, in that some equipment and/or instrumentation may be out of service, but it does not cue the operators into expected events. ~

rre

,p

~

2. The scenarios consist mostly of related events. f#:. tJc..o eMf

3. Each event description consists of

the point in the scenario when it is to be initiated

the malfunction(s) that are entered to initiate the event 4/? /h.o I'

the symptoms/cues that will be visible to the crew

the expected operator actions (by shift position)

the event termination point (if applicable) tM'f

~

4. No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario without a credible preceding incident such as a seismic event. ~ (}I.AJ

5. The events are valid with regard to physics and thermodynamics. &If!.. 1M) dk

6. Sequencing and timing of events is reasonable, and allows the examination team to obtain

\e. M ItHf ~

complete evaluation results commensurate with the scenario objectives.

7. If time compression techniques are used, the scenario summary clearly so indicates. Na Na Na Operators have sufficient time to carry out expected activities without undue time constraints.

Cues are given.

8. The simulator modeling is not altered. ¥-- M I~

9. The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator performance deficiencies or deviations from the referenced plant have been evaluated 'I!- /Jttl ~

to ensure that functional fidelity is maintained while running the planned scenarios. cAK

10. Every operator will be evaluated using at least one new or significantly modified scenario.

All other scenarios have been altered in accordance with Section D.5 of ES-301. ~ f>'-o ,;,

~

11. All individual operator competencies can be evaluated, as verified using Form ES-301-6 (submit the form along with the simulator scenarios). "- pt.o

~

12. Each applicant will be significantly involved in the minimum number of transients and events {f(

specified on Form ES-301-5 (submit the form with the simulator scenarios). ()vJ

13. The level of difficulty is appropriate to support licensing decisions for each crew position. CI!. (JJJ I~

Target Quantitative Attributes (Per Scenario; See Section D.S.d) Actual Attributes -- -- --

1. Total malfunctions (5-8) 8/9 ~ /W'> I~ ~

2. Malfunctions after EOP entry (1-2) 3/3 1ft ~

_n

3. Abnormal events (2-4) 3/4 ~ ~ !J

4. Major transients (1-2) 2/1 Ctc:. Ik2 1J '(

5. EOPs entered/requiring substantive actions (1-2) 2/1 IcC- IIiW ~

6. EOP contingencies requiring substantive actions (0-2) 1/0 ~ 1M i~

7. Critical tasks (2-3) 3/4 fiL tJ.u> I~

ES-301-4 Simulator Scenario Quality Checklist Faciltv: Farley Date of Exam: Oct 27 - Nov 10, 2008 Scenario Numbers: 4/5/6 ODeratina Test No.: FA2008-3 QUALITATIVE ATTRIBUTES Initials a b* c#

~

1. The initial conditions are realistic, in that some equipment and/or instrumentation may be out of service, but it does not cue the operators into expected events. Iue !~

2. The scenarios consist mostly of related events. If/j! I~

3. Each event description consists of

the point in the scenario when it is to be initiated

the malfunction(s) that are entered to initiate the event

the symptoms/cues that will be visible to the crew

the expected operator actions (by shift position) ~ /JLO

~

the event termination point (if applicable) e41(,

4. No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario without a credible preceding incident such as a seismic event. we-

5. The events are valid with regard to physics and thermodynamics. ttfL

6. Sequencing and timing of events is reasonable, and allows the examination team to obtain complete evaluation results commensurate with the scenario objectives. 'f:- ~ t

7. If time compression techniques are used, the scenario summary clearly so indicates. Na Na Na Operators have sufficient time to carry out expected activities without undue time constraints.

Cues are given.

8. The simulator modeling is not altered. we- (J...;o fit

9. The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator performance deficiencies or deviations from the referenced plant have been evaluated to ensure that functional fidelity is maintained while running the planned scenarios.

re. (ff

~ I(Ill(

f4<!-

e

10. Every operator will be evaluated using at least one new or significantly modified scenario.

All other scenarios have been altered in accordance with Section 0.5 of ES-301. (M) 11 . All individual operator competencies can be evaluated, as verified using Form ES-301-6 (submit the form along with the simulator scenarios).

V!- ~

12. Each applicant will be significantly involved in the minimum number of transients and events specified on Form ES-301-5 (submit the form with the simulator scenarios).

fF.- ~ ~

13. The level of difficulty is appropriate to support licensing decisions for each crew position. ~ ~

Target Quantitative Attributes (Per Scenario; See Section D.5.d) Actual Attributes -- -- --

i. Total malfunctions (5-8) 6/5/7 ~ (I.;JJ .~l}

2. Malfunctions after EOP entry (1-2) 2/4/3 t4?- ,"",I,) ~

3. Abnormal events (2-4) 4/5/4 42- ALAI

4. Major transients (1-2) 1/2/3 ~ ~

~-I

5. EOPs entered/requiring substantive actions (1-2) 1 /2/2

6. EOP contingencies requiring substantive actions (0-2) 1 /1 /1 4(Z.~

7. Critical tasks (2-3) 3/2/4 If"'"'p A **...,

~,

ES-301-5 Transient and Event Checklist Facility: Farley Nuclear Plant Date of Exam: Oct 27 - Nov 10, 2008 operating Test No.: FA2008-301 A E Scenarios P V 3 4 5 6 T M P E o I L N CREW CREW CREW CREW T N I T POSITION POSITION POSITION POSITION I A

M C S A B S A B S A B S A B L U A T R T o R T o R T o R T o M(*)

N Y o C P o C P o C P o C P R U T P E

RX 5 0 SRO-I x NOR 2 SRO-U I/C 21 4 4 2 x

MAJ 8 2 2 TS 8 o 2 2 RX 5 o RO NOR x

SRO-I IIC 12 4 4 2 x

SRO-U MAJ 722 x

TS 8 0 2 2 RO RX 2 0 x

SRO-I NOR 2 x IIC 15 4 4 2 MAJ 8 2 2 TS 8 o 2 2 Instructions:

1. Check the applicant level and enter the operating test number and Form ES-O-1 event numbers for each event type; TS are not applicable for RO applicants. ROs must serve in both the "at-the-controls (ATC)" and "balance-of-plant (BOP)"

positions; Instant SROs must serve in both the SRO and the ATC positions, including at least two instrument or component (I/C) malfunctions and one major transient, in the ATC position. If an Instant SRO additionally serves in the BOP position, one I/C malfunction can be credited toward the two I/C malfunctions required for the ATC position.

2. Reactivity manipulations may be conducted under normal or controlled abnormal conditions (refer to Section O.5.d) but must be significant per Section C.2.a of Appendix O. (*) Reactivity and normal evolutions may be replaced with additional instrument or component malfunctions on a 1-for-1 basis.

3. Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to the applicant's competence count toward the minimum requirements specified for the applicant's license level in the right-hand columns.

ES-301-5 Transient and Event Checklist Facilitll: Farley Nuclear Plant Date of Exam: Oct 27 - Nov 10, 2008 ORerating Test No.: FA2008-301 A E Scenarios P V 2 T M P E 0 I L N CREW T N I T POSITION A I

M C S A B L U A T R T 0 M(*)

N Y 0 C P R U T P E

SRO-I RX 0 x

NOR SRO-U x IIC 5 4 4 2 MAJ 2 2 2 TS 2 0 2 2 RO RX 0 x NOR SRO-I x I/C 3 4 4 2 SRO-U MAJ 2 2 2 x

TS 2 0 2 2 RO RX 0 x

NOR SRO-I x I/C 5 4 4 2 MAJ 2 2 2 TS 2 o 2 2 Instructions:

1. Check the applicant level and enter the operating test number and Form ES-D-1 event numbers for each event type; TS are not applicable for RO applicants. ROs must serve in both the "at-the-controls (ATC)" and "balance-of-plant (BOP)"

positions; Instant SROs must serve in both the SRO and the ATC positions, including at least two instrument or component (I/G) malfunctions and one major transient, in the ATC position. If an Instant SRO additionally serves in the BOP position, one I/C malfunction can be credited toward the two I/C malfunctions required for the ATC position.

2. Reactivity manipulations may be conducted under normal or controlled abnormal conditions (refer to Section D.S.d) but must be significant per Section C.2.a of Appendix D. (*) Reactivity and normal evolutions may be replaced with additional instrument or component malfunctions on a 1-for-1 basis.

3. Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to the applicant's competence count toward the minimum requirements specified for the applicant's license level in the right-hand columns.

ES-301-6 Competencies Checklist Facility: Farley Nuclear Plant Date of Examination: Oct 27 - Nov 10, 2008 Operating Test No.: FA2008-301 APPLICANTS Com petencies RO X SRO-I X SRO-U X SCENARIO SCENARIO SCENARIO 3 4 5 6 3 4 5 6 3 4 5 6 Interpret/Diagnose 23 13 12 12 23 13 12 12 234 13 12 12 Events and Conditions 45 45 45 34 45 45 45 34 567 45 45 34 67 78 6 56 67 78 6 56 8910 78 6 56 89 78 89 78 78 10 9 10 910 910 10 Comply With and 12 12 12 12 12 12 12 12 123 12 12 12 Use Procedures (1) 34 34 34 34 34 34 34 34 456 34 34 34 56 56 56 56 56 56 56 56 78 56 56 56 78 7 7 78 7 710 7 710 10 Operate Control 12 12 12 12 Boards (2) 34 34 34 34 56 57 56 56 78 89 78 9 9 10 10 Communicate 12 12 12 12 12 12 12 12 123 12 12 12 and Interact 34 34 34 34 34 34 34 34 456 34 34 34 56 56 56 56 56 56 56 56 7 56 56 56 7 7 78 7 7 78 7 78 9 910 910 10 Demonstrate 12 13 12 12 123 13 12 12 Supervisory Ability (3) 34 45 34 34 456 45 34 34 56 67 56 56 7 67 56 56 7 9 79 9 79 10 10 34 35 14 35 34 35 14 35

Notes:

(1 ) Includes Technical Specification compliance for an RO.

(2) Optional for an SRO-U.

(3) Only applicable to SROs.

Instructions:

Check the applicants' license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.

ES-301-6 Competencies Checklist Facility: Farley Nuclear Plant Date of Examination: Oct 27 - Nov 10, 2008 Operating Test No.: FA2008-301 APPLICANTS Competencies RO X SRO-I X SRO-U X n___S=-.;CENARIO SCENARIO _ _S::--CENARIO 2

Interpret/Diagnose Events and Conditions Comply With and Use Procedures (1)

Operate Control Boards (2)

Communicate and Interact Demonstrate Supervisory Ability (3)

Notes:

(1 ) Includes Technical Specification compliance for an RO.

(2) Optional for an SRO-U.

(3) Only applicable to SROs.

Instructions:

Check the applicants' license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.

ES-401 1 Rev. 9 PWR Examination Outline Form ES-401" Facility:.n;/'I'~ v Date of Exam: /(J / // /Z<:ra 8-'"

RO KIA Category Points SRO-Only Points Tier Group K K K K K K A A A A G A2 G* Total 1 2 3 4 5 6 1 2 3 4

Total

1. 1 3 2 4 3 4 2 18 3 3 6 Emergency &

2 2 2 1 1 2 1 9 2 2 4 Abnormal Plant N/A N/A Evolutions Tier Totals 5 4 5 4 6 3 27 5 5 10 1 3 2 2 3 3 3 2 3 2 2 3 28 2 3 5 2.

2 1 1 1 1 1 1 1 1 1 0 1 10 1 2 3 Plant Systems Tier Totals 4 3 3 4 4 4 3 4 3 2 4 38 3 5 8

3. Generic Knowledge and Abilities 1 2 3 4 10 1 2 3 4 7 Categories 3 2 2 3 1 2 2 2

1. Ensure that at least two topics from every applicable KIA category are sampled within each tier ofthe RO and SRO-only outlines (i.e., exceptfor one category in Tier 3 ofthe SRO-only outline, the "Tier Totals" in each KIA category shall not be less than two).

2. The point total for each group and tier in the proposed outline must match that specified in the table.

The final point total for each group and tier may deviate by +/- 1 from that specified in the table based on NRC revisions. The final RO exam must total 75 points and the SRO-only exam must total 25 points.

3. Systems/evolutions within each group are identified on the associated outline; systems or evolutions that do not apply at the facility should be deleted and justified; operationally important, site-specific systems that are not included on the outline should be added. Refer to ES-401, Attachment 2, for guidance regarding the elimination of inappropriate KIA statements.

4. Select topics from as many systems and evolutions as possible; sample every system or evolution in the group before selecting a second topic for any system or evolution.

5. Absent a plant-specific priority, only those KIAs having an importance rating (IR) of 2.5 or higher shall be selected. Use the RO and SRO ratings for the RO and SRO-only portions, respectively.

6. Select SRO topics for Tiers 1 and 2 from the shaded systems and KIA categories.

7. *The generic (G) KIAs in Tiers 1 and 2 shall be selected from Section 2 of the KIA Catalog, but the topics must be relevant to the applicable evolution or system.

8. On the following pages, enter the KIA numbers, a brief description of each topic, the topics' importance ratings (IRs) for the applicable license level, and the point totals (#) for each system and category. Enter the group and tier totals for each category in the table above; if fuel handling equipment is sampled in other than Category A2 or G* on the SRO-only exam, enter it on the left side of Column A2 for Tier 2, Group 2 (Note

1 does not apply). Use duplicate pages for RO and SRO-only exams.

9. For Tier 3, select topics from Section 2 of the KIA catalog, and enter the KIA numbers, descriptions, IRs, and point totals (#) on Form ES-401-3. limit SRO selections to KIAs that are linked to 10 CFR 55.43.

ES-401, REV 9 T1G1 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME I SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO 007EA2.01 Reactor Trip - Stabilization - Recovery 4.1 4.3 DDDDDDD~DDD Decreasing power level from available indications 11 Ability to determine and interpret the following as they apply to (EMERGENCY PLANT EVOLUTION):(CFR: 41.10/43.51 45.13) 00BAK2.03 Pressurizer Vapor Space Accident 1 3 2.5 2.4 0 ~ 0 0 0 0 0 0 0 0 0 Controllers and positioners Knowledge of the interrelations between (ABNORMAL PLANT EVOLUTION) and the foliowing:(CFR: 41.7/45.7/45.8) 009EK3.16 Small Break LOCA 1 3 3.8 4.1 0 0 ~ 0 0 0 0 0 0 0 0 Containment temperature, pressure, humidity and level limits Knowledge of the reasons for the following responses as they apply to (EMERGENCY PLANT EVOLUTION):(CFR: 41.5 1 41.101 45.61 45.13) 015AG2.4.50 RCP Malfunctions 14 4.2 4.0 0 0 0 0*0 0 0 0 0 0 ~ Ability to verify system alarm setpoints and operate controls identified in the alarm response manual.

This is a Generic, no stem statement is associated.

025AA2.05 Loss of RHR System 1 4 3.1 3.5 0 0 0 0 0 0 0 ~ 0 0 0 Limitations on LPI flow and temperature rates of change Ability to determine and interpret the following as they apply to ABNORMAL PLANT EVOLUTION):(CFR: 41.10 1 43.5 1 45.13) 026AK3.03 Loss of Component Cooling Water 1 8 4 4.2 DD~DDDDDDDD Guidance actions contained in EOP for Loss of CCW Knowledge of the reasons for the following responses as they apply to (ABNORMAL PLANT EVOLUTION):(CFR: 41.5 1 41.10 1 45.6/45.13)

Page 1 of3 9/24/2008 3:17 PM

ES-401, REV 9 T1G1 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME / SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO 027AA2.06 Pressurizer Pressure Control System Malfunction I 3 3.5 3.9 DDDDDDD~ DDD Conditions requiring plant shutdown Ability to (a) predict the impacts of the following on the (SYSTEM) and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those abnormal operation:(CFR: 41.5 143.5 I 45.3 I 45.13) 029EK1.05 ATWS 11 2.8 3.2 ~DDDDDDDDDD definition of negative temperature coefficient as applied to large PWR coolant systems Knowledge of the operational implications of the following concepts as they apply to the EMERGENCY PLANT EVOLUTION):(CFR: 41.8 to 41.10145.3) 038EA1.34 Steam Gen. Tube Rupture I 3 4.2 4.3 DDDDDD~DDDD Obtaining shutdown with natural circulation Ability to operate and lor monitor the following as they apply to (EMERGENCY PLANT EVOLUTION):(CFR: 41.71 45.5 I 45.6) 040AK2.02 Steam Line Rupture - Excessive Heat 2.6 2.6 D~DDDDDDDDD Sensors and detectors Transfer 14 Knowledge of the interrelations between (ABNORMAL PLANT EVOLUTION) and the following:(CFR: 41.7145.7145.8) 055EK3.02 Station Blackout I 6 4.3 4.6 DD~ DDDDDDDD Actions contained in EOP for loss of offsite and onsite power Knowledge of the effect that a loss or malfunction of the (SYSTEM) will have on the following:(CFR: 41.7145.6) 056AG2.4.1 Loss of Off-site Power 16 4.6 4.8 DDDDDDDDDD~ Knowledge of EOP entry conditions and immediate action steps.

This is a Generic, no stem statement is associated.

Page 2 of 3 9/24/2008 3:17 PM

ES-401, REV 9 T1G1 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME I SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO 057AK3.01 Loss of Vital AC Inst. Bus 1 6 4.1 4.4 0 0 ~ 0 0 0 0 0 0 0 0 Actions contained in EOP for loss of vital ac electrical instrument bus Knowledge of the reasons for the following responses as they apply to (ABNORMAL PLANT EVOLUTION):(CFR: 41.5 1 41.10 1 45.6/45.13) 058AK1.01 Loss of DC Power 1 6 2.8 3.1 ~ 0 0 0 0 0 0 0 0 0 0 Battery charger equipment and instrumentation Knowledge of the operational implications of the following concepts as they apply to the (ABNORMAL PLANT EVOLUTION):(CFR: 41.8 to 41.10/45.3) 062AA1.07 Loss of Nuclear Svc Water 1 4 2.9 3 0 0 0 0 0 0 ~ 0 0 0 0 Flow rates to the components and systems that are serviced by the SWS; interactions among the components Ability to operate and 1 or monitor the following as they apply to (ABNORMAL PLANT EVOLUTION):(CFR: 41.7 I 45.5 1 45.6) 065AA2.03 Loss of Instrument Air 1 8 2.6 2.9 0 0 0 0 0 0 0 ~ 0 0 0 Location and isolation of leaks Ability to determine and interpret the following as they apply to ABNORMAL PLANT EVOLUTION):(CFR: 41.10 1 43.51 45.13) 077AK1.01 Generator Voltage and Electric Grid 3.3 3.5 ~ 0 0 0 0 0*0 0 0 0 0 Definition of the terms: volts, watts, amps, VARS, power Disturbances 1 6 factor Knowledge of the operational implications of the following concepts as they apply to the (ABNORMAL PLANT EVOLUTION):(CFR: 41.8 to 41.10/45.3)

WE11EA1.2 Loss of Emergency Coolant Recirc. 14 3.5 3.8 0 0 0 0 0 0 ~ 0 0 0 0 Operating behavior characteristics of the facility.

Ability to predict and/or monitor changes in parameters associated with operating the (SYSTEM) controls including:(CFR: 41.5 1 45.5)

Page 3 of3 9/24/2008 3:17 PM

ES-401, REV 9 T1G2 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME / SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO 001AK3.02 Continuous Rod Withdrawal I 1 3.2 4.3 0 0 ~ 0 0 0 0 0 0 0 0 Tech-Spec limits on rod operability Knowledge of the reasons for the following responses as they apply to (ABNORMAL PLANT EVOLUTION):(CFR: 41.5 I 41.10 I 45.6 I 45.13) 003AK2.05 Dropped Control Rod 11 2.5 2.8 0 ~ 0 0 0 0 0 0 0 0 0 Control rod drive power supplies and logic circuits Knowledge of the interrelations between (ABNORMAL PLANT EVOLUTION) and the following:(CFR: 41.7 I 45.7 I 45.8) 033AA2.02 Loss of Intermediate Range NI/7 3.3 3.6 0 0 0 0 0 0 0 ~ 0 0 0 Indications of unreliable intermediate-range channel operation Ability to (a) predict the impacts of the following on the (SYSTEM) and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those abnormal operation:(CFR: 41.5/43.5/45.3/45.13) 059AG2.4.49 Accidental Liquid RadWaste ReI. I 9 4.6 4.4 0 0 0 0 0 0 0 0 0 0 ~ Ability to perform without reference to procedures those actions that require immediate operation of system This is a Generic, no stem statement is components and controls.

associated.

061AK2.01 ARM System Alarms I 7 2.5 2.6 0 ~ 0 0 0 0 0 0 0 0 0 Detectors at each ARM system location Knowledge of the interrelations between (ABNORMAL PLANT EVOLUTION) and the following:(CFR: 41.7 I 45.7 I 45.8) 076AA1.04 High Reactor Coolant Activity I 9 3.2 3.4 0 0 0 0 0 0 ~ 0 0 0 0 Failed fuel-monitoring equipment Ability to operate and I or monitor the following as they apply to (ABNORMAL PLANT EVOLUTION):(CFR: 41.7/45.51 45.6)

Page 1 of2 9/24/2008 3:17 PM

ES-401, REV 9 T1G2 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME / SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO WE03EK1.2 LOCA Cooldown - Depress. /4 3.6 4.1 ~ D D D D D D D D D D Normal, abnormal and emergency operating procedures associated with (LOCA Cooldown and Depressurization).

Knowledge of the operational implications of the following concepts as they apply to the EMERGENCY PLANT EVOLUTION):(CFR: 41.B to 41.10 / 45.3)

WEOBEK1.2 RCS Overcooling - PTS / 4 3.4 4.0 ~ D D D D D D D D D D Normal, abnormal and emergency operating procedures associated with (Pressurized Thermal Shock).

Knowledge of the operational implications of the following concepts as they apply to the EMERGENCY PLANT EVOLUTION):(CFR: 41.B to 41.10/45.3)

WE15EA2.1 Containment Flooding / 5 2.7 3.2 D D D D D D D ~ D D D Facility conditions and selection of appropriate procedures during abnormal and emergency operations.

Ability to determine and interpret the following as they apply to (EMERGENCY PLANT EVOLUTION):(CFR: 41.10 /43.5 /

45.13)

Page 2 of2 9/24/200B 3:17 PM

ES-401, REV 9 T2G1 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME I SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO 003K5.01 Reactor Coolant Pump 3.3 3.9 0 0 0 0 ~ 0 0 0 0 0 0 The relationship between the RCPS flow rate and the nuclear reactor core operating parameters (quadrant Knowledge of the operational implications power tilt, imbalance, ONB rate, local power density, of the following concepts as they apply to difference in loop T-hot pressure) the (SYSTEM):(CFR: 41.5/45.7) 004A3.02 Chemical and Volume Control 3.6 3.6 0 0 0 0 0 0 0 0 ~ 0 0 Letdown isolation Ability to monitor automatic operations of the (SYSTEM) including:(CFR: 41.7/45.5) 004K6.02 Chemical and Volume Control 2.5 2.6 0 0 0 0 0 ~ 0 0 0 0 0 Oemineralizers and ion exchangers Knowledge of the effect that a loss or malfunction of the following will have on the (SYSTEM):(CFR: 41.7/45.7) 005K5.05 Residual Heat Removal 2.7 3.1 0 0 0 0 ~ 0 0 0 0 0 0 Plant response during "solid plant": pressure change due to the relative incompressibility of water Knowledge of the operational implications of the following concepts as they apply to the (SYSTEM):(CFR: 41.5/45.7) 005K6.03 Residual Heat Removal 2.5 2.6 0 0 0 0 0 ~ 0 0 0 0 0 RHR heat exchanger Knowledge of the effect that a loss or malfunction of the following will have on the (SYSTEM):(CFR: 41.7/45.7) 006K1.08 . Emergency Core Cooling 3.6 3.9 ~ 0 0 0 0 0 0 0 0 0 0 CVCS Knowledge of the physical connections and/or cause-effect relationships between (SYSTEM) and the foliowing:(CFR: 41.2 to 41.9/45.7 to 45.8) 007G2.1.7 Pressurizer Relief/Quench Tank 4.4 4.7 000 000 0 0 0 0 ~ Ability to evaluate plant performance and make operational judgments based on operating This is a Generic, no stem statement is characteristics, reactor behavior and instrument associated. interpretation.

Page 1 of 5 9/24/2008 3:17 PM

ES-401, REV 9 T2G1 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME / SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO 008K4.09 Component Cooling Water 2.7 2.9 0 0 0 ~ 0 0 0 0 0 0 0 The "standby" feature for the CCW pumps Knowledge of (SYSTEM) design feature(s) and or interlock(s) which provide for the foliowing:(CFR: 41.7) 010A4.01 Pressurizer Pressure Control 3.7 3.5 0 0 0 0 0 0 0 0 0 ~ 0 PZR spray valve Ability to manually operate andlor monitor in the control room:(CFR: 41.7/45.5 to 45.8) 010K1.05 Pressurizer Pressure Control 3.4 3.6 ~ 0 0 0 0 0 0 0 0 0 0 PRTS Knowledge of the physical connections andlor cause-effect relationships between (SYSTEM) and the foliowing:(CFR: 41.2 to 41.9/45.7 to 45.8) 012K3.02 Reactor Protection 3.2 3.3 0 0 ~ 0 0 0 0 0 0 0 0 T/G Knowledge of the effect that a loss or malfunction of the (SYSTEM) will have on the foliowing:(CFR: 41.7 1 45.6) 013A2.01 Engineered Safety Features Actuation 4.6 4.8 0 0 0 0 0 0 0 ~ 0 0 0 LOCA Ability to (a) predict the impacts of the following on the (SYSTEM) and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those abnormal operation:(CFR: 41.5/43.5/45.3/45.13) 013K5.02 Engineered Safety Features Actuation 2.9 3.3 0 0 0 0 ~ 0 0 0 0 0 0 Safety system logic and reliability Knowledge of the operational implications of the following concepts as they apply to the (SYSTEM):(CFR: 41.5/45.7)

Page 2 of 5 9/24/2008 3:17 PM

ES-401, REV 9 T2G1 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME I SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO 022A2.01 Containment Cooling 2.5 2.7 0 0 0 0 0 0 0 ~ 0 0 0 Fan motor over-current Ability to (a) predict the impacts of the following on the (SYSTEM) and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those abnormal operation:(CFR: 41.5 143.5 1 45.3 1 45.13) 022G2.4.20 Containment Cooling 3.8 4.3 0 0 0 0 0 0 0 0 0 0 ~ Knowledge of operational implications of EOP warnings, cautions and notes.

This is a Generic, no stem statement is associated.

026K3.02 Containment Spray 4.2 4.3 0 0 ~ 0 0 0 0 0 0 0 0 Recirculation spray system Knowledge of the effect that a loss or malfunction of the (SYSTEM) will have on the foliowing:(CFR: 41.7/45.6) 039A4.04 Main and Reheat Steam 3.8 3.9 0 0 0 0 0 0 0 0 0 ~ 0 Emergency feedwater pump turbines Ability to manually operate andlor monitor in the control room:(CFR: 41.7/45.5 to 45.8) 059A3.03 Main Feedwater 2.5 2.6 0 0 0 0 0 0 0 0 ~ 0 0 Feedwater pump suction flow pressure Ability to monitor automatic operations of the (SYSTEM) inciuding:(CFR: 41.7/45.5) 061K6.01 Auxiliary/Emergency Feedwater 2.5 2.8 0 0 0 0 0 ~ 0 0 0 0 0 Controllers and positioners Knowledge of the effect that a loss or malfunction of the following will have on the (SYSTEM):(CFR: 41.7/45.7) 062K4.01 AC Electrical Distribution 2.6 3.2 0 0 0 ~ 0 0 0 0 0 0 0 Bus lockouts Knowledge of (SYSTEM) design feature(s) and or interlock(s) which provide for the foliowing:(CFR: 41.7)

Page 3 of 5 9/24/2008 3:17 PM

ES-401, REV 9 T2G1 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME / SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 Ai A2 A3 A4 G TOPIC:

RO SRO 062K4.03 AC Electrical Distribution 2.8 3.1 0 0 0 ~ 0 0 0 0 0 0 0 Interlocks between automatic bus transfer and breakers Knowledge of (SYSTEM) design feature(s) and or interlock(s) which provide for the foliowing:(CFR: 41.7) 063K2.01 DC Electrical Distribution 2.9 3.1 0 ~ 0 0 0 0 0 0 0 0 0 Major DC loads Knowledge of electrical power supplies to the foliowing:(CFR: 41.7) 064A1.02 Emergency Diesel Generator 2.5 2.8 0 0 0 0 0 0 ~ 0 0 0 0 Fuel consumption rate with load Ability to predict andlor monitor changes in parameters associated witi'l operating the (SYSTEM) controls including:(CFR: 41.5/

45.5) 073A2.02 Process Radiation Monitoring 2.7 3.2 0 0 0 0 0 0 0 ~ 0 0 0 Detector failure Ability to (a) predict the impacts of the following on the (SYSTEM) and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those abnormal operation:(CFR: 41.5/43.5 1 45.3 1 45.13) 076A1.02 Service Water 2.6 2.6 0 0 0 0 0 0 ~ 0 0 0 0 Reactor and turbine building closed cooling water temperatures.

Ability to predict andlor monitor changes in parameters associated with operating the (SYSTEM) controls inciuding:(CFR: 41.5/

45.5) 076K2.08 Service Water 3.1 3.3 0 ~ 0 0 0 0 0 0 0 0 0 ESF-actuated MOVs Knowledge of electrical power supplies to the foliowing:(CFR: 41.7)

Page 4 of 5 9/24/2008 3:17 PM

ES-401, REV 9 T2G1 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME I SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO 078K1.03 Instrument Air 3.3 3.4 ~ D D D D D D D D D D Containment air Knowledge of the physical connections andlor cause-effect relationships between (SYSTEM) and the foliowing:(CFR: 41.2 to 41.9/45.7 to 45.8) 103G2.4.4 Containment 4.5 4.7 D D D D D D D D D D ~ Ability to recognize abnormal indications for system operating parameters which are entry-level conditions for This is a Generic, no stem statement is emergency and abnormal operating procedures.

associated.

Page 5 of 5 9/24/2008 3:17 PM

ES-401, REV 9 T2G2 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME / SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO 001K5.07 Control Rod Drive 3.3 4.0 0 0 0 0 ~ 0 0 0 0 0 0 Effects of an asymmetric rod configuration on power distribution Knowledge of the operational implications of the following concepts as they apply to the (SYSTEM):(CFR: 41.5 I 45.7) 002A2.04 Reactor Coolant 4.3 4.6 0 0 0 0 0 0 0 ~ 0 0 0 Loss of heat sinks Ability to (a) predict the impacts of the following on the (SYSTEM) and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those abnormal operation:(CFR: 41.5/43.5 I 45.31 45.13) 011K2.01 Pressurizer Level Control 3.1 3.2 0 ~ 0 0 0 0 0 0 0 0 0 Charging pumps Knowledge of electrical power supplies to the following:(CFR: 41.7) 015K6.01 Nuclear Instrumentation 2.9 3.2 0 0 0 0 0 ~ 0 0 0 0 0 Sensors, detectors and indicators Knowledge of the effect that a loss or malfunction of the following will have on the (SYSTEM):(CFR: 41.7/45.7) 027G2.1.27 Containment Iodine Removal 3.9 4 DDDDDDDDDD~ Knowledge of system purpose and or function.

This is a Generic, no stem statement is associated.

029K4.02 Containment Purge 2.9 3.1 0 0 0 ~ 0 0 0 0 0 0 0 Negative pressure in containment Knowledge of (SYSTEM) design feature(s) and or interlock(s) which provide for the following:(CFR: 41.7) 035A3.01 Steam Generator 4.0 3.9 0 0 0 0 0 0 0 0 ~ 0 0 SI G water level control Ability to monitor automatic operations of the (SYSTEM) including:(CFR: 41.7 I 45.5)

Page 1 of2 9/24/2008 3:17 PM

ES-401, REV 9 T2G2 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME / SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO 041A1.01 Steam Dump/Turbine Bypass Control 2.9 2.9 D D D D D D ~ D D D D T-ave. verification above low/low setpoint Ability to predict and/or monitor changes in parameters associated with operating the (SYSTEM) controls inciuding:(CFR: 41.51 45.5) 055K3.01 Condenser Air Removal 2.5 2.7 D D ~ D D D D D D D D Main condenser Knowledge of the effect that a loss or malfunction of the (SYSTEM) will have on the foliowing:(CFR: 41.7/45.6) 056K1.03 Condensate 2.6 2.6 ~ D D D D D D D D D D Knowledge of the physical connections and/or cause-effect relationships between (SYSTEM) and the foliowing:(CFR: 41.2 to 41.9/45.7 to 45.8)

Page 2 of 2 9/24/2008 3:17 PM

ES-401, REV 9 T3 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME / SAFETY FUNCTION: IR K1 K2 K3 K4 KS K6 A1 A2 A3 A4 G TOPIC:

RO SRO G2.1.17 Conduct of operations 3.9 4.0 0 0 0 0 0 0 0 0 0 0 ~ Ability to make accurate, clear and concise verbal reports.

G2.1.21 Conduct of operations 3.S 3.6 0 0 0 0 0 0 0 0 0 0 ~ Ability verify the controlled procedure copy.

G2.1.36 Conduct of operations 3.0 4.1 0 0 0 0 0 0 0 0 0 0 ~ Knowledge of procedures and limitations involved in core alterations G2.2.13 Equipment Control 4.1 4.3 0 0 0 0 0 0 0 0 0 0 ~ Knowledge of tagging and clearance procedures.

G2.2.17 Equipment Control 2.6 3.8 0 0 0 0 0 0 0 0 0 0 ~ Knowledge of the process for managing maintenance activities during power operations.

G2.3.13 Radiation Control 3.4 3.8 0 0 0 0 0 0 0 0 0 0 ~ Knowledge of radiological safety procedures pertaining to licensed operator duties G2.3.4 Radiation Control 3.2 3.7 0 0 0 0 0 0 0 0 0 0 ~ Knowledge of radiation exposure limits under normal and emergency conditions Page 1 of 2 9/24/2008 3:17 PM

ES-401, REV 9 T3 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME I SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO G2.4.26 Emergency Procedures/Plans 3.1 3.6 0 0 0 0 0 0 0 0 0 0 ~ Knowledge of facility protection requirements including fire brigade and portable fire fighting equipment usage.

G2.4.28 Emergency Procedures/Plans 3.2 4.1 0 0 0 0 0 0 0 0 0 0 ~ Knowledge of procedures relating to emergency response to a security event (non-safeguards information ).

G2.4.5 Emergency Procedures/Plans 3.7 4.3 0 0 0 0 0 0 0 0 0 0 ~ Knowledge of the organization of the operating procedures network for normal, abnormal and emergency evolutions.

Page 2 of2 9/24/2008 3:17 PM

ES-401, REV 9 SRO T1G1 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME I SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO 008AG2.1.32 Pressurizer Vapor Space Accident 13 3.8 4.0 0 0 0 0 0 0 0 0 0 0 ~ Ability to explain and apply all system limits and precautions.

This is a Generic, no stem statement is associated.

011EA2.10 Large Break LOCA 1 3 4.5 4.7 0 0 0 0 0 0 0 ~ 0 0 0 Verification of adequate core cooling Ability to (a) predict the impacts of the following on the (SYSTEM) and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those abnormal operation:(CFR: 41.5/43.5/45.3/45.13) 025AA2.02 Loss of RHR System 14 3.4 3.8 0 0 0 0 0 0 0 ~ 0 0 0 Leakage of reactor coolant from RHR into closed cooling water system or into reactor building atmosphere Ability to determine and interpret the following as they apply to ABNORMAL PLANT EVOLUTION):(CFR: 41.10/43.51 45.13) 056AG2.2.40 Loss of Oft-site Power 1 6 3.4 4.7 0 0 0 0 0 0 0 0 0 0 ~ Ability to apply technical specifications for a system.

This is a Generic, no stem statement is associated.

065AA2.06 Loss of Instrument Air 1 8 3.6 4.2 0 0 0 0 0 0 0 ~ 0 0 0 When to trip reactor if instrument air pressure is de-creasing Ability to determine and interpret the following as they apply to ABNORMAL PLANT EVOLUTION):(CFR: 41.10 143.5 1 45.13) we04EG2.1.7 LOCA Outside Containment 13 4.4 4.7 0 0 0 0 0 0 0 0 0 0 ~ Ability to evaluate plant performance and make operational judgments based on operating This is a Generic, no stem statement is characteristics, reactor behavior and instrument associated. interpretation.

Page 1 of 1 9/24/2008 3:17 PM

ES-401, REV 9 SRO T1G2 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME / SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO 028AG2.1.7 Pressurizer Level Malfunction 1 2 4.4 4.7 0 0 0 0 0 0 0 0 0 0 ~ Ability to evaluate plant performance and make operational judgments based on operating This is a Generic. no stem statement is characteristics. reactor behavior and instrument associated. interpretation.

068AA2.02 Control Room Evac. 1 8 3.7 4.2 0 0 0 0 0 0 0 ~ 0 0 0 Local boric acid flow Ability to determine and interpret the following as they apply to ABNORMAL PLANT EVOLUTION):(CFR: 41.10 1 43.5 1 45.13) 076AG2.1.32 High Reactor Coolant Activity 1 9 3.8 4.0 0 0 0 0 0 0 0 0 0 0 ~ Ability to explain and apply all system limits and precautions.

This is a Generic. no stem statement is associated.

WE10EA2.1 Natural Circ. With Seam Void/4 3.2 3.9 0 0 0 0 0 0 0 ~ 0 0 0 Facility conditions and selection of appropriate procedures during abnormal and emergency operations.

Ability to determine and interpret the following as they apply to (EMERGENCY PLANT EVOLUTION):(CFR: 41.10/43.51 45.13)

Page 1 of 1 9/24/2008 3:17 PM

ES-401, REV 9 SRO T2G1 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME / SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO 012G2.4.2 Reactor Protection 4.5 4.6 0 0 0 0 0 0 0 0 0 0 ~ Knowledge of system set points, interlocks and automatic actions associated with EOP entry conditions.

This is a Generic, no stem statement is associated.

039G2.4.1 Main and Reheat Steam 4.6 4.8 0 0 0 0 0 0 0 0 0 0 ~ Knowledge of EOP entry conditions and immediate action steps.

This is a Generic, no stem statement is associated.

062G2.2.36 AC Electrical Distribution 3.1 4.2 0 0 0 0 0 0 0 0 0 0 ~ Ability to analyze the effect of maintenance activities, such as degraded power sources, on the status of This is a Generic, no stem statement is limiting conditions of operations associated.

063A2.02 DC Electrical Distribution 2.3 3.1 0 0 0 0 0 0 0 ~ 0 0 0 Loss of ventilation during battery charging Ability to (a) predict the impacts of the following on the (SYSTEM) and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those abnormal operation:(CFR: 41.5 1 43.5 1 45.3 1 45.13) 103A2.02 Containment 2.2 3.2 0 0 0 0 0 0 0 ~ 0 0 0 Necessary plant conditions for work in containment Ability to (a) predict the impacts of the following on the (SYSTEM) and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those abnormal operation:(CFR: 41.5/43.5/45.3/45.13)

Page 1 of 1 9/24/2008 3:17 PM

ES-401, REV 9 SRO T2G2 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME I SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO 016G2.2.40 Non-nuclear Instrumentation 3.4 4.7 D D D D D D D D D D ~ Ability to apply technical specifications for a system.

This is a Generic, no stem statement is associated.

033A2.01 Spent Fuel Pool Cooling 3.0 3.5 D D D D D D D ~ D D D Inadequate SDM Ability to (a) predict the impacts of the following on the (SYSTEM) and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those abnormal operation:(CFR: 41.5/43.5/45.3/45.13) 055G2.1.19 Condenser Air Removal 3.9 3.8 D D D D D D D D D D ~ Ability to use plant computer to evaluate system or component status.

This is a Generic, no stem statement is associated.

Page 1 of 1 9/24/2008 3:17 PM

ES-401, REV 9 SRO T3 PWR EXAMINATION OUTLINE FORM ES-401-2 KA NAME I SAFETY FUNCTION: IR K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G TOPIC:

RO SRO G2.1.3 Conduct of operations 3.7 3.9 0 0 0 0 0 0 0 0 0 0 ~ Knowledge of shift or short term relief turnover practices.

G2.2.1 Equipment Control 4.5 4.4 0 0 0 0 0 0 0 0 0 0 ~ Ability to perform pre-startup procedures for the facility, including operating those controls associated with plant equipment that could affect reactivity.

G2.2.36 Equipment Control 3.1 4.2 0 0 0 0 0 0 0 0 0 0 ~ Ability to analyze the effect of maintenance activities, such as degraded power sources, on the status of limiting conditions of operations G2.3.6 Radiation Control 2.0 3.8 0 0 0 0 0 0 0 0 0 0 ~ Ability to aprove release permits G2.3.7 Radiation Control 3.5 3.6 0 0 0 0 0 0 0 0 0 0 ~ Ability to comply with radiation work permit requirements during normal or abnormal conditions G2.4.35 Emergency Procedures/Plans 3.8 4.0 0 0 0 0 0 0 0 0 0 0 ~ Knowledge of local auxiliary operator tasks during emergency and the resultant operational effects G2.4.38 Emergency Procedures/Plans 2.4 4.4 0 0 0 0 0 0 0 0 0 0 ~ Ability to take actions called for in the facility emergency plan, including supporting or acting as emergency coordinator.

Page 1 of 1 9/24/2008 3:17 PM

ES-401 Record of Rejected K/As Form ES-401-4 Tier / Group Randomly Reason for Rejection Selected KIA T1Gl/SRO 008AG2.1.32 008AG2.2.4 was replaced with concurrence from Mark Bates with a randomly generated KA. This KA was not valid for FNP. 5/29/2008 per phone call.

T1G2/SRO 028AG2.1.7 028AG2.4.8 was replaced with concurrence from Mark Bates with a randomly generated KA. This KA was not valid for FNP. 5/29/2008 per phone call.

T1G2/RO 033AA2.02 060AA2.03 was replaced with concurrence from Mark Bates with a randomly generated KA. This KA had overlap concerns with another area of the exam. Also to write a multiple choice question to this KA using P&IDs and that a candidate can answer in 3 to 5 minutes that does not give away answers to other questions and to make it discriminatory for this exam is a concern among the exam authors.

T2G2/RO 001K5.04 071'K5.04 03 was replaced with concurrence from Mark Bates with a randomly generated KA. This question could only be written at a low level of difficulty for an RO, (i.e., level 1) or written at a SRO level.

This topic is being tested on the SRO portion of the exam which would be in conflict with the RO portion of the exam.

T3/SRO G2.2.36 G2.2.15 was replaced with concurrence from Craig Kontz and a peer checker with a randomly generated KA. This KA did not yield a suitable written exam question, but would have been more suitable for an ADMIN JPM, since several references would have needed to be

.provided.

T3/RO G2.3.13 G2.3.15 does not apply to FNP since the KA tests knowledge of radiation monitoring systems, such as fixed radiation monitors and alarms, portable survey instruments, personnel monitoring equipment, etc., and FNP does not require that detailed knowledge for our radiation workers. (We do not have Advanced radiation workers) A question written to the detail of the knowledge at FNP would constitute either minutia or a level 1 question.

T1Gl/RO EllEA1.2 E04EA1.1 directly conflicted with the SRO exam and a different KA but same topic. Since there is not much in the way of procedural guidance and background, there is little to examine on without incurring overlap.

T2G2/RO 001K5.07 001K5.04 since there are no operational implications of the RIL on the rods.

T1Gl/RO 027AA2.06 027AG2.4.6 since there are no EOP mitigation strategies for PRZR I pressure control malfunctions, only AOP strategies.

T2G2/SRO 016G2.2.40 028A2.01 could not meet the KA at an SRO level.

T2G2/SRO 033A2.01 071G2.1.20 could not meet the KA at an SRO level without many references.

ES-401, Page 27 of 33

ES-401-6 Written Examination Quality Checklist Facility: FA200B-301 Date of Exam: Oct 27 - Nov 10, 200B Exam Level: RO X SROX Initial Item Description a b* c#

il\~

1. Questions and answers are technically accurate and applicable to the facility.

"- IN:>

(\\1)

2. a.

b.

NRC K/As are referenced for all questions.

Facilitv learninq obiectives are referenced as available.

UI- ~ c#(

3. SRO questions are appropriate in accordance with Section D.2.d of ES-401 Cf2- ~ (f\~

4. The sampling process was random and systematic (If more than 4 RO or 2 SRO questions [.if:" If:::!1 I~ ~

were repeated from the last 2 NRC licensing exams, consult the NRR OL program office).

5. Question duplication from the license screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate:

~

X. the audit exam was systematically and randomly developed; or X the audit exam was completed before the license exam was started; or

_ the examinations were developed independently; or ~

X the licensee certifies that there is no duplication; or ~

_ other (explain) c#C

6. Bank use meets limits (no more than 75 percent from the bank, at least 10 percent new, and the rest Bank Modified 10/0 New 42/23 ~

I/I~

new or modified); enter the actual RO / SRO-only 23/2 ~

question distribution(s) at right. c#{

7. Between 50 and 60 percent of the questions on the RO Memory CIA I\\~

exam are written at the comprehension/ analysis level; the SRO exam may exceed 60 percent if the randomly selected K/As support the higher cognitive levels; enter 32/6 43/19 u:: (J<<J the actual RO / SRO question distribution(s) at riqht. ~

\~

B. References/handouts provided do not give away answers or aid in the elimination of distractors.

~ f'4 I'#{

9. Question content conforms with specific K/A statements in the previously approved examination outline and is appropriate for the tier to which they are assigned; deviations are justified.

~ /JNJ ~~

~

10. Question psychometric quality and format meet the quidelines in ES Appendix B. ~ 4JJ:)

11. The exam contains the required number of one-point, multiple choice items; the total is correct and agrees with the value on the cover sheet.

I~ f!'lP

\

eN{

Printed Name / ~.X Date

a. Author C. Vince Richter / ~"" I()-i~

b. Facility Reviewer (*) Ga'" Ohm,I"', I ~ IQl.t.LitOdg

c. NRC Chief Examiner (#) *N>,M..~A.t3A/iis/, . I . ; (\p...J.7(..,ne j '" Ie Ob l{~Cffl

. .~- "PI

d. NRC Regional Supervisor JplJf.;}rH"!.\!;ln~IAAlj1J l f., !N~/~

. /7. oJ:<:::: ~ ~'" ,

Note:

The facility reviewer's initials/signature are not applicable for NRC-developed examinations.

Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

0# LOK LOD ~--~~~--~--~--~~~----~--~--~--~~

(F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- 0= SRO B/M/N U/EIS Explanation Focus Dist. Link units ward KIA Only Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (Le., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if ajob content error is identified:

The question is not linked to the job requirements (Le., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (Le., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check Questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer's jUdgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

Page 1 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 LOK I I I I I I Q# LOD (F/H) (1-5) Stem Focus Cues T/F Cr.ed*1 Partial Dist.

J~b-Link Mnutia #/

units Back-ward 0=1 KIA SRO Only B/M'N U/EiS 1 H 2 o o o o o o o o N E 001AK3.02 The question is required to be enhanced to address the following:

DA indicates that 3.1.4 Cond B would be applicable - but with 2 rods out of alignment condition D would be applicable. Basis for D.1.1 indicates that SDM is the concern.

This potentially allows for 2 or 0 correct answers.

Options include information not required for answering the question. Additional information could be used by unknowledgeable applicant to eliminate distractors.

Including the diagnosis after each TS makes the first part of each answer unique.

This lowers the discriminatory value of the question by removing the diagnosis requirement of the question and making the second part of the question irrelevant.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Section C.1.f. of NUREG 1021 Appendix B - Limit question to one concept or topic, unless synthesis of concepts is being tested.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Remove second bullet from questions Question Sat 2 H 3 o o o 0 o o o o o o M u 001 K5.04 The question is unsatisfactory and is required to be changed to address:

Question does meet the intent of the KA statement. The question tests the effect of rod position on RIL and SDM not the Knowledge of the operational implications of RIL as they apply to the Control Rod Drive system.

Section C.1.b. of NUREG 1021 Appendix B - The question needs to match the intent of the KIA.

KA to be replaced - not able to derive a discriminatory question new KA 001 K5.07 New Question Sat Page 2 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Flaw s 4. Job Content Flaw s 5. Other 6 7 I I I I I I Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cr.ed*1 Partial J~b- Mnutial #1 8ack- 0= SRO 8/tvVN U/EIS Focus Dist. Link units ward KIA Only 3 H 2 o o o o o o o o o o N E 002A2.04 The question is required to be enhanced to address question clarity:

Having FRP-H.1 in the same sentence questioning about a loss of heat sink could cause some confusion as if it is asking for the requirements of FRP-H.1. Need to split into 2 sentences to avoid confusion.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

. Question will be reworded to prevent possible confusion.

Sat 4 F 3 o o o o o o o o o o N E 003AK2.05 The question is required to be enhanced to address:

By using the terms "Affected" and "Unaffected" in the options, it does not require the applicant to identify which cabnite(s) is/are affected and decreases the plausibility of a distractor by labeling it "Unaffected". Replacing with actual cabinet designations would alleviate this.

Section C.2.m.5 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Distractor B will have actual cabinet substituted for "Affected" Distractor D may refer to multiple cabinets - if so then add "s" to cabinet - if not change to address specific cabinet SAT Page 3 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 Q# LOK LOD (F/H) (1-5) Stem I Cues I T/F I Cr.ed*1 Partial J~b-I Mnutial #/ I Back- 0= I SRO B/flNN U/E/S Focus Dist. Link units ward KIA Only 5 F 2 o o 0 o o o o o o o N E 003K5.01 The question is required to be enhanced to address:

Use of the phrase "Significant Period Of Time" is too ambiguous to ensure the question has only one correct answer.

Exceeding QPTR limits is not a credible distractor for the given plant conditions and question asked.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Section C.2.m.5 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

"Significant time period" is to be removed from the question. QPTR left as distractor.

SAT 6 H 3 0 0 0 0 0 0 0 0 0 0 0 M S 004A3.02 7 H 2 ~ 0 0 ~ 0 0 0 0 0 0 0 N E 004K6.02 The question is required to be enhanced to address question clarity:

The stem specifies that the demineralizer was not flushed but does not give a condition to that would indicate to the applicant that the demineralizer was not vented.

This diminishes the plausibility of distractors C & D. Clarification of exactly what portion(s) of SOP-2.1 were or were not performed before placing cation bed in service in order to eliminate any potential confusion regarding system status and enhance distractor plausibility.

Section C.1.a. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Correct DA with correct procedure (SOP-2.5)

Focus stem to include actual procedure reference.

Page 4 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 Q#

I I LOK LOD (F/H) (1-5) Stem. 1 Cues T/F Cr.ed*1 Partial J~b- IMnutia I #1 1Back- 0= I SRO B/IINN U/E/S Focus Dist. Link units ward KIA Only 8 H 3 D D D D D D D D D B E 005K5.05 The question is required to be enhanced to address the following:

Options include information not required for answering the question. Additional information could be used by unknowledgeable applicant to eliminate distractors. The inclusion of the response of PCV-145 to changes in Letdown pressure and the purpose for the response are informational and not required for selection of the correct answer.

Section C.1.f. of NUREG 1021 Appendix B - Limit question to one concept or topic, unless synthesis of concepts is being tested.

Section C.2.d. of NUREG 1021 Appendix B - Include as much information as possible about the problem or situation in the stem. Leaving only the solution, action, or effect for the answer options.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Remove PCV-45 from choices SAT 9 H 3 D D D D D D D D D D D B s 005K6.03 Page 5 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I LOK LOD I

Q#

(F/H) (1-5) Stem Focus Cues T/F Cr.ed*1 Partial Dist.

J~b-Link Mnutial #1 units Back-ward 0=

KIA I SRO Only B/M'N U/E/S 10 F 3 o o o o o o o o o M E 006K1.08 The question is required to be enhanced to address the following:

Options include information not required for answering the question. Additional information could be used by unknowledgeable applicant to eliminate distractors.

Question statement containg conditions that should be incorporated into the plant conditions portion of the stem.

Options are long and confusing.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Section C.1.f. of NUREG 1021 Appendix B - Limit question to one concept or topiC, unless synthesis of concepts is being tested.

SectionC.2.d. of NUREG 1021 Appendix B - Include as much information as possible about the problem or situation in the stem. Leaving only the solution, action, or effect for the answer options.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Remove unnecessary information from choices.

Use suction source information only.

SAT Page 6 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I LOK LOD I I 1 Q#

(F/H) (1-5) Stem Cues T/F cr.ed*1 Partial J~b- Mnutial #1 Back- 0= SRO B/M'N U/6'S Focus Dist. Link units ward KIA Only 11 F 2 D D D D D D D D D D D N E 007EA2.01 The question is required to be enhanced to address:

Options include information not required for answering the question. Additional information could be used by unknowledgeable applicant to eliminate distractors.

Stem statement uses "was operating" in start of sentence and "is lost" at end - tense usage.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Section C.1.f. of NUREG 1021 Appendix B - Limit question to one concept or topic, unless synthesis of concepts is being tested.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Remove SUR from all choices.

SAT Page 7 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I I QIt LOK LOD (F/H) (1-5) Stem Focus Cues T/F Cr.ed*l Partial Dist.

J~b-Link Mnutia #1 units 8ack-ward 0=

KIA I SAO Only 8/M'N U/EIS 12 H 3 o o 0 '0 o o o o o o B E 007G2.1.7 The question is required to be enhanced to address the following:

Plant conditions in stem are not necessary for answering question. Only PRT conditions relevant.

SOP-1.2 is in all options - should be in stem HE3 is in 2 answers - other 2 are unique.

Section C.2.d. of NUREG 1021 Appendix B - Include as much information as possible about the problem or situation in the stem. Leaving only the solution, action, or effect for the answer options.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Remove first part of choices.

Move procedure reference to stem Reword question SAT 13 H 3 o o o o o o o o o B E 008AK2.03 The question is required to be enhanced to address the following:

Weather or not operator actions have been taken needs to be addressed in the stem.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

- - - - ~.************************************************

Add "no operator actions have been taken" statement to stem SAT Page 8 of 59

ES-401, Rev. 9 ' Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychorretric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I 1 Q# LOK LOD (FfH) (1-5) Stem Cues TfF Cr~d'l Partial J~b- Mnutial #f Back- 0= SRO BffIIVN U/E/S Focus Dist. Link units ward KIA Only 14 F 3 o o o 0 o o o o o o B E 008K4.09 The question is required to be enhanced to address the following:

A,B, and C are subsets of D. ONLY is needed for each to ensure clarity of choices Section C.2.m.4 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including options that include another option.

Add only to choices A,B, and C.

SAT 15 H 5 o o o 0 o o o o o o N u 009EK3.16 The question is unsatisfactory and is required to be changed to address:

The first half of all options are unique. Therefore, the second half of each option is not required to correctly answer the question and decreases the discriminatory value of the question by increasing the opportunity to eliminate distractors.

Question appears to be testing procedure transition requirements at an SRO level.

Question is not testing the reasons for the (Containment temperature, pressure, humidity and level limits) as they apply a small break LOCA and therefore does not meet the intent of the KA.

Section C.1.a. of NUREG 1021 Appendix B - The concept being measured needs to have a direct, important relationship to the ability to perform the job.

Section C.1.b. of NUREG 1021 Appendix B - The question needs to match the intent of the KIA.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

.Change Question to 2X2 Changed SAT Page 9 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Flaw s 4. Job Content Flaw s 5. Other 6 7 I I I I I Q# LOK LOD (F/H) (1-5) Stem Focus Cues T/F cr.ed*1 Partial Dist.

J~b-Link Mnutial #1 units Back-ward 0=

KIA I SRO Only B/M'N U/EIS 16 H 3 D D D D D D D D D D D N E 010A4.01 The question is required to be enhanced to address the following:

Distractor analysis for A does not agree with question stem; DA says 40% stem says 35%

DA will be changed SAT Page 10 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I Q# LOD I

LOK (F/H) (1-5) Stem Focus I I I Cues T/F Cr.ed*1 Partial Dist.

J~b-Link Mnutial #1 units Back-ward 0=

KIA SRO Only B/M'N U/6'S 17 H 2 o o o o o o o B u 010K1.05 The question is unsatisfactory and needs to be changed to address multiple implausible distractors:

Implausible distractors

1. The PRT does not have TS limits, therefore making distractor A referencing TS limits for the PRT is not plausible.

2. The PRT is designed to accept the lifting from a PORV without creating an explosive environment, therefore creating an explosive mixture from a PORV "blowing by" would not be plausible.

3. The rate of PORV leakage is not specifically stated in the stem of the question.

Depending on the leakrate, the temperature and pressure effects on the PRT would be very different. DA for distractor D is premised on the applicants knowledge of isenthalpic processes, the PRT is a saturated steam system, and that the leakrate would be sufficient enough for temperature in the PRT to rise above 216F prior to pressure increasing above rupture disc setpoint. The PRT is NOT a saturated steam system, it has a Nitrogen atmosphere. Even if the leakrate was given, knowledge of specific heat transfer characteristics of the PRT with varied leakrates is beyond the scope of operator requirements.

4. The stem of the question states that the temperature and pressure of the PRT are rising, decreasing the plausibility of distractors A & B.

Section C.1.a. of NUREG 1021 Appendix B - The concept being measured needs to have a direct, important relationship to the ability to perform the job.

Section C.1.a. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Section C.2.m.S of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Change Q to a 2X2 format Add Tailpiece temp SAT 18 F 2 o o o 0 o o o o o o o B s 011 K2.01 Page 11 of 59

ES-401, Rev. 9 Written Examination Review Worksheet I Form ES-401-9 1 2 3. Psychometric Flaw s 4. Job Content Flaw s 5. Other 6 7 I I I I I Q# LOK LOD (F/H) (1-5) Stem Focus Cues T/F cr~d'l Partial Dist.

J~b-Link Mnutial #1 units Back-ward 0=

KIA I SAO Only B/M'N U/5'S 19 H 3 D D D D D D B E 012K3.02 The question is required to be enhanced to address the following:

Distractor A is not a plausible distractor at 25% PWR.

Options include information not required for answering the question. Additional information could be used by unknowledgeable applicant to eliminate distractors.

Section C.2.m.5 of NUREG 1021 Appendix B - Specific determiners:that give cues as to the correct answer should be avoided, including implausible distractors.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient count~rbalance in questions with multi-part answers.

Add "Maintaining level on FRV's" to stem Remove unnecessary information from choices (second half) 20 F 3 D D D D D D D D D D N E 013A2.01 The question is required to be enhanced to address the following:

Question is testing knowledge of relay operation and ability to reset SI. ESP-1.2, specifically discusses resetting SI and MLB1, 1-1 and 1-11. One of the lights would have to not extinguished to apply RNO as described in distractor analysis.

Question is very cumbersome and requires editing to increase clarity.

Section C.1.G. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible. .

I Question/stem will be reworded to remove unnecessary information.-

Stem will have one annunciator staying lit Remove MOV references SAT Page 12 of 59

ES-401, Rev. 9 Written Examination Fteview Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I I Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cr.ed*1 Partial J~b- Mnutial #1 8ack- 0= SRO 8/M'N U/EIS Focus Dist. L.lnk units ward KIA Only 21 F 3 D D D D D D D D D D D B E 013K5.02 The question is required to be enhanced to address the following:

Need to clarify the definition of "remove from service" Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Change to "Place in Test" SAT 22 H 3 D D D D D D D D D D D B u 015AG2.4.5 The question is unsatisfactory and needs to be changed to address the following:

Question does not meet intent of KA. Question does not test ability to verify system setpoints.

Stem includes indications of #1 Sealleakoff flow and #1 Seal DP making alarms irrelevant in determining required actions of AOP. This makes knowledge of alarm setpoints unnecessary for determining correct answer.

Section C.1.b. of NUREG 1021 Appendix B - The question needs to match the intent of the KIA.

Insert actual indication Remove Annunciator refference Add pump vibration to make 2X2 format Page 13 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I Q# LOK LOD (F/H) (1-5) Stem Focus Cues T/F cr.ed*1 Partial

. Dist.

J~b-Link Mnutial #1 units 8ack-ward 0=

KIA 1 SRO Only 8/M'N U/EIS 23 F 2 o o 0 o o o o o o o B E 015K6.01 The question is required to be enhanced to address the following:

Distracto:

Distractor A has "Operable" in the distractor and the other options do not.

Stem of question contains information not required to answer the question. Failure is a "Red Herring" - doesn't effect answer Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible but provide all necessary information.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Remove Operable Cannot be manually reinstated ~ rework question SAT Page 14 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychorretric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I I I Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cr.ed*1 Partial J~b- Mnutia #1 8ack- 0= SRO 8/M'N U/EIS Focus Dist. Link units ward KIA Only 24 H 2 o o o o o o o o N u 022A2.01 The question is unsatisfactory and needs to be changed to address multiple implausible distractors:

Implausible distractors

1. Starting B Fan in Fast speed with A fan available and the stem indicating B fan has a fault alarm make distractors A & C implausible.

2. Knowledge that fast speed fans will not run after an SI is the only piece of information needed to answer question. This allows the applicant to eliminate all distractors without determining if the applicant has the knowledge required by the KIA.

Section C.2.m.5 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Section C.1.B. of NUREG 1021 Appendix B - The question needs to match the intent of the KIA.

Question changed to affect 1A fan Change format to 2 part first half and second half using first half as qualifier for 2X2 SAT 25 F 2 o o o 0 o o o o o o o 8 s 022G2.4.20 Page 15 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 LOD I I Q# LOK (F/H) (1-5) Stem Focus I I I Cues T/F cr~d*1 Partial Dist.

J~b-Link Mnutial #1 units Back-ward 0=

KIA I SRO Only B/M'N U/6'S 26 H 4 D D D D D D D D B E 025AA2.05 The question is required to be enhanced to address the following:

Distractor A reads opposite of P&L even though P&L does not apply in this situation.

Decreases plausibility of distractors.

The first half of all options are unique. Therefore, the second half of each option is not required to correctly answer the question and decreases the discriminatory value of the question by increasing the opportunity to eliminate distractors.

Section C.2.f of NUREG 1021 Appendix 8 - Provide sufficient counterbalance in questions with multi-part answers.

Section C.1.f. of NUREG 1021 Appendix 8 - Lim it question to one concept or topic, unless synthesis of concepts is being tested.

Section C.2.m.S of NUREG 1021 Appendix 8 - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Changed distractors to give band for flow rate Removed explinations SAT 27 F 2 D D D D D D D D D D D N s 026AK3.03 28 H 4 D D D D D D D D D D D N s 026K3.02 Potential issue with the premise that the check valve will prevent potential air introduction into the CS pump and cause air binding. Suction head on pump not very significant. Need to discuss.

Discussed with licensee and assured that static head on pump suction sufficient to hold check valve closed.

SAT Page 16 of 59

ES-401, Rev. 9 Written Examination Review Worksheet . Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I I Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cr.ed*1 Partial J~b- Mnutial #1 Back- 0= SRO B/M'N U/E/S Focus Dist. LInk units ward KIA Only 29 F 3 D D D D D D D D D M u 027AG2.4.6 The question is unsatisfactory and needs to be changed to address the following:

Multiple Implausible distractors

1. Procedures require that the reactor is to always be tripped prior to securing RCPs.

This makes C & D distractors implausible.

KIA Match

1. KIA addresses EOP strategies - -Question written for AOP procedure steps.

Section C.2.m.S of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Section C.1.B. of NUREG 1021 Appendix B - The question needs to match the intent of the KIA.

Resampled KA - 027AA2.06 SAT 30 F 3 D D D D D D D D D D D N E 027G2.1.27 The question is suggested to be enhanced to address the following:

Symmetry between distractors - 2 use "so that" and 2 use "to ensure that" Changed choises to make symetrical use of "in order to" Page 17 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I LOK LOD I I QIt (F/H) (1-5) Stem Focus Cues T/F Cr:d'l Partial Dist.

J~b-Link Mnutial #1 units Back-ward 0=

KIA I SRO Only B/M'N U/E/S 31 H 3 o o o o o o o o o B u 029EK1.05 The question is unsatisfactory and needs to be changed to address multiple implausible distractors:

1. distractor A - Not plausible to provide full AFW flow to add negative reactivity. Basic knowledge of reactivity coefficients makes this distractor implausible.

2. distractor 0 - Not Plausible to depressurize RCS to 200# during ATWS - fouled by Q 39 - Concept of distractor is correct - only 200# is incorrect Section C.2.m.S of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

- - - - ~***************** ************************

Replaced question with one from 2008 NA exam SAT Page 18 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I I Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cr.ed*1 Partial J~b- Mnutial #1 Back- 0= SRO BIM'N U/EIS Focus Dist. Link units ward KIA Only 32 F 2 D D D D D D D D N E 029K4.02 The question is required to be enhanced to address the following:

The first half of all options are unique. Therefore, the second half of each option is not required to correctly answer the question and decreases the discriminatory value of the question by increasing the opportunity to eliminate distractors.

Options include information not required for answering the question. Additional information decreases the discriminatory value of the question by increasing the opportunity to eliminate distractors.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Section C.2.d. of NUREG 1021 Appendix B - Include as much information as possible about the problem or situation in the stem. Leaving only the solution, action, or effect for the answer options.

Changed to a 2X2 format using B&C first halves Reworded question to focus stem.

SAT Page 19 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I LOK LOD I

Q#

(FfH) (1-5) Stem Cues TfF Cr.ed*1 Partial J~b- Mnutial #f 8ack- 0= SRO 8fM'N Uf6'S Focus Dist. Link unITS ward KIA Only 33 H 4 D D D D D D D D D D N E 033AA2.02 The question is required to be enhanced to address the following:

Stem needs to specify that IR channel indications have "leveled off" or stabilized.

Options indicate which channel of IR indication is faulty, KA requires a diagnosis of an unreliable indication.

Section C.1.b. of NUREG 1021 Appendix B - The question needs to match the intent of the KIA.

Added "IR indication leveled off" Removed second part of choices Changed 2 of the choises to N-35 SAT 34 H 3 D D D D D D D D D B E 035A3.01 The question is required to be enhanced to address the following:

Distractor B is not plausible for FRV to end up in a position more open than started.

Also the symmetry between AlB is not consistent with symmetry of C/D. Change to agree with A - return to a position more closed than original position.

Section C.2.m.5 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Changed B distractor to "return to a more closed position" SAT Page 20 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I

Q# LOK LOD (FfH) (1-5) Stem Focus I I I Cues TfF cr.ed*1 Partial Dist.

J~b-Link I Mnutial #f units Back-ward 0=

KIA 1 SAO Only BfM'N U/E/S 35 F 3 D D D D D D D D B E 038EA1.34 The question is required to be enhanced to address the following:

The options need to specify which loop the distractors are addressing. In tact or ruptured.

Distractor C - Stagnant loop temperature rising is not a plausible concept for the given conditions. Suggest change "slow rise in cold leg temp due to loop being stagnant" to "slow rise in cold leg temp due to reversing flow in ruptured loop". - Actual effect is rapid cooling due to SI flow.

Distractor D - Rapid rise in containment pressure from PRT rupture not a plausible concept.

Section C.2.m.S of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Identify which loops are rerfernced in choixes Change to a 2X2 format Effect / Cause SAT Page 21 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Flaw s 5. Other 6 7 I I I I I I Off LOK LOD (F/H) (1-5) Stem Cues T/F Cr:d*l. Partial J~b- Mnutial #1 Back- 0= SRO B/M'N U/EIS Focus Dist. Link units ward KIA Only 36 H 3 o o o o o o o o N E 039A4.04 The question is required to be enhanced to address the following:

Options include information not required for answering the question. Additional information could be used by unknowledgeable applicant to eliminate distractors. The position of HV-3235A/B does not appear to be able to be determined from information given in stem. Also their valve positions are not relevant in the options. Options give a choice between HV-3226 being "open" or "leaking by".

The use of "and/or" inherently adds ambiguity to the question.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible but provide all necessary information.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Replaced question SAT 37 H 3 o o o 0 o o o o o o o B s 040AK2.02 Page 22 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Flaw s 5. Other 6 7 I I I I I Q#

I LOK LOD (F/H) (1-5) Stem CUes T/F Cr.ed*1 Partial J~b- Mnutial #/ Back- 0= SRO B/MIN U/~S Focus Dist. Link units ward KIA Only 38 F 2 D D D D D D D D D D M E 041A1.01 The question is required to be enhanced to address the following:

Question cues applicant to evaluate conditions for P-12 being met.

Options include information not required for answering the question. Additional information could be used by unknowledgeable applicant to eliminate distractors.

Distractors C & D have 3 bullets and A & B have 2.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible but provide all necessary information.

Remove P-12 interlock from stem Remove 3rd bullet from choices C/D SAT 39 F 2 D D D D D D D D D 8 s 055EK3.02 40 F 2 D D D D D D D D D D M E 055K3.01 The question is required to be enhanced to address the following:

"Get Better" and "Get Worse" provide cueing to applicant as to the nature of the change in condenser pressure.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Add PSIA to pressure Removw "Get Better & Get Worse" SAT Page 23 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 Q# LOK LOD (F/H) (1-5) Stem I Cues I T/F I Cr.ed*1 Partial J~b-I Mnutial #/ I Back- 0= I SRO B/M'N U/6'S Focus Dist. Link units ward KIA Only 41 H 2 D D D D D D D D D N u 056AG2.4.1 The question is unsatisfactory and needs to be changed to address multiple im plausible distractors:

Question asks for "Immediate Operator Actions" A is the only option that contains ANY lOA's, making the question answerable without knowing stem. This makes the other 3 choices implausible.

Section C.2.m.S of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Section C.2.m.2 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including options that can be judged correct or incorrect without reading the stem.

Removed "immediate operator actions" from question SAT 42 F 2 D D D D D D D D D D D B s 056K1,03 Page 2401 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychorretric Raw s 4. Job Content Raw s 5. Other 6 7 I

Q# LOK LOD (F/H) (1-5) Stem Focus I I I Cues T/F Cr.ed*1 Partial Dist.

J~b-Link I Mnutial #1 units Back-ward 0=

KIA I SAO Only B/M'N U/EIS 43 H 3 D D D D D D D N E 057AK3.01 The question is required to be enhanced to address the following:

Clarity of answer choices - answers are long and cumbersome.

Stem indicates "8 Train is On Service train" , appears to be excess information in stem not required for applicant to answer question.

Distractor 8 lacks plausibility - having a valve opened on the "on-service" train Question doesn't address the reason for a procedural response, it appears to addresses the reason a pump has power.

Section C.1.b. of NUREG 1021 Appendix 8 - The question needs to match the intent of the KIA.

Section C.2.d. of NUREG 1021 Appendix 8 - Include as much information as possible about the problem or situation in the stem. Leaving only the solution, action, or effect for the answer options.

Section C.2.m.5 of NUREG 1021 Appendix 8 - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Changed options to a 2X2 format Which pump to start - LC shedding SAT 44 F 2 D D D D D D D D B E 058AK1.01 The question is required to be enhanced to address the following:

Need to review wording in question - ... implications ON the aux building 125V DC system and maximum time to recharge the batteries to full power.

Reword question SAT Page 25 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I Q# LOK LOD (F/H) (1-5) Stem Focus Cues T/F cr~d'l Partial Dist.

J~b-Link Mnutial #1 units 8ack-ward 0=

KIA I SRO Only 8/M'N U/EIS 45 F 2 D D D D D D D D M E 059A3.03 The question is required to be enhanced to address the following:

3 options are T/F statements not requiring information contained in the stem of the question.

Distractor C is a subset of distractor D.

Section C.2.m.2 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including options that can be judged correct or incorrect without reading the stem.

Section C.2.m.4 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including options that include other options.

Section C.2.c of NUREG 1021 Appendix B - Do not present a collection of true-false (T/F) statements as a multiple choice item.

Changed question to 295# and decreasing Changed options to a 2X2 Pressure and time for SCP to start SAT Page 26 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I LOK LOD I

Q#

(FtH) (1-5) Stem Cues TtF cr.ed*1 Partial J~b- Mnutial #t 8ack- 0= SAO BlM'N UtE/S Focus Dist. Link units ward KIA Only 46 H 3 D D D D D D D D D M E 059AG2.4.4 The question is required to be enhanced to address the following:

SOP-50.1 sequence for securing discharge has pump secured, RCV-18 shut, then Manual discharge shut after other valve manipulations. Securing the pump or shutting RCV-18 would appear meet the requirement of IMMEDIATELY stopping the discharge. Closing a manual valve in the field seems to not be an immediate method.

Need clarification on expectation of IMMEDIATELY, potential for argument of no correct answer.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible but provide all necessary information.

Section C.2.d. of NUREG 1021 Appendix B - Include as much information as possible about the problem or situation in the stem. Leaving only the solution, action, or effect for the answer options.

Changing to a 2X2 format Method to secure discharge x SAT Page 27 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 Ott LOK LOD (F/H) (1-5) Stem Focus I I I Cues T/F Cr~d'l Partial Dist.

J~b-Link I Mnutial #1 units I8ack-ward 0=

KIA J SRO Only 8/M'N U/E/S 47 F 2 D D D D D D D D M E 061AK2.01 The question is required to be enhanced to address the following:

Options include information not required for answering the question. Additional information could be used by unknowledgeable applicant to eliminate distractors.

Distractors C & D have unique information not in other options. There are a total of 3 different RAD monitor combinations and 4 different ventilation combinations. This would potentially enable an individual to select the correct answer without adequate knowledge of the RMs.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Change to a 2X2 format Remove 3rd bullets Change D first part to ageree W C SAT 48 F 3 D D D D D D D D D D N E 061 K6.01 The question is required to be enhanced to address the following:

The answer includes information not required for answering the question. The answer contains a specific determiner that is a qualifier for the answer. It includes the phrase "and possibly trip" at the end of the statement which is unnecessary to correctly answer the question.

Section C.2.m.7 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including qualifiers in the correct answer unless they are also used in the distractors.

Remove "and possibly trip" & "And there will be no speed control" SAT Page 28 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other .6 7 I I I I I Q# LOK LOD (F/H) (1-5) Stem Focus Cues T/F cr.ed*1 Partial Dist.

J~b-Link Mnutial #1 units Back-ward 0=

KIA I SAO Only B/IvVN U/EIS 49 H 3 o o o o o o o o o N E 062AA1.07 The question is required to be enhanced to address the following:

It is not plausible for a dropping level in the pond to cause pressure to increase in the SW system.

The second half of all options are unique. Therefore, the first half of each option is not required to correctly answer the question and decreases the discriminatory value of the question by increasing the opportunity to eliminate distractors.

Section C.2.m.5 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Changed "increase" to "decrease" in C & D Deleted "to ..... " statements from options SAT Page 29 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I Q# LOK LOD (F/H) (1-5) Stem Focus Cues T/F Cr.ed*1 Partial Dist.

J~b-Link Mnutial #1 units Back-ward er-KIA I SRO Only B/M'N U/E/S 50 F 2 o o o o o o o o o N E 062K4.01 The question is required to be enhanced to address the following:

Potential for two correct answers. Definition of minimum could be questioned -

options are for different methods to accomplish same thing.

Options are not balanced.

Distractor A not plausible. Attempting to start pump is not a plausible distractor for clearing the fault on the pump.

Turning pump switch to start and releasing it is in every option.

Distractor D is not specific as to which Start Switch should be taken to start.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible but provide all necessary information.

Section C.2.m.5 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Changed "send operator. .. " to "Locally" SAT 51 F 3 o o o o o o o o o N s 062K4.03 Page 30 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I Q#

I LOK LOD (F/H) (1-5) Stem Cues T/F Cr.ed*1 Partial J~b- Mnutial #1 Back- 0= SRO B/IVYN U/EtS Focus Dist. Link units ward KIA Only 52 H 3 D D D D D D D D D D B E 063K2.01 The question is required to be enhanced to address the following:

Options have up to 5 different elements Options B & C reference "sequencers" and option D references "sequencer" Options include information not required for answering the question. Additional information could be used by unknowledgeable applicant to eliminate distractors.

Section C.2.d. of NUREG 1021 Appendix B - Include as much information as possible about the problem or situation in the stem. Leaving only the solution, action, or effect for the answer options.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Change options to a 2X2 Start location X Sequencer response SAT 53 H 3 D D D D D D D D D D N E 064A1.02 The question is required to be enhanced to address the following:

TS limits are different for dipstick and EPB readings stem doesn't indicate what reading is to be used.

TS mentions 21000 gal as fuel level to declare diesel inoperable and 25000 for needing to fill. Stem needs to specify which TS min level is referenced.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible but provide all necessary information.

Add "As read on EPS" to stem Page 31 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 Q# LOK LOD (F/H) (1-5) Stem I Cues I T/F I Cr.ed*1 Partial J~b-I Mnutial #1 I Back- 0= I SRO B/M'N U/6'S Focus Dist. Link units ward KIA Only 54 H 3 D D D D D D D D D N E 065AA2.03 The question is required to be enhanced to address the following:

The first half of all options are unique. Therefore, the second half of each option is not required to correctly answer the question and decreases the discriminatory value of the question by increasing the opportunity to eliminate distractors.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible but provide all necessary information.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Remove "has closed to isolate leak" Remove second bullet Remove"and provides location of the leak" from stem SAT Page 32 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I I Q# LOK LOD (F/H) (1-5) Stem CUes T/F Cr.ed*1 Partial J~b- Mnutial #1 Back- 0= SRO B/M'N U/E'S Focus Dist. Link units ward KIA Only 55 H 3 o o o o o o o o N E 073A2.02 The question is required to be enhanced to address the following:

The first half of all options are unique. Therefore, the second half of each option is not required to correctly answer the question and decreases the discriminatory value of the question by increasing the opportunity to eliminate distractors Options include information not required for answering the question. Additional information could be used by unknowledgeable applicant to eliminate distractors.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Section C.2.d. of NUREG 1021 Appendix B - Include as much information as possible about the problem or situation in the stem. Leaving only the solution, action, or effect for the answer options.

Change options to a 2X2 format Valve reaction X Follow-up actions SAT 56 H 3 o o 0 o o o o o o o N E 076A1.02 The question is required to be enhanced to address the following:

Stem indicates FCV-3009C is "Fully Opened" but opening is mechanically limited --

could cause confusion to applicants.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible but provide all necessary information.

Stem will be clarified to "open valve to Open-Stop" SAT Page 33 of 59

ES-401, Rev. 9 Written Examination Review Worksheet ,Form ES-401-9 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 Q# LOK LOD (F/H) (1-5) ~St-e-m"l-cu~e-s""IT,~l~F-I""Cr~.ed~."I~Pa!"'"r-'tia-'I+-J!"'"~b!"'"--I~M~n-u-'-tia"ll-#~t-~IBa-c~k--+--'o=~-rI-S~IRO~ BfMlN ut6'S Focus Dist. Link I units ward KIA Only 57 F 2 o o o 0 o o o o o o N s 076AA1.04 58 F 3 o o o 0 o o o o o o o 8 s 076K2.08 59 H 1 o o o o o o o o N u 077AK1.01 The question is unsatisfactory and needs to be changed to address the following:

Taking voltage switch to lower then asking what voltage will do cues applicant to correct answer and makes 2 distractors implausible.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible but provide all necessary information.

Section C.2.m.S of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Focus stem and remove operator action.

Change options to 2X2 Operator will change Voltage in which direction X How will Amps respond SAT Page 34 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 Q#

I LOD I I I I I LOK (F/H) (1-5) Stem CUes T/F cr.ed*1 Partial J~b- Mnutial #1 Back- 0= SRO B/M'N U/EtS Focus Dist. Link units ward KIA Only 60 H 3 o o o o o o o o o B E 078K1.03 The question is required to be enhanced to address the following:

Distractor A not credible with "Pressure will rise until" given in the stem.

Section C.2.m.5 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Remove "Pressure will rise until statement from stem Change distractors Add the reactor will trip on high PZR level The pzr will go solid Remove Code safety's will lift Reword PORV opening.

SAT 61 H 3 o o o 0 o o o o o o o N s 103G2.4.4 Page 35 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I LOD I

Q# LOK (F/H) (1-5) Stem Cues T/F cr.ed*1 Partial J~b- MnUtiat #1 18ack- 0= SRO 8/M'N U/8'S Focus Dist. Link units ward KIA Only 62 H 4 D D D D D D D D D D B E WE03EK1.2 The question is required to be enhanced to address the following:

Option D gives "how" SI will be reduced while other options give "why" Question appears to require detailed knowledge of ESP - 1.2 procedure for a specific set of circumstances. Potentially SRO knowledge.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Section C.1.a. of NUREG 1021 Appendix B - The concept being measured needs to have a direct, important relationship to the ability to perform the job.

Knowlede of soak requirements is required RO knowledge at Farley Remove "by securing HHSI" from D SAT Page 36 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I LOK LOD I

Q#

(F/H) (1-5) Stem Cues T/F cr.ed*1 Partial J~b- Mnutial #1 8ack- 0= SAO 8/M'N U/E/S Focus Dist. link units ward KIA Only 63 F 3 o o 0 o o o o o o N u WEOBEK1.2 The question is unsatisfactory and needs to be changed to address the following:

Need time from accident start to establish the> 1OOF in one hour for soak requirement Action to be performed during soak not RO knowledge Holding temp constant is in all distractors Section C.1.a. of NUREG 1021 Appendix B - The concept being measured needs to have a direct, important relationship to the ability to perform the job.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible but provide all necessary information.

Section C.2.d. of NUREG 1021 Appendix B - Include as much information as possible about the problem or situation in the stem. Leaving only the solution, action, or effect for the answer options.

Change options to a 2X2 format Cooldown X Depressurization requirements SAT Page 37 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 LOD I I I J LOK I

Q#

(F/H) (1-5) Stem Focus Cues T/F Cr.ed'J Partial Dist.

J~b-link Mnutial #1 units Back-ward 0=

KIA J SRO Only B/M'N U/8'S 64 H 3 o o o o o o o o o o N u WE11EA1.2 The question is unsatisfactory and needs to be changed to address the following:

Question used backward logic to ask question.

Pictures are nice but not viewed as necessary.

Section C.1.h. of NUREG 1021 Appendix B - Avoid "backward logic" questions that ask for what should be provided in the question and provide what should be required in the examinee's response.

Remove 2 pictures - leave one picture unlabled as to which valve it corresponds Add - Amber light lit for RHR pump description Reference picture for valve corresponded to in option Change question wording to be a positive question SAT 65 H 2 o o o o o o o o o o o N s WE15EA2.1 Procedural transitions are SRO LOK normally but indicated that for this procedure it is a requirement of the RO to know how to step through procedure and that transition out would be inappropriate.

Procedure flowpath and exit criteria is RO LOK at Farley for this procedure Page 38 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 Q# LOK LOD (F/H) (1-5) Stem 1 Cues Focus J T/F .1 Cr.ed*1 Partial Dist. .

J~b- 1Mnutia 1 #1 Link units 1 8ack-ward 0= 1 SAO KIA Only 8/M'N U/E/S 66 H 2 D D D D D D D D D N E G2.1.17 The question is required to be enhanced to address the following:

Distractor C is not plausible.

Does the SM "call for" or "provide" an update?

Section C.2.m.5 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Change option C to Thunderstorm warnings have been issued for the area.

Actually changed to STA Announced he is stationed in CR SAT 67 F 2 D D D D D D D D D D D B E G2.1.21 The question is recommended to be enhanced to address the following:

Separate conditions in the stem from the question.

Option A is a subset of Option C.

Section C.2.d. of NUREG 1021 Appendix B - Include as much information as possible about the problem or situation in the stem. Leaving only the solution, action, or effect for the answer options.

Section C.2.m.5 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Reformat stem Remove last sentence in C.

SAT Page 39 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychorretric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I Q# LOK LOD (F/H) (1-5) Stem Focus Cues T/F cr.ed*1 Partial Dist.

J~b-Link Mnutial #1 units Back-ward 0=

KIA I SAO Only B/M'N U/EtS 68 F 2 D D D D D D D D D D D B E G2.1.36 The question is recommended to be enhanced to address the following:

Remove "but" from distractor A.

Reword distractor D to be a direct statement like the other options.

Remove But from A Reword distractor D SAT 69 F 3 D D D D D D D D D B E G2.2.13 The question is required to be enhanced to address the following:

Options include information not required for answering the question. Additional information could be used by unknowledgeable applicant to eliminate distractors.

Options C & D have qualifying statements in the second half not contained in A & B and not required to correctly answer the question.

The second half of all options are unique. Therefore, the first half of each option is not required to correctly answer the question and decreases the discriminatory value of the question by increasing the opportunity to eliminate distractors.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Change to a 2X2 format Can position be changed X Status of system/component Changed "equipment to either system/component SAT Page 40 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I I Q# LOK LOD (F/H) (1-5) Stem Focus Cues T/F Cr.ed*1 Partial Dist.

J~b-Link Mnutia #1 units Back-ward 0=

KIA I SRO Only B/M'N U/EtS 70 F 3 D D D D D D D D D D N E G2.2.17 The question is required to be enhanced to address the following:

The first and second halves of all options are unique. Therefore, only one of 12 pieces of information is required to correctly answer the question and decreases the discriminatory value of the question by increasing the opportunity to eliminate distractors.

Options include information not required for answering the question. Additional information could be used by unknowledgeable applicant to eliminate distractors.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

White status added to stem Remove reference to NRC PI Use only 4 individual components 71 F 3 D D D D D D D D D D D B u G2.3.13 The question is unsatisfactory and needs to be changed to address the following:

Distractors C & D are not plausible. Staying in area is not plausible with alarm.

Verify not basic radworked knowledge.

Section C.2.m.S of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Replaced question SAT 72 F 3 D D D D D D D D D D D N s G2.3.4 The question is recommended to be enhanced to address the following:

Movement should be capitalized in stem.

Page 41 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I I Off LOK LOD (FfH) (1-5) Stem Cues TfF Cr~d*1 Partial J~b- Mnutial #f Back- 0= SRO BfM'N UfE/S Focus Dist. Link units ward KIA Only 73 F 2 D D D D D D D D D M E G2.4.26 The question is required to be enhanced to address the following:

Security on hose support not plausible.

Fire Marshal - -only one - not plausible for shift Fire Brigade make-up The first half of all options are unique. Therefore, the other information in the options is not required to correctly answer the question and decreases the discriminatory value of the question by increasing the opportunity to eliminate distractors.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Removed tanker from question Changed to a 2X2 format Personnel support X Agent type SAT Page 42 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I I Q# LOK LOD (F/H) (1-5) Stem Cues T/F cr:d'l Partial J~b- Mnutial #1 Back- 0= SRO B/M'N U/EtS Focus Dist. Link units ward KIA Only 74 H 1 D D D D D D D D D D D B u G2.4.28 This question may need to be withheld from ADAMS - need to discuss licensee expectations for disposition of exam hard copies and control of information in the event of appeal.

The question is unsatisfactory and needs to be changed to address the following:

Distractor Analysis states "only Unit 1 is affected so far" but stem indicates all service water is lost to both units.

Potential for 2 correct answers - use of AOP-1 0 would be applicable as well as AOP-49.

'Question doesn't specifically ask for actions lAW AOP-49.

Option assumes 1B DG is running and not specified in stem.

Options include information not required for answering the question. Additional information could be used to eliminate distractors.

Complete loss os SW to both units would also require a dual unit trip - knowledge of unit security procedure not required to select correct answer.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible but provide all necessary information.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Section C.2.mA of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, an option that includes another option.

Change to a 2X2 format RX Trip X EDG actions Remove excess information Add 1B EDG supplying buss to stem SAT 75 F 2 D D D D D D D D D D D M s G2.4.5 Page 43 of 59

ES-401, Rev. 9 Written Examination Review Worksheet . Form ES-401-9 1 2 3. Psychorretric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I I Q# LOK LOD (F/H) (1-5) Stem CUes T/F Cr~d'l Partial J~b- Mnutial #1 Back- 0= SRO B/lvVN U/6'S Focus Dist. Link units ward KIA Only 76 H 3 o o 0 o o o o o o o N E 00BAG2.1.3 The question is required to be enhanced to address the following:

Distractors/Answer includes information not required for answering the question.

Additional information cues applicant regarding content of procedure, which is required knowledge.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Section C.2.d. of NUREG 1021 Appendix B - Include as much information as possible about the problem or situation in the stem. Leaving only the solution, action, or effect for the answer options.

Remove "since ..... " from options Reworded option to focus information 77 H 3 o o 0 o o o o o o o N E 011 tA2.10 The question is required to be enhanced to address distractor plausibility:

Distractor D specifies to "maintain" Core exit TIC's falling, when the stem indicates they are "stable". Suggest using a term to indicate a change in current condition. (e.g.

"establish")

Section C.2.m.1 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including distractors that do not follow grammatically from the stem.

Change "maintain" to "establish" SAT Page 44 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Flaw s 4. Job Content Raw s 5. Other 6 7 I I I I I I Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cr:d'l Partial J~b- Mnutial #1 Back- 0= SRO 8/M'N U/E/S Focus Dist. Link units ward KIA Only 78 H 4 o o 0 o o o o o o o N E 012G2.4.2 The question is required to be enhanced to address question clarity:

Question states it is asking for 3 knowledge aspects; 1 - System Operation 2 - TS Applicability 3 - TS Basis Knowledge of system operation and TS applicability are not interrelated. TS applicability is tied to system operation in distractors/answer. Direct tie between information pieces cues applicant regarding the other. .

TSB is unique for each answer. Question is only testing information on TS 3.3.1 applicability below 8%.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Section C.1.f. of NUREG 1021 Appendix B - Limit question to one concept or topic, unless synthesis of concepts is being tested.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Change stem initial conditions Changed to a 2X2 format Action X Basis 79 H 3 o o o 0 o o o o o o o N s 025AA2.02 Page 45 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I LOK LOD I I Q#

(F/H) (1-5) Stem Focus Cues T/F Cr.ed*1 Partial Dist.

J~b-Link Mnutial #1 units Back-ward 0=

KIA 1 SAO Only B/MIN U/EtS 80 H 2 o o o 0 o o o o o N u 028A2.01 The question is unsatisfactory and is required to be changed to address:

Non-discriminatory level of knowledge required to correctly answer question.

(Distinguishing between 3.4% and 3.9% for starting recombiners is not a discriminating piece of knowledge and potentially a systems knowledge requirement.)

SRO-Only knowledge is not required to correctly answer question. ( Neither the knowledge of the starting limit or reading of the graph meet the requirements to be considered disc rim inating for an SRO applicant)

Section C.1.a. of NUREG 1021 Appendix B - The concept being measured needs to have a direct, important relationship to the ability to perform the job. (Not trivial in nature)

Section C.1.e. of NUREG 1021 Appendix B - Avoid questions that are unnecessarily difficult or irrelevant.

Replaced KA W 016G2.2.40 SAT Page 46 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Flaw s 4. Job Content Flaw s 5. Other 6 7 Q#

I LOK LOD (F/H) (1-5) Stem 1 Cues 1 T/F 1 Cr:d*.1 Partial Focus Dist.

J~b- 1Mnutial Link

1 units J 8ack-ward 0=

KIA SRO

<?nly 8/M'N U/EtS 81 H 3 D D D D D D D D N E 028AG2.1.7 The question is required to be enhanced to address the following:

Clarity of distractors - the word "for" should be inserted between "channel" and "up" in distractors B & C.

Cueing in the stem - indicating the TS evaluations that are in progress is expected SRO knowledge and not required to answer the question.

Distractor analysis should indicate that question is SRO Only because application of generic LeO requirements, including 3.0.3, is SRO Only knowledge. NOT because instrument failure is more than 1 hours1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> action statement.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Added "for" to optiorls as necessary Remove TS from stem SAT Page 47 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychorretric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I Q#

I LOK LOD (F/H) (1-5) Stem Cues T/F Cr.ed*1 Partial J~b- Mnutial #1 8ack- 0= SRO 8/I\NN U/E/S Focus Dist. Link units ward KIA Only 82 H 3 D D D D D D D D D N E 039G2.4.1 The question is required to be enhanced to address the following:

Distractor Bis not a credible distractor with no indications that SI may be required.

Answer D is significantly longer than the distractors.

Distractor analysis indicates "as long as 2 operators are present to allow one operator to complete immediate actions" - reference needed to support specific requirement and remove potential ambiguity.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Section C.2.m.6 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including the correct answer being longer than the distractors.

Focused stem Reworded options for clarity and removed unnecessary information Changed to a 2X2 format Trip vs SD X lA's VS shut MSIV's SAT 83 H 3 D D D D D D D D D D N E 055G2.1.19 The question is required to be enhanced to address the following:

Distractors/Answer includes information not required for answering the question.

Additional information could be used by unknowledgeable applicant to eliminate distractors.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Remove "When .... " from C & D SAT Page 48 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 Q# LOK LOD (F/H) (1-5) Stem I Cues I T/F I Cr~d'l Partial J~b-I Mnutial #1 I Back- 0= I SRO BIM'N U/5'S Focus Dist. Link units ward KIA Only 84 H 3 D D D D D D D D D D D N E 056AG2.2.4 The question is suggested to be enhanced to improve question clarity:

As worded the question uses TS 3.8.1 in the stem and in only 2 of the distractors.

Potentially incorrectly suggests that 3.8.1 is not applicable and therefore NOT entered at all. Suggest rewording question without referencing TS 3.8.1 in stem.

Section C.1.c. of NUREG 1021 Appendix 8 - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Remove TS reference from stem SAT 85 F 2 D D D D D D D D D D D N E 062G2.2.36 The question is suggested to be enhanced to improve question clarity:

Question statement "At 1000 prior to any maintenance or further actions:" might cause confusion if it means prior to or after bypass switch manipulation. Suggest clarifying that the question is referencing after the inverter is in bypass.

Section C.1.c. of NUREG 1021 Appendix 8 - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Changed reference time in question to 1002 SAT Page 49 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I Q# LOK LOD (F/H) (1-5) Stem Focus Cues T/F cr~d*1 Partial Dist.

J~b-Link Mnutial #1 units Back-ward 0=

KIA I SAO Only B/MN U/E/S 86 H 3 D D D D D D D D D D D N E 063A2.02 The question is required to be enhanced to address the following:

Distractors/Answer includes information not required for answering the question.

Additional information could be used by unknowledgeable applicant to eliminate distractors.

Question contains the term "potential" which allows for the potential of 2 correct answers. The question needs to be focused to ensure there is only 1 correct answer.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Section C.2.d. of NUREG 1021 Appendix B - Include as much information as possible about the problem or situation in the stem. Leaving only the solution, action, or effect for the answer options.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Change to a 2X2 format Remove from equalization X Ventillation requirements Page 50 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 LOD I I Q# LOK (F/H) (1-5) Stem Focus I I I Cues T/F cr~d'l Partial Dist.

J~b-Link Mnutial #1 units Back-ward 0=

KIA I SRO Only BIlvVN U/EtS 87 H 3 D D D D D D D D N E 065AA2.06 The question is required to be enhanced to address the following:

Consideration of PORV not operable after Back-up Nitrogen in lined up is not a credible distractor.

Without written definition of critical valves, it is not clear that the requirement to trip would be met at 1002 and therefore possibly having 2 correct answers. The question needs to be focused to ensure there is only 1 correct answer.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Section C.2.m.5 of NUREG 1021 Appendix B - Specific determiners that give cues as to the correct answer should be avoided, including implausible distractors.

Changed time to 1016 removed "and backup ..... " from question Critical valves listed SAT 88 F 3 D D D D D D D D o D D N s 068AA2.02 Page 51 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I J LOK LOD J

Q#

(F/H) (1-5) Stem Cues T/F Cr~d. Partial J~b- JMnutial #/1 Back- 0= SRO B/M'N U/EIS Focus Dist. Link units ward KIA Only 89 H 1 D D D D D D D D N u 071G2.1.20 The question is unsatisfactory and is required to be changed to address:

Question is direct look up, except for knowledge that monitors are always NON-Operable.

NON- Discriminatory level of knowledge required for SRO applicant.

Section C.1.a. of NUREG 1021 Appendix B - The concept being measured needs to have a direct, important relationship to the ability to perform the job.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Replaced KA 033A2.01 SAT Page 52 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I Q# LOK LOD (F/H) (1-5) Stem Focus Cues T/F Cr.ed*1 Partial Dist.

J~b-Link Mnutial #1 units 8ack-ward 0=

KIA I SRO Only 8/M'N U/EIS 90 H 1 D D D D D D D D D D D N u 076AG2.1.3 The question is unsatisfactory and is required to be changed to address:

Question is direct look up with the references given.

NON- Discriminatory level of knowledge required for SRO applicant.

Question asks for longest power operation, but some answer options also address restoration of DEI.

First part of each answer is unique - second part only decreases discriminatory value /

plausibility Section C.1.a. of NUREG 1021 Appendix B - The concept being measured needs to have a direct, important relationship to the ability to perform the job.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Section C.1.f. of NUREG 1021 Appendix B - Limit question to one concept or topic, unless synthesis of concepts is being tested.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with mUlti-part answers.

Change to a 2X2 format Delete B&D SAT Page 53 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I I Q# LOK LOD (F/H) (1-5) Stem Cues T/F cr.ed*1 Partial Job- Mnutial #1 Back- 0= SRO B/M'N U/EIS Focus Dist. Link units ward KIA Only 91 F 3 D D D D D D D D D D B E 103A2.02 The question is required to be enhanced to address the following:

Distractors/Answer includes information not required for answering the question.

Additional information could be used by unknowledgeable applicant to eliminate distractors.

RO LOK can be used to eliminate all distractors. A & C can be eliminated due to not mentioning closing an airlock door. The "must" in the stem of the question allows the elimination of B due to equipment hatch is not "required" to be bolted.

Section C.1.a. of NUREG 1021 Appendix B - The concept being measured needs to have a direct, important relationship to the ability to perform the job.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Change question statement Changed to 2X2 format FHS location requirement X What hatch closed (equipment or airlock)

SAT Page 54 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Flaw s 4. Job Content Flaw s 5. Other 6 7 I I I I I Q#: LOK LOD (F/H) (1-5) Stem Focus CUes T/F Cr:d'l Partial Dist.

J~b-Link Mnutial #1 units 8ack-ward 0=

KIA I SRO Only 8/M'N utEiS 92 H 2 o o o o o o o o N E we04EG2.1. The question is required to be enhanced to address the following:

Question asks "which procedure will provide mitigating strategies when the leak is isolated"? Is the intention to ask which procedure will the crew go to following ECP-1.2?

The first and second part of each answer option is unique. If the applicant knows a single piece of information, the rest is redundant.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers. .

Change to a 2X2 format Parameter change X Procedure selection SAT 93 H 4 o o o 0 o o D D D D D N E WE10EA2.1 The question is required to be enhanced to address the following:

Distractors/Answer includes information not required for answering the question.

Additional information could be used by unknowledgeable applicant to eliminate distractors. Transitions to UOP in Distractors A&B is unnecessary to answer question.

"When directed to by ESP- 0.2" is redundant in distractors C&D Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Remove UOP 2.2 and ESP 2.2 from stem and options SAT Page 55 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I Q# LOK LaD (F/H) (1,-5) Stem Cues T/FI Cr.ed*1 Partial J~b- Mnutial #1 ~ Back- 0= ~ SAO B/M'N U/6IS Focus Dist. Link units ward KIA Only 94 F 1 D D D D D D D D D N u G2.1.3 The question is unsatisfactory and is required to be changed to address:

Fire brigade leader in stem is not relevant to the question SSS Plant is correct in all the distractors/answer Question can be clarified to ask question more simply Inadequate operational relevance of SS maintaining all responsibility while out of control room.

Section C.1.a. of NUREG 1021 Appendix B - The concept being measured needs to have a direct, important relationship to the ability to perform the job.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Section C.1.e. of NUREG 1021 Appendix B - Avoid questions that are unnecessarily difficult or irrelevant.

Section C.2.d. of NUREG 1021 Appendix B - Include as much information as possible about the problem or situation in the stem. Leaving only the solution, action, or effect for the answer options.

Reworded stem Changed distractors SAT Page 56 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I I QIt LOK LOD (F/H) (1-5) Stem Cues T/F Cr~d'l Partial J~b- Mnutial #1 Back- 0= SRO B/M'N U/EIS Focus Dist. Link units ward KIA Only 95 H 2 D D D D D D D D D D N u G2.2.1 The question is unsatisfactory and is required to be changed to address:

As stated in the distractor analysis, the first part of each question can be answered using system knowledge which is RO knowledge. Changing reactivity by one method at a time is a general precaution of UOP 1.1 P&L 3.4.2, which is RO Knowledge.

Question does not meet requirements for SRO Only knowledge.

Question construction places an unnecessary reading burden on applicants.

Section C.1.a. of NUREG 1021 Appendix B - The concept being measured needs to have a direct, important relationship to the ability to perform the job.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguoUSly, precisely, and as concisely as possible.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers Change to a 2X2 format Change second half to brief requirements SAT 96 F 2 D D D D D D D D D D D N E G2.2.36 The question is suggested to be enhanced to improve question clarity:

Distractor C states - "since it was not time stroked" - possible change to "since is has not been time stroked" to maintain consistency with B & D.

Changed distractor SAT 97 H 3 D D D D D D D D D D D N s G2.3.12 Page 57 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I 1 LOK LOD I

Q#

(F/H) (1-5) Stem Cues T/F cr.ed*1 Partial J~b- Mnutia #1 Back- 0= SRO B/M'N u/es Focus Dist. Link units ward KIA Only 98 F 3 o o o 0 o o o o o o o B s G2.3.6 99 H 3 o o o o o o o o N E G2.4.35 The question is required to be enhanced to address the following:

The first half of all options are unique. Therefore, the second half of each option is not required to correctly answer the question and decreases the discriminatory value of the question by increasing the opportunity to eliminate distractors.

Distractor B is the only one that does not address the procedure for establishing FW to a CCP. This procedure is identified in the stem and used in all the other distractors/answer. A, C, & D have effects of lining up FW and B has a requirement to reestablish cooling.

Potential 2 correct answers - AOP-09 does not appear to prohibit establishing FW to running pump. This would make C also a correct answer.

Section C.2.d. of NUREG 1021 Appendix B - Include as much information as possible about the problem or situation in the stem. Leaving only the solution, action, or effect for the answer options.

Section C.1.f. of NUREG 1021 Appendix B - Limit question to one concept or topic, unless synthesis of concepts is being tested.

Section C.2.f of NUREG 1021 Appendix B - Provide sufficient counterbalance in questions with multi-part answers.

Remove reason in second part of options Shorten answers Reword question.

SAT Page 58 of 59

ES-401, Rev. 9 Written Examination Review Worksheet Form ES-401-9 1 2 3. Psychometric Raw s 4. Job Content Raw s 5. Other 6 7 I I I I I LOD I

Off LOK (F/H) (1-5) Stem Cues T/F cr:d'l Partial J~b- Mnutial #1 Back- 0= SRO B/IINN U/EIS Focus Dist. Link units ward KIA Only 100 F 2 D D D D D D D D D D N E G2.4.38 The question is unsatisfactory and is required to be changed to address:

Who can approve exposure limits in excess of 10CFR Requirements or above site administrative limits is not discriminating knowledge for an SRO applicant.

. Q asks for SM's responsibility in beginning but no reference to SM in any answers.

Stem is unclear as to if the 5500MR is for each person or for the entire team.

Assumption eithr way changes answer to the question.

Section C.1.a. of NUREG 1021 Appendix B - The concept being measured needs to have a direct, important relationship to the ability to perform the job.

Section C.1.c. of NUREG 1021 Appendix B - The question needs to be stated unambiguously, precisely, and as concisely as possible.

Focus stem Change question to ask for responsible approval authority.

SAT Page 59 of 59

ES-403, Rev. 9 Written Examination Grading Form ES-403-1 Quality Checklist Facility: Farley Date of Exam: 11/10/08 Exam Level: RO and SRO Initials a b c Item Description

1. Clean answer sheets copied before grading dill %- ;1f!;

2. Answer key changes and question deletions justified and documented cIl6( Itt- /J;f/:;

3. Applicants' scores checked for addition errors lreviewers spot check> 25% of examinations) al61.

1+ IrI[3

4. Grading for all borderline cases (80 +/-2% overall and 70 or 80, as applicable, +/-4% on the SRO-only) reviewed in detail cHi 11+ tJl$

5. All other failing examinations checked to ensure that grades are justified C4<< 1+- ;11$

6. Performance on missed questions checked for training deficiencies and wording problems; evaluate validity of tft.. 1f1. !}1{7 questions missed by half or more of the applicants Printed Name/Signature Date

a. Grader

~~lrkrJNh /2IfftlY

b. Facility Reviewer(*) #//J--

c. NRC Chief Examiner (*) MfMGK~, "&A/£S/ !72/C: )~ 1:?-/71o'Y

~

J

d. NRC Supervisor (*) J.,JN.at.IL-r:'Ml:>~ / rll£'L J/f.JJ-'r

- 'l/U/6[J I'- )

(*) The facility reviewer's signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.

ML091480156

Text

Navigation menu

Search