ML120480078

Initial Exam 2011-302 Final Administrative Documents

ML120480078
Person / Time
Site:	Robinson
Issue date:	02/10/2012
From:	NRC/RGN-II
To:
Mark Bates
Shared Package
ML120470109	List: ML120470096 ML120470120 ML120470144 ML120480068 ML120480074 ML120480078 ML120480083 ML120480088 ML120480093 ML120480100 ML120480104
References
Download: ML120480078 (52)

Text

ES-201 Examination Preparation Checklist Form ES-201-1 Facility: Date of Examination: 6C ,zoll Developed by: Written - Facility NRC D II Operating - Facility NRC Target Chief Date* Task Description (Reference) Examiners Initials

-180 1. Examination administration date confirmed (C.1.a; C.2.a and b)

-120 2. NRC examiners and facility contact assigned (C.1.d; C.2.e) )7Ø

-120 3. Facility contact briefed on security and other requirements (C.2.c) 3

)fl

-120 4. Corporate notification letter sent (C.2.d)

[-90] [5. Reference material due (C.1.e; C.3.c; Attachment 3)]

{-75) 6. Integrated examination outline(s) due, including Forms ES-201-2, ES-201-3, ES-301-1, ES-301-2, ES-301-5, ES-D-ls, ES-401-1/2, ES-401-3, and ES-401-4, as applicable (C.1.e and f; C.3.d)

{-70} {7. Examination outline(s) reviewed by NRC and feedback provided to facility licensee (C.2.h; C.3.e)}

{-45} 8. Proposed examinations (including written, walk-through JPMs, and scenarios, as applicable), supporting documentation (including Forms ES-301-3, ES-301-4, ES-301-5, ES-301-6, and ES-401-6, and any Form ES-201-3 updates), and reference materials due (C.1.e, f, g and h; C.3.d)

-30 9. Preliminary license applications (NRC Form 398s) due (C.1.l; C.2.g; ES-202)

-14 10. Final license applications due and Form ES-201-4 prepared (C.1.l; C.2.i; ES-202)

-14 11. Examination approved by NRC supervisor for facility licensee review (C.2.h; C.3.f)

-14 12. Examinations reviewed with facility licensee (C.1.j: C.2.f and h; C.3.g)

-7 13. Written examinations and operating tests approved by NRC supervisor (C.2.i; C.3.h)

-7 14. Final applications reviewed; 1 or 2 (if >10) applications audited to confirm qualifications I eligibility; and examination approval and waiver letters sent (C.2.i; Attachment 5; ES-202, C.2.e; ES-204)

-7 15. Proctoring/written exam administration guidelines reviewed with facility licensee (C.3.k)

-7 16. Approved scenarios, job performance measures, and questions distributed to NRC examiners (C.3.i)

Target dates are generally based on facility-prepared examinations and are keyed to the examination date identified in the corporate notification letter. They are for planning purposes and may be adjusted on a case-by case basis in coordination with the facility licensee.

[Applies only] {Does not apply) to examinations prepared by the NRC.

ES-201, Page 25 of 28

ES-201 Examination Outline Quality Checklist Form ES-201-2 Facility: H B Robinson Date of Examination: 11/28/11 Initials Item Task Description

1. a. Verify that the outline(s) fit(s) the appropriate model, in accordance with ES-401.

R b. Assess whether the outline was systematically and randomly prepared in accordance with /1.1 j I Section D.1 of ES-401 and whether all K/A categories are appropriately sampled.

4 T c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics.

ci. Assess whether the justifications for deselected or rejected K/A statements are appropriate.

2. a. Using Form ES-301-5, verify that the proposed scenario sets cover the required number of normal evolutions, instrument and component failures, technical specifications, 71J S and major transients.

M b. Assess whether there are enough scenario sets (and spares) to test the projected number U and mix of applicants in accordance with the expected crew composition and rotation schedule L without compromising exam integrity, and ensure that each applicant can be tested using ,

A at least one new or significantly modified scenario, that no scenarios are duplicated T from the applicants audit test(s), and that scenarios will not be repeated on subsequent days.

c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative 7)/,/

and quantitative criteria specified on Form ES-301-4 and described in Appendix D.

3. a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2:

(1) the outline(s) contain(s) the required number of control room and in-plant tasks W distributed among the safety functions as specified on the form

/ (2) task repetition from the last two NRC examinations is within the limits specified on the form T (3) no tasks are duplicated from the applicants audit test(s) .

(4) the number of new or modified tasks meets or exceeds the minimums specified on the form (5) the number of alternate path, low-power, emergency, and RCA tasks meet the criteria on the form.

b. Verifythatthe administrative outline meets the criteria specified on Form ES-301-1:

(1) the tasks are distributed among the topics as specified on the form 4j (2) at least one task is new or significantly modified (3) no more than one task is repeated from the last two NRC licensing examinations

c. Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days.

4. a. Assess whether plant-specific priorities (including PRA and IPE insights) are covered .7 in the appropriate exam sections.

E b. Assesswhetherthe 10 CFR 55.41/43 and 55.45 sampling is appropriate.

c. Ensure that K/A importance ratings (except for plant-specific priorities) are at least 2.5. 4j P W2 R d. Check for duplication and overlap among exam sections. 2 i \t32 L e. Check the entire exam for balance of coverage. .

11 P

f. Assess whether the exam fits the appropriate job level (RO or SRO).

Ee.

/ eture b FacWty Reviewer (*) J:m:s Conder

c. NRC Chief Examiner (#) Mark Bates / /7 7r

d. NRC Supervisor A4 (%&i 4

gpuJ1ocP3A(LD,,4 Note: # Independent NRC reviewer initial items in Column c; chief examiner concurrence required.

Not applicable for NRC-prepared examination outlines

e S ci PA_L ES-201 Examination Security Agreement Form ES-201-3

1. Pre-Examination iiI2Il1, iLI)

I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of °__2aif the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensees procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2. Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME JOB TITLE / RESPONSIBILITY SIGNATUR DATE SIGNAT. . E(2 A ATE NOTE

(

i. 4m FJA,V? /]-

3. thLif Sj:.J ri.#,1 Sit4trj1-Tc,E.... //

4. 7c ceI- EV AlA.) vC iwc /i/,f/ If

5. 5tr Ki-fAL PAY s,LA-ro 0 t2-/I4L/II

/7

6. PE/-67 . / iL///k/

7. (. A Id c5 c;; j7* / z/ / / /

8.py(,r- //& ,/S /542:c) ei LLL.....

1I. #t 4jtc ba1 -J.

fAIn()tIS!ILI 3j

12. P -

, 4_ft/h L1Z..(tL 1 -

13. . lLJ4 0 /-

.4VL

14. Fc/4 k-tA c-/.1,

15. Sila g2- /s 1/

NOTES:

e e ....L c*

ES-201 Examination Security Agreement Form ES-201 -3

1. Pre-Examination I acknowledge that have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) ofA ) 213s of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensees procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2. Post-Examination To the best of my knowledge I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME JOB TITLE / RESPONSIBILITY SJG ATURE(1) DATE SIGNATURE (2) DATE NOTE

1. !flicilAe,.. /1- lflc 12s.r ci )R,rft 2.Ai R.Pt-L,ev1- R,er

3. cg oc.) gAAs 2l_ iziuf

4. ,41.4,i tz:1.JAJ,cl (fftf l,cM.i k),a+f,

5. ((Q,LC dU,itW_ s.,ct) OPá7cK. 10(1

6. (Q& bA4JbLf 5Lo joN/u 7.7y fjI / ,eL,.W((

7/

8. v-t MAcSJLAJLL+/-

/4///

4,i /i, .&W ()

9. c,5ft- *Jt(fi. (3

10. c. 1< c )t/I1dt C

11. 4\&(R COJ 4ç. rOcDps Yn4a sD)u ILl 72/i 4/li _-.

12. t, 4 1

jZ/ff( (5_)

7

/

l3?4.j t

14. Oci-4r.Q tz. 4I NOTES:ü P:? &,icj IL

ES-201 Examination Security Agreement Form ES-201-3

1. Pre-Examination iz.III:

I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of4M of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensees procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2. Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME JOB TITLE / RESPONSIBILITY NATURE (1) DATE SIGNATURE (2) DATE NOTE

1. -

G øi t) (

2. Vi,t Lti (

1 4 S _/4)(

3. cIT ,44 qJ (2:

I- 1 t C 0s f--- _,fll_/f/

5. 11yjLJ. s4r sfIcPO i7Jç IL I 1,-il,

6. SLJç,Jc/ 4

,c4ød 7, ff

/

_/2/,,/*

8.

.A ILI/4tt/ I .A.__._. ._&..... I,_/,sfi (

10. 4

?f 11.

12.

13.

14.

NOTESQ 1

ES-301 Operating Test Quality Checklist Form ES-301-3 Facility: H B Robinson Date of Examination: 1 1/28/11 Operating Test Number:

Initials

1. General Criteria a b*

a. The operating test conforms with the previously approved outline; changes are consistent with sampling requirements (e.g., 10 CFR 55.45, operational importance, safety function distribution).

b. There is no day-to-day repetition between this and other operating tests to be administered ,,.

.14 during this examination.

c. The operating test shall not duplicate items from the applicants audit test(s). (see Section D.1 .a.)

d. Overlap with the written examination and between different parts of the operating test is within ,.. f/

acceptable limits.

e. It appears that the operating test will differentiate between competent and less-than-competent applicants at the designated license level.

2. Walk-Through Criteria --

a. Each JPM includes the following, as applicable:

• initial conditions

• initiating cues

• references and tools, including associated procedures

• reasonable and validated time limits (average time allowed for completion) and specific designation if deemed to be time-critical by the facility licensee

• operationally important specific performance criteria that include:

detailed expected actions with exact criteria and nomenclature system response and other examiner cues statements describing important observations to be made by the applicant criteria for successful completion of the task identification of critical steps and their associated performance standards restrictions on the sequence of steps, if applicable

b. Ensure that any changes from the previously approved systems and administrative walk-through outlines (Forms ES-3d-i and 2) have not caused the test to deviate from any of the acceptance criteria (e.g., item distribution, bank use, repetition from the last 2 NRC examinations) specified on those forms and Form ES-201-2. 7

3. Simulator Criteria --

The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES-301-4 and a copy is attached. -:1___

Printed Name / Signatur Date

a. Author Jeffrey Smith / )1/Z/1 i

b. Facility Reviewer(*) James Conder / //,/_ 3//i

c. NRC Chief Examiner (#) Mark Bates / /2.214. /,- ii / /21 /

d. NRC Supervisor 1ttG -

NOTE:

• The facility signature is not applicable for NRC-developed tests.

1. Independent NRC reviewer initial items in Column c; chief examiner concurrence required.

imuIator scenario uuaiity i..fleckiist 1-orm S-U1 -4 Facilty: H B Robinson Date of Exam: 11/28/11 Scenario Numbers: 1 /2 / 4 Operating Test No.:

QUALITATIVE ATTRIBUTES Irtials a b*

1. The initial conditions are realistic, in that some equipment and/or instrumentation may be out -

of service, but it does not cue the operators into expected events.

2. The scenarios consist mostly of related events. )jjJ

3. Each event description consists of

• the point in the scenario when it is to be initiated

• the malfunction(s) that are entered to initiate the event

/1

. the symptoms/cues that will be visible to the crew 741

• the expected operator actions (by shift position) (

. the event termination point (if applicable)

4. No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario without a credible preceding incident such as a seismic event.

5. The events are valid with regard to physics and thermodynamics. 47/4 1 -

6. Sequencing and timing of events is reasonable, and allows the examination team to obtain A complete evaluation results commensurate with the scenario objectives.

7. If time compression techniques are used, the scenario summary clearly so indicates.

Operators have sufficient time to carry out expected activities without undue time constraints. ,,j( .

Cues are given.

8. The simulator modeling is not altered. 13

9. The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator performance deficiencies or deviations from the referenced plant have been evaluated ,74ij to ensure that functional fidelity is maintained while running the planned scenarios. /

10. Every operator will be evaluated using at least one new or significantly modified scenario. /1 All other scenarios have been altered in accordance with Section D.5 of ES-301. ..

11. All individual operator competencies can be evaluated, as verified using Form ES-301-6 (submit the form along with the simulator scenarios). I

12. Each applicant will be significantly involved in the minimum number of transients and events specified on Form ES-301-5 (submit the form with the simulator scenarios).

13. The level of difficulty is appropriate to support licensing decisions for each crew position. 1 Target Quantitative Attributes (Per Scenario; See Section D.5.d) .

Actual Attributes -- --

1. Total malfunctions (58) 7 / 8 / 5

/ ,

i)ø

2. Malfunctions after EOP entry (12) 2 / 3/ 3

3. Abnormal events (24) 4 / 5/ 4 a

4. Major transients (12) 1 / 1 / 1

5. EOPs entered/requiring substantive actions (12) 1 / 2 / 1

6. EOP contingencies requiring substantive actions (02) 0/ 1 / 1

7. Critical tasks (23) 3/ 2 / 3 I

imuator cenar;o Uuaity cfleckllst 1-orm S-SU1-4 Facilty: H B Robinson Date of Exam: 11/28/11 Scenario Number: 5 Operating Test No.:

QUALITATIVE ATTRIBUTES Initials a b*

1. The initial conditions are realistic, in that some equipment and/or instrumentation may be out of service, but it does not cue the operators into expected events.

ii Y) 1)9

2. The scenarios consist mostly of related events.

3. Each event description consists of

. the point in the scenario when it is to be initiated

• the malfunction(s) that are entered to initiate the event the symptoms/cues that will be visible to the crew i7j

• the expected operator actions (by shift position)

. the event termination point (if applicable)

4. No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario without a credible preceding incident such as a seismic event. I

5. The events are valid with regard to physics and thermodynamics.

6. Sequencing and timing of events is reasonable, and allows the examination team to obtain /

3 j complete evaluation results commensurate with the scenario objectives. if

7. If time compression techniques are used, the scenario summary clearly so indicates. 4, Operators have sufficient time to carry out expected activities without undue time constraints. 71/1 Cues are given. 1

8. The simulator modeling is not altered.

9. The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator performance deficiencies or deviations from the referenced plant have been evaluated (If!

to ensure that functional fidelity is maintained while running the planned scenarios. .

10. Every operator will be evaluated using at least one new or significantly modified scenario.

All other scenarios have been altered in accordance with Section D.5 of ES-301. ,

7Th--

11. All individual operator competencies can be evaluated, as verified using Form ES-301-6 (submit the form along with the simulator scenarios).

fr

12. Each applicant will be significantly involved in the minimum number of transients and events specified on Form ES-301-5 (submit the form with the simulator scenarios).

13. The level of difficulty is appropriate to support licensing decisions for each crew position.

Target Quantitative Attributes (Per Scenario; See Section D.5.cI) Actual Attributes -- -- --

1. Total malfunctions (58) 9-_ ,

,4

2. Malfunctions after EOP entry (12) 2

3. Abnormal events (24) 5 , /

4. Major transients (12) 1 i4

5. EOP5 entered/requiring substantive actions (12) 1

6. EOP contingencies requiring substantive actions (02) 1

7. Critical tasks (23) 2

/ ,

ES-301 Transient and Event Checklist Form ES-301-5 Facility: H B Robinson Date of Exam: 11/28/11 Operating Test No.:

A E Scenarios P V 1 2 3 4 T M P E o I L N CREW CREW CREW CREW T N I T POSmON POSITION POSITION POSITION A C S A B S A B S A B S A B L u A T R T 0 R T 0 R T 0 R T 0 M*

0 C P 0 C P 0 C P 0 C P N Y R IU T

E RX 11 1 1 0 NOR 2 26 3 111 SRO-l 442 134 134 5 9 I/C 57 MAJ 5 8 6 3 221 TS 134 15 5 022 RX 1 1 110 NOR 2 26 3 111 SRO-l I/C 1 34 1 34 5 9 4 4 2 2 57 MAJ 5 8 6 3 221 TS 134 15 5 022 RX 110 NOR 111 I/C 442 MAJ 2 2 1 TS 022 RX 110 NOR 111 I/C 442 MAJ 221 TS 022

ES-301 Transient and Event Checklist Form ES-301-5 Facility: H B Robinson Date of Exam: 11/28/11 Operating Test No.:

A E Scenarios P V 1 2 3 4 TM P E o I L N CREW CREW CREW CREW T N I T POSmON POSfl1ON POSmON A C S A B S A B SA B S A B L u A T R T 0 R T 0 R T 0 R T 0 M(*)

N 0 C P 0 C P 0 C P 0 C P T P E

RX 2 1 110 NOR 26 2 1 1 1 RO-1 I/C 14 14 4 442 MAJ 5 8 2 221 TS 0 022 RX 2 1 110 NOR 12 2 1 1 1 RO-2 I/C 13 35 237 442 7

MAJ 5 8 6 3 221 TS 0 022 RX 2 1 110 NOR 26 2 1 1 1 1414 4 442 RO-3 MAJ 5 8 2 221 TS 0 022 RX 2 1 110 NOR 12 2 1 1 1 RO-4 I/C 1 3 35 23 7 4 4 2 7

MAJ 5 8 6 3 221 TS 0 022

ES-301 Transient and Event Checklist Form ES-301-5 Facility: H B Robinson Date of Exam: 11/28/11 Operating Test No.:

A E Scenarios P V 1 TM 2 3 4 P E o L N CREW CREW CREW CREW N T

I T IiPiL PJi°L PI!L A C S A B S A B S A B S A B L R T 0 R u

A T T 0 R T 0 R T 0 M(*)

N y 0 C P 0 C P 0 C P 0 C P T P E

RX 2 1 110 NOR 26 2 1 1 1 RO-5 I/C 14 14 4 442 MAJ 5 8 2 221 TS 0 022 RX 2 1 110 NOR 12 2 1 1 1 RO-6 I/C 13 35 5 4 4 2 7

MAJ 5 8 2 221 TS 0 022 RX 110 NOR lii I/C 442 MAJ 2 2 1 TS 022 Instructions:

Check the applicant level and enter the operating test number and Form ES-D-1 event numbers for each event type; TS are not applicable for RO applicants. ROs must serve in both the at-the-controls (ATC) and balance-of-plant (BOP) positions; Instant SROs must serve in both the SRO and the ATC positions, including at least two instrument or component (I/C) malfunctions and one major transient, in the ATC position. If an Instant SRO additionally serves in the BOP position, one I/C malfunction can be credited toward the two I/C malfunctions required for the ATC position.

2. Reactivity manipulations may be conducted under normal or controlled abnormal conditions (refer to Section D.5.d) but must be significant per Section C.2.a of Appendix D. (*) Reactivity and normal evolutions may be replaced with additional instrument or component malfunctions on a 1-for-i basis.

3. Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to the applicants competence count toward the minimum requirements specified for the applicants license level in the right-hand columns.

ES-301 Competencies Checklist Form ES-301-6 Facility: H B Robinson Date of Examination: 1 1/28/1 1 Operating Test No.:

APPLICANTS SRO-I1 SRO-12 Competencies SCENARIO SCENARIO SCENARIO SCENARIO 1234123412341234 Interpret/Diagnose 56 5 Events and Conditions 6 Comply With and 1 2 1 2 1 5 1 2 1 2 1 Use Procedures (1) 34 34 6 34 34 56 5 56 5 56 78 78 Operate Control 1 5 1 5 Boards(2) 66 Communicate 1 2 1 2 1 5 1 2 1 2 1 and Interact 34 34 6 34 34 56 5 56 5 56 78 78 Demonstrate 1 2 1 2 1 2 1 2 Supervisory Ability (3) 34 34 34 34 5 56 5 56 78 78 Comply With and 1 3 1 5 1 3 1 5 Use Tech. Specs. (3) 4 4 Notes:

(1) Includes Technical Specification compliance for an RO.

(2) Optional for an SRO-U.

(3) Only applicable to SROs.

Instructions:

Check the applicants license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.

ES-301 Competencies Checklist Form ES-301-6 Facility: H B Robinson Date of Examination: 11/28/11 Operating Test No.:

APPLICANTS RO-1 RO-2 RO-3 RO-4 Competencies SCENARIO SCENARIO SCENARIO SCENARIO 12 34123412341234 Interpret/Diagnose 1 2 1 2 1 2 23 23 1 2 1 2 1 2 23 23 Events and Conditions 45 46 35 57 46 45 46 35 57 46 8 8 8 8 Comply With and 12 12 12 23 23 12 12 12 23 23 UseProcedures(1) 45 46 35 57 46 45 46 35 57 46 8 8 8 8 Operate Control 1 2 1 2 1 2 23 23 1 2 1 2 1 2 23 23 Boards(2) 45 46 35 57 46 45 46 35 57 46 8 8 8 8 Communicate 1 2 1 2 1 2 23 23 1 2 1 2 1 2 23 23 andlnteract 4546 35 57 46 4546 35 57 46 8 8 8 8 Demonstrate Supervisory Ability (3)

Comply With and Use Tech. Specs. (3)

Notes:

(1) Includes Technical Specification compliance for an RD.

(2) Optional for an S RD-U.

(3) Only applicable to SROs.

Instructions:

Check the applicants license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for even, applicant.

ES-301 Competencies Checklist Form ES-301-6 Facility: H B Robinson Date of Examination: 11/28/11 Operating Test No.:

APPLICANTS RO-5 RO-6 Competencies SCENARIO SCENARIO SCENARIO SCENARIO 1234 1 2341234 1234 Interpret/Diagnose 1 2 1 2 1 2 23 Events and Conditions 45 46 35 57 8 8 Comply With and 1 2 1 2 1 2 23 Use Procedures (1) 45 46 35 57 8 8 Operate Control 1 2 1 2 1 2 23 Boards (2) 45 46 35 57 8 8 Communicate 1 2 1 2 1 2 23 and Interact 45 46 35 57 8 8 Demonstrate Supervisory Ability (3)

Comply With and Use Tech. Specs. (3)

Notes:

(1) Includes Technical Specification compliance for an RO.

(2) Optional for an SRO-U.

(3) Only applicable to SROs.

Instructions:

Check the applicants license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.

ES-401 Record of Rejected KIAs Form ES-401-4 Tier / Randomly Reason for Rejection Group Selected K/A 6/15/1 1 K/A addresses the knowledge of the SRDs T3 G2 4 40 responsibilities in emergency plan implementation. KIA replaced with G2.4.39 on 6/15/1 1.

T2G1 022A4.03 6/20/1 1 K/A addresses controls or indications of Containment Cooling dampers in the Control Room. The emergency dampers are failed in the open position. RNP has minimal containment parameter indication in the control room that would lend itself to identifying a damper malfunction.

K/A replaced with 022A4.05 on 6/20/1 1.

T2G2 071 Ki .05 6/20/1 1 K/A addresses the relationship between the meteorological tower and the waste gas disposal system.

RNP does not have equipment nor procedural relationship between the two. KIA replaced with 071 KI .06 on 6/20/1 1.

T1G2 OO1AG2.1.28 7/14/11 K/A addresses the knowledge of the purpose and SRO function of major system components and controls associated with a continuous rod withdrawal. RNP has physically disconnected the rod withdrawal circuitry such that automatic rod withdrawal will not function. A malfunction would have to occur in the IN-HOLD-OUT switch with the rods in manual for a continuous rod withdrawal to occur. K/A replaced with WEOI 5EGZI .32.

Ti Gi 068AG2.4.2 7/14/11 K/A addresses the knowledge of system set points, SRO interlocks and automatic actions associated with EOP entry conditions. RNP does not have any system set points, interlocks or automatic actions associated with control room evacuation. Additionally, EOP entry conditions are RD level knowledge. KIA replaced with 037AG2.4.41.

T3 G2.3.7 9/29/1 1 K/A addresses the ability to comply with radiation work permit requirements during normal or abnormal S RD conditions. Difficulty was experience in developing an SRO level question for this K/A based on the current number of Rad Protection / Rad Monitor K/As that were randomly selected. K/A replaced with G2.4.30.

T1G1 058AA1.02 10/4/11 K/A addresses the ability to operate and/or monitor static inverter dc input breaker, frequency meter, ac output breaker, and ground fault detector as they apply to a loss of DC power. Difficulty was experienced in constructing a valid question at the appropriate license level. K/A replaced with 058AA1 .03.

T2G2 056A2.04 10/4/1 1 K/A addresses the ability to predict the impact of a loss of condensate pumps on the condensate system and SRO based on the impact to use procedures to correct, control, or mitigate the consequences. Difficulty was experience in constructing a valid question at the SRO license level. KIA replaced with 029AZ01.

Ti/Gi O11EK1.01 10/21/11 K/A requires testing the operational implications of natural circulation and cooling, including reflux boiling as they apply to Large Break LOCA knowledge. Reflux boiling and natural circulation are more related to a SBLOCA. A new K/A was provided. KA replaced with 011EK2.02.

ES-40 I Written Examination Quality Checklist Form ES-401-6 Facility: H B Robinson Date of Exam: 11/28/11 Exam Level: RD X SRO X Initial Item Description a b c#

1. Questions and answers are technically accurate and applicable to the facility.  ?-

2. a.

b.

NRC K/As are referenced for all questions.

Facility learning objectives are referenced as available.

,.f4,

3. SRO questions are appropriate in accordance with Section D.2.d of ES-401

4. The sampling process was random and systematic (If more than 4 RD or 2 SRD questions were repeated from the last 2 NRC licensing exams, consult the NRR CL program office).

5. Question duplication from the license screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate:

the audit exam was systematically and randomly developed; or the audit exam was completed before the license exam was started; or the examinations were developed independently; or f)

X the licensee certifies that there is no duplication; or other (explain) 1

6. Bank use meets limits (no more than 75 percent Bank Modified New from the bank, at least 10 percent new, and the rest new or modified); enter the actual RD / SRO-only 37/ 4 1./ 0 624/ 96 :4 question distribution(s) at right.

(28)! (1) (1) / (0) (46) /(24) t

7. Between 50 and 60 percent of the questions on the RD Memory C/A exam are written at the comprehension! analysis level; the SRO exam may exceed 60 percent ifthe randomly J) 47(/ / 8Y 53 / 92 I selected K/As support the higher cognitive levels; enter the actual RD! SRD question distribution(s) at right. (35) / (2) (40) / (23)

8. References/handouts provided do not give away answers or aid in the elimination of distracters.

1 S

9. Question content conforms with specific K/A statements in the previously approved examination outline and is appropriate for the tier to which they are assigned; deviations are justified. 1

10. Question psychometric quality and format meet the guidelines in ES Appendix B.

11. The exam contains the required number of one-point, multiple choice items; the total is correct and agrees with the value on the cover sheet.

,.4ft

-r

a. Author Jeffrey Smith /

b. Facility Reviewer (*)

c. NRC Chief Examiner (#)

d. NRC Regional Supervisor Note:

• The facility reviewers initials/signature are not applicable for NRC-developed examinations.

1. Independent NRC reviewer initial items in Column c; chief examiner concurrence required.

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W

GENERAL COMMENT

S B= Bank / M=Modified / N=New / F=Fundamental Level (I.E. Memory) / H=Higher Cognitive Level (I.E. C/A)

Test what operators are required to do by procedure - not what they will do.

Generally, extra information that does not serve a purpose, should be deleted from the stem and answer choices.

Note: For Bank questions, at a minimum change the location of the correct answer.

Answer choices should only contain the information that makes it unique. Several questions contained the flaw of having too many items in the answer choices.

For each Bank question, list the bank from which it was taken AND list any previous NRC exams on which it appeared. I just want to ensure that the bank was not limited to recently administered NRC exams and that it was not limited to a one-for-one KA match (all questions meet several KAs). NUREG-1021, ES-401, Page 8 of 33, states, If the bank contains more than one question that fits a specific KA statement, randomly select from among the available questions unless there is an appropriate basis for selecting a specific question (e.g., higher cognitive level, better discrimination validity, more operationally oriented, or site-specific priority).

I think 6 questions may have been repeated from the 2007 NRC exam - This still needs to be verified. If this is the case, some of them may need to be modified. Simply addressing some of the comments throughout the exam may resolve the issue.

RO EXAM 1 007EG2.4.11 B H 1 X U Is it operationally valid to provide a reactor trip in the stem without 2 providing the power level at the time of trip? This seems a bit artificial, S like you are withholding obvious information just to try to make the question work.

Addressed. MAB 29NOV2011 Not tripped does not appear to be plausible. When all turbine stop valves are closed, then the turbine is effectively tripped, regardless of what the basis document states.

Addressed. MAB 29NOV2011 2 008AK2.03 N H 2 X E What other meaningful pressure values can be used other than 1715? Your S chosen value seems so low that it becomes not plausible. It does not make Page 1 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W sense that the PORV would be allowed to remain open until well after your reactor trip setpoint of 1844 psig and at the point of SI. This question will be satisfactory if you can address this plausibility concern with two of the distractors. This question will be considered an E due to the fix being relatively easy, although technically two answer choices are affected. 2185 psig may be a better choice to consider - at least it is prior to a reactor trip and SI.

Addressed. MAB 29NOV2011 3 009EK2.03 B H 2 S No comments.

4 011EK1.01 N F 2 U This question will not be counted toward the number of unsat questions.

S The KA requires testing Large Break LOCA knowledge. I am not sure how natural circulation cooling and reflux boiling impact a LBLOCA. The question was written for a very small LOCA - a break size for which NC cooling / reflux boiling actually makes sense. I think a new KA is required. We can discuss further if necessary, otherwise you will need to write a question to the replacement KA.

Addressed. MAB 29NOV2011 NEW KA: 011EK2.02 (IR=2.6)

Addressed. MAB 29NOV2011 5 015AK2.08 N H 2 S Previously reviewed and approved. No further comments.

6 022AA2.03 B H 1 x x x x U Question is backward logic, which presents some of the following 2 S concerns.

Addressed. MAB 29NOV2011 How do you know what HAS caused the alarm? Could a different failure cause 113B to fail closed? Would it be more appropriate to test what a possible, or viable, single failure could cause the alarms?

Addressed. MAB 29NOV2011 The second part of B and C is not needed - it is just extra information.

Those two choices should just state that the air line has been completely severed. Delete , causing the valve to fail CLOSED.

Addressed. MAB 29NOV2011 How does letdown have any impact on the two stated alarms? These alarms appear to be only applicable to the makeup. D does not appear to be plausible. Discuss with licensee to enhance understanding of plausibility.

Addressed. MAB 29NOV2011 How would the charging pump suction impact the two stated alarms?

Page 2 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W Discuss plausibility of A. Also, since the applicant is forced to assume failures, is there a failure or combination of failures that could make A potentially correct.

Addressed. MAB 29NOV2011 7 025AK3.01 N F 2 E Discussion with previous CE noted.

S This question appears to be (F)undamental LOK, not (H)igher. Memory item for the reasons for doing the steps. Discuss.

Addressed. MAB 29NOV2011 No further comments. Intent of the KA is considered to be met.

8 027AA1.05 N H 2 S No comments.

9 029EG2.4.49 B F 2 x E In the correct answer, why would the applicant push the GV Down and GV S Fast buttons if the turbine manual pushbutton was successful? Are the applicants forced to make an assumption here? NUREG-1021 states that applicants are not to make assumptions, therefore they could conclude the turbine trip was successful.

OK - CE read incorrectly.

The question statement asks for the next (singular) required action, yet the correct answer is a list of several actions. Same comment can be applied to other answer choices where more than one action is provided.

Addressed. MAB 29NOV2011 What immediate actions have already been performed? The stem does not provide this information. This forces the applicant to make assumptions as to where they are in the procedure. This creates multiple correct answers.

Reviewed - OK. MAB 29NOV2011 The question wording and presentation needs to be tightened to ensure one and only one correct answer.

Reviewed - OK. MAB 29NOV2011 10 038EK1.04 B F 2 E The licensee should produce supporting documentation for flow returning S to the core via the hot leg. The provided documentation only states that reflux boiling will occur, but does not define that reflux boiling is entirely limited to returning flow to the core via the hot leg.

Discussions and further documentation satisfied CE - OK. MAB 29NOV2011 11 054AA1.04 N F 2 x E The procedure has the operator start at least one SI pump and open both S PORVs. Will opening one PORV provide adequate cooling, which is what the question is asking? No supporting documentation is provided to Page 3 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W support that two PORVs are required for adequate cooling. This question will be rated as satisfactory if sufficient documentation can show one and only one correct answer.

Supporting documentation reviewed - OK. MAB 29NOV2011 12 055EG2.1.31 B F 2 S Previously reviewed and approved. No further comments.

13 056AK1.03 N H 1 x E Plausibility needs to be enhanced by using 1785 psig without converting to 2 S psia. For example, 61.85 F would be the resultant SCM when 1785 psia is used in conjunction with 558F.

Addressed. MAB 29NOV2011 Also, does your temperature display have a mechanism for notifying an operator that the T/C reading is not reliable and should not be used (I.E.

turns a different color)? If so, I would like this to be used along with failing to convert the 1785 psig to psia.

No. MAB 29NOV2011 In summary, make the answer choices similar to the following convention:

A. Lowest P and Highest Temp B. Lowest P without converting to psia and Highest Temp C. Lowest P and Lowest Temp (or preferably a higher invalid T/C temp)

D. Lowest P without converting to psia and Lowest Temp (or preferably a higher invalid T/C temp)

Addressed. MAB 29NOV2011 The plausibility can be fixed with a small effort, therefore this question is rated as an E.

Addressed. MAB 29NOV2011 14 057AA2.04 N F 2 S No comments.

15 058AA1.03 N H 2 X E Why are so many columns used? Generally good question writing practice S would suggest that only the info needed to make 4 unique answer choices should be used. With that stated, can the answer choices be limited to only the first and last column (SI Actuation and Exciter Field Breaker)? The extra information typically just weakens the plausibility of the distractors by providing additional ways to disqualify those distractors.

Addressed. MAB 29NOV2011 16 065AA2.05 B F 2 x E IA pressure is 83 psig. Therefore, I am having difficulty understanding S how MFR valves would be affected. Typically MFR valves will control at most plants at pressures much lower than 83 psig (maybe even down to 50 or 60 psig).

Addressed. MAB 29NOV2011 Page 4 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W Add to prior to the blank.

Addressed. MAB 29NOV2011 C is not plausible. Everything in the stem points to instrument air as being the problem (Instrument Air Alarms are annunciating). So if IA is the problem, taking manual control does not have much credibility.

Addressed. MAB 29NOV2011 17 W/E 05EK3.1 N H 2 E In a loss of heat sink scenario, how long would it take to get to 25% wide S range SG levels - assuming EOC, long operating run, trip from full power?

After this time has elapsed, what would pressurizer level likely be? Is 12%

a reasonable pressurizer level for these circumstances? Is this situation operationally valid?

Addressed. MAB 29NOV2011 18 W/E 12EK3.3 N H 1 x U Why would locally throttling AFW flow be plausible when nothing in the 2 S stem would cause an applicant to doubt control of AFW from the control room. Some complication needs to be added to the stem to make A(2) and B(2) plausible. There may be several ways to accomplish this - like failing a power supply that has no impact, but would add credibility.

Addressed. MAB 29NOV2011 19 005AK2.02 B L 2 E Add to the question stem, in accordance with AOP-001.

S Addressed. MAB 29NOV2011 20 028AA2.04 B H 2 x U Why are three columns of information provided in the answer choices?

S Generally good question writing practice would suggest that only the info needed to make 4 unique answer choices should be used. Suggest deleting the last column because it does not help to distinguish the answer choices.

Because of this knowledge required by the KA is not needed to answer the question. I can answer the question with or without that third column just by knowing how the two level indicators respond.

Addressed. MAB 29NOV2011 To try to fix the question, I would suggest breaking the reference leg on the LT that is not selected for control. Test how that LI responds with a broken reference leg and then test how the C charging pump speed controller responds. By doing this, the KA is met and the previous comment would also be addressed.

Addressed. MAB 29NOV2011 21 032AA2.09 B H 2 S Suggest deleting the extra words in the answer choices unless they are necessary to remain. I.E. A(2) pulse height discriminator; B(2) detector.

Addressed. MAB 29NOV2011 22 033AG2.2.44 B H 2 S No comments.

Page 5 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W 23 060AK1.02 N F 2 x U Why is all the peripheral information provided to start out the question?

S Delete all of the unnecessary information. Window dressing that has no material impact does not help match the KA. The only thing it does is add more for the applicant to read. In this question you are asking for parts of the definition of a DAC and an ALI. I do not think that only testing knowledge of those two definitions is testing the operational implications part of the KA.

Addressed. MAB 29NOV2011 Suggestion: Any operational implication could be tested. This could include procedural actions based on exceeding an exposure limit, etc. You do not need to test every aspect of the list provided in the KA, but you do need to test the operational implications.

Addressed. MAB 29NOV2011 24 067AA1.06 N H 2 x E C is not plausible. A is a subset of C; therefore, if C was correct, S then A would also be correct. A guessing man would always choose the smaller of answers A and C.

Addressed. MAB 29NOV2011 D is not plausible. B is a subset of D.

Addressed. MAB 29NOV2011 Suggest adding words in parentheses to A and B stating that placing control room in pressurization mode is not required. This will address the subset issue.

Addressed. MAB 29NOV2011 Question was rated as E due to the simplicity of the needed corrections, even though there were two distractors that were not plausible, Addressed. MAB 29NOV2011 25 076AK2.01 N H 2 E No documentation was supplied that supports both R-11 and R-12 having S elevated readings. The question is likely satisfactory, but to ensure the technical accuracy, some documentation needs to be supplied to show that both of these would alarm.

Documentation reviewed. OK MAB 29NOV2011 26 W/E 08EG2.4.2 N F 2 E Currently the question stem asks for what conditions will meet the entry S conditions. This causes the two choices with 320F to not be plausible, because if 320F meets the criteria, then 290F will obviously also meet the criteria. This can be addressed by testing specifically what the procedure (Status Tree) states for P.1 entry criteria. I.E. The CSFST for CSF-4 states that .

Addressed. MAB 29NOV2011 Page 6 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W 27 W/E 10EK3.1 N H 2 S No comments.

28 003A1.07 N H 2 x E Appendix E of NUREG-1021 directs the applicants to not assume S anything. The question directly contradicts the Appendix E direction by stating that they are required to assume. Is there a credible pressure control malfunction that results in a linear pressure decrease?

Addressed. MAB 29NOV2011 Suggestion: Discard the math and just test the values. This may also allow the stem to be simplified. Testing the values for when actions need to occur and then testing what those actions are will meet the KA.

Addressed. MAB 29NOV2011 29 004K5.30 B H 2 X? U Some supporting information was supplied with the question, but no S justification for plausibility was written in the question analysis. The correct answer makes perfect sense to me, but I am struggling to see plausibility in the other answer choices. I placed a question mark on the question rating because no justification was supplied, so maybe there is something I am missing. Plausibility will need to be discussed to see if the question is acceptable.

Addressed. MAB 29NOV2011 Suggestion: Test controller manipulation or controller response. How does the operator adjust letdown pressure if being controlled in manual and how would the operator adjust temperature.

Addressed. MAB 29NOV2011 Suggestion: How does the plant respond to a failure of valves or controllers for the bypass around the RHR heatexchanger? Then how does the letdown controller respond (demand increases or decreases)?

Addressed. MAB 29NOV2011 30 004K6.14 N H 2 S Previously reviewed and approved. No further comments.

31 005K3.01 N H 1 x x U In the stem - how many RCPs have been secured? Would it be better to 2 S state that all RCPs have been secured?

OK after reading. MAB 29NOV2011 Do procedures allow more than one RHR pump to be in operation? The stem states that the operating RHR pump trips. Does this leave open the possibility that an RHR pump is still operating?

Addressed. MAB 29NOV2011 Delete all the extra unnecessary stuff in the answer choices - the only effect it has is to reduce plausibility. Just include the amount of Page 7 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W information needed to make 4 unique answer choices. The answer choices should look something like:

A. RCS pressure will rise; RCS temperature will rise B. RCS pressure will lower; RCS temperature will rise C. RCS pressure will rise; RCS temperature will lower D. RCS pressure will rise; RCS temperature will lower Addressed. MAB 29NOV2011 There may be an overlap issue with Q 29 (004K5.30). Knowledge from Q29 includes knowing that an increase in temperature will result in an increase in pressure during solid operating conditions. This question tests what will happen to RCS pressure when your cooling pump is tripped (I.E.

temp increase). I understand the argument that the pump has tripped and this could impact pressure - but there still may be some overlap / double jeopardy issues.

Addressed. MAB 29NOV2011 No explanation was provided to help me understand why a temperature decrease is plausible. Currently I do not see plausibility in temperature lowering when my cooling pump trips and stops moving coolant.

Therefore, unless convinced otherwise, C and D are not plausible.

Addressed. MAB 29NOV2011 32 006K5.10 B H 2 x E Do procedures ever allow terminating SI when termination criteria are not S met? If so, then I will view C as plausible, if not, then C will not be plausible and will need to be modified or replaced.

OK after discuss ions. MAB 29NOV2011 33 007A1.01 N F 2 E Why is all the stuff at the beginning of the question provided? What S importance or function does it have? Would this be the same question if it started with Which ONE (1) of the following?

Addressed. MAB 29NOV2011 Considering the above comment, is this a (F)undamental LOK vs. a (H)igher LOK? It seems like the applicant just needs to recognize the setpoint and know the basis, which would be a lower cognitive question.

Addressed. MAB 29NOV2011 34 007K3.01 N H 2 x U Analysis of answer choice A: What would PRT pressure do if CL Inj S RV failed open? Your analysis states that the high PRT pressure alarm will not come in - is this true?

Addressed. MAB 29NOV2011 None of the second set of conditions is needed for this question. Delete all of it unless there is a reason that it needs to stay. Remember that adding unnecessary information does not help match a KA, etc. If it is not needed Page 8 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W to answer the question, then it has no impact on the KA match.

Addressed. MAB 29NOV2011 Modification Idea: I would consider deleting the PRT Hi Temp alarm to make the source the CL inj RV and test whether sump level will go up and whether R-2 will rise.

Addressed. MAB 29NOV2011 Are there any differences in indications between a PORV failing open and a Safety Valve failing open?

Addressed. MAB 29NOV2011 The question states, assuming no operator action. Appendix E disallows assumptions. Simply state that no operator actions have taken place, rather than telling the applicants to make an assumption. I would suggest doing a word search on the entire exam and try to remove the word assume wherever possible. There is no harm in stating that no operator actions have occurred, but this is also part of the rules for taking the test as stated in Appendix E.

Addressed. MAB 29NOV2011 35 008G2.4.50 N H 2 x U Test what the operator is required to do - not what he will do. Do a search of the entire exam (I know the word will appears elsewhere).

Change the second part of the question prefix to: APP-002-E5 requires the operator to _______________.

Make similar changes throughout the exam. We must test what operators are required to do because who knows what they will do.

Addressed. MAB 29NOV2011 Distractors A(2) and B(2) are not plausible. From a common sense perspective, a safety related pump has no cooling during a surveillance test (non-emergency), so it only makes sense to protect that pump by stopping it. There is not much plausibility here.

Addressed. MAB 29NOV2011 36 010K1.03 B H 2 S No comments.

37 010K4.01 B F 2 x E Is A arguably correct? Will water continually pass through the spray S nozzle? Could the nozzle be considered full? How is full defined for the spray nozzle? We need to ensure that A is not a correct answer choice.

If needed, it will be replaced. We can discuss if you have the documentation that supports it as being incorrect. I have concerns because the mechanism for reducing thermal stresses is to have a continuous stream of water. Are we then testing the definition of what a full spray nozzle is to differentiate between A and D?

Addressed. MAB 29NOV2011 Page 9 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W 38 012A3.05 N H 1 X U C and D not plausible because they are not mutually exclusive answer 2 S choices. Knowing that there cannot be two correct answers makes these two choice not plausible.: How are C and D different? N-42 is bypassed for both answer choices and power is above P-10 for both answer choices. N-42 being bypassed would play into both answer choices, thereby creating a subset issue. These two answer choices are not mutually exclusive. Suggest making D simply: Reactor will not trip. Then replace C to another credible mechanism that could trip the reactor.

Addressed. MAB 29NOV2011 39 013K2.01 B F 2 S No comments.

40 022A3.01 B H 2 S No comments.

41 022A4.05 N F 2 S No comments.

42 026K1.01 N F 2 U The second part of the answer choices re confusing. It appears that the A S RHR pump will in fact supply suction to allow operation of both spray pumps. It is true that the B spray pump does not have power, but its operation is not disallowed due to the RHR supply. It also strikes me as strange to see operation underlined here. Seeing that underlined may, in fact, validate my comment here in that the RHR supply has no impact on the operation of the Spray Pump. This ambiguity could lend itself to more than one correct answer because the RHR supply does allow operation of both spray pumps.

Addressed. MAB 29NOV2011 This question appears to test two knowledge items: (1) the power supply to the B Spray Pump, and (2) the position of SI-844A&B.

Q Modified. MAB 29NOV2011 43 026K3.02 N H 1 x U RCS water is always acidic. PWRs use boric acid as one of the two 2 S primary means of reactivity control that are available to licensed operators.

So when faced with a situation where chemicals cannot be added due to the Add Tank Outlet failing closed, why would an operator believe that the sump water would be caustic? If Robinson also uses TSP baskets in the sump or something like that, then there may be some plausibility for the water to be caustic.

Addressed. MAB 29NOV2011 I also struggle in seeing plausibility in the gas binding.

Addressed. MAB 29NOV2011 44 039K4.06 N F 2 S No comments.

Page 10 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W 45 059A2.04 M F 2 S Previously reviewed and approved. No further comments.

46 059A3.02 N F 2 S No comments.

47 061K5.01 B H 2 E It looks like this question was used on a recent NRC exam (ILT 11-1).

S What methodology was used to select this question? How did you ensure that you randomly selected among all of your bank questions that met this KA?

OK MAB 29NOV2011 No comments on the question itself.

48 061K6.02 B H 2 S No comments.

49 062A2.10 N H 2 S No comments.

50 063A1.01 N F 2 U The question statement does not address both parts of the question - it only S asks for the time limitations. The question should solicit everything being asked for in the fill in the blank and answer choices.

Addressed. MAB 29NOV2011 C(1) and D(1) are not plausible. You have provided the applicants with two kinds of loads to be shed, both of which would reduce load on the batteries. It is not credible that the applicant would choose to shed an inverter when the other choice is a LOW priority load.

Addressed. MAB 29NOV2011 51 064K6.07 N F 2 E What benefit does it add to test their ability to extrapolate to a point in time S that coincides with a certain value? This is at least the third question that has done this. Does your air compressor raise pressure at a linear rate of 1 psig/min? If not, then this is not an operationally valid question. Also, would you expect an operator to know the rate at which a compressor will charge your EDG air receivers and then calculate the time that will support 8 cold starts? Is this testing meaningful information?

Addressed. MAB 29NOV2011 I see no supporting documentation that supports the air receivers raising pressure at 1 psig/min.

Addressed. MAB 29NOV2011 Suggest just testing the various pressures (100, 210, 216, and 220).

Addressed. MAB 29NOV2011 52 073G2.2.12 N H 2 E Add to the question statement that you are asking the question with the Page 11 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W S CKT TEST pushbutton depressed iaw OST-924-2. I.E. Given the above conditions, which one of the following completes the statement?

Addressed. MAB 29NOV2011 53 076G2.4.18 N H 2 E ONLY does not work in this case. Will PATH-1 have the operators close S ANYTHING else? More precise language needs to be used here. I understand why you tried to use ONLY, because just closing 16A, without also closing 16B, will not isolate SW to the turbine building. By wording it in this fashion, there is also an argument that there is no correct answer because closing 16C is not the only valve that PATH-1 will direct to be closed.

Addressed. MAB 29NOV2011 54 078K2.02 N F 2 S No comments.

55 103A4.09 B F 2 S It looks like this question was used on a recent NRC exam (ILT 11-1).

How was this question selected ES-401: If the bank contains more than one question that fits a specific K/A statement, randomly select from among the available questions unless there is an appropriate basis for selecting a specific question (e.g., higher cognitive level, better discrimination validity, more operationally oriented, or site-specific priority).

OK MAB 29NOV2011 No further comments.

56 001A3.05 N H 2 S No comments.

57 011K2.02 N H 2 S No comments.

58 015K6.01 N H 2 x E D is not plausible. The interplay between the first and second half does S not make sense. If the lower section fails low, then all (or most) power would be indicated in the top. Therefore making the delta high. Even if an applicant has a misconception that the delta was an absolute value, the number would still be high, not low.

Addressed. MAB 29NOV2011 A also has minimal credibility.

Addressed. MAB 29NOV2011 The alarm in question should be referred to by its official designation and name as you have on other questions. What is the official designation for the Section Deviation Alarm?

Addressed. MAB 29NOV2011 59 016K3.09 B H 1 x U A and C are not plausible. We are evaluating high dP between steam 2 S lines. Therefore, if the same failures, whether it be high or low, occur on Page 12 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W each steam line, then how does that impact differences between the steam lines?

Addressed. MAB 29NOV2011 60 028A4.02 N F 2 S No comments.

61 041G2.4.11 N H 2 E D does not appear to be plausible. No justification is supplied for its S plausibility. Would BYPASS TAVG INTERLOCK ever be used at power with higher RCS temperatures?

Addressed. MAB 29NOV2011 62 071K1.06 B F 2 S No comments.

63 072K5.01 N H 2 x U Why is the second part of the answer choices included? Just with part (1)

S alone, you have provided 4 unique answer choices, therefore none of part (2) is needed to arrive at the correct answer. Only information that is needed to make that answer choice unique should be included. Extra information only acts to lower the credibility of the distracters by providing additional ways to disqualify that distractor. Knowledge of the KA must be required to arrive at the correct answer in order to meet the KA. In this case, all I need to know is what is detected and how does the instrument display it. Operational implications needs to be tested to meet the KA.

Addressed. MAB 29NOV2011 64 079K4.01 N H 2 S No comments.

65 086A1.05 N F 2 x x E The supporting documentation suggests that the MDFP may not start until S 95 or 96 psig. The documentation suggests that the MDFP could start anywhere between 95 and 105 psig. Does this mean that it is possible that someone could argue that A is an alternate correct answer?

Addressed. MAB 29NOV2011 C is not plausible. I am having trouble believing that someone would think that an engine would be started before an electrical motor would be used.

Addressed. MAB 29NOV2011 66 G2.1.15 N F 2 x x U OPS-NGGC-1000, Section 9.17.5.1 (f) states that for site-specific standing S instructions may use a database OR Attachment 8 (which could be placed in a book). The question appears to pertain to site-specific standing instructions - therefore, could A be an alternate correct answer? Is there a procedure contradiction here? Does this warrant a procedure change request? I would think that most standing instructions would be site specific; therefore (f) would be the applicable section to follow.

Addressed. MAB 29NOV2011 Page 13 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W B is not plausible. I understand what you have used for plausibility justification, but it still does not seem credible that someone could pull a procedure off the shelf and begin to use it - not knowing that it has been altered by a standing order. It is only logical to believe that the procedure itself must be changed to ensure that a task is performed correctly.

Addressed. MAB 29NOV2011 67 G2.1.17 B F 1 x U C is not plausible. This is a plant announcement - an announcement to 2 S the entire plant. When an announcement is made to the entire plant, would it be credible that everyone be required to respond?

Addressed. MAB 29NOV2011 D also is not plausible. Nothing in the stem even indicates that the TSC and EOF have been activated. Even if the EOF was activated - it is located off site. Why would there be a requirement for the announcement to be in a continuous do-loop until an off-site acknowledgement is made for an announcement?

Addressed. MAB 29NOV2011 For a plant announcement, what mechanism exists for three way communications? Three way communications typically is done only one to one or in small groups.

Addressed. MAB 29NOV2011 I also noted that this question was used on the 2007 NRC exam.

OK. MAB 29NOV2011 68 G2.1.27 B F 2 x x E The wording of this question does not sound right to me. The design S function of the ICCM is a SB LOCA? I think what you are trying to test is that the ICCM is designed to monitor for inadequate core cooling during a SB LOCA. I think some modification to the wording of the question and/or answer choices is necessary.

Addressed. MAB 29NOV2011 C is not plausible. A steam line break will cool the core - cooling will not be inadequate.

Addressed. MAB 29NOV2011 69 G2.2.6 B F 1 x U What is PI-2089 at Robinson?

2 S Modified. MAB 29NOV2011 B is not plausible. The procedure in question is an AOP. Therefore, the plant is in an abnormal condition - potentially on a backshift. Is it credible to think that the procedure owner would be required to approve the temporary procedure change?

Page 14 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W Addressed. MAB 29NOV2011 C is not plausible for similar reasons. How long does the normal procedure change process take? The plant is in an abnormal condition. Is it reasonable that a typo would prevent the crew from taking care of the plant?

Addressed. MAB 29NOV2011 70 G2.2.7 N F 2 x E This type of information is usually tested on the SRO portion of the exam.

S Is this RO required knowledge at Robinson? Is there an RO learning objective for this information? If this question appears in its current form in the final submittal, then it will be considered as RO knowledge at Robinson.

Licensee assures that this is RO knowledge at HBR. OK MAB 29NOV2011 No comments on the material aspects of the question.

71 G2.3.13 B F 1 x U Nuclear Shift Manager does not appear to contain plausibility for a LHRA 2 S entry approval. A normal evolution is occurring and the issue is with high radiation levels. Suggest replacing the approval part. What about equipment verifications in a high dose area - are their alternate methods that can be used to maintain does ALARA?

Addressed. MAB 29NOV2011 Question was on the ILC-09 NRC Exam.

OK MAB 29NOV2011 How was this question selected? Have the licensee walk through in detail, the selection process used for this question.

OK MAB 29NOV2011 NUREG-1021, ES-401, Page 8 of 33, states, If the bank contains more than one question that fits a specific KA statement, randomly select from among the available questions unless there is an appropriate basis for selecting a specific question (e.g., higher cognitive level, better discrimination validity, more operationally oriented, or site-specific priority).

OK MAB 29NOV2011 72 G2.3.4 B F 2 E Why are three columns of information included? Only include information S in the distractors that is needed to make the answer choices unique. In this case you can completely delete the first column (Extremities) and you still have 4 unique answer choices. The only purpose the extra information serves is to reduce the plausibility of distractors by providing more ways to disqualify the distractors.

Page 15 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W Addressed. MAB 29NOV2011 73 G2.4.25 B F 2 S Previously reviewed and approved.

74 G2.4.39 B F 2 S No comments.

2007 NRC Exam 75 G2.4.9 N H 2 E How would the operators know that there were no indications that the leak S was in the RHR system? No sump/rad alarms in aux bld would not tell the entire story? I just want to ensure that the conditions provided are operationally valid to place them in the right procedure location to support the correct answer.

Addressed. MAB 29NOV2011 SRO EXAM 76 038EG2.4.11 N H 2 S No comments.

77 054AG2.4.11 N H 2 X? X? E Why is 2385 MWth plausible? What documents refer to 2385 MWth? No S documentation was provided to justify its plausibility. (I am assuming 102% of rated?) There may be two acceptable options: (1)To enhance plausibility, consider making UFMs out of service and iterate on 2346 MWth and 2300 MWth. OR (2) provide supporting documentation for the plausibility of 2385 MWth.

Addressed. MAB 29NOV2011 Why is OP-105 an incorrect choice? I see no explanation or documentation with the question that supports OP-105 being incorrect.

Consider iterating on (1) AOP-10 contains steps to reduce power. AND (2)

AOP-10 does not contain steps to reduce power.

Addressed. MAB 29NOV2011 The last sentence in the analysis of answer choice B does not appear to be correct. It states that 2385 MWth is correct - yet the correct answer has 2346 MWth as correct.

Addressed. MAB 29NOV2011 Knowledge of max licensed power limits is RO knowledge - after all the ROs are operating the plant.

Addressed. MAB 29NOV2011 78 055EA2.01 B H 2 x x U KA/SRO-only: The KA is not met at the SRO level. EPP-1 is a major S EOP as defined in the SRO guidance. RO knowledge can be used to know that EPP-1 does not apply. Knowing failure modes for a valve is RO Page 16 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W knowledge.

Addressed. MAB 29NOV2011 Could e-plan be used to hit the KA at the SRO level?

Addressed. MAB 29NOV2011 How does the question map through the SRO guidance document?

Addressed. MAB 29NOV2011 79 058AA2.03 N H 2 x U SRO-only: If the plant conditions do not result in an SI, then using RO S knowledge, the applicant would conclude that EPP-7 is not a valid answer choice. Then system response knowledge is the only thing needed to analyze the second part of the answer choices also.

Addressed. MAB 29NOV2011 How does the question map through the SRO guidance document?

Addressed. MAB 29NOV2011 80 W/E 04EG2.4.3 N H 2 S Previously worked with the licensee to develop a satisfactory question.

81 W/E 05EA2.1 N H 2 x U SRO-only: Red path procedure entry is RO knowledge. Mitigating S strategy is RO knowledge. No SRO level procedure selection is tested in the question. Part of the H.1 strategy is to limit heat input to the RCS -

I.E. stop RCPs. Testing details of a procedure is not SRO knowledge.

Details of a procedure could only be SRO only knowledge if that information is needed to make a procedure selection. In this case, stopping an RCP is not a procedure selection - it is a single action that also happens to be part of the overall mitigating strategy.

Addressed. MAB 29NOV2011 How did this question map through the SRO guidance document?

Addressed. MAB 29NOV2011 82 W/E15EG2.1.32 N H 2 E Prior to the STA reporting to the control room, who is responsible for S monitoring safety function status trees? Where are these responsibilities defined in administrative procedures? At most plants, monitoring of safety functions is RO required knowledge. Does Robinson have an SRO-learning objective for knowing the hierarchy of safety functions?

Addressed. MAB 29NOV2011 I see no supporting documentation to justify the second part of the answer choices. Before this question can be rated as satisfactory, I will need to see documentation that justifies plausibility and incorrectness of distractors as well as the correctness of the answer. The basis for the prioritization potentially could be used to justify SRO-only because the guidance document is silent on EOP basis information.

Page 17 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W Addressed. MAB 29NOV2011 83 037 AG2.4.41 N H 1 x E Do Robinson procedures actually allow a SG to be isolated prior to 2 S shutting down? If a SG is isolated at 100% power, how would the plant respond? My guess is that isolating the C SG at 100% power would result in a trip. Therefore, the first part of C and D are not plausible unless the licensee can provide sufficient justification for plausibility.

Addressed. MAB 29NOV2011 Have the licensee show the leakrate calc that results in greater than 77 gpm leakage.

Addressed. MAB 29NOV2011 84 076AA2.05 N H 2 S No comments.

85 W/E 09EA2.1 B H 2  ?  ?  ? The display of information is a little confusing. The second to last bullet S states that it is 15 minutes after the trip. Is the last bullet also 15 minutes after the trip? The second bullet lists conditions after the trip, yet the third bullet list conditions at the time of trip. To clarify, consider providing sets of conditions at various times - like INITIAL CONDITIONS, then CURRENT CONDITIONS.

Addressed. MAB 29NOV2011 In this case, I believe that procedure selection is being tested in that direction for controlling temperature is found in both procedures. The applicant must know more than just one action and the applicant must know more than the mitigation strategy. Also, more than just entry conditions for the AOP and major EOPs are needed.

OK MAB 29NOV2011 The KA must be met at the SRO level. The KA requires selection of procedures for natural circ. How is the procedure selection in the form of temperature control associated with natural circ. It seems like the procedure selection may just be associated with a loss of air. Discuss with licensee to enhance my understanding.

OK MAB 29NOV2011 86 003A2.03 N H 2 S Previously worked with the licensee to develop a satisfactory question.

87 007A2.05 N H 2 S Previously worked with the licensee to develop a satisfactory question.

88 022G2.4.50 N H 2  ? x  ? U The question tests what should be done. Why would it be wrong to stop S HVH-3 when it has high vibes? Would it be possible for me to find an admin procedure that would allow an operator to secure a piece of equipment to protect it against further damage? The concern here is having one and only one correct answer. Why would C really be wrong?

Page 18 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W Suggestion: Reword the first part of the question: To reset the High Vibration Alarm, APP-002-A7 directs: stopping HVH-3 prior to resetting the vibration alarm vs. resetting the high vibration alarm without stopping HVH-3.

Addressed. MAB 29NOV2011 Does a motor calculation error make the B CV Pump inoperable? Does it depend on the specifics of the calculational error? How does your procedures define OOS? Can a piece of equipment simply be off and be considered OOS?

Addressed. MAB 29NOV2011 Does the High Vibe alarm make HVH-3 inoperable? The question is set up to leave HVH-3 running, therefore, would it be wrong to assume operable until an operability recommendation could be produced by engineering? If it is clearly inoperable, then why would an operator try to restart them as stated in two of the distractors?

Addressed. MAB 29NOV2011 89 059A2.03 N H 2 E Why is the second part of the distractors incorrect? I do not see any S documentation that supports Supplement G being incorrect. Discuss with licensee to ensure that there is only one correct answer.

Addressed. MAB 29NOV2011 H.3 is a yellow path. The question does not include a reference for the applicant. Is yellow path info required closed book knowledge at Robinson?

Addressed. MAB 29NOV2011 90 064G2.4.46 N H 2 S Previously worked with the licensee to develop a satisfactory question.

91 011G2.2.38 N H 2 S No comments.

92 045A2.11 N H 2 S No comments.

93 029A2.01 N H 2 E Add in accordance with Tech Specs to the end of the first question S statement.

Addressed. MAB 29NOV2011 Add in accordance with ODCM to the end of the first question statement.

Addressed. MAB 29NOV2011 94 G2.1.31 N H 2 S Previously worked with the licensee to develop a satisfactory question.

95 G2.1.41 N F 1 x X? U RO knowledge can be used to disqualify AOP-013. AOP entry conditions Page 19 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W 2 S are RO required knowledge. ROs also likely move fuel in the spent fuel pool at various times, where AOP-013 would still be applicable.

Addressed. MAB 29NOV2011 ROs are licensed to move fuel. Union contracts, etc., may allow contractors to do it, but those contracts are not permanent restrictions.

Does Robinson require fuel movers (whether they be ROs or contract help such as Westinghouse) to know where to place an assembly when level is lowering? I would want to see the learning objectives for fuel movers to ensure that this is not required knowledge. If fuel movers are not required to know this, then it may be OK to allow fuel placement requirements to be SRO-only knowledge.

Addressed. MAB 29NOV2011 Placing an assembly in its original location is always correct - it was OK for the assembly to be there prior to lifting it, so it is fairly basic to understand that it would also be OK to put it back in the same spot. This causes B(1) and D(1) to be not plausible.

Addressed. MAB 29NOV2011 96 G2.2.35 N H 2 E Suggest changing A(2) and B(2) to Allowed. I do not think S conditionalizing the answer choices adds plausibility.

Addressed. MAB 29NOV2011 97 G2.2.42 N H 2 x x U A reference is required to be provided to the applicant if testing greater S than one hour Tech Spec actions. I think the reference will cause the Mode 3 entry piece to not be plausible.

Addressed. MAB 29NOV2011 Also, the KA does not appear to be met at the SRO level. The Tech Spec entry is the part that meets the KA, but that is RO knowledge.

Addressed. MAB 29NOV2011 Suggestion: write a question where the PORV is on its backup nitrogen supply. I would expect the RO to know that the PORV will work, but it would be the SRO that would be required to know whether the PORV was operable. This is just one idea - there may be others.

Addressed in another way. MAB 29NOV2011 98 G2.3.4 B H 2 x x U The KA is not met at the SRO level. Radiation exposure limits are being S tested at the basic rad worker (or RO) level. To hit the KA at the appropriate level, I would suggest testing emergency dose approval authorization for saving life or equipment. This is a common SRO question for this KA.

Addressed. MAB 29NOV2011 Page 20 of 21

ES-401, Rev. 9 Written Exam Review Worksheet Form ES-401-9 Robinson 2011-302 Q# K/A# B L L Psychometric Flaws Content Flaws U Comment M O O Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred K/A Only S Dist Dist B/W The E-Plan classification is not related to the KA.

Addressed. MAB 29NOV2011 99 G2.4.30 N H 2 E Does Robinson have a learning objective that supports asking this question S in a closed book format? I am asking this to ensure that Robinson Training and Operations Management all agree that this is closed book knowledge at your facility. This question appearing in its current form in your Final Submittal will be viewed as the licensee officially agreeing that this is closed book knowledge for SROs at Robinson and therefore acceptable to appear in that manner on their NRC exam.

Addressed. MAB 29NOV2011 100 G2.4.43 N F 2 x E This question tests what the HOO will do. Maybe the licensee should not S test what the HOO might do? The HOO is not required to follow your OMM procedures. Would it not be possible that the HOO could ask if you were ready for the code, or wanted the code, prior to providing it? Maybe they would do that to ensure you were ready to receive the information.

Would that be wrong?

Suggestion: Test where the code is stored or whether or not the code is safeguards information, or something of that nature. The second part of the answer choices are OK. Just the first part needs some work. There appears to be enough info in the OMM procedure to test something that will ensure only one correct answer to a further extent.

Addressed. MAB 29NOV2011 Page 21 of 21

ES-403 Written Examination Grading Form ES-403-1 Quality Checklist L)

[Facility: Date of Exam: Exam Level: RO SROL1 Initials Item Description a b c

1. Clean answer sheets copied before grading 1 N/, inQ

2. Answer key changes and question deletions justified and documented -p i

3. Applicants scores checked for addition errors (reviewers_spot_check>_25%_of examinations)

4. Grading for all borderline cases (80 +/-2% overall and 70 or 80, NJ as applicable, +/-4% on the SRO-only) reviewed in detail I

5. All other failing examinations checked to ensure that grades are justified iti 1 I

6. Performance on missed questions checked for training deficiencies and wording problems; evaluate validity 1 N

of questions missed by half or more of the applicants Printed Name/Signature Date

a. Grader 1) 3

b. Facility Reviewer(*) t/V\ N/

c. NRC Chief Examiner(*) I/o3/oi

d. NRC Supervisor (*) uj.o Cguei2 3 /2

(*) The facility reviewers signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.

/

ES-403, Page 6 of 6

ES-301 Control Roomlln-Plant Systems Outline Form ES-301-2 Facility: H B Robinson Date of Examination: 1 1/28/1 1 Exam Level: RO X SRO-I X SRO-U Operating Test No.:

Control Room Systems@ (8 for RO); (7 for SRO-l); (2 or 3 for SRO-U, including 1 ESF)

System / JPM Title Type Code* Safety Function

a. Withdrawing Control Rod Shutdown Bank B A, L, M, S 1

b. Align SI System for Cold Leg Recirculation D, EN, L, S 2

c. PZR Pressure Control Malfunction A, D, S 3

d. Startup, Parallel, and Load the Main Generator A, M, S 4S

e. Respond to RHR Leakage with the Unit on RHR A, EN, L, Cooling M, S f.CV Isolation Phase B and CV Spray Allgnment A,M,S 5 (RO ONLY)

g. Remove Source Range Instrument From Service D, L, 5 7

h. Respond to a Loss of CCW to the RCP Motor Coolers D, 5 8 In-Plant Systems@ (3 for RO); (3 for SRO-l); (3 or 2 for SRO-U)

i. Align Deepwell Pump D to Supply Cooling Water to CCWHXs D E R 4S

j. Startup of Dedicated Shutdown UPS Inverter lAW OP-N 6 602

k. Respond to Control Room Inaccessibility D, E, L, R 2

@ All RO and SRO-I control room (and in-plant) systems must be different and serve different safety functions; all 5 SRO-U systems must serve different safety functions; in-plant systems and functions may overlap those tested in the control room.

Type Codes Criteria for RO I SRO-l I SRO-U (A)lternate path 4-6 I 4-6 I 2-3 (C)ontrol room (D)irectfrombank (E)mergency or abnormal in-plant 1 I 1 / 1 (EN)gineered safety feature - / - / l (control room system)

(L)ow-PowerlShutdown 1 I1 I 1 (N)ew or (M)odified from bank including 1 (A) 2/ 2/ 1 (P)revious 2 exams 3 I 3 / 2 (randomly selected)

(R)CA 1I1I1 (S)imulator JPM A: Withdrawing Control Rod Shutdown Bank B K/A 003 AK3.04 Knowledge of the reasons for the following responses as they apply to the Dropped Control Rod: Actions contained in EOP for dropped control rod.

(CFR: 41.5/41.10/45.6/45.13)

(Control Rod Drive System / 001) The candidate will be directed to withdraw Shutdown Bank B rods to support the upcoming reactor startup.

Once the control rods reach 70 steps withdrawn, Group 2 of Shutdown Bank B (4 control rods) will drop into the core. The candidate will be expected to enter AOP-001, Malfunction of Reactor Control System, and take the actions for dropped rods while the plant is in Mode 3. This will require that the remaining shutdown bank rods be driven into the core.

(CR-044 Bank JPM modified to drop the 4 control rods during withdrawal).

JPM B: Align SI System for Cold Leg Recirculation K/A 006 A4.05 Ability to manually operate and/or monitor in the control room: Transfer of ECCS flowpaths prior to recirculation.

(CFR: 41.7 / 45.5 to 45.8)

(Emergency Core Cooling System (ECCS) / 006) Candidate will transfer to cold leg recirculation lAW EPP-9, Transfer to Cold Leg Recirculation.

(CR-007 Bank JPM)

JPM C: PZR Pressure Control Malfunction K/A 010 A2.02 Ability to (a) predict the impacts of the following malfunctions or operations on the PZR PCS; and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those malfunctions or operations: Spray valve failures.

(CFR: 41.5/43.5/45.3 / 45.13)

(Pressurizer Pressure Control System (PZR PCS) /010) Plant is operating in Mode 1 with the candidate directed to respond to plant conditions. The Auxiliary Spray valve will fail open, causing PZR pressure to lower. The candidate will be expected to take the immediate actions of AOP-01 9, Malfunction of RCS Pressure Control, and enter the procedure to analyze and respond to the lowering pressure. Once the failure is recognized, actions will be taken to isolate letdown and charging flow to isolate the auxiliary spray flow into the PZR. (CR-099 Bank JPM)

JPM D: Startup, Parallel, and Load the Main Generator k/A 045 A4.02 Ability to manually operate and/or monitor in the control room: T/G controls, including breakers. (CFR: 41.7 / 45.5 to 45.8)

(Main Turbine Generator (MT/G) System / 045) Plant is in Mode I with the turbine at 1800 RPM and ready for the Voltage Regulator to be placed in service and the unit synchronized to the electrical grid. The candidate will be directed to continue with GP-005, Power Operation. Once the unit OCB is closed to parallel the unit to the electrical grid, the minimum load on the generator will not be picked up and the candidate will have to take actions to pick up the necessary load to prevent a generator motoring lockout from occurring on a 1 minute timer. (CR-046 Bank JPM modified with the turbine minimum load pickup defeated).

JPM E: Respond to RHR Leakage with the unit on RHR Cooling K/A 025 AA2.02 Ability to determine and interpret the following as they apply to the Loss of Residual Heat Removal System: Leakage of reactor coolant from RHR into closed cooling water system or into reactor building atmosphere. (CFR: 43.5 /45.13)

(Loss of Residual Heat Removal System (RHRS) / 005) Plant is currently in Mode 5 with RHR supplying core cooling. When the RHR pumps are swapped, the RHR discharge relief valve lifts and fails to reseat. This results in a loss of RCS inventory and requires entry into AOP-020, Loss of Residual Heat Removal (Shutdown Cooling). The actions of AOP-020 will require that the RHR Pumps and Reactor Coolant Pumps be secured, along with the isolation of the RHR system. (CR-030 Bank JPM modified to change the leak location from an RCS pipe break to the RHR relief valve).

JPM F: CV Isolation Phase B and CV Spray Alignment (RO ONLY)

K/A 026 A4.0l Ability to manually operate and/or monitor in the control room: CSS controls.

(CFR: 41.7 / 45.5 to 45.8)

(Containment Spray System (CSS) / 026) Candidate will be directed to perform Supplement B, Phase B and CV Spray Component Alignment, following a Large Break LOCA during the implementation of PATH-i.

Several of the CV Spray and Phase B valves will fail to actuate and the candidate will have to take manual actions to align the valves.

(CR-003 Bank JPM modified with the valve failures).

JPM G: Remove Source Range Instrument from Service K/A 015 A4.03 Ability to manually operate and/or monitor in the control room: Trip bypasses. (CFR: 41.7 / 45.5 to 45.8)

(Nuclear Instrumentation System (NIS) /015) The plant is in Mode 3 when a failure occurs on Source Range Channel N-31. The candidate will be directed to remove the failed channel from service using OWP-01 1, Nl-5.

This will remove the channel from scan on the ERFIS computer, bypass the channel trip signal and align the audio count rate channel to the operable Source Range channel. (CR-062 Bank JPM)

JPM H: Respond to a Loss of CCW to the RCP Motor Coolers K/A 008 K3.03 Knowledge of the effect that a loss or malfunction of the COWS will have on the following: RCP. (CFR: 41.7)

(Component Cooling Water System (CCWS) / 008) Candidate will respond to a loss of CCW flow to the Containment when supply valve CC-716B inadvertently closes. Attempts to re-open the valve will be unsuccessful and will lead to entry into AOP-014, Component Cooling Water System Malfunction, and result in manually tripping the reactor and securing the Reactor Coolant Pumps. (2008 NRC Exam JPM)

JPM I: Align Deepwell Pump D to Supply Cooling Water to COW HXs K/A 076 A2.01 Ability to (a) predict the impacts of the following malfunctions or operations on the SWS; and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those malfunctions or operations: Loss of SWS.

(CFR: 41.5 I 43.5 / 45.3 /45.13)

(Service Water System (SWS) / 076) Candidate will simulate establishing Deepwell Pump D flow to the CCW HXs lAW EPP-28, Loss of Ultimate Heat Sink. This will require that manual valve alignments be performed in EDG A Room and the CCW HX Room. The deep well flow to EDG A will be throttled to allow cooling to the CCW HXs. The CCW HXs discharge will be throttled to ensure that Deepwell Pump D does not reach runout conditions and to support heat removal from other plant components. (lP-164 Bank JPM)

JPM J: Startup of Dedicated Shutdown UPS Inverter lAW OP-602 K/A 062 G2. 1.20 Ability to interpret and execute procedure steps: AC Electrical Distribution System (AC Electrical Distribution System / 062) Candidate will simulate placing the Dedicated Shutdown UPS Inverter back in service following maintenance activities. (New JPM written for 2011-2 NRC Exam)

JPM K: Respond to Control Room Inaccessibility K/A 068 AA1 .06 Ability to operate and/or monitor the following as they apply to the Control Room Evacuation: Charging pump.

(CFR: 41.7 / 45.5 /45.6)

(Chemical and Volume Control System I 004) Candidate will simulate performing the breaker manipulations and local controls for inventory control lAW AOP-004, Control Room Inaccessibility, Attachment 1.

(lP-063 Bank JPM)

ES-301 Administrative Topics Outline Form ES-301-1 Facility: H B Robinson Date of Examination: 11/28/11 Examination Level: RO X SRO Operating Test Number:

Administrative Topic I Type Describe activity to be peormed (see Note) Code*

M, R Perform the RCS Leakage Surveillance Conduct of Operations Procedure D, R Calculate the boron addition required prior to Conduct of Operations initiating a natural circulation cooldown to CSD Perform Section 8.2.3 of OST-020, Shiftly M, R Surveillances Equipment Control N, R Calculate the maximum permissible stay time Radiation Control with emergency dose limits Emergency N/A Procedures/Plan NOTE: All items (5 total) are required for SROs. RO applicants require only 4 items unless they are retaking only the administrative topics, when all 5 are required.

Type Codes & Criteria: (C)ontrol room, (S)imulator, or Class(R)oom (D)irect from bank ( 3 for ROs; 4 for SROs & RO retakes)

(N)ew or (M)odified from bank ( 1)

(P)revious 2 exams ( 1; randomly selected)

2011-2 NRC RO Admin JPM Summary 2011-2 NRC JPM Admin RO Al-I Perform the RCS Leakage Surveillance procedure.

G2.1 .7 Ability to evaluate plant performance and make operational judgments based on operating characteristics, reactor behavior, and instrument interpretation.

(CFR: 41.5 / 43.5 /45.12 /45.13) RO 4.4, SRO 4.7 The candidate will be directed to complete an RCS leakage evaluation lAW OST-051, Reactor Coolant System Leakage Evaluation. The evaluation will include both unidentified and identified leakage. The necessary calculations will be performed to demonstrate that the plant leakage is within the ITS 3.4.13.b limit of unidentified leakage of less than or equal to 1 Gallon Per Minute (GPM).

2011-2 NRC JPM Admin RO AI-2 Calculate the boron addition required prior to initiating a natural circulation cooldown to CSD

, G2.1 .25 Ability to interpret reference materials, such as graphs, curves, tables, etc.

(CFR: 41.10 / 43.5/45.12) RO 3.9, SRO 4.2 The candidate will be expected to calculate the boron addition needed for the plant to be placed in the cold shutdown condition while in natural circulation. This boration includes calculating the change in boric acid storage tank level.

2011-2 NRC JPM Admin RO A2 Perform Section 8.2.3 of OST-020, Shiftly Surveillances G2.2.37 Ability to determine operability and/or availability of safety related equipment.

(CFR: 41.7 /43.5 /45.12) RO 3.6, SRO 4.6 The candidate will be directed to complete OST-020, Shiftly Surveillances, Section 8.2.3. Several instruments will be out of tolerance for the parameters measured. The candidate will be expected to identify the out of tolerance instruments and make the appropriate log entries to identify the failures.

2011-2 NRC JPM Admin RO A3 Calculate the maximum permissible stay time with emergency dose limits G2.3.4 Knowledge of radiation exposure limits under normal or emergency conditions.

(CFR:41.12/43.4145.1O) R03.2, SRO3.7 The candidate will be given specific tasks to be performed inside the Containment Vessel during a declared emergency event. He will be expected to calculate the dose to be received and apply the proper emergency dose limits to the allowed dose.

ROBINSON 2011-302 FINAL SAMPLE PLAN The final sample plan is the combination of the Form ES-401-4 and the Draft Sample Plan.

ES-301 Administrative Topics Outline Form ES-3d-I Facility: H B Robinson Date of Examination: 11/28/11 Examination Level: RO SRO X Operating Test Number:

Administrative Topic Type Describe activity to be performed (see Note) Code*

Conduct of Operations M, R Heat Stress Work Limits Conduct of Operations N, R Complete Equipment Inoperable Record Perform Section 8.2.3 of OST-020, Shiftly Equipment Control M, R Surveillances Calculate emergency dose exposure time Radiation Control M, R limits Emergency M, R Classify an Emergency Event Procedures/Plan NOTE: All items (5 total) are required for SROs. RO applicants require only 4 items unless they are retaking only the administrative topics, when all 5 are required.

Type Codes & Criteria: (C)ontrol room, (S)imulator, or Class(R)oom (D)irect from bank ( 3 for ROs; 4 for SROs & RO retakes)

(N)ew or (M)odified from bank ( 1)

(P)revious 2 exams ( 1; randomly selected)

2011-2 NRC SRO Admin JPM Summary 2011-2 NRC JPM Admin SRO Al-I Heat Stress Work Limits G2.1.26 Knowledge of industrial safety procedures (such as rotating equipment, electrical, high temperature, high pressure, caustic, chlorine, oxygen and hydrogen).

(CFR: 41.10 /45.12) RD 3.4, SRO 3.6 The candidate will be expected to evaluate the heat stress work limits lAW AP-020. This determination will include the type of work to be performed, stay time and recovery time period.

2011-2 NRC JPM Admin SRO Al-2 Complete Equipment Inoperable Record G2. 1.18 Ability to make accurate, clear, and concise logs, records, status boards, and reports. (CFR: 41.10/45.12/45.13) RD 3.6, SRO 3.8 The candidate will be expected to complete the OMM-007, Equipment Inoperable Record, for Component Cooling Water Pump B being inoperable. He will complete the necessary attachments, determine the allowed time to Modes 3 and 5, and determine whether a safety function determination is required for the equipment failure.

2011-2 NRC JPM Admin SRO A2 Perform Section 8.2.3 of OST-020, Shiftly Surveillances G2.2.37 Ability to determine operability and/or availability of safety related equipment.

(CFR: 41.7 /43.5 /45.12) RD 3.6, SRO 4.6 The candidate will be directed to complete OST-020, Shiftly Surveillances, Section 8.2.3. Several instruments will be out of tolerance for the parameters measured. The candidate will be expected to identify the out of tolerance instruments and make the appropriate log entries to identify the failures. Dnce the out of tolerance instruments are identified, the candidate will be required to identify the applicable Technical Specification action statements for the affected instruments.

2011-2 NRC JPM Admin SRO A3 Calculate emergency dose exposure time limits.

G2.3.4 Knowledge of radiation exposure limits under normal or emergency conditions.

(CFR: 41.12/43.4/45.10) RO 3.2, SRO 3.7 The candidate will be given specific tasks to be performed and will be expected to apply the appropriate emergency exposure limits to the specified jobs.

2011-2 NRC JPM Admin SRO A4 Classify an Emergency Event.

G2.4.41 Knowledge of the emergency action level thresholds and classifications.

(CFR: 41.10 /43.5 /45.11) RO 2.9, SRO 4.6 The candidate will be given the necessary plant conditions to classify that an emergency event has occurred. This classification is required to be determined within 15 minutes of the onset of the event. Once the classification is communicated to the examiner, the candidate will be expected to fill out the Emergency Notification Form for communication to the state and counties within 15 minutes. Both portions of this JPM are time critical with a 15 minute completion criteria on each section.