ML16007A473

301 Final Administrative Documents

ML16007A473
Person / Time
Site:	Surry
Issue date:	01/07/2016
From:	Division of Reactor Safety II
To:	Virginia Electric & Power Co (VEPCO)
References
Download: ML16007A473 (80)

Text

ES-201 Examination Preparation Checklist Form ES-201-1 Facility: Surry 2015-301 Date of Examination: 09/21/2015 and 10/06/2015 Developed by: Written: Facility NRC // Operating Facility NRC Chief Target Task Description (Reference) Examiners Date*

Initials

-180 1. Examination administration date confirmed (C.1.a; C.2.a and b) JAT 01/06/2015

-150 2. NRC examiners and facility contact assigned (C.1.d; C.2.e) JAT 01/06/2015

-150 3. Facility contact briefed on security and other requirements (C.2.c) JAT 03/30/2015

-150 4. Corporate notification letter sent (C.2.d) JAT 04/02/2015

[-120] 5. Reference material due (C.1.e; C.3.c; Attachment 3) JAT 05/08/2015

{-90} 6. Integrated examination outline(s) due, including Forms ES-201-2, ES-201-3, ES-301-1, ES-301-2, ES-301-5, ES-D-1, ES-401-1/2, ES-401N-1/2, ES-401-3, ES-401N-3, JAT 05/22/2015 ES-401-4, and ES-401N-4, as applicable (C.1.e and f; C.3.d)

{-85} 7. Examination outline(s) reviewed by NRC and feedback provided to facility licensee JAT 06/01/2015 (C.2.h; C.3.e)

{-60} 8. Proposed examinations (including written, walk-through JPMs, and scenarios, as applicable), supporting documentation (including Forms ES-301-3, ES-301-4, ES-301-5, ES-301-6, and ES-401-6, ES-401N-6, and any Form ES-201-2, ES-201-3, JAT 06/26/2015 ES-301-1, or ES-301-2 updates), and reference materials due (C.1.e, f, g and h; C.3.d)

-45 9. Written exam and operating test reviews completed. (C.3.f) JAT 07/17/2015

-30 10. Preliminary license applications (NRC Form 398's) due (C.1.l; C.2.g; ES-202) JAT 08/21/2015

-21 11. Examination approved by NRC supervisor for facility licensee review (C.2.h; C.3.f) JAT08/21/2015

-21 12. Examinations reviewed with facility licensee (C.1.j; C.2.f and h; C.3.g) JAT 08/24/2015

-14 13. Final license applications due and Form ES-201-4 prepared (C.1.l; C.2.i; ES-202) JAT 09/07/2015

-14 14. Written examinations and operating tests approved by NRC supervisor (C.2.i; JAT 09/16/2015 C.3.h)

-7 15. Facility licensee management queried regarding the licensees views on the JAT 09/16/2015 examination. (C.2 j)

-7 16. Final applications reviewed; 1 or 2 (if >10) applications audited to confirm qualifications / eligibility; and examination approval and waiver letters sent (C.2.i; JAT 08/24/2015 Attachment 5; ES-202, C.2.e; ES-204)

-7 17. Proctoring/written exam administration guidelines reviewed with facility licensee JAT 08/27/2015 (C.3.k)

-7 18. Approved scenarios, job performance measures, and questions distributed to NRC JAT09/16/2015 examiners (C.3.i)

• Target dates are generally based on facility-prepared examinations and are keyed to the examination date identified in the corporate notification letter. They are for planning purposes and may be adjusted on a case-by-case basis in coordination with the facility licensee.

[Applies only] {Does not apply} to examinations prepared by the NRC.

ES-201 Examination Outline Quality Checklist Form ES-201-.2 Facility: Surry Power Station Date of Examination: 9/21/2015

. Initials Item Taak Deacripton a b c#

I. a. Verify that the outline(s) fit(s) the appropriate model! in accordance with ES-401 or ES-401 N.

F b. Assess whether the outline was systematically and randomly prepared in accordance with

. Section D.1 of ES401 or ES-401N and whether all K/A categories are appropriately sampled. 1 t

c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics. U?

N

d. Assess whether the justifications for deaelected or rejected K/A statements are appropriate.

2 a. Using Form ES-301-5, verify that the proposed scenario sets cover the required number of normal evolutions, instrument and component failures, technical specifications, and major S transients.

M b. Assess whether there are enough scenario sets (and spares) to test the projected number and U mix of applicants in accordance with the expected crew composition and rotation schedule L without compromising exam integrity, and ensure that each applicant can be tested using at A least one new or significantly modified scenario, that no scenarios are duplicated from the T appticants audit teat(s), and that scenarios will not be repeated on subsequent days.

c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative and R

quantitative criteria specified on Form ES-301-4 and described in Appendix D. 4

3. a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2:

(1) the outline(s) contain(s) the required number of control room and in-plant tasks distributed W among the safety functions as specified on the form A (2) task repetition from the lest two NRC examinations is within the limits spocitied on the form L (3) no tasks are duplicated from the applicants audit test(s)

K (4) the number of new or modified tasks meets or exceeds the minimums specified on the form (5) the number of altemate path, low-power, emergency, end RCA teaks meet the criteria on T the form.

H R b. Verify that the administrative outline meets the criteria specified on Form ES-301 -1:

o (1) the tasks are diatributed among the topics as specified on the form u (2) at least one task is new or significantly modified 7t Cc?

C (3) no more than one task is repeated from the last two NRC licensing examinations H c. Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days. It

4. a. Assess whether plant-specific priorities (including PRA and IPE inaighta) are covered in the appropriate exam sections. /

b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate. t?f

c. Ensure that K/A importance ratings (except for plant-specific priorities) are at least 2.5. (2?

R d. Check for duplication and overiap among exam sections. 2 9-2 cY?l L e. Check the entire exam for balance of coverage. rZ!i

f. Assess whether the exam fita the appropriate job level (RO or SRO). 7 ft9 f4

s. Author

b. Facility Reviewer (j)

c. NRC Chief Examiner (#)

d. NRC Supervisor Note: 4 Independent NRC reviewer initial items in Column Pc; chief examiner concurrence required.

Not appliceble for NRC-prepared examination outlines.

ES-301 Administrative Topics Outline Form ES-301-1 Facility: Surry Date of Examination: 9/21/15 Examination Level: RO SRO Operating Test Number: SR 15-301 Administrative Topic (see Note) Type Describe activity to be performed Code*

Complete Reactivity Summary Sheet Conduct of Operations K/A: G2.1.37 Knowledge of procedures, guidelines, or R/D limitations associated with reactivity management.

RO: 4.3/SRO: 4.7 Calculate Primary to Secondary leakage and determine ramp rate in accordance with 0-OSP-Conduct of Operations R/N RC-002 K/A: G2.1.23 Ability to perform specific system and integrated plant procedure during all modes of plant operation. RO: 4.3/SRO: 4.4 Equipment Control Calculate stay time Radiation Control K/A: G2.3.4 Knowledge of radiation exposure limits under R/M normal or emergency conditions. RO: 3.2/SRO: 3.7 Complete EPIP-2.01 for Alert, and Transmit Emergency Plan K/A: G2.4.39 Knowledge of RO responsibilities in 5/D emergency plan implementation. RO: 3.9/SRO: 3.8 NOTE: All items (five total) are required for SROs. RO applicants require only four items unless they are retaking only the administrative topics (which would require all five items).

Type Codes & Criteria: (C)ontrol room, (S)imulator, or Class(R)oom (D)irect from bank ( 3 for ROs; 4 for SROs & RO retakes)

(N)ew or (M)odified from bank ( 1)

(P)revious 2 exams ( 1; randomly selected)

ES-301 Administrative Topics Outline Form ES-301-1 Facility: Surry Date of Examination: 9/21/15 Examination Level: RO SRO Operating Test Number: SR 1 5-301 Administrative Topic (see Note) Type Describe activity to be performed Code*

Complete Reactivity Summary Sheet K/A: G2.1.37 Knowledge of procedures, guidelines, or Conduct of Operations R/D limitations associated with reactivity management.

RO: 4.3/SRO: 4.7 Calculate Primary to Secondary leakage and determine ramp rate in accordance with 0-OSP-Conduct of Operations R/M RC-002 K/A: G2.1.23 Ability to perform specific system and integrated plant procedure during all modes of plant operation. RO: 4.3/SRO: 4.4 Review 1 -OPT-C H-002 K/A: G2.2.37 Ability to determine operability and/or Equipment Control R/D availability of safety related equipment.

Calculate stay time Radiation Control KJA: G2.3.4 Knowledge of radiation exposure limits under R/M normal or emergency conditions. RO: 3.2/SRO: 3.7 Classify EAL and Approve Notification Emergency Plan R/M K/A: 2.4.41 Knowledge of the emergency action level thresholds and classifications. RO: 2.9 / SRO: 4.6 NOTE: All items (five total) are required for SROs. RO applicants require only four items unless they are retaking only the administrative topics (which would require all five items).

Type Codes & Criteria: (C)ontrol room, (S)imulator, or Class(R)oom (D)irect from bank ( 3 for ROs; 4 for SROs & RO retakes)

(N)ew or (M)odified from bank ( 1)

(P)revious 2 exams ( 1; randomly selected)

ES-301 Control Roomlln-Plant Systems Outline Form ES-301-2 Facility: Surrv Date of Examination: 9/21/15 Exam Level: RO SRO-l SRO-U Operating Test No.: SR 15-301 Control Room Systems:8 for RO; 7 for SRO-I; 2 or 3 for SRO-U System / JPM Title Safety Type Code Function

a. Perform AP-3.00 to Emergency Borate the RCS During ES-0.1 (Faulted) [ 024AA1.17 (3.9/3.9)] N/S/A/L

b. Makeup to RWST from Blender (pre-brief) 1SYS004A2.13 (3.6/3.9)] N/S 2

c. Respond to spurious Containment Spray Actuation. [EPEE02EA1.3 D/S/EN 3 (3.8/4.0)]

d. Initiate RCS Feed and Bleed [WE05EA1.1 (4.1/4.0)] M/S/A/L 4P

e. Respond to Secondary Transient [SYSO16A2.01, (3.0/3.1)] M/S/A 7

f. Manually start Cont. Spray pumps per Attachment 1 (WE14EA1.1 M/S/EN/L 5 (3.7/3.7)]

g. Waste Gas Tank Release (Faulted) [APE06OAA2.05 (3.7/4.2)] D/S 6

h. Align the Emergency Bus in accordance with AP-10.07 9 Attachment 4 [APEO56AA2.49 (3.0 / 3.4)] N/S/A In-Plant System& (3 for RO); (3 for SRO-I); (3 or 2 for SRO-U)

i. Manually makeup to SFP [5YS033A2.03 (3.1/3.5)] D/E/R 8

j. Locally trip the Unit 2 Reactor Trip Breakers [EPEO29.EA1.12 (4.1/4.0] N/E/L 1

k. Transfer Semi-vital bus power supply [APEO56AA2.44 94.3/4.4))) D/E 6 AM RO and SRO-l control room (and in-plant) systems must be different and serve different safety functions; aM five SRO-U systems must serve different safety functions; in-plant systems and functions may overlap those tested in the control room.

Type Codes

} Criteria for RO I Actual A)lternate path 4-6 I 4 (C)ontrol room (D)irectfrom bank 9 I 4 (E)mergency or abnormal in-plant 1 I 3 (EN)gineered safety feature 1 I 2 (control room system)

(L)ow-Power I Shutdown 1 I 4 (N)ew or (M)odifled from bank including I (A) 2 / 7 (P)revious 2 exams 3 I 0 (randomly selected)

(R)CA 1 / 1 (S)imulator

ES-301 Control Roomlln-Plant Systems Outline Form ES-301-2 Facility: Surry Date of Examination: 9/21/15 Exam Level: RO SRO-l SRO-U Operating Test No.: SR 15-301 Control Room Systems:*8 for RO; 7 for SRO-l; 2 or 3 for SRO-L3 System I JPM Title Type Code* Safety Function

a. Perform AP-3.00 to Emergency Borate the RCS During ES-0.1 (Faulted) [KA: 024AA1.17 39/39] N/S/A/L

b. Makeup to RWST from Blender (pre-brief) ISYSOO4A2.13 (3.6/3.9)] N/S 2

c. Respond to spurious Containment Spray Actuation. [EPEEO2EA1.3 D/S/EN 3 (3.8/4.0)]

d. Initiate RCS Feed and Bleed [WE05EA1.1 4.1/4.0] M/S/A/L 4

e. Respond to Secondary Transient [SYS016A2.O1, (3.0/3.1)1 M/S/A 7

f. Manually start Cont. Spray pumps per Attachment 1 [WE14EA1.1 M/S/EN/L 5 (3.7/3.7)]

g. Waste Gas Tank Release (Faulted) [APEO6OAA2.05 (3.7/4.2)] N/S/A 9 h.

In-Plant Systems (3 for RO); (3 for SRO-l); (3 or 2 for SRO-U)

i. Manually makeup to SFP SYS033A2.03 (3.1 / 3.5)] D/E/R 8

j. Locally trip the Unit 2 Reactor Trip Breakers [EPEO29.EA1.12 (4.1/4.0] N/E/L 1

k. Transfer Semi-vital bus power supply [APEO56AA2.44 94.3/4.4))) D/E 6 All RO and SRO-l control room (and in-plant) systems must be different and serve different safety functions; all five SRO-U systems must serve different safety functions; in-plant systems and functions may overlap those tested in the control room.

A)lternate path Type Codes

] Criteria for SRO-l / Actual 4-6 I 4 (C)ontrol room (D)irect from bank 8 I 3 (E)mergency or abnormal in-plant 1 I 3 (EN)gineered safety feature I I 2 (control room system)

(L)ow-Power I Shutdown 1 I 4 (N)ew or (M)odified from bank including 1 (A) 2 I 7 (P)revious 2 exams 3 / 0 (randomly selected)

(R)CA l I 1 (S)imulator

ES-301 Control Roomlln-Plant Systems Outline Form ES-301-2 Facility: Surry Date of Examination: 9/21/15 Exam Level: RO SRO-l SRO-U Operating Test No.: SR 15-301 Control Room Systems:*8 for RO; 7 for SRO-l; 2 or 3 for SRO-L System / JPM Title Type Code* Safety Function

a. Perform AP-3.00 to Emergency Borate the RCS During ES-0.1 N/s/A/L (Faulted) (pre-brief) [KA: 024AA1.17 3.9/3.9]

b. Makeup to RWST from Blender (pre-brief) [SYS004A2.13 (3.6/3.9)] D/S/A/L 2

c. Respond to spurious Containment Spray Actuation. [EPEEO2EA1.3 D/S/EN 3 (3.8/4.0)]

d.

e.

f.

g.

h.

In-Plant Systems* (3 for RO); (3 for SRO-l); (3 or 2 for SRO-U)

i. Manually makeup to SFP [SYSO33A2.03 (3.1 / 3.5)] D/E/R 8

j. Locally trip the Unit 2 Reactor Trip Breakers [EPEO29.EA1.12 (4.1/4.0] N/E/L 1 k.

All RO and SRO-l control room (and in-plant) systems must be different and serve different safety functions; all five SRO-U systems must serve different safety functions; in-plant systems and functions may overlap those tested in the control room.

A)lternate path Type Codes

] Criteria for SRO-U / Actual 2-3 / 2 (C)ontrol room (D)irect from bank 4 / 3 (E)mergency or abnormal in-plant 1 / 2 (EN)gineered safety feature 1 / 1 (control room system)

(L)ow-Power / Shutdown 1 I 3 (N)ew or (M)odified from bank including 1 (A) 1 I 2 (P)revious 2 exams 2 / 0 (randomly selected)

(R)CA 1 / 1 (S)imulator

ES-301 Operating Test Quality Checklist Form ES-301-3 Facility: Surry Date of Examination: 9/21/2015 Operating Test Numben SR3O1-2015

, Initials

1. General Criteria a b* 4

a. The operating test conforms with the previously approved outline; changes are consistent with sampling requirements (e.g., 10 CFR 55.45, operational importance, safety function distribution).

b. There Is no day-to-day repetition between this and other operating tests to be adminIstered during this examination.

c. The operating test shall not duplicate items from the applicants audit test(s). (see Section D. I a.) 1

d. Overlap with the written examination and between different parts of the operating test is within -

acceptable limits.

e. It appears that the operating test will differentiate between competent and less-than-competent y.. .411 applicants at the desIgnated license level. =

2. Walk-Through Criteria

a. Each .JPM includes the following, as applicable:

initial conditions

• initiating cues

• references and tools, including associated procedures

• reasonable and valIdated time limits (average time allowed for completion) and specific designation if deemed to be time-critical by the facility licensee operationally important specific performance criteria that include:

detailed expected actions with exact criteria and nomenclature system response and other examiner cues statements describing important observations to be made by the applicant criteria for successful completion of the task identification of critical steps and their associated performance standards restrictions on the sequence of steps, if applicable

b. Ensure that any changes from the previously approved systems and administrative walk-through ouflines (Forms ES-301-1 and 2) have not caused the test to deviate from any of the acceptance v criteria (eg. item distribution bank use, repetition from the last 2 NRC examinations) specified on those forms and Form ES-201-2.

3. SImulator Criteria The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES- ,.- Y 4

301-4 and a copy is attached.

Printed Name / Signature Date

a. Author Paul Orrison / / . -

b. Facility Reviewer(*) MIchael R. Meyer!

c. NRC Chief Examiner (#) Amanda Toth / j )

d. NRC Supervisor C

(

NOTE: The facility signature is not applicable for NRC-developed tests.

1. Independent NRC reviewer initial items in Column c; chief examiner concurrence required.

ES-301 Simulator Scenario Quality Checklist Form ES-301-4 Facility: Surry Date of Exam: 9/21/15 Scenario Numbers: 1/2/3/4 Operating Test No.SR3O1-2015 QUALITATIVE ATTRIBUTES Initials a b* 4 1, The initial conditions are realistic, in that some equipment and/or instrumentation may be out of service, but it does not cue the operators into expected events.

2. The scenarios consist mostly of related events. I-.

3. Each event description consists of

. the point in the scenario when it is to be initiated

. the malfunction(s) or conditions that are entered to initiate the event

. the symptoms/cues that will be visible to the crew

. the expected operator actions (by shift position)

. the event termination point (if applicable)

4. The events are valid with regard to physics and thermodynamics. t.

5. Sequencing and timing of events is reasonable, and allows the examination team to obtain complete evaluation results commensurate with the scenario objectives. I

6. If time compression techniques are used, the scenario summary clearly so indicates.

Operators have sufficient time to carry out expected activities without undue time constraints. /

Cues are given.

7. The simulator modeling is not altered.

8. The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator performance deficiencies or deviations from the referenced plant have been evaluated to ensure that functional .V- I fidelity is maintained while running the planned scenarios.

9. Every operator will be evaluated using at least one new or significantly modified scenario. All other ._ -vi scenarios have been altered in accordance with Section D.5 of ES-301.

10, All individual operator competencies can be evaluated, as verified using Form ES-301-6 (submit the ,

form along with the simulator scenarios).

11. The scenario set provides the opportunity for each applicant to be evaluated in each of the applicable rating factors. (Competency Rating factors as described on forms ES-303-1 and ES-303-3.)

12. Each applicant will be significantly involved in the minimum number of transients and events specified pS.

4V7 p on Form ES-301 -5 (submit the form with the simulator scenarios).

13. The level of difficulty is appropriate to support licensing decisions for each crew position.

Target Quantitative Attributes {Per Scenario; See Section D.S.d) Actual Attributes -- --

1. Malfunctions after EOP entry (1-2) 2/1/1/2 1

2. Abnormal events (2-4) 5/4/4/3 j

3. Major transients (12) 2/11111 tA.. W7

4. EOPs entered/requiring substantive actions (12) 2/3/2/2

5. EOP contingencies requiring substantive actions (0-2) 1/1/1/1 *N1

6. EOP based Critical tasks (23) 4/3/3/3 CL NOTE:

• The facility signature is not applicable for NRC-developed tests.

1. Independent NRC reviewer initial items in Column c; chief examiner concurrence required.

ES-301 Transient and Event Checklist Form ES-301-5 Facility: Surry Date of Exam: Operating Test No.: SR 15-301 A E Scenarios 1 2 3 4 T M 0 I L N CREW CREW POSITION CREW POSITION CREW POSITION I T POSfl1ON T C A S A B S A B S A B S A B M R T 0 R T 0 R T 0 R T o L U 0 C P 0 C P 0 C P 0 C P M(*)

T P SRO-I I/C 2 3 2 2 2,3,442 L__i 2,2 SRO-U MAJ 1 2 1 1 1,2, 2 2 1 D TS 0 0 0 0 0022 RO RX 0 0 0 0 0110 NOR 1 2 22 SRO-I I/C 333i3,442 D 3,1 MAJ 1 2 1 1 1,2, 2 2 1 SRO-U 1,1 D TS 0 0 0 0 0022 RO RX 1 1 1 1 1,1, 1 1 0 D

NOR 0 1 1 1 0,1, 1 1 1 SRO-I 1,1

• I/C 5 6 5 4 5,6,442 5,4 SRO-U MAJ 12 11t2,221 D

1,1, TS 1 3 3 2 1,3, 0 2 2 3,2 RO RX 1 1 1 1 1,1, 1 1 0 D

NOR 0 1 1 1 0,1, 1 1 1 SRO-I 1,1 SROU D I/C 5 6 5 ti::zzzzz 4 5,6,442

ELEZI

ES-301 Competencies Checklist Form ES-301-6 Facility: Surry Date of Examination: 9/21/15 Operating Test No.: SR3O1-2015 APPLICANTS RO RO El RO El SRO-I El SRO-l SRO-I El SRO-U El SRO-U El SRO-U Competencies SCENARIO 1 2 34 1 2 34 1 2 3 4 Interpret/Diagnose 1,2,3, 13,4, 2,3, 23, 1,2,3, 1,3,4, 2,3, 2,3, 1,2,3, 1,3,4, 2,3,4 2,3,

. 4,6,7, 5,6 4,5, 4,5, 4,6,7, 5,6 4,5, 4,5, 4,6,7, 5,6 5,6, 4,5, Events and Conditions 8 6,7 8 6,7 8 7 Comply With and ALL ALL ALL ALL ALL ALL ALL ALL ALL ALL ALL ALL Use Procedures (1)

Operate Control ALL ALL ALL ALL Boards (2)

Communicate ALL ALL ALL ALL ALL ALL ALL ALL ALL ALL ALL ALL and Interact Demonstrate ALL ALL ALL ALL ALL ALL ALL ALL Supervisory Ability (3)

Comply With and 2 1,3 2,3, 3,4 2 1,3 2,3,4 3,4 Use Tech. Specs. (3)

Notes:

(1) Includes Technical Specification compliance for an RO.

(2) Optional for an SRO-U.

(3) Only applicable to SROs.

Instructions:

Check the applicants license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant. (This includes all rating factors for each competency.) (Competency Rating factors as described on forms ES-303-1 and ES-303-3.)

ES-401 Record of Rejected KIAs Form ES-401-4 Tier / Randomly Reason for Rejection Grour Selected K/A 2/1 RO 059A4.0l Surry Power station does not have this type of equipment.

Replace vi1h 05QA4.08.

2/2 RO 027A4.04 Surly Power station does not have indication of this parameter.

Replace v ith 027A4.0 I 2/2 RO 029G2.4.3 No tie or link between Containment purge and post-accident instrumentation.

Replace with 029G2. 1 .25.

1/1 SRO 065AG2.4.3 No tie or link between Loss of instrument air and post-accident instrumentation.

Replace with 065AG2.4.9.

2/I SRO 012G2.4.1 K/A does not lend itself to an SRO question Rerlace with 01 2G2. 1 .23.

ES-401 Written Examination Quality Checklist Form ES-401-6 Facility: Surry Power Station Date of Exam: 9/21/2015 Exam Level: RO SRO Initial Item Description a b c*#

1. Questions and answers are technically accurate and applicable to the facility. ..

2. a. NRC K/As are referenced for all questions.

b. Facility learning objectives are referenced as available.

3. SRO questions are approprIate In accordance with Section D.2.d of ES-401 .._ A) 4 4 The sampling process was random and systematic (If more than 4 RD or2 SRO questions were repealed from the last two NRC licensing exams, consult the NRRJNRO DL. program attica). NOTE 1

5. Question duplication from the licensee screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate The audit exam was systematically arid randomly developed; or the audit exam was completed before the license exam was started: or X the examinations were developed independently; or the licensee certifies that there Is no duplication; or other (explain)

6. Bank use meets limits (no more than 75 percent from the Bank Modified New bank, at least 10 percent new, and the rest new or modified), enter the actual RD / SRO-onty question 11)8 21/28 68)64 distribution(s) at right

7. Between 50 and 60 percent of the questions on the RD Memory C/A exam are written at the comprehension] analysis level: the SRQ exam may exceed 80 percent if the randomly selected K/As support the higher cognitive levels: enter 42/20 58/80 the actaal RO / SRO question distribution(s) at right.

8. ReferencesThandouts provided do not give away answers or aid in the elimination of yL.

distractors.

9. Question content conforms to specific K/A statements In the previously approved examination outtine and_Is_appropriate_for_the tier to which_they_are_assigned:_deviations_are justified.

10. Question psychometric quality arid format meet the guidelines in ES Appendix 8.

11. The exam contains the required number of one-point, multiple choice items: the total is correct and agrees with the value on lhe cover sheet.

Printed Name) Signature Date

a. Aathor Paul Orrison/ (of/(c__ 9)3/15

b. Facility Reviewer (*) Michael R Meyer) 5_

c. NRC Chief Examtner(#) Amanda Toth/ . 9 II

d. NRC RegionalSupervisor (x ttt*.

(

Note: The facility reviewers initials or signature are not applicable for NRC-developed examinations.

1. Independent NRC reviewer initials items in Column c; chief examiner concurrence required, Note 1: Sample Plan provided by Chief Examiner Amanda Toth

ES-401 Surry Early Review - Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

• The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

• The answer choices are a collection of unrelated true/false statements.

• The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

• One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

• The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

• The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) B/M/N U/E/S Explanation Key/Summary S Indicates SAT: 15 SRO, 33 RO (initial submittal) 25 SRO, 75 RO SAT in final submittal Question distribution:

RO SRO E Indicates Enhancement: 1 SRO, 27 RO (initial submittal)

Bank 13 initial 3 initial U Indicates UNSAT: 5 SRO, 12 RO (initial submittal)

Modified 16 initial 16 initial U/E Indicates UNSAT/ENHANCEMENT (likely due to two non-plausible New 46 initial 6 initial distractors that are the same, as in a 2x2, i.e., the same distractor counted 2x as non-plausible) 3 SRO, 3 RO (initial submittal) (1 SRO KA replacement in initial submittal)

(pre) Indicates presubmitted question comments

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 2 X N E 001AA1.02 Question appears to meet the KA.

Are there any competing factors during the power change that could change the answer? (i.e., rods moved a different number of steps because of normal rod motion) What is the Tave/Tref difference before the failure occurs? Tave/Tref mismatch is 0 before failure. Added Tave for 100%

power in IC. No other competing factors. You should give them a starting point for the difference prior to the failure so that there is no basis for two correct answers or no correct answers. Divided Initial and Current conditions to clarify starting point.

Recommend adding the word immediately to the first question to ask, the direction of the rod motion immediately before the rods were placed in manual to tighten up the answer possibilities. Added immediately to first question.

For the second question - need to make sure the question is clear enough.

Original in this case means, immediately prior to the Tave failure.

Would it be better to just ask that? Changed ORIGINAL to previous and added (prior to Tave failure).

Im a little bit confused on the math - should the equation be: [8.0 steps/min +(32 steps/min-F)(1.0F)]*[0.5min] ? When I first read the explanation, it looked like you would end up with 16+8 or 16. The explanation originally included basis for correct answer and distractor.

Deleted distractor basis from explanation and added to Distractor comments.

JAT 06/26/2015 S SR made the corrections annotated in red, above.

Updated question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 2 H 2 N E 001K6.03 Question appears to meet the KA.

Can the question be answered without giving the information but only the B RPS train actuates in the stem? I.e., if A trip breaker is undergoing testing, would only the B; train be expected to actuate? If so, then the statement that only B actuates is not necessary, and the applicant should be able to determine whether the picture accurately reflects B RPS train actuating (which it does not). Changed 3rd bullet to An automatir reactor trip signal occurs.

The second half of the question almost seems to be a tack-on, and very easy. It would make sense that as a back up to an undervoltage coil, that something would energize. Recommend an alternative question (i.e.,

possibly what actions an operator would be required to take first with respect to the breaker under test). Changed second question as denoted.

JAT 06/26/2015 SR made the corrections annotated in red, above.

S The second question needs a comma added: under test, the Updated question appears to be SAT.

JAT 08/04/2015 3 H 2 X N U 003K4.04 (pre)

Question does not appear to match the KA. Question queries on the impact of a loss of IA on CC valves to the RCP. The CC valves are not specifically a design feature of the pump.

Question appears to be at the RO level.

JAT 2/4/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only E

(pre2) Newly submitted question appears to match the KA, in that the candidate must have knowledge of the RCP precaution on limiting the number of starts of the RCP in a given time frame due to motor cooling concerns.

Addtionally, the applicant must have knowledge of the precaution on which cooling flow to reestablish first, which is based on the design of the RCP (restoring the wrong subsystem first has the potential to warp the pump shaft due to the injection point). Comments have been addressed.

Question appears to be at the RO level.

To remove all ambiguity from the question, I would recommend stating that three starts have been attempted and query on whether an additional start is allowed. Even better would be to give a timeline to show that the starts had been attempted IAW with the precaution statement. For example: At time 1000 RCP A was attempted to be started and did not start. At time 1045, RCP A was again attempted to be started and did not start. At time 1115, RCP A was again attempted to be started and did not start.

At time 1200 [or 1220, depending on what you want the right answer to be], a start attempt on RCP A _[may/may not]__ be made.

[OR what is the earliest time RCP A may be attempted to be restarted].

The second question is fine, but it seems somewhat unrelated to the first part of the question. If you set up the question in the timeline fashion above, you can state that the pump is being restarted after the crew has recovered from a loss of all RCP seal cooling. You can ask which subsystem they restored first IAW 1-OP-RC-001, and then ask the question about whether the fourth start is/is not allow IAW 1-OP-RC-001.

Ensure that a loss of all seal cooling means a loss of CC Water and Seal Injection Flow. Or you can just state those two independently.

JAT 2/13/2015 Newly modified question incorporates the above suggestions. Only E recommendation is to use the phrase earliest time when asking when (pre3) the pump can be started so as to not be able to eliminate a distractor based on logic (i.e., if 1058 is correct, then 1128 would also be correct if

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only the question asked is when can the pump be started - if it can be started S

at 1058 it can also be started at 1128. Since there is only one correct answer, 1058 cannot be the correct answer.)

JAT 4/2/2105 Officially submitted question incorporates recommendation. Question appears to be SAT.

JAT 06/26/2015 No changes made. JAT 08/04/2014 4 F 2 X N E? 004K4.03 Question appears to match the KA.

For the second part of the question, Im not sure of the likelihood of someone choosing positive reactivity addition due to boron absorption (since the higher temperature doesnt cause boron to be removed from the coolant) as an answer when also presented with resin damage due to high letdown temperature. Another place to go on a second question may be an operator action that is taken or (RO-level) conditions that must be met to restore flow to the IX. Changed distractors B2 and D2 to make more plausible.

JAT 06/26/2015 SR made the corrections annotated in red, above.

S Updated distractor states what will physically happen when the temperature goes up, and states that the action is required to mitigate negative reactivity from release of boron. From a reactor safety standpoint, it seems unlikely someone would choose this distractor.

However, the question places the applicant in a situation where the reactor is stable at 100% operation, so there is at least some chance that someone might choose this to prevent a transient on the unit.

Updated question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 5 F 2 X N E/U 005K2.03 Question appears to match the KA.

Is there more than one open interlock for that valve? If not, the 460 psig needs to be removed so as to not teach in the stem. There is only one interlock, comment incorporated. It would be fine to refer to it as a minimum pressure or maximum pressure open interlock (or something similar). Referred as maximum pressure.

Is there another pressure transmitter that would be more plausible than the PRZR pressure control transmitter? The name PRZR pressure control transmitter implies that the range of pressures it covers is nowhere near the low pressure open interlock setpoint. Changed distractor pressure indicator to another RCS loop pressure indicator that is in the vicinity of the referenced indicators.

In the second question, recommend specifying that the open interlock is for 1-RH-MOV-1700. Comment incorporated.

JAT 06/26/2015 SR made the corrections annotated in red, above.

S Updated question appears to be SAT.

JAT 08/04/2015 6 H 2  ? B S? 005K6.03 Question appears to match the KA.

Just need to make sure there is no delay in the effects (i.e., does the stem question need to have a specific time associated with it?). There is no time delay in effect. Otherwise, question appears to be SAT.

The 401-5 indicates that this question was not on the last NRC exam.

However, we track the last TWO NRC exams, so I need to know if it was on either the 2014 or the 2012 exams. This question was not on the last two NRC exams, updated ES-401-5.

JAT 06/26/2015 S No corrections were required. SR comments are annotated in red, above.

Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 7 H 2 X N E 006A2.05 Question appears to match the KA.

What is the plausibility of putting the pump in pull-to-lock on the normal header? The justification provided is merely that it is a possible configuration if the applicant fails to apply the note correctly. However, the note says to align the highest priority pump to the normal header, but what would be accomplished by aligning a pump in pull-to-lock to the normal header? This distractor either needs an enhanced explanation of when this would be encountered or may need to reformat the question.

Re-wrote the question using similar concept for part 1.

JAT 06/26/2015 SR made the corrections annotated in red, above.

S Updated question appears to be SAT.

JAT 08/04/2015 8 H 2 X N S 006K3.01 Question appears to match the KA.

Regarding distractor D: What is the plausibility that a single train malfunction of the ECCS system would result in no ECCS flow to the RCS?

Additionally, in the stem of the question, it states that spurious safety injection signal has been initiated. Would it make more sense to say that the crew experienced a spurious SI? Comment incorporated.

One option for the question would be to reformat the question to ask: in the given alignment, the VCT is/is not a source of HHSI flow, and the RWST is/is not a source of HHSI flow. This still allows for another possible source of water, and there is at least some plausibility lent to neither one of them being a source. Question changed as recommended.

JAT 06/26/2015 SR made the corrections annotated in red, above.

The first question needs a comma: ...alignment, the VCT Updated question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 9 C 2 X N U 007A2.02 This question does not appear to meet the KA. The question essentially focuses on the open PORV (due to a controller malfunction) and the procedure that addresses the open PORV. The question needs to address the procedures that mitigate the abnormal pressure in the PRT, not the procedure that addresses the open PORV.

Recommend changing the question to:

1) The PRZR PORV block valve that must be closed to isolate the open PORV is ____.

2) The procedure that will direct returning the PRT to normal level

[or temperature, or pressure] is _____.

The above suggestion for question 1 incorporates a slight modification.

The answer choices that were submitted (e.g., close MOV-XXX block valve to isolate XXXX) lean toward teaching in the stem, unless the name of the PORV that is isolated is in the label for the block valve. Changed question as follows: Part 1 now asks for which PORV must be closed (more operationally relevant since block valve is right below PORV). Part 2 changed as recommended.

As an example, question 7 from the 2014 RO exam appears to be a good fit for this KA.

JAT 07/01/2015 S SR made the corrections annotated in red, above.

Updated question appears to match the KA. Updated question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 10 C 2 X M E 008AA1.04 Question appears to meet the KA.

Recommend adding the word first or initially to the second question so that there are not possibly two correct answers. Comment incorporated.

Is a temperature under Initial Conditions needed to ensure one and only one correct answer? (i.e., to make certain that Tave was not low enough to cause a FWI before an SI). Added normal Tave indication in Initial conditions.

Since this is a modified question, I need to see a copy of the original question for comparison. Original question is in References section of question 10 (Commanche Peak 2005, Q1-75).

JAT 07/01/2015 SR made the corrections annotated in red, above.

S Verified question is significantly modified. Updated question appears to be SAT.

JAT 08/04/2015 11 F 2 N S 008K1.01 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/01/2015 Question unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 12 C 2 N E 009EK3.11 The question appears to meet the KA.

The reason for tripping the RCPs should be neutral (as in, not imply that the correct answer to the first question is met) and should be tied to a procedure. For example, the second question could read, the reason RCP trip criteria are established in 1-E-1, in accordance with the basis for 1-E-1, is _____. Comment incorporated.

Also - please ensure that knowledge of EOP basis and reasons is RO knowledge at Surry. Yes, this is required knowledge.

JAT 07/01/2015 SR made the corrections annotated in red, above.

Updated question appears to match the KA. Updated question appears to S be SAT.

JAT 08/04/2015 13 C 2 N S 010K6.02 The question appears to meet the KA.

The question appears to be SAT.

JAT 07/01/2015 Question unchanged. JAT 08/04/2015 14 C 2 X B U 011EK2.02 The question does not appear to meet the KA. The question requires the applicant to demonstrate knowledge of pump operation and transfer to the RMT mode, rather than the reason for the pump response, as required by the KA.

To fix, I would leave the first of the two proposed questions, and then ask a question related to reasoning for the second question. This would change the question from a bank question to a modified question.

Changed part 2 to Per ES-1.3 CAP, after step 5, the operator will monitor for oscillating LHSI pump amps and flow to check for _____ .[ sump blockage, RWST < 3%]. Original Bank question was included in submittal.

JAT 07/01/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only M S SR made the corrections annotated in red, above.

Distractor B2 needs the . removed.

Question meets criteria for significantly modified. Updated question appears to be SAT.

JAT 08/04/2015 15 C 2 N S 011K2.02 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/02/2015 Question is unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 16 H 2 X M U/E 012K5.02 (NA14)

Question appears to match the KA.

I would recommend breaking the stem information into two pieces: Initial Conditions, Current Conditions. The part about N44 failing low can then go as a bullet point under current conditions.

Im not sure how plausible the IR High Flux trip is as a distractor when it is placed against the correct answer of PR high flux high setpoint trip. At

~35% power, the unit is well into the power range. Additionally, the justification that the setpoint for the IR trip may be confused with the block point seems to be a stretch, and with a trip setpoint at 35%, the unit would trip given the conditions in the stem.

Instead, it would be more plausible to ask whether or not the IR High Flux trip is currently protecting the reactor against a positive reactivity excursion (i.e., Given the current conditions, the Intermediate Range High Flux Trip [is/is not] protecting the core against a positive reactivity excursion.)

The plausibility of is lies in someone confusing the block setpoint with the trip setpoint. If someone thinks its not blocked until 35%, then N44 failing low would unblock it, and may be led to select the wrong answer.

There would be no cue from the other answer choice that the distractor is wrong.

Question 62 appears to contain information that would also render two of the existing distractors non-plausible, and could also overlap with the proposed modification.

Since this is a modified question, I need to see a copy of the original question for comparison. Rewrote the question entirely to better meet the K/A and eliminate fouling with question 62. Focused on Reactor protection against a high power density. This is now a NEW question.

JAT 07/02/2015 S

N SR made the corrections annotated in red, above.

New question appears to meet the KA. New question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 17 C 4 X X N U 013G2.2.36 The question appears to meet the KA. This question appears to be at the SRO-only job level, in that it asks the applicant to evaluate whether the application of 3.02 was correct for the current plant conditions.

At time 2130, I would recommend putting the entire position in quotations. The description seems to indicate that it reads >10 rather than spelling out the words greater than. If this is the case, I would use the >10 nomenclature. Otherwise, I would include the words greater than inside the quotation marks.

Is the EDG being inoperable associated with a TS action that is less than 1h? If so, that would be RO knowledge and a good place to take this question. Rewrote entire question to better conform to K/A.

JAT 07/02/2015 SR made the corrections annotated in red, above.

The new question does not appear to match the KA, in that the PORV block valve does not really relate to ESF actuation. However, the idea of the first question is good for the part of the KA relating to analyzing the effect of maintenance activities on LCOs. It just needs to be a maintenance activity on an ESF component or the actuation signals themselves.

The second question appears to be a tack-on. The TS definition of hot shutdown is not related to the first question at all, other than they both show up in TS.

JAT 08/04/2015 S The new question appears to match the KA.

Do you need to give the LCO number?

Otherwise, question appears to be SAT.

JAT 08/06/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 18 C 2  ? N E 013K4.10 The question appears to meet the KA.

Recommend the following minor edit to the first question: 1) After depressing both CLS RESET buttons, CLS [is/is not] reset. Made change as recommended.

(These do not need to be operated simultaneously, correct?)

The plausibility of the combination of CLS not resetting with the valve opening is shaky. However, I am accepting that SI is reset as an ok justification of plausibility.

What is the plausibility of 1-IA-TV-100 remaining closed if the applicant mistakenly thinks CLS does reset in the first half question? There may be some enhancement to the stem required to make choice A plausible.

Added to stem CTMT IA indicates 25 psig. This makes choice A plausible because IA-TV-100 requires IA pressure to be > 30 psig to open, therefore even if candidate thought CLS was reset the valve still will not open. This does not affect plausibility of B and D because IA Comp suction valves will still be able to be opened if SI was reset. Also updated plausibility statements and added references.

JAT 07/02/2015 S SR made the corrections annotated in red, above.

Need a comma in the second question: open 1-IA-TV-100, the valve Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 19 C X N E 014K5.01 This question appears to meet the KA: The KA talks about operational implication, which usually suggests a procedural direction of some sort. In this case, the diagnosis of the cabinet with the failure is the operational implication. Although the question does not address a reason for the difference between the RPIS and step counter, which is specifically addressed in the KA, the reason for the difference would be considered when developing operating procedures, and by appropriately diagnosing the event, the appropriate procedural actions could be implemented, and thus knowledge of the operational implication of the reason will be demonstrated by the applicant.

The second question appears to be a tack-on. I would recommend replacing the second question with a question that asks something from the ARP or possibly which abnormal procedure will require entry to mitigate the failure. Part 2 replaced with question asking for AP that will be used to mitigate failure.

JAT 07/02/2015 SR made the corrections annotated in red, above.

S A2 and C2 need to say Rod rather than Ron in the procedure titles.

Question appears to be SAT.

JAT 08/04/2015 20 H 2 B S 015AK2.10 The question appears to meet the KA.

The question appears to be SAT.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. This question has not been used on either the 2014 or the 2012 exam. Question form annotated.

JAT 07/02/2015 Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 21 F 2 N S 017A1.01 The question appears to meet the KA.

The question appears to be SAT.

JAT 07/06/2015 Question remains unchanged. JAT 08/04/2015 22 H 2 N S 022A4.03 The question appears to meet the KA.

The question appears to be SAT.

JAT 07/06/2015 Question remains unchanged. JAT 08/04/2015 23 H 2 N S 022AK1.04 The question appears to meet the KA. Based on the KA match description provided on the 401-5, which specifically addresses the reason portion of the KA, application of the knowledge that CH flow is limited by design while in automatic control is required to answer the question. The operational implications aspect of the KA is demonstrated by recognizing the low flow setpoint and the pzr level setpoint that the unit would be maintaining while in automatic.

JAT 07/06/2015 Question remains unchanged. JAT 08/04/2015 24 H 2 M S 022K1.02 The question appears to meet the KA.

The question appears to be SAT.

Since this is a modified question, I need to see a copy of the original question for comparison. Original question was included in submittal, relabeled as Original question to better annotate question.

JAT 07/06/2015 Question is significantly modified. Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 25 H 2 N S 025AA1.04 The question appears to meet the KA.

The question appears to be SAT.

JAT 07/06/2015 Question remains unchanged. JAT 08/04/2015 26 F 2 M S 026A3.02 The question appears to meet the KA.

The question appears to be SAT.

Since this is a modified question, I need to see a copy of the original question for comparison. Original question was included in submittal, relabeled as Original question to better annotate question.

JAT 07/06/2015 Question is significantly modified. Question remains unchanged. JAT 08/04/2015 27 F 2 N E 027A4.01 The question appears to meet the KA.

Instead of asking what positions there are for the switch to the Iodine filter fans, I would ask how the fans would be started. For example, since the stem of the question has the applicant start the fan per the request of HP, the first question can be rephrased to ask:

• The Iodine Filter Fans [can/cannot] be started by placing the control switch in the Auto position.

Or

• The operator will place the control switch for the Iodine Filter Fans to

[Auto/On] to start the fans.

The second option would also be more plausible if the third bullet of the stem information were revised to state: HP has requested the Unit 1 Iodine filter fans, 1-VS-F-3A and 3B, be placed in service.

(rather than started). Made changes to P1 question using second option. Also reworded P2 to fill in the blank to be consistent with P1.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S JAT 07/06/2015 SR made the corrections annotated in red, above.

The second question needs a 2 in front.

The question appears to be SAT.

JAT 08/04/2015 28 H 2 X N U 027AA2.02 This question does not appear to match the KA. The KA is specific to a pressurizer Pressure Control System malfunction, and the proposed question relates to normal operation of the pressurizer pressure control system.

To meet the KA, a similar question can be asked by having a POT setting in the stem initial conditions, and then stating a pressure under current conditions, and asking about whether a malfunction occurred or what type of malfunction has occurred. (There are other ways to meet this KA, that is just one option). Modified question as follows. New question provides conditions of a failed A spray valve controller. Part 1 asks for POT setting for 2235 psig, and Part 2 asks for operator action to correct. P2 requires candidate to diagnose current conditions and determine what has failed.

This has also changed the COG level from Low to High.

JAT 07/06/2015 S SR made the corrections annotated in red, above.

The new question is a much closer match to the KA. I consider it a match because the applicant must be able to identify what POT setting is associated with normal pressure and determine the failure that occurred in the given conditions.

You can use the pictures of the controller indications for the current conditions if you prefer.

Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 29 H 2 X M E 028AG2.4.47 The question appears to meet the KA.

I would either state that Channel I is selected for control OR that Channel I is the upper channel. The way it is written now, you are explaining in the stem of the question that selected for control means upper channel.

(teaching in the stem).

All options for the second half question include, place CH flow in manual. The only difference between the two options is the addition of and isolate letdown to the second answer choice option. This creates a subset issue, because an applicant can choose either answer and be correct since the stem only states, the expected operator action. Thus, there are two correct answers. To fix, change the second question to: In accordance with 0-AP-53.00, the operator [is/is not] required to isolate letdown for this failure. Made changes as recommended.

Since this is a modified question, I need to see a copy of the original question for comparison. Original question was included in submittal in Reference section of question.

JAT 07/06/2015 S SR made the corrections annotated in red, above.

Need to add a comma to the first question: ..leg leak, the level Also change indication to indicated.

Question is significantly modified.

Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 30 H 2 X X M U 029EA2.01 The question appears to meet the KA. This question appears to be at the SRO-only job level, since it asks the applicant to make a determination on a procedure transition that is not solely related to knowing the entry conditions for the EOP in question. The relevant information is not contained within a procedural note, nor is it contained in a foldout page.

Therefore, this question does not appear to be at the appropriate job level.

The 4th bullet (Power range instruments are all indicating 8%) needs a period at the end.

Since this is a modified question, I need to see a copy of the original question for comparison. This question has been rewritten. Previous Q 30 has been moved to Q 78. New question has both parts relating to nuclear instrumentation and ATWS. Note this is now a NEW question.

N JAT 07/06/2015 S SR made the corrections annotated in red, above.

The question appears to meet the KA at the RO level.

The first question needs a comma: ..less than 5%, the reactor There may be an overlap issue with current question 78 (i.e., old question 30 that this question replaced). However, Q78 asks what is required to return to 1-E-0 from FR-S.1. This question asks what 1-E-0 considers tripped. These may have similar answers, but since they are two distinct pieces of knowledge, Im going to say that the questions are not overlapped; if an applicant misses one of the questions, that lack of knowledge will not necessarily cause them to miss the other question.

Similarly, although the answers contain similar information, knowing one answer does not give you the answer to the other.

Question appears to be SAT.

JAT 08/04/2015 31 F 2 N S 029G2.1.25 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/06/2015 Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 32 H 2  ? N E? 035A2.05 The question appears to meet the KA.

If the entry condition for AP-21.00 states, The reactor is in the power range and a reduction in Main FW flow occurs as indicated by one or more of the following: 1H-G6, STM GEN 1B ERROR, would this also not be a correct answer? Need to provide a justification for why this answer is definitely wrong to avoid the possibility of two correct answers. Changed the stem of the question by adding B Feed Reg valve demand begins to slowly lower due to an instrument failure.

JAT 07/06/2015 S SR made the corrections annotated in red, above.

The question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 33 F 1 X B U 038EK1.02 The question appears to meet the KA.

This question is a direct lookup, LOD=1. All that is required is to find the PZR level stated in the stem and the SG level trend stated in the stem and look up the answer, rendering all distractors not plausible. If there is some plausibility allowed for =35% vs >35%, there are still 2 non-plausible distractors.

The stem question asks should, which is not a definitive question and leaves some room for interpretation.

To fix: 1) Have the applicant diagnose which SG has the tube leak; 2) instead of stating affected SG NR level is 68% and lower, give a level and trend for each SG; 3) change the stem question to Which ONE of the following is the MINIMUM actions required to be taken by the operating team in accordance with step 36 of E-3?. This will allow your answer choices to remain as-is. Made changes as recommended.

The first sentence has a minor grammatical error. It should read: The Operating Team is in E-3, Steam Generator Tube Rupture, and has completed RCS cooldown and depressurization. Made change as recommended.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. Believe this is now modified enough to no longer be a Bank question. Also Note: Found similar question on 2012 exam and have included the question in Reference section. We believe it is different enough to be considered a Modified question but we request Chief Examiner to also review. New (Modified) question is NOT a 2-part question like the 2012 exam question, and stem is markedly different.

S JAT 07/06/2015 H 2 M SR made the corrections annotated in red, above.

Question is significantly modified (and higher cog now).

Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 34 H 2 N S 039K3.04 The question appears to meet the KA.

The question appears to be SAT.

JAT 07/06/2015 Question remains unchanged. JAT 08/04/2015 35 F 2 N E 040AG2.1.30 The question appears to meet the KA.

The second question should end in a . rather than a ?. Change made as recommended.

Otherwise, the question appears to be SAT.

JAT 07/06/2015 SR made the corrections annotated in red, above.

S Question remains SAT. JAT 08/04/2015 36 H 2 M S 045K3.01 The question appears to meet the KA.

The question appears to be SAT.

Since this is a modified question, I need to see a copy of the original question for comparison. Original question was included in Question references section.

JAT 07/10/2015 Question is significantly modified. Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 37 H 2 X X M U 054AK3.05 This question does not appear to meet the KA. The question requires knowledge of PORV cycling to eliminate a distractor rather than to arrive at the correct answer. An applicant can arrive at the correct answer without demonstrating knowledge of the KA the way the question is currently proposed.

This question does appear to require RO knowledge to ANSWER, in that the determination for whether RCS bleed and feed is required appears on the continuous actions page. However, two of the distractors (B and D) appear to require SRO-only knowledge to eliminate in that they involve detailed procedure knowledge and are not associated with a procedural note, caution, or foldout page and are more specific than the overall mitigation strategy. Thus, two of the distractors for this question are not plausible on the RO portion of the exam.

Since this is a modified question, I need to see a copy of the original question for comparison. Changed question to more closely match K/A. P1 requires knowledge of CAP Bleed and Feed due to PORV open. P2 requires understanding of reason for PORV remaining open vs. cycling during loss of heat sink. Significantly modified question therefore this is now a NEW question.

JAT 07/10/2015 SR made the corrections annotated in red, above.

S Per the FAQs, if you based your question, even loosely, on another question, even it looks almost nothing like the previous version, it is still significantly modified rather than new. The only way for a question to be considered new is if you developed it without reference to another question.

Question appears to be SAT.

JAT 08/04/2015 38 H 2 N S 055A3.03 The question appears to meet the KA.

The question appears to be SAT.

JAT 07/10/2015 Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 39 F 2 M E 055EK3.02 The question appears to meet the KA. The distractors are contained within the notes and maintaining SG pressures greater than 200 psig is referenced in a caution of the procedure, indicating that this knowledge is RO level. Is preventing nitrogen injection also referenced in the background document for the EOP, and is EOP background information also RO-level knowledge at Surry? (Asking because the caution references 200 psig and the step references 300 psig. My assumption is that 300 psig allows you to be greater than the 200 psig called out in the caution - Im wondering if there is a specific tie between those two numbers in the background document.)

In ECA-0.0 Rev 39, which was provided as part of the reference package, it is step 24 rather than step 21 (which is referenced in the question) which directs SG depressurization to 300 psig. Changed to step 24.

Since this is a modified question, I need to see a copy of the original question for comparison. Original question was included with submittal and is in reference section of question.

JAT 07/10/2015 SR made the corrections annotated in red, above.

S Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 40 X M U/E 057AG2.4.50 The question appears to meet the KA.

There is a potential subset issue. The way the question is worded, if there is no failure, the SG could be controlled using either method presented.

The applicant could use logic to determine that there cannot be more than one correct answer, and since both methods will result in controlling SG level, then there must not be power to the controller to allow that choice as an option. This results in two non-plausible distractors. Thus, the first question needs to be more specific - you either need to specify the preferred method or the method IAW the applicable AP (which is answered in part 2), etc. This can be avoided entirely by simply asking:

B SG [can/cannot] be controlled using the pushbuttons on the FRV controller. Made changes as recommended.

Since this is a modified question, I need to see a copy of the original question for comparison. Question was submitted and is included in Question reference section.

JAT 07/10/2015 S SR made the corrections annotated in red, above.

This question is significantly modified. This question appears to be SAT.

JAT 08/04/2015 41 F 2 N E 058AA1.02 The question appears to meet the KA.

Should the question stem ask for the alternate source, rather than the alternant source? Changes made as recommended.

Answer choice D is the only one with the word breaker in it, and the stem question specifically calls out that the choices are breakers.

Recommend removing the word breaker from choice D so that it is not an outlier. Changes made as recommended.

JAT 07/10/2015 SR made the corrections annotated in red, above.

S Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 42 F 2 X X X N U 059A1.07 (pre)

Question appears to match the KA.

Question appears to be at the RO level.

• Credible Distractors:

o There is a subset issue that renders two distractors implausible. If Recirc flow is <3000gpm, it is clearly <3600gpm, so an applicant that knows there is one and only one correct answer can eliminate distractor B. Similarly, if feed flow is <3000gpm, it is clearly

<3600gpm, so D can be eliminated.

• Partial:

o If the recirc valve is not fully open, it could give a flow of <3000gpm.

Therefore, an applicant may be able to argue A as a partially correct answer.

• Recommend including valve number - recirc valve appears to be non-specific.

• Backward: The question is written in a backward logic way (observation only)

• Possible way to fix: Set up a timeline with feed flows, recirc flows, and recirc valve positions, and query on MFP status at two different points in time.

JAT 2/4/2015

? E Question appears to match the KA. Question appears to be at the RO level.

(pre2) Comments have been addressed.

Recommend definitively specifying which MFP trips.

Would lowering the feed pump suction pressure make D more plausible?

Otherwise, question appears SAT.

JAT 2/13/2015 Comments have been address. The question still appears to match the KA.

S The question appears to be SAT.

JAT 07/10/2015 Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 43 F 2 X B E 059A4.08 The question appears to meet the KA.

The answer choices do not appear analgous - options C and D offer an effect on the controller rather the control available. Recommend slightly altering the question to ask: 1) the effect on the output of the controller (shift to manual and remain the same as prior to the loss of power/raise to 100%), 2: where control can be from (in MCR/cannot be controlled from control room). Made changes as recommended.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. With the changes made, this question has changed from Bank to Modified Bank. Original question added to Question reference. Also checked the last 2 NRC exams and this question was not asked in either exam.

JAT 07/10/2015 SR made the corrections annotated in red, above.

S I would still lean towards calling this a bank question. There werent really any changes made to the stem, and although the question is now in a 2x2 format that was mostly to enhance plausibility of the distractors since the answer to the original question is still the same two pieces of information.

You are well within limits of Bank questions on the exam, so erring on the conservative side is best.

Question appears to be SAT.

JAT 08/04/2015 44  ? N E? 060AK1.02 The question appears to meet the KA.

Consideration - is it more likely an applicant will confuse an admin limit with the federal limit, or forget to apply the 85%? (There is no distractor that accounts for: [(2000)-(700)]/ (1000 mr/hr) = 1.3 h). This distractor would mirror the one with the fleet admin limit, and would likely make the distractors together more plausible overall, rather than having the federal limit as an outlying distractor. Changed wording of question, and distractor A to 0.50 hour5.787037e-4 days <br />0.0139 hours <br />8.267196e-5 weeks <br />1.9025e-5 months <br />, based on limit for other Dominion sites.

Recommend these changes to avoid potential for two correct answers.

JAT 07/10/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S SR made the corrections annotated in red, above.

Question appears to be SAT.

JAT 08/04/2015 061K1.04 45 H 2 X M U (pre) Question appears to match the KA (cause-effect relationship: the RCS flowing into one SG that is also faulted has the effect of requiring AFW to be maintained to the other SGs at a certain level)

• Credible Distractors: It is not plausible that an applicant would choose to feed both a faulted and ruptured SG (since they were told that was the case in the stem), so choices A and C are implausible, given that they both have AFW flow to B SG.

• Stating that a SG is faulted and ruptured may be teaching based on the conditions you give them in the stem of the question. Given the containment pressure, PZR level/P, Tave, and B SG parameters, it is fairly obvious that both a fault and an RCS rupture of some type is in progress. The fact that B is both faulted and ruptured does not seem to make a difference to the correct answer.

• Concerned there may not be a correct answer: the procedure states that the minimums are 350gpm normally and 450gpm in adverse containment. None of the answer choices are technically at the minimum flows, per the procedure, but the question asks which of the following states the minimum required flow.

• Recommendations:

o Remove statement about ruptured/faulted SGyou only need to say that the team is performing 1-E-0 and the plant conditions are as follows.

o Recommend putting a trend into the SG parameters to make the rupture obvious.

o Query on the actual minimum required AFW flow (and cite the procedure/step/attachment) in one part of the question.

(350gpm/450gpm) o Query on which SGs should be aligned to receive the flow in the second part of the question. (AandC/A,B,C) o Although one of the answers will still have B receiving flow vs. no flow, it is more plausible after the applicant is no longer specifically told they have a faulted/ruptured SG.

• I will need to see the parent question in the draft submittal since this is listed a modified bank question.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only B JAT 2/4/2015 S

(pre2) Question appears to match the KA. Question appears to be at the RO level. Comments have been addressed.

Ensure that lowering slowly means that level is controllable.

Parent question provided. Question may actually be Bank rather than significantly modified. (OK, bc not near bank limit.)

JAT 2/13/2015 S Question remains SAT.

07/10/2015 Question remains unchanged. JAT 08/04/2015 46 F 2 N S 061K5.05 The question appears to meet the KA.

The question appears to be SAT.

JAT 07/10/2015 Question remains unchanged. JAT 08/04/2015 47 F 2 N E 062A4.04 The question appears to meet the KA.

For the first question, it would be more precise to ask: The red breaker position indicating light [can/cannot] be used by the control room team to identify 1-CS-P-1A has been successfully started. This would avoid any potential confusion with the choice, 1-CS-P-1A amps only, since there may be another indication that can be used to confirm flow (i.e., flow meter, change in temperatures, etc.), which could lead to a condition where an applicant thinks there are no correct answers. The words control room team were used to more clearly tie the question to the KA and to ensure there is no other light that could be confused for the indicating light in question. Made recommended changes to question.

JAT 07/10/2015 S SR made the corrections annotated in red, above.

The question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 48 H 2 X N E 062AA2.04 The question appears to meet the KA.

Need to add the words, a minimum of to question 2 to avoid two potential correct answers. Made recommended changes to question.

JAT 07/10/2015 SR made the corrections annotated in red, above.

S Question appears to be SAT.

JAT 08/04/2015 49 F 2 M E 063K2.01 The question appears to meet the KA.

Question 2 needs to have a slight grammatical revision: It should ask, The Black Battery is supplies the ____ or The black battery is the supplyies for the _____

Otherwise, question appears to be SAT.

Since this is a modified question, I need to see a copy of the original question for comparison. Made recommended changes to question.

Original bank question was included in draft submittal, and is located in question reference section.

JAT 07/14/2015 S SR made the corrections annotated in red, above.

Question is significantly modified. Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 50 F 2 X B U 064K2.03 This question does not appear to meet the KA. The KA requires the applicant to demonstrate knowledge of bus power supplies to EDG control power. The question is set up based on the exciter field circuit breaker for the EDG tripping, and seems to be able to be answered entirely from knowing whether the EDG loads (the failure of the EDG to load is a given condition of the stem). The applicant does not need to know the power supply to the EDG control power to answer the question.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. Revised question to better meet the K/A. This is now a NEW question.

JAT 07/14/2015 S SR made the corrections annotated in red, above.

N The new question appears to meet the KA. The new question appears to be SAT.

JAT 08/04/2015 51 F 2 X N U 067AK3.02 The question does not appear to match the KA. The KA requires the applicant to demonstrate knowledge of the reason for the steps in the procedure. The question simply ask the type of fire protection and whether it is manual or automatic.

To have a better KA match, it might be possible to modify the existing question to ask whether the method of CO2 action is manual or automatic and why. If you go this route, ensure that the combinations of answers are plausible together. - Rewrote question to match K/A.

JAT 07/14/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only E SR made the corrections annotated in red, above.

Need to add the word maximum to the time limit in the first part of the question.

I would add the procedure reference containing the caution with the basis for the time for the second question. Additionally, since the distractor is for an EAL call, it seems implausible that the max time would be 15 minutes when the EAL call must be made within 15 minutes. Recommend changing the first part distractor to something lower (3 minutes, 10 minutes, etc).

Question requires enhancement. JAT 08/04/2015 Question SAT. JAT 9/9/2015 S

52 F 2  ? N U? 068AA2.10 Not clear if this question meets the KA. Can the applicant answer the first question without knowing any information other than it has been 20 minutes after reactor trip? If so, this half of the question does not meet the KA, unless there is a unique type of meter on the remote monitoring panel. A better KA match would be to provide information up to the point where the operator reports that both meters are offscale low at the remote monitoring panel, and then ask a question based on that (i.e.,

what does it mean, what do they do, which procedure must they enter keep it RO level, though). This would meet the interpret the source range count piece of the KA.

On the other hand, if the SR meter at the remote monitoring panel reads differently than in the CR, the query can be what they expect to see and the distractor can be an indication in the control room that is not available at the remote monitoring panel. Rewrote the stem to include a picture of Remote monitoring panel with flawed SR indication to better meet the K/A.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S JAT 07/14/2015 SR made the corrections annotated in red, above.

Question appears to match the KA.

Was the wording of the first question validated? Might be easier to just ask: The NI indication at the Remote Monitoring panel [is/is not]

expected.

Otherwise, question appears to be SAT. Wording choice is up to SRthere is one and only one correct answer with the current wording.

JAT 08/04/2015 53 F 2 N E 068K1.07 The question appears to meet the KA.

It looks like the VCT may go to the HL/LL tanks after going through a few other tanks to tighten up the answer choices, I recommend adding the word directly to the stem question: Which ONE of the following water sources is collected directly into the High/Low Level Liquid Waste Tanks prior to transfer to the SRF?

Otherwise, this question appears to be SAT.

Added Directly Into to stem of question as recommended.

JAT 07/14/2015 SR made the corrections annotated in red, above.

S Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 54 H 2 M E 073A1.01 The question appears to meet the KA.

Recommend adding the word automatic to the stem question: Which ONE of the following describes the automatic plant response?

Otherwise, this question appears to be SAT.

Since this is a modified question, I need to see a copy of the original question for comparison.

Added automatic to Stem of question. Pre-modified question is part of full file.

JAT 07/14/2015 S SR made the corrections annotated in red, above.

Question appears to be significantly modified. Question appears to be SAT.

JAT 08/04/2015 55 H 2 N S 076AK2.01 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/14/2015 Question remains unchanged. JAT 08/04/2015 56 F 2 N S 076G2.1.30 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/14/2015 Question remains unchanged. JAT 08/04/2015 57 F 2 N S 076G2.2.22 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/14/2015 Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 58 H 2 N S 077AK1 .02 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/14/2015 Question remains unchanged. JAT 08/04/2015 59 F 2 B S 078A3.01 The question appears to meet the KA.

This question appears to be SAT.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. Checked last 2 NRC exams (2014, 2012) and this question was not on either of those exams.

JAT 07/14/2015 Question remains unchanged. JAT 08/04/2015 60 H 2 N S 103A1.01 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/14/2015 Question remains unchanged. JAT 08/04/2015 61 F 2 N S G2.1.23 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/15/2015 Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 62 M E G2.1.7 This question appears to meet the KA.

There may be some overlap between this question and question 16. If question 16 remains as is (and additional distractor justification is provided) or the recommended change is implemented, the first half of this question would overlap. One or the other needs to change. Keep in mind that this KA is a much broader topic and can likely accommodate more possibilities.

Since this is a modified question, I need to see a copy of the original question for comparison. Question 16 was modified, therefore this question no longer overlaps question 16. Original question was included in submittal and is located in question reference section.

JAT 07/15/2015 S Question remains unchanged. Question 16 was modified and no longer overlaps with this question.

Question is significantly modified. Question is SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 63 F 2 B E G2.2.6 This question appears to meet the KA.

Recommend stating the procedure that the change is in accordance with in the stem question (AD-AA-100).

Recommend making the following change to simplify the question and leave less room for interpretation in the answer:

Which ONE of the following completes the following statement regarding the actions to be performed in accordance with AD-AA-100 to address the issue?:

After making a pen and ink change, the Operator [is/is not] required to have a supervisor initial and date the correction before proceeding. After completing the OPT, a procedure change [is/is not] required to be immediately processed. Made changes recommended.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. Changes made make this a Modified question. Question included in reference section of question.

Also checked last two NRC exams and did not find this question.

JAT 07/15/2015 M SR made the corrections annotated in red, above.

S Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 64 F 2 X M U G2.2.7 This question does not appear to meet the KA. The KA requires knowledge of the process for conducting special or infrequent tests. The question asks which procedure IS an infrequently conducted test or complex evolution.

To meet the KA, I recommend giving a procedure that governs a special/infrequent test and then soliciting knowledge (at least in half of the question) as to any special briefing or preparatory requirements in OP-AA-106, which governs Infrequently Conducted or Complex Evolutions.

You could ask two different procedures (one of which meets conditions of OP-AA-106 and one that does not) and then ask for each test whether a specific requirement is required. This would meet the process part of the KA.

Since this is a modified question, I need to see a copy of the original question for comparison. Rewrote the question to better meet the K/A.

Question should now be considered a NEW question. Original question in question reference section for review by NRC Chief Examiner.

JAT 07/15/2015 SR made the corrections annotated in red, above.

S Question appears to meet the KA. Need to ensure this question is at the correct license level, as it appears to ask specific details of the admin procedure. Provided ROs are expected to know these two pieces of information at Surry, this question is likely okay. Considerations: ROs performing these tests must know contingency plans/termination criteria.

Need to discuss second question.

JAT 08/04/2015 Surry states these are RO level questions and will attest to that in the 401-5 (ops expectation for ROs to know). JAT 08/05/2015 65 F 2 N S G2.3.13 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/15/2015 Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 66 H 2 M S G2.3.14 The question appears to meet the KA.

Need a better justification for why the two choices in distractor A are plausible together. Why would ventilation remain in normal but containment evacuation be required? The plausibility currently given for maintaining CR ventilation in normal is that no rad alarms are in, so the applicant may think there are no hazards. If they have this misconception, why, then, would they choose to evacuate containment? If an appropriate justification of these two choices together can be provided, the question is likely SAT. Otherwise, this distractor may need to be revised.

This question appears to overlap with SRO question 85, which also discusses a fuel handling incident, escaping bubbles. Question 85 elicits knowledge on establishing containment isolation and the basis for when emergency ventilation is required. The correct answer to this question (RO66) is that containment evacuation is required and that emergency ventilation is required. The way question SRO Q85 is asked implies the correct answer to this question. For that reason, either SRO Q85 requires replacement or this question requires replacement.

Since this is a modified question, I need to see a copy of the original question for comparison. Modified question Part 1), closure of MCR ventilation dampers secures MCR normal supply or exhaust fan to exclude overlap to question 85. Modified question included in Master file.

JAT 07/15/2015 SR made the corrections annotated in red, above.

Question is significantly modified. Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 67 F 2 B E G2.3.4 This question appears to meet the KA.

There are 4 different answer choices for the first question and 4 different answer choices for the second question. Recommend choosing one or the other.

(This is modified from a NAPS question although it still meets the criteria for being a bank question. Thus, it was not on a previous Surry NRC exam.)

Revised to limit to Federal limit for Lens of the Eye.

JAT 07/15/2015 SR made the corrections annotated in red, above.

S Distractor analysis for answer A needs updating (no longer a part 2).

Question appears to be SAT.

JAT 08/04/2015 68 F 2 B E G2.4.21 This question appears to meet the KA.

To make all distractors more plausible, I would recommend changing the stem question to: Which of the following Critical Safety Function RED paths is the highest priority CSF that can be directly caused by procedurally directed operator action?

This would lend credibility to the fact that more than one of them could be caused by a procedurally directed operator action.

(This is modified from a Diablo Canyon question although it still meets the criteria for being a bank question. Thus, it was not on a previous Surry NRC exam.)

Revised per recommendation.

JAT 07/15/2015 S SR made the corrections annotated in red, above.

Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 69 F 2 N S G2.4.29 The question appears to meet the KA.

Please ensure that this question is RO level knowledge at Surry. Otherwise, question appears to be SAT.

JAT 07/15/2015 Question remains unchanged. JAT 08/04/2015 70 F 2 N E/S G2.4.43 The question appears to meet the KA.

Recommend adding a procedural tie to the two questions. I.e., add, In accordance with EPIP-2.01 in front of When obtaining meteorological data from the MET panel Otherwise, question appears to be SAT.

Revised Stem as recommended.

JAT 07/15/2015 S SR made the corrections annotated in red, above.

Question appears to be SAT.

JAT 08/04/2015 71 F 2 M E/S WE04EK2.2 The question appears to meet the KA.

Recommend adding a procedural tie to the two questions. Change the stem question to: In accordance with 2-E-1, which ONE of the following identifies:

Otherwise, question appears to be SAT.

Since this is a modified question, I need to see a copy of the original question for comparison. - located in Master File.

Revised Stem as recommended. Added procedure title.

JAT 07/15/2015 S SR made the corrections annotated in red, above.

Question is significantly modified. Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 72 H 2 N S WE08EK3.3 The question appears to meet the KA.

Need to ensure that this is expected knowledge for an RO at Surry.

Additionally, need to verify that EOP and FR background knowledge is RO level knowledge. EOP and FR background knowledge is RO level knowledge. Specific knowledge included in reference of original submittal.

Otherwise, question appears to be SAT.

JAT 07/15/2015 Question remains unchanged. JAT 08/04/2015 73 H 2 B S WE10EA1.3 Question appears to meet the KA.

Question appears to be SAT.

JAT 07/15/2015 Question remains unchanged. JAT 08/04/2015 74 H 2 X X B U WE11EG2.1 .2 Question appears to meet the KA.

Question does not appear to be at the RO level. The first half of the question asks for the procedure that should be used for the next step, and the options are ECA-1.1 and ES-1.3. This is procedure determination, and is SRO-only.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. (In part, because if this was an approved RO-level question on a previous NRC exam, it would not be considered UNSAT, even though modification would still be required.)

Rewrote question to better meet RO level. One of actions in ES-1.3 is to manually initiate RMT if it fails to actuate automatically. Wrote questions testing the ROs ability to manually actuate RMT. Question is now a NEW question, but left original bank question in the reference section of question 74.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S JAT 07/15/2015 SR made the corrections annotated in red, above.

Question appears to match the KA at the appropriate license level.

Second question needs a . at the end.

Question appears to be SAT.

JAT 08/04/2015 75 H 2 N S WE15EG2.4.8 This question appears to meet the KA.

This question appears to be SAT.

JAT 07/15/2015 Question remains unchanged. JAT 08/04/2015 76 C 2 B S 002G2.2.25 The question appears to meet the KA.

This question appears to be SAT.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. This question was not asked in either the 2014 or the 2012 NRC exam.

JAT 07/16/2015 Question remains unchanged. JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 77 H 3 X N E 004A2.35 (pre)

Question appears to match the KA. Question appears to be at the SRO-only level. KA appears to be met at the SRO-only level.

• Regarding 1-CH-TV-1204A, either tell them that the valve is closed, or tha the green light is lit and the red light is out. Telling them both is teaching in the stem. Also, the valve name should be included in the question stem.

• For question 1), is it common practice to deactivate the OPERABLE trip valve when the INOPERABLE valve has the ability to be verified in the closed position?

• For question 2), remove the (MET / NOT MET) in the stem of the question. Also - typically only the NOT is emphasized. Currently all part 2) answer choices are in all capital letters, whether they contain the word not or not.

• Does deactivating the automatic isolation valve render it inoperable?

The note regarding maintaining at least one isolation valve OPERABLE in each affected penetration is unclear to me as to whether the deactivated valve would be OPERABLE or INOPERABLE.

• Instead of specifically differentiating between the OPERABLE trip valve and the affected trip valve, I would recommend just naming the valve that you intend to close and deactivate, and then the applicant can determine if that was the OPERABLE or INOPERABLE valve. This will also get rid of any potential cues in the stem.

JAT 2/4/2015

? Question appears to match the KA. Question appears to be SRO-only.

(pre2) Question apperas to be met at the SRO-only level. Comments have been addressed.

My only remaining concern is the 4h clock - typically we give a reference for >1h action statements, and in this case, the reference cannot be provided due to being a direct lookup. Is there a learning objective for this that can justify no reference? If not, you could simply ask whether the deactivated valve is OPERABLE or INOPERBLE.

S JAT 2/15/2015 (pre3) Comment incorporated. Question appears to be SAT at this time.

JAT 4/2/2015 S

Question remains SAT. JAT 07/16/2015 Question is unchanged. 08/05/15

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 78 H 2 X N U 007EA2.06 The question does not appear to meet the KA at the SRO-only job level.

Both questions are SRO-only, but neither meet the KA. The KA requires the applicant to demonstrate knowledge of the occurrence of a reactor trip. The question relates to restart following a reactor trip. Additionally, the second question, while at the SRO-only level because of being TS basis knowledge, does not relate to the first part of the question. If the computer-calculated leak rate calculation that is present in the stem of the question related to or complicated the reactor trip, it would be fine. But as-is, these are two unrelated questions and neither appear to relate to the KA. This question was replaced from Q 30. This is a modified question and original came from Harris 2004 exam, q 47. Copy of q included in Reference section of Q 78.

JAT 07/16/2015 B

S SR made the corrections annotated in red, above.

This question appears to be bank. There were some modification made, but it doesnt look significantly modified.

Question appears to meet the KA. Question appears to be at the SRO-only level. Question appears to be SAT.

JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 79 H 2 N 012G2.4.1 The question does not appear to meet the KA at the SRO-only level. The second question appears to be at the SRO-only level, but does not appear to meet the KA. Upon further review, this KA does not appear to lend itself to an SRO question, so I am replacing the KA with randomly selected KA: 012 G2.1.23 Reactor Protection / 7, Ability to perform specific system and integrated plant procedures during all modes of plant operation.

The current proposed question, if you choose to keep it, would need to have a slight modification because the part of the question that meets the KA appears to be able to be answered with RO knowledge. If you modify the question such that SRO-knowledge of procedure transitions is required, the question would likely be SAT.

The current question will not be counted as UNSAT because the KA is being replaced.

Question rewritten based on change to K/A.

JAT 07/16/2015 SR made the corrections annotated in red, above.

X U N The question appears to match the KA. The question does not appear to be at the SRO-only level. The question does not appear to meet the KA at the SRO-only level.

The first question meets the KA but can be answered with systems knowledgeplacing the rods in manual is an immediate action of AP-53.00, and knowing the system configuration that would cause rod movement is RO-level knowledge. The second question also meets the KA, but it is associated with a 1h or less TS required action, and is therefore RO-level knowledge. Additionally, the second question appears to be a direct lookup.

JAT 08/05/2015 Look at 2014 exam. What makes this SRO is the applicability of the 1h S

clock (what makes them have to go there).

Based on discussions with Surry, and the format of their TS, although there is a one-hour action associated with the number of channels less than the minimum, the question does require the applicant to make an operability call, and is therefore SRO-only. This question appears to be acceptable as-is.

JAT 08/06/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 80 H 2 N S 015G2.4.47 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

This question appears to be SAT.

JAT 07/16/2015 Question remains unchanged. JAT 08/05/2015 81 H 2 N S 022A2.04 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level (using procedures aspect of KA is met through TS).

This question appears to be SAT.

JAT 7/16/2015 Question remains unchanged. JAT 08/05/2015 82 H 2 X N U 024AA2.01 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA does NOT appear to be met at the SRO-only level.

The part of the question that meets the KA only requires systems knowledge to answer. Although the second part of the question is TS basis knowledge, it is not related to the KA of determining whether boron flow and/or MOVs are malfunction during emergency boration.

To meet the KA at the SRO-only level, recommend asking the appropriate procedural transitions to mitigate the failure. Changed Part 1) to allow closer match to K/A. Changed Part 2) of question to query Operability of required Boration paths per TS 3.2.

Additionally, does the third bullet point need to spell out Emergency (versus Emrg), or is that how 1-CH-FI-1110 is labelled? Spelled out emergency.

JAT 07/16/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only E SR made the corrections annotated in red, above.

The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

Need to ensure that a failure of 1-CH-FI-1110 could not also give the same indications.

Otherwise, question appears SAT. JAT 08/05/2015 S Tank level would be changing. No additional info needed. Question is SAT.

83 H 2 X N E/U 026A2.04 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

Recommend changing the first question to: The operator [is/is not]

required to locally open the supply breaker after placing A ORS pump in PTL. Otherwise, there is a subset issue because an applicant could use logic to eliminate two of the distractors rather than knowledge of the KA (Placing A ORS in PTL is a subset of Placing A ORS in PTL and opening the supply breaker.) Recommended changes to Part 10 of question made.

JAT 07/16/2015 SR made the corrections annotated in red, above.

S Question appears to be SAT. JAT 08/05/2015 84 H 2 N S 026AG2.2.44 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

This question appears to be SAT.

JAT 07/16/2015 Question remains unchanged. JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 85 F 2 N  ? 036AG2.2.37 This question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

This question appears to overlap with RO question 66, which also discusses a fuel handling incident, escaping bubbles, containment evacuation and whether normal or emergency ventilation is required. The correct answer to RO Q66 is that containment evacuation is required and that emergency ventilation is required. The way this question is asked (SRO Q85) implies what the correct answer to RO Q66 should be. For that reason, either RO Q66 requires replacement or this question requires replacement. Modified question 66 to eliminate overlap.

JAT 07/16/2015 SR made the corrections annotated in red, above.

X This question does not appear to overlap with question 66 (although there are similar stem conditions). This assessment was made because someone missing one of the questions would not necessarily miss the other due to the same lack of knowledge (double jeopardy), nor would knowing the answer to one (or eliminating a distractor from one) allow an applicant to eliminate a distractor from the other (i.e., the answer to one question cannot be used to eliminate a distractor from another).

This question appears to meet the KA. This question appears to be at the SRO-only level. The question does not appear to meet the KA at the SRO-only level. However, this was not indicated in the initial 401-9.

Recommend to put information in the stem of the question regarding the emergency ventilation, and then ask if there are enough operable trains to keep it SRO only and hit the KA.

JAT 08/05/2015 S

Question SAT. JAT 09/09/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 86 H 2 M E 054AG2.4.18 (pre)

Question appears to match the KA. Question appears to be at the SRO-only level. KA appears to be met at the SRO-only level.

• It is a little confusing to state that a loss of all feedwater occurs with the plant at 100%, and THEN state the initial conditions. I would call the events that are listed after the 100% power/LOFW current conditions, subsequent conditions, or given conditions (really, anything other than initial conditions).

• Part 1) question refers to this analysis, without specifying an analysis.

Recommend specifying which analysis the question is referring to.

Similarly, state what basis document you are referring to in the part 2) question: what is the basis, IAW with the FR-S.1 background document, for

• I will need to see the parent question in the draft submittal since this is listed a modified bank question.

JAT 2/4/2015 S

Question appears to match the KA. Question appears to be at the SRO-(pre2) only level. KA appears to be met at the SRO-only level. Comments have been addressed.

Based on parent question, question is significantly modified.

Question appears to be SAT.

JAT 2/13/15 Question remains SAT. JAT 07/16/2015 S

Question remains unchanged. JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 87 H 2 X N U 056AA2.75 The question appears to meet the KA. The question appears to be SRO-only. The KA does NOT appear to be met at the SRO-only level. The KA relates to determining and interpreting CVCS makeup/lineup during a LOOP. The part of the question that addresses the CH pump configuration can be answered with systems knowledge, and is therefore not SRO-only.

The second part of the question regarding the EAL call for a LOOP does not require any knowledge of what happened in the CVCS system, and therefore does not meet the KA at the SRO-only level.

To fix, I would focus on keeping the first half of the question (regarding the CH pump running after the EDG loads on 1J) and then ask a second question involving procedures (beyond EOP and AOP entry conditions) that are based on the situation in the stem of the question. This will help meet the KA at the SRO-only level. Rewrote stem of questions to focus Part 1). Changes Part 2) to query TS 3.01 applicability with CH configuration.

JAT 07/17/2015 SR made the corrections annotated in red, above.

While on the surface, the modification to the second question appears to be at the SRO-only level, it can be answered based on knowledge of a note in AP-10.07, and is therefore not SRO-only. Are there any other TS that would require entry or you could apply TS rules of usage to? Any transitions from AP-10.07?

JAT 08/05/2015 S

The modified question appears to meet the KA. The question appears to be at the SRO-only. The question appears to meet the KA at the SRO-only level. The updated question appears to be SAT.

JAT 08/06/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 88 H 2 N S/E 061AA2.03 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

Is the name of the ARP for annunciator 1-RMA-A1 also just 1-RMA-A1?

Does ARP need to be in the first question somewhere? Added annunciator noun name to Part 1) question.

Otherwise, question appears to be SAT.

JAT 07/16/2015 SR made the corrections annotated in red, above.

Question appears to be SAT. JAT 08/05/2015 89 F 2 B S 064G2.1.32 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

This question appears to be SAT.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. This question was not asked on either the 2014 or the 2012 NRC exams.

JAT 07/16/2015 Question remains unchanged. JAT 08/05/2015 90 H 2 M S 065AG2.4.9 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

Since this is a modified question, I need to see a copy of the original question for comparison.

The question appears to be SAT. This question was not asked on either the 2014 or the 2012 NRC exams.

JAT 07/16/2015 Question remains unchanged. JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 91 H 2 X N U 069AG2.4.21 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA does NOT appear to be met at the SRO-only level.

The part of the question that meets the KA requires the applicant to determine if the CSF is red or orange, which is RO-level knowledge. The part of the question that is at the SRO-level, the classification, does not meet the KA of parameters or logic used to assess the status of safety functions for a Loss of CTMT Integrity.

Assessing parameters and determining a yellow path is at the SRO-only level. You can modify the question to give parameters in the stem and have the applicant determine which procedure to enter (FR-Z.3 or FR-Z.4, which would require them to make a determination or a yellow path based on containment radiation or containment pressure). Changed Part

2) of question to query whether FR-Z.1 required re-entry following exit from procedure with steps completed.

JAT 07/17/2015 SR made the corrections annotated in red, above.

Is knowledge of the background document for FRPs (which is needed to answer the second question) SRO-only? EOP background and AP background were previously stated as RO level. It cannot be both ways. If it is SRO-only, then the question appears to be SAT. Otherwise, I would recommend addressing the question as suggested above.

JAT 08/05/2015 Question SAT, JAT 09/09/2015 S

92 H 2 M S 086A2.04 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

Since this is a modified question, I need to see a copy of the original question for comparison.

The question appears to be SAT. Original question was included in draft submittal.

JAT 07/17/2015 Question is significantly modified. Question remains unchanged.

JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 93 F 3 X X N U? G2.1.14 (pre)

Not sure if this question meets the KA - KA is knowledge of conditions/criteria that require plant announcements, rather than WHO makes the announcement.

Question appears to be at the SRO-only level.

• Partial:

o Why is it wrong for the unaffected Unit SRO to make the announcement? May be two correct answers.

• Part 2) needs to refer to the MAXIMUM amount of time allowed to avoid any potential subset issues.

• Is it likely any applicant will need to ask what tab HA-2.1 specifically refers to?

JAT 2/4/2015 U/E Question appears to match the KA. Question appears to be at the SRO-X only level. KA appears to be met at the SRO-only level. Comments have (pre2) been addressed.

Not sure how plausible A1/B1 are: why would an applicant think that station personnel are notified via beeper only when the alternative is a plant announcement, which can reach all employees, whether they have been issued a beeper or not? Could possible solve by asking the FIRST method used to contact plant personnel. If you go for this approach, ensure that you can justify a casualty wherein telephone notification occurs prior to a plant announcement.

JAT 2/13/2015 S Comments have been incorporated. Question appears to be SAT.

JAT 07/16/2015 Question remains unchanged. JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 94 F 3  ? N E G2.1.38 (pre)

Question appears to match the KA. Question appears to be at the SRO-only level. KA appears to be met at the SRO-only level.

• Recommend adding the attachment from which the need for open communication is cited (OP-AA-106, Att 3) so that it cannot be argued that open communication is similar to establishing the lines of communication steering the test, as specified in OP-AA-106, Attachment 2. Also - ensure this is not minutia.

• In question stem, the comma after the quotation the need for open communication is unnecessary.

• The (1) and (2) in the question stem are inconsistent with how other questions were formatted (previously, the stem had 1) or 2) to reflect how the choices were presented). It doesnt matter which way you choose to do them, I would just make them consistent throughout the test.

• Is there any chance the test coordinator could be the second line supervisor? Need to ensure this is not possible so as to avoid two correct answers.

JAT 2/4/2015 Question appears to match the KA. Question appears to be at the SRO-S only level. KA appears to be met at the SRO-only level. Comments have (pre2) been addressed.

Question appears to be SAT.

JAT 2/13/2015 S

Question remains SAT 07/16/2015 Question remains unchanged. JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only G2.2.1 95 H 2 X M U The question appears to meet the KA. The question appears to be at the SRO-only level. The KA does NOT appear to be met at the SRO-only level, in that the first half question requires knowledge of a P&L 1-OP-RX-006 (an operating procedure) which is RO level knowledge, and the second half question is a <1h TS action, which is also RO knowledge.

This KA is very broad, and can encompass any pre-startup procedure. It gives an example of systems that affect reactivity, but is not limited to those systems affecting reactivity. I would focus on asking about a pre-startup requirement contained an administrative procedure, which would be SRO-only knowledge.

Since this is a modified question, I need to see a copy of the original question for comparison. Replaced this question with question 78.

Modified Part 2 to specify minimum authority.

JAT 07/17/2015 S SR made the corrections annotated in red, above.

The replaced question appears to meet the KA. The question appears to be at the SRO-only level. The question appears to meet the KA at the SRO-only level.

The question appears to be SAT.

JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only G2.2.21 96 F 1 X N U (pre) Question appears to match the KA. Question appears to be at the SRO-only level. KA to be met at the SRO-only level.

• Capitalize Sat in the part 1) question to keep it consistent with the previous part of the question.

• Credible Distracters:

o C.1) and D.1) do not make sense rework must be performed Sat followed by PMT or PMT deferred. Why would an applicant choose that as an answer when the alternative is that engineering justifies that the failure does not affect operability? It would be more plausible to query on whether an engineering justification can/can not be provided to allow the diesel to be returned OPERABLE status if no rework is done.

o A.2) and C.2) seem implausible why would you waive PMT after testing is starting or complete? Also, this question doesnt appear to directly tie to the conditions above it regarding the EDG test.

Recommend asking a specific question about the administrative requirements for the failed EDG already referenced in the question (or about whether another EDG can be tested while the work for this EDG is complete but not test post-maintenance).

JAT 2/4/2015 Question appears to match the KA. Question appears to be at the SRO-only level. KA appears to be met at the SRO-only level. Comments have been addressed.

Ensure there is documentation for the second question that states that Operations has the final call on operability.

S Question appears to be SAT.

(pre2) JAT 2/13/2015 S This question remains SAT.

JAT 07/17/2015 Question remains unchanged. JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 97 F 2 X M U G2.3.11 (pre)

Question appears to match the KA. Question does NOT appear to be at the SRO-only level. KA does NOT appear to be met at the SRO-only level.

• Not SRO-Only o Querying on the response of the process monitor and the nuclide of concern in the gas storage tanks are both system-level/RO-level knowledge. Although the specific nuclide is in TS, it appears to be the equivalent of above-the-line information, in that the spec is The quantity of radioactivity contained in each gas storage tank shall be limited to less than or equal to 24,600 curies of noble gases (considered as Xe-133). This would make it RO knowledge.

o Similarly, the state the monitor stays in is systems knowledge.

However, if you ask the process by which the normal range monitor is returned to service, that could be SRO-only knowledge (i.e., query on whether the SRO can direct the return to service using instructions contained in procedure 0-RMA-C7 ONLY, or if I&C required).

o To hit the KA at the SRO-only level, you probably need to ask not only how to return the monitor to service, but also who has permission to authorize recommencing the release. The permission/authorization aspect would more directly hit the control part of the KA.

• I will need to see the parent question in the draft submittal since this is listed a modified bank question.

S/E? JAT 2/4/2015 (pre2)

Question appears to match the KA. Question appears to be at the SRO-only level. KA appears to be met at the SRO-only level. Comments have been addressed.

When does Attachment 2 of the WGDT procedure get used? Need to ensure that O2 is a plausible distractor.

Double check the distractor analysis - it looks like B2 might not have the correct justification (since it is correct, but the analysis says it is incorrect).

Question is modified, rather than new.

Assuming the plausibility of O2 as a distractor can be bolstered, this question appears to be SAT. JAT 2/13/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S Question modified to change first question and distractor.

(pre3) Question appears to be SAT.

JAT 4/2/2015 S

Question remains SAT. JAT 07/16/2015 Question remains unchanged. JAT 08/05/2015 98 H 2 B E G2.4.12 (pre)

Question appears to match the KA. Question appears to be at the SRO-only level. KA appears to be met at the SRO-only level.

Recommend a singular time for removal of the individual from the area.

Recommend changing either the dose rate or the time such that the minimum dose is closer to the 25 Rem threshold. You could also change this to a modified question if you made the time a maximum of 10 minutes and adjusted the dose rate so that the dose received was less than 25 Rem (changing the question so that a distractor becomes the answer is considered significantly modified).

If this question remains a Bank question, I will need to know which NRC exam (if any) it was previously used on.

E JAT 2/4/2015 (pre2) Question appears to match the KA. Question appears to be at the SRO-only level. KA appears to be met at the SRO-only level. Comments have X been addressed.

In order to ensure there is only one correct answer, change the stem to state a maximum of 10 minutes rather than at least 10 minutes.

Leaving it at least 10 minutes allows someone to argue that there is the possibility that the total dose could exceed 25 Rem and therefore volunteers are required.

Question is modified, rather than new.

Question will be SAT with the one minor modification.

JAT 2/13/2015 Comment has been incorporated. Question appears to be SAT.

S JAT 07/17/2015 Question remains unchanged. JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 99 H 2 X M E G2.4.23 The question appears to meet the KA. The question appears to be at the SRO-only level. The question appears to meet the KA at the SRO-only level, in that the applicant must use knowledge of the bases for prioritizing emergency procedure response when choosing the appropriate procedural flowpath.

Recommend rewording distractors A and B to: Go directly to to avoid any possible subset issues, since at least 1-ES-0.3 will be entered eventually. Adding the word directly will eliminate any potential subset issues. Recommended Modifications made, added directly to to distractors A and B.

Since this is a modified question, I need to see a copy of the original question for comparison. Original question was included with draft submittal, and is in the reference section of the question.

With recommended modification, question appears to be SAT.

JAT 07/17/2015 SR made the corrections annotated in red, above.

S There are some modifications to the distractors and at least one difference in stem conditions. The answer is the same, but I would consider this modified.

Question appears to be SAT.

JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 100 H 2 X N U/E WE05EA2.1 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

It does not seem plausible that an applicant would choose the answer choice that enters FR-H.1 after transitioning through both E-2 and E-2.1. Is there a point in E-2 where the applicant would NOT make a transition to FR-H.1 and instead would remain in E-2? (Two non-plausible distractors, but for the same reason.) Additionally, the first question implies that the appropriate procedure to be in is FR-H.1.

You may be able to meet the KA and have plausible distractors if you added information to the stem stating that the SRO announced entry into E-2 and has gotten to the point where he has read step 2, and then state the STA reports a red-path on Heat Sink. Then the question can ask, which procedure will they transition to from FR-H.1: E-2 or ECA-2.1. Since step 2 was started but not completed, the correct answer will be E-2 (step 2 is where the transition to ECA-2.1 occurs when the step cannot be completed). You will need to ensure that there would only be one correct answer if you pursue this path. Changes made as recommended to changes in Part 2), changes made, to Stem and distractors.

JAT 07/17/2015 S SR made the corrections annotated in red, above.

The question appears to be SAT.

JAT 08/05/2015

Final Sample Plan (ES-401-2/3)

The final sample plan is the combination of the draft sample plan and the Record of Rejected K/As (ES-401-4)

ES-403 Rev 10 Written Examination Grading Form ES-403-1 Quality Checklist Facility: Sç 2OOi Date of Exam11291 Exam Level: RO SRO Item Description Initials a b c

1. Clean answer sheets copied before grading .

2. Answer key changes and question deletions justified and documented

3. Applicants scores checked for addition errors (v ,

(reviewers spot check> 25% of examinations)

4. Grading for all borderline cases (80 +/-2% overall and 70 or 80, as applicable, +/-4% on the SRO-only) reviewed in detail J)

5. All other failing examinations checked to ensure that grades (9.i) are justified

6. Performance on missed questions checked for training deficiencies and wording problems; evaluate validity of questions missed by half or more of the applicants Printed Name/Signature Date

a. Grader M.PoNrcP

b. Facility Reviewer(*) il)Pc

c. NRC Chief Examiner (*) Thi /

d. NRC Supeisor(*) 10! lj

(*) The facility reviewers signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.

i

ES-201 Examination Preparation Checklist Form ES-201-1 Facility: Surry 2015-301 Date of Examination: 09/21/2015 and 10/06/2015 Developed by: Written: Facility NRC // Operating Facility NRC Chief Target Task Description (Reference) Examiners Date*

Initials

-180 1. Examination administration date confirmed (C.1.a; C.2.a and b) JAT 01/06/2015

-150 2. NRC examiners and facility contact assigned (C.1.d; C.2.e) JAT 01/06/2015

-150 3. Facility contact briefed on security and other requirements (C.2.c) JAT 03/30/2015

-150 4. Corporate notification letter sent (C.2.d) JAT 04/02/2015

[-120] 5. Reference material due (C.1.e; C.3.c; Attachment 3) JAT 05/08/2015

{-90} 6. Integrated examination outline(s) due, including Forms ES-201-2, ES-201-3, ES-301-1, ES-301-2, ES-301-5, ES-D-1, ES-401-1/2, ES-401N-1/2, ES-401-3, ES-401N-3, JAT 05/22/2015 ES-401-4, and ES-401N-4, as applicable (C.1.e and f; C.3.d)

{-85} 7. Examination outline(s) reviewed by NRC and feedback provided to facility licensee JAT 06/01/2015 (C.2.h; C.3.e)

{-60} 8. Proposed examinations (including written, walk-through JPMs, and scenarios, as applicable), supporting documentation (including Forms ES-301-3, ES-301-4, ES-301-5, ES-301-6, and ES-401-6, ES-401N-6, and any Form ES-201-2, ES-201-3, JAT 06/26/2015 ES-301-1, or ES-301-2 updates), and reference materials due (C.1.e, f, g and h; C.3.d)

-45 9. Written exam and operating test reviews completed. (C.3.f) JAT 07/17/2015

-30 10. Preliminary license applications (NRC Form 398's) due (C.1.l; C.2.g; ES-202) JAT 08/21/2015

-21 11. Examination approved by NRC supervisor for facility licensee review (C.2.h; C.3.f) JAT08/21/2015

-21 12. Examinations reviewed with facility licensee (C.1.j; C.2.f and h; C.3.g) JAT 08/24/2015

-14 13. Final license applications due and Form ES-201-4 prepared (C.1.l; C.2.i; ES-202) JAT 09/07/2015

-14 14. Written examinations and operating tests approved by NRC supervisor (C.2.i; JAT 09/16/2015 C.3.h)

-7 15. Facility licensee management queried regarding the licensees views on the JAT 09/16/2015 examination. (C.2 j)

-7 16. Final applications reviewed; 1 or 2 (if >10) applications audited to confirm qualifications / eligibility; and examination approval and waiver letters sent (C.2.i; JAT 08/24/2015 Attachment 5; ES-202, C.2.e; ES-204)

-7 17. Proctoring/written exam administration guidelines reviewed with facility licensee JAT 08/27/2015 (C.3.k)

-7 18. Approved scenarios, job performance measures, and questions distributed to NRC JAT09/16/2015 examiners (C.3.i)

• Target dates are generally based on facility-prepared examinations and are keyed to the examination date identified in the corporate notification letter. They are for planning purposes and may be adjusted on a case-by-case basis in coordination with the facility licensee.

[Applies only] {Does not apply} to examinations prepared by the NRC.

ES-201 Examination Outline Quality Checklist Form ES-201-.2 Facility: Surry Power Station Date of Examination: 9/21/2015

. Initials Item Taak Deacripton a b c#

I. a. Verify that the outline(s) fit(s) the appropriate model! in accordance with ES-401 or ES-401 N.

F b. Assess whether the outline was systematically and randomly prepared in accordance with

. Section D.1 of ES401 or ES-401N and whether all K/A categories are appropriately sampled. 1 t

c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics. U?

N

d. Assess whether the justifications for deaelected or rejected K/A statements are appropriate.

2 a. Using Form ES-301-5, verify that the proposed scenario sets cover the required number of normal evolutions, instrument and component failures, technical specifications, and major S transients.

M b. Assess whether there are enough scenario sets (and spares) to test the projected number and U mix of applicants in accordance with the expected crew composition and rotation schedule L without compromising exam integrity, and ensure that each applicant can be tested using at A least one new or significantly modified scenario, that no scenarios are duplicated from the T appticants audit teat(s), and that scenarios will not be repeated on subsequent days.

c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative and R

quantitative criteria specified on Form ES-301-4 and described in Appendix D. 4

3. a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2:

(1) the outline(s) contain(s) the required number of control room and in-plant tasks distributed W among the safety functions as specified on the form A (2) task repetition from the lest two NRC examinations is within the limits spocitied on the form L (3) no tasks are duplicated from the applicants audit test(s)

K (4) the number of new or modified tasks meets or exceeds the minimums specified on the form (5) the number of altemate path, low-power, emergency, end RCA teaks meet the criteria on T the form.

H R b. Verify that the administrative outline meets the criteria specified on Form ES-301 -1:

o (1) the tasks are diatributed among the topics as specified on the form u (2) at least one task is new or significantly modified 7t Cc?

C (3) no more than one task is repeated from the last two NRC licensing examinations H c. Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days. It

4. a. Assess whether plant-specific priorities (including PRA and IPE inaighta) are covered in the appropriate exam sections. /

b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate. t?f

c. Ensure that K/A importance ratings (except for plant-specific priorities) are at least 2.5. (2?

R d. Check for duplication and overiap among exam sections. 2 9-2 cY?l L e. Check the entire exam for balance of coverage. rZ!i

f. Assess whether the exam fita the appropriate job level (RO or SRO). 7 ft9 f4

s. Author

b. Facility Reviewer (j)

c. NRC Chief Examiner (#)

d. NRC Supervisor Note: 4 Independent NRC reviewer initial items in Column Pc; chief examiner concurrence required.

Not appliceble for NRC-prepared examination outlines.

ES-301 Administrative Topics Outline Form ES-301-1 Facility: Surry Date of Examination: 9/21/15 Examination Level: RO SRO Operating Test Number: SR 15-301 Administrative Topic (see Note) Type Describe activity to be performed Code*

Complete Reactivity Summary Sheet Conduct of Operations K/A: G2.1.37 Knowledge of procedures, guidelines, or R/D limitations associated with reactivity management.

RO: 4.3/SRO: 4.7 Calculate Primary to Secondary leakage and determine ramp rate in accordance with 0-OSP-Conduct of Operations R/N RC-002 K/A: G2.1.23 Ability to perform specific system and integrated plant procedure during all modes of plant operation. RO: 4.3/SRO: 4.4 Equipment Control Calculate stay time Radiation Control K/A: G2.3.4 Knowledge of radiation exposure limits under R/M normal or emergency conditions. RO: 3.2/SRO: 3.7 Complete EPIP-2.01 for Alert, and Transmit Emergency Plan K/A: G2.4.39 Knowledge of RO responsibilities in 5/D emergency plan implementation. RO: 3.9/SRO: 3.8 NOTE: All items (five total) are required for SROs. RO applicants require only four items unless they are retaking only the administrative topics (which would require all five items).

Type Codes & Criteria: (C)ontrol room, (S)imulator, or Class(R)oom (D)irect from bank ( 3 for ROs; 4 for SROs & RO retakes)

(N)ew or (M)odified from bank ( 1)

(P)revious 2 exams ( 1; randomly selected)

ES-301 Administrative Topics Outline Form ES-301-1 Facility: Surry Date of Examination: 9/21/15 Examination Level: RO SRO Operating Test Number: SR 1 5-301 Administrative Topic (see Note) Type Describe activity to be performed Code*

Complete Reactivity Summary Sheet K/A: G2.1.37 Knowledge of procedures, guidelines, or Conduct of Operations R/D limitations associated with reactivity management.

RO: 4.3/SRO: 4.7 Calculate Primary to Secondary leakage and determine ramp rate in accordance with 0-OSP-Conduct of Operations R/M RC-002 K/A: G2.1.23 Ability to perform specific system and integrated plant procedure during all modes of plant operation. RO: 4.3/SRO: 4.4 Review 1 -OPT-C H-002 K/A: G2.2.37 Ability to determine operability and/or Equipment Control R/D availability of safety related equipment.

Calculate stay time Radiation Control KJA: G2.3.4 Knowledge of radiation exposure limits under R/M normal or emergency conditions. RO: 3.2/SRO: 3.7 Classify EAL and Approve Notification Emergency Plan R/M K/A: 2.4.41 Knowledge of the emergency action level thresholds and classifications. RO: 2.9 / SRO: 4.6 NOTE: All items (five total) are required for SROs. RO applicants require only four items unless they are retaking only the administrative topics (which would require all five items).

Type Codes & Criteria: (C)ontrol room, (S)imulator, or Class(R)oom (D)irect from bank ( 3 for ROs; 4 for SROs & RO retakes)

(N)ew or (M)odified from bank ( 1)

(P)revious 2 exams ( 1; randomly selected)

ES-301 Control Roomlln-Plant Systems Outline Form ES-301-2 Facility: Surrv Date of Examination: 9/21/15 Exam Level: RO SRO-l SRO-U Operating Test No.: SR 15-301 Control Room Systems:8 for RO; 7 for SRO-I; 2 or 3 for SRO-U System / JPM Title Safety Type Code Function

a. Perform AP-3.00 to Emergency Borate the RCS During ES-0.1 (Faulted) [ 024AA1.17 (3.9/3.9)] N/S/A/L

b. Makeup to RWST from Blender (pre-brief) 1SYS004A2.13 (3.6/3.9)] N/S 2

c. Respond to spurious Containment Spray Actuation. [EPEE02EA1.3 D/S/EN 3 (3.8/4.0)]

d. Initiate RCS Feed and Bleed [WE05EA1.1 (4.1/4.0)] M/S/A/L 4P

e. Respond to Secondary Transient [SYSO16A2.01, (3.0/3.1)] M/S/A 7

f. Manually start Cont. Spray pumps per Attachment 1 (WE14EA1.1 M/S/EN/L 5 (3.7/3.7)]

g. Waste Gas Tank Release (Faulted) [APE06OAA2.05 (3.7/4.2)] D/S 6

h. Align the Emergency Bus in accordance with AP-10.07 9 Attachment 4 [APEO56AA2.49 (3.0 / 3.4)] N/S/A In-Plant System& (3 for RO); (3 for SRO-I); (3 or 2 for SRO-U)

i. Manually makeup to SFP [5YS033A2.03 (3.1/3.5)] D/E/R 8

j. Locally trip the Unit 2 Reactor Trip Breakers [EPEO29.EA1.12 (4.1/4.0] N/E/L 1

k. Transfer Semi-vital bus power supply [APEO56AA2.44 94.3/4.4))) D/E 6 AM RO and SRO-l control room (and in-plant) systems must be different and serve different safety functions; aM five SRO-U systems must serve different safety functions; in-plant systems and functions may overlap those tested in the control room.

Type Codes

} Criteria for RO I Actual A)lternate path 4-6 I 4 (C)ontrol room (D)irectfrom bank 9 I 4 (E)mergency or abnormal in-plant 1 I 3 (EN)gineered safety feature 1 I 2 (control room system)

(L)ow-Power I Shutdown 1 I 4 (N)ew or (M)odifled from bank including I (A) 2 / 7 (P)revious 2 exams 3 I 0 (randomly selected)

(R)CA 1 / 1 (S)imulator

ES-301 Control Roomlln-Plant Systems Outline Form ES-301-2 Facility: Surry Date of Examination: 9/21/15 Exam Level: RO SRO-l SRO-U Operating Test No.: SR 15-301 Control Room Systems:*8 for RO; 7 for SRO-l; 2 or 3 for SRO-L3 System I JPM Title Type Code* Safety Function

a. Perform AP-3.00 to Emergency Borate the RCS During ES-0.1 (Faulted) [KA: 024AA1.17 39/39] N/S/A/L

b. Makeup to RWST from Blender (pre-brief) ISYSOO4A2.13 (3.6/3.9)] N/S 2

c. Respond to spurious Containment Spray Actuation. [EPEEO2EA1.3 D/S/EN 3 (3.8/4.0)]

d. Initiate RCS Feed and Bleed [WE05EA1.1 4.1/4.0] M/S/A/L 4

e. Respond to Secondary Transient [SYS016A2.O1, (3.0/3.1)1 M/S/A 7

f. Manually start Cont. Spray pumps per Attachment 1 [WE14EA1.1 M/S/EN/L 5 (3.7/3.7)]

g. Waste Gas Tank Release (Faulted) [APEO6OAA2.05 (3.7/4.2)] N/S/A 9 h.

In-Plant Systems (3 for RO); (3 for SRO-l); (3 or 2 for SRO-U)

i. Manually makeup to SFP SYS033A2.03 (3.1 / 3.5)] D/E/R 8

j. Locally trip the Unit 2 Reactor Trip Breakers [EPEO29.EA1.12 (4.1/4.0] N/E/L 1

k. Transfer Semi-vital bus power supply [APEO56AA2.44 94.3/4.4))) D/E 6 All RO and SRO-l control room (and in-plant) systems must be different and serve different safety functions; all five SRO-U systems must serve different safety functions; in-plant systems and functions may overlap those tested in the control room.

A)lternate path Type Codes

] Criteria for SRO-l / Actual 4-6 I 4 (C)ontrol room (D)irect from bank 8 I 3 (E)mergency or abnormal in-plant 1 I 3 (EN)gineered safety feature I I 2 (control room system)

(L)ow-Power I Shutdown 1 I 4 (N)ew or (M)odified from bank including 1 (A) 2 I 7 (P)revious 2 exams 3 / 0 (randomly selected)

(R)CA l I 1 (S)imulator

ES-301 Control Roomlln-Plant Systems Outline Form ES-301-2 Facility: Surry Date of Examination: 9/21/15 Exam Level: RO SRO-l SRO-U Operating Test No.: SR 15-301 Control Room Systems:*8 for RO; 7 for SRO-l; 2 or 3 for SRO-L System / JPM Title Type Code* Safety Function

a. Perform AP-3.00 to Emergency Borate the RCS During ES-0.1 N/s/A/L (Faulted) (pre-brief) [KA: 024AA1.17 3.9/3.9]

b. Makeup to RWST from Blender (pre-brief) [SYS004A2.13 (3.6/3.9)] D/S/A/L 2

c. Respond to spurious Containment Spray Actuation. [EPEEO2EA1.3 D/S/EN 3 (3.8/4.0)]

d.

e.

f.

g.

h.

In-Plant Systems* (3 for RO); (3 for SRO-l); (3 or 2 for SRO-U)

i. Manually makeup to SFP [SYSO33A2.03 (3.1 / 3.5)] D/E/R 8

j. Locally trip the Unit 2 Reactor Trip Breakers [EPEO29.EA1.12 (4.1/4.0] N/E/L 1 k.

All RO and SRO-l control room (and in-plant) systems must be different and serve different safety functions; all five SRO-U systems must serve different safety functions; in-plant systems and functions may overlap those tested in the control room.

A)lternate path Type Codes

] Criteria for SRO-U / Actual 2-3 / 2 (C)ontrol room (D)irect from bank 4 / 3 (E)mergency or abnormal in-plant 1 / 2 (EN)gineered safety feature 1 / 1 (control room system)

(L)ow-Power / Shutdown 1 I 3 (N)ew or (M)odified from bank including 1 (A) 1 I 2 (P)revious 2 exams 2 / 0 (randomly selected)

(R)CA 1 / 1 (S)imulator

ES-301 Operating Test Quality Checklist Form ES-301-3 Facility: Surry Date of Examination: 9/21/2015 Operating Test Numben SR3O1-2015

, Initials

1. General Criteria a b* 4

a. The operating test conforms with the previously approved outline; changes are consistent with sampling requirements (e.g., 10 CFR 55.45, operational importance, safety function distribution).

b. There Is no day-to-day repetition between this and other operating tests to be adminIstered during this examination.

c. The operating test shall not duplicate items from the applicants audit test(s). (see Section D. I a.) 1

d. Overlap with the written examination and between different parts of the operating test is within -

acceptable limits.

e. It appears that the operating test will differentiate between competent and less-than-competent y.. .411 applicants at the desIgnated license level. =

2. Walk-Through Criteria

a. Each .JPM includes the following, as applicable:

initial conditions

• initiating cues

• references and tools, including associated procedures

• reasonable and valIdated time limits (average time allowed for completion) and specific designation if deemed to be time-critical by the facility licensee operationally important specific performance criteria that include:

detailed expected actions with exact criteria and nomenclature system response and other examiner cues statements describing important observations to be made by the applicant criteria for successful completion of the task identification of critical steps and their associated performance standards restrictions on the sequence of steps, if applicable

b. Ensure that any changes from the previously approved systems and administrative walk-through ouflines (Forms ES-301-1 and 2) have not caused the test to deviate from any of the acceptance v criteria (eg. item distribution bank use, repetition from the last 2 NRC examinations) specified on those forms and Form ES-201-2.

3. SImulator Criteria The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES- ,.- Y 4

301-4 and a copy is attached.

Printed Name / Signature Date

a. Author Paul Orrison / / . -

b. Facility Reviewer(*) MIchael R. Meyer!

c. NRC Chief Examiner (#) Amanda Toth / j )

d. NRC Supervisor C

(

NOTE: The facility signature is not applicable for NRC-developed tests.

1. Independent NRC reviewer initial items in Column c; chief examiner concurrence required.

ES-301 Simulator Scenario Quality Checklist Form ES-301-4 Facility: Surry Date of Exam: 9/21/15 Scenario Numbers: 1/2/3/4 Operating Test No.SR3O1-2015 QUALITATIVE ATTRIBUTES Initials a b* 4 1, The initial conditions are realistic, in that some equipment and/or instrumentation may be out of service, but it does not cue the operators into expected events.

2. The scenarios consist mostly of related events. I-.

3. Each event description consists of

. the point in the scenario when it is to be initiated

. the malfunction(s) or conditions that are entered to initiate the event

. the symptoms/cues that will be visible to the crew

. the expected operator actions (by shift position)

. the event termination point (if applicable)

4. The events are valid with regard to physics and thermodynamics. t.

5. Sequencing and timing of events is reasonable, and allows the examination team to obtain complete evaluation results commensurate with the scenario objectives. I

6. If time compression techniques are used, the scenario summary clearly so indicates.

Operators have sufficient time to carry out expected activities without undue time constraints. /

Cues are given.

7. The simulator modeling is not altered.

8. The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator performance deficiencies or deviations from the referenced plant have been evaluated to ensure that functional .V- I fidelity is maintained while running the planned scenarios.

9. Every operator will be evaluated using at least one new or significantly modified scenario. All other ._ -vi scenarios have been altered in accordance with Section D.5 of ES-301.

10, All individual operator competencies can be evaluated, as verified using Form ES-301-6 (submit the ,

form along with the simulator scenarios).

11. The scenario set provides the opportunity for each applicant to be evaluated in each of the applicable rating factors. (Competency Rating factors as described on forms ES-303-1 and ES-303-3.)

12. Each applicant will be significantly involved in the minimum number of transients and events specified pS.

4V7 p on Form ES-301 -5 (submit the form with the simulator scenarios).

13. The level of difficulty is appropriate to support licensing decisions for each crew position.

Target Quantitative Attributes {Per Scenario; See Section D.S.d) Actual Attributes -- --

1. Malfunctions after EOP entry (1-2) 2/1/1/2 1

2. Abnormal events (2-4) 5/4/4/3 j

3. Major transients (12) 2/11111 tA.. W7

4. EOPs entered/requiring substantive actions (12) 2/3/2/2

5. EOP contingencies requiring substantive actions (0-2) 1/1/1/1 *N1

6. EOP based Critical tasks (23) 4/3/3/3 CL NOTE:

• The facility signature is not applicable for NRC-developed tests.

1. Independent NRC reviewer initial items in Column c; chief examiner concurrence required.

ES-301 Transient and Event Checklist Form ES-301-5 Facility: Surry Date of Exam: Operating Test No.: SR 15-301 A E Scenarios 1 2 3 4 T M 0 I L N CREW CREW POSITION CREW POSITION CREW POSITION I T POSfl1ON T C A S A B S A B S A B S A B M R T 0 R T 0 R T 0 R T o L U 0 C P 0 C P 0 C P 0 C P M(*)

T P SRO-I I/C 2 3 2 2 2,3,442 L__i 2,2 SRO-U MAJ 1 2 1 1 1,2, 2 2 1 D TS 0 0 0 0 0022 RO RX 0 0 0 0 0110 NOR 1 2 22 SRO-I I/C 333i3,442 D 3,1 MAJ 1 2 1 1 1,2, 2 2 1 SRO-U 1,1 D TS 0 0 0 0 0022 RO RX 1 1 1 1 1,1, 1 1 0 D

NOR 0 1 1 1 0,1, 1 1 1 SRO-I 1,1

• I/C 5 6 5 4 5,6,442 5,4 SRO-U MAJ 12 11t2,221 D

1,1, TS 1 3 3 2 1,3, 0 2 2 3,2 RO RX 1 1 1 1 1,1, 1 1 0 D

NOR 0 1 1 1 0,1, 1 1 1 SRO-I 1,1 SROU D I/C 5 6 5 ti::zzzzz 4 5,6,442

ELEZI

ES-301 Competencies Checklist Form ES-301-6 Facility: Surry Date of Examination: 9/21/15 Operating Test No.: SR3O1-2015 APPLICANTS RO RO El RO El SRO-I El SRO-l SRO-I El SRO-U El SRO-U El SRO-U Competencies SCENARIO 1 2 34 1 2 34 1 2 3 4 Interpret/Diagnose 1,2,3, 13,4, 2,3, 23, 1,2,3, 1,3,4, 2,3, 2,3, 1,2,3, 1,3,4, 2,3,4 2,3,

. 4,6,7, 5,6 4,5, 4,5, 4,6,7, 5,6 4,5, 4,5, 4,6,7, 5,6 5,6, 4,5, Events and Conditions 8 6,7 8 6,7 8 7 Comply With and ALL ALL ALL ALL ALL ALL ALL ALL ALL ALL ALL ALL Use Procedures (1)

Operate Control ALL ALL ALL ALL Boards (2)

Communicate ALL ALL ALL ALL ALL ALL ALL ALL ALL ALL ALL ALL and Interact Demonstrate ALL ALL ALL ALL ALL ALL ALL ALL Supervisory Ability (3)

Comply With and 2 1,3 2,3, 3,4 2 1,3 2,3,4 3,4 Use Tech. Specs. (3)

Notes:

(1) Includes Technical Specification compliance for an RO.

(2) Optional for an SRO-U.

(3) Only applicable to SROs.

Instructions:

Check the applicants license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant. (This includes all rating factors for each competency.) (Competency Rating factors as described on forms ES-303-1 and ES-303-3.)

ES-401 Record of Rejected KIAs Form ES-401-4 Tier / Randomly Reason for Rejection Grour Selected K/A 2/1 RO 059A4.0l Surry Power station does not have this type of equipment.

Replace vi1h 05QA4.08.

2/2 RO 027A4.04 Surly Power station does not have indication of this parameter.

Replace v ith 027A4.0 I 2/2 RO 029G2.4.3 No tie or link between Containment purge and post-accident instrumentation.

Replace with 029G2. 1 .25.

1/1 SRO 065AG2.4.3 No tie or link between Loss of instrument air and post-accident instrumentation.

Replace with 065AG2.4.9.

2/I SRO 012G2.4.1 K/A does not lend itself to an SRO question Rerlace with 01 2G2. 1 .23.

ES-401 Written Examination Quality Checklist Form ES-401-6 Facility: Surry Power Station Date of Exam: 9/21/2015 Exam Level: RO SRO Initial Item Description a b c*#

1. Questions and answers are technically accurate and applicable to the facility. ..

2. a. NRC K/As are referenced for all questions.

b. Facility learning objectives are referenced as available.

3. SRO questions are approprIate In accordance with Section D.2.d of ES-401 .._ A) 4 4 The sampling process was random and systematic (If more than 4 RD or2 SRO questions were repealed from the last two NRC licensing exams, consult the NRRJNRO DL. program attica). NOTE 1

5. Question duplication from the licensee screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate The audit exam was systematically arid randomly developed; or the audit exam was completed before the license exam was started: or X the examinations were developed independently; or the licensee certifies that there Is no duplication; or other (explain)

6. Bank use meets limits (no more than 75 percent from the Bank Modified New bank, at least 10 percent new, and the rest new or modified), enter the actual RD / SRO-onty question 11)8 21/28 68)64 distribution(s) at right

7. Between 50 and 60 percent of the questions on the RD Memory C/A exam are written at the comprehension] analysis level: the SRQ exam may exceed 80 percent if the randomly selected K/As support the higher cognitive levels: enter 42/20 58/80 the actaal RO / SRO question distribution(s) at right.

8. ReferencesThandouts provided do not give away answers or aid in the elimination of yL.

distractors.

9. Question content conforms to specific K/A statements In the previously approved examination outtine and_Is_appropriate_for_the tier to which_they_are_assigned:_deviations_are justified.

10. Question psychometric quality arid format meet the guidelines in ES Appendix 8.

11. The exam contains the required number of one-point, multiple choice items: the total is correct and agrees with the value on lhe cover sheet.

Printed Name) Signature Date

a. Aathor Paul Orrison/ (of/(c__ 9)3/15

b. Facility Reviewer (*) Michael R Meyer) 5_

c. NRC Chief Examtner(#) Amanda Toth/ . 9 II

d. NRC RegionalSupervisor (x ttt*.

(

Note: The facility reviewers initials or signature are not applicable for NRC-developed examinations.

1. Independent NRC reviewer initials items in Column c; chief examiner concurrence required, Note 1: Sample Plan provided by Chief Examiner Amanda Toth

ES-401 Surry Early Review - Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

• The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

• The answer choices are a collection of unrelated true/false statements.

• The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

• One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

• The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

• The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) B/M/N U/E/S Explanation Key/Summary S Indicates SAT: 15 SRO, 33 RO (initial submittal) 25 SRO, 75 RO SAT in final submittal Question distribution:

RO SRO E Indicates Enhancement: 1 SRO, 27 RO (initial submittal)

Bank 13 initial 3 initial U Indicates UNSAT: 5 SRO, 12 RO (initial submittal)

Modified 16 initial 16 initial U/E Indicates UNSAT/ENHANCEMENT (likely due to two non-plausible New 46 initial 6 initial distractors that are the same, as in a 2x2, i.e., the same distractor counted 2x as non-plausible) 3 SRO, 3 RO (initial submittal) (1 SRO KA replacement in initial submittal)

(pre) Indicates presubmitted question comments

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 2 X N E 001AA1.02 Question appears to meet the KA.

Are there any competing factors during the power change that could change the answer? (i.e., rods moved a different number of steps because of normal rod motion) What is the Tave/Tref difference before the failure occurs? Tave/Tref mismatch is 0 before failure. Added Tave for 100%

power in IC. No other competing factors. You should give them a starting point for the difference prior to the failure so that there is no basis for two correct answers or no correct answers. Divided Initial and Current conditions to clarify starting point.

Recommend adding the word immediately to the first question to ask, the direction of the rod motion immediately before the rods were placed in manual to tighten up the answer possibilities. Added immediately to first question.

For the second question - need to make sure the question is clear enough.

Original in this case means, immediately prior to the Tave failure.

Would it be better to just ask that? Changed ORIGINAL to previous and added (prior to Tave failure).

Im a little bit confused on the math - should the equation be: [8.0 steps/min +(32 steps/min-F)(1.0F)]*[0.5min] ? When I first read the explanation, it looked like you would end up with 16+8 or 16. The explanation originally included basis for correct answer and distractor.

Deleted distractor basis from explanation and added to Distractor comments.

JAT 06/26/2015 S SR made the corrections annotated in red, above.

Updated question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 2 H 2 N E 001K6.03 Question appears to meet the KA.

Can the question be answered without giving the information but only the B RPS train actuates in the stem? I.e., if A trip breaker is undergoing testing, would only the B; train be expected to actuate? If so, then the statement that only B actuates is not necessary, and the applicant should be able to determine whether the picture accurately reflects B RPS train actuating (which it does not). Changed 3rd bullet to An automatir reactor trip signal occurs.

The second half of the question almost seems to be a tack-on, and very easy. It would make sense that as a back up to an undervoltage coil, that something would energize. Recommend an alternative question (i.e.,

possibly what actions an operator would be required to take first with respect to the breaker under test). Changed second question as denoted.

JAT 06/26/2015 SR made the corrections annotated in red, above.

S The second question needs a comma added: under test, the Updated question appears to be SAT.

JAT 08/04/2015 3 H 2 X N U 003K4.04 (pre)

Question does not appear to match the KA. Question queries on the impact of a loss of IA on CC valves to the RCP. The CC valves are not specifically a design feature of the pump.

Question appears to be at the RO level.

JAT 2/4/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only E

(pre2) Newly submitted question appears to match the KA, in that the candidate must have knowledge of the RCP precaution on limiting the number of starts of the RCP in a given time frame due to motor cooling concerns.

Addtionally, the applicant must have knowledge of the precaution on which cooling flow to reestablish first, which is based on the design of the RCP (restoring the wrong subsystem first has the potential to warp the pump shaft due to the injection point). Comments have been addressed.

Question appears to be at the RO level.

To remove all ambiguity from the question, I would recommend stating that three starts have been attempted and query on whether an additional start is allowed. Even better would be to give a timeline to show that the starts had been attempted IAW with the precaution statement. For example: At time 1000 RCP A was attempted to be started and did not start. At time 1045, RCP A was again attempted to be started and did not start. At time 1115, RCP A was again attempted to be started and did not start.

At time 1200 [or 1220, depending on what you want the right answer to be], a start attempt on RCP A _[may/may not]__ be made.

[OR what is the earliest time RCP A may be attempted to be restarted].

The second question is fine, but it seems somewhat unrelated to the first part of the question. If you set up the question in the timeline fashion above, you can state that the pump is being restarted after the crew has recovered from a loss of all RCP seal cooling. You can ask which subsystem they restored first IAW 1-OP-RC-001, and then ask the question about whether the fourth start is/is not allow IAW 1-OP-RC-001.

Ensure that a loss of all seal cooling means a loss of CC Water and Seal Injection Flow. Or you can just state those two independently.

JAT 2/13/2015 Newly modified question incorporates the above suggestions. Only E recommendation is to use the phrase earliest time when asking when (pre3) the pump can be started so as to not be able to eliminate a distractor based on logic (i.e., if 1058 is correct, then 1128 would also be correct if

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only the question asked is when can the pump be started - if it can be started S

at 1058 it can also be started at 1128. Since there is only one correct answer, 1058 cannot be the correct answer.)

JAT 4/2/2105 Officially submitted question incorporates recommendation. Question appears to be SAT.

JAT 06/26/2015 No changes made. JAT 08/04/2014 4 F 2 X N E? 004K4.03 Question appears to match the KA.

For the second part of the question, Im not sure of the likelihood of someone choosing positive reactivity addition due to boron absorption (since the higher temperature doesnt cause boron to be removed from the coolant) as an answer when also presented with resin damage due to high letdown temperature. Another place to go on a second question may be an operator action that is taken or (RO-level) conditions that must be met to restore flow to the IX. Changed distractors B2 and D2 to make more plausible.

JAT 06/26/2015 SR made the corrections annotated in red, above.

S Updated distractor states what will physically happen when the temperature goes up, and states that the action is required to mitigate negative reactivity from release of boron. From a reactor safety standpoint, it seems unlikely someone would choose this distractor.

However, the question places the applicant in a situation where the reactor is stable at 100% operation, so there is at least some chance that someone might choose this to prevent a transient on the unit.

Updated question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 5 F 2 X N E/U 005K2.03 Question appears to match the KA.

Is there more than one open interlock for that valve? If not, the 460 psig needs to be removed so as to not teach in the stem. There is only one interlock, comment incorporated. It would be fine to refer to it as a minimum pressure or maximum pressure open interlock (or something similar). Referred as maximum pressure.

Is there another pressure transmitter that would be more plausible than the PRZR pressure control transmitter? The name PRZR pressure control transmitter implies that the range of pressures it covers is nowhere near the low pressure open interlock setpoint. Changed distractor pressure indicator to another RCS loop pressure indicator that is in the vicinity of the referenced indicators.

In the second question, recommend specifying that the open interlock is for 1-RH-MOV-1700. Comment incorporated.

JAT 06/26/2015 SR made the corrections annotated in red, above.

S Updated question appears to be SAT.

JAT 08/04/2015 6 H 2  ? B S? 005K6.03 Question appears to match the KA.

Just need to make sure there is no delay in the effects (i.e., does the stem question need to have a specific time associated with it?). There is no time delay in effect. Otherwise, question appears to be SAT.

The 401-5 indicates that this question was not on the last NRC exam.

However, we track the last TWO NRC exams, so I need to know if it was on either the 2014 or the 2012 exams. This question was not on the last two NRC exams, updated ES-401-5.

JAT 06/26/2015 S No corrections were required. SR comments are annotated in red, above.

Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 7 H 2 X N E 006A2.05 Question appears to match the KA.

What is the plausibility of putting the pump in pull-to-lock on the normal header? The justification provided is merely that it is a possible configuration if the applicant fails to apply the note correctly. However, the note says to align the highest priority pump to the normal header, but what would be accomplished by aligning a pump in pull-to-lock to the normal header? This distractor either needs an enhanced explanation of when this would be encountered or may need to reformat the question.

Re-wrote the question using similar concept for part 1.

JAT 06/26/2015 SR made the corrections annotated in red, above.

S Updated question appears to be SAT.

JAT 08/04/2015 8 H 2 X N S 006K3.01 Question appears to match the KA.

Regarding distractor D: What is the plausibility that a single train malfunction of the ECCS system would result in no ECCS flow to the RCS?

Additionally, in the stem of the question, it states that spurious safety injection signal has been initiated. Would it make more sense to say that the crew experienced a spurious SI? Comment incorporated.

One option for the question would be to reformat the question to ask: in the given alignment, the VCT is/is not a source of HHSI flow, and the RWST is/is not a source of HHSI flow. This still allows for another possible source of water, and there is at least some plausibility lent to neither one of them being a source. Question changed as recommended.

JAT 06/26/2015 SR made the corrections annotated in red, above.

The first question needs a comma: ...alignment, the VCT Updated question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 9 C 2 X N U 007A2.02 This question does not appear to meet the KA. The question essentially focuses on the open PORV (due to a controller malfunction) and the procedure that addresses the open PORV. The question needs to address the procedures that mitigate the abnormal pressure in the PRT, not the procedure that addresses the open PORV.

Recommend changing the question to:

1) The PRZR PORV block valve that must be closed to isolate the open PORV is ____.

2) The procedure that will direct returning the PRT to normal level

[or temperature, or pressure] is _____.

The above suggestion for question 1 incorporates a slight modification.

The answer choices that were submitted (e.g., close MOV-XXX block valve to isolate XXXX) lean toward teaching in the stem, unless the name of the PORV that is isolated is in the label for the block valve. Changed question as follows: Part 1 now asks for which PORV must be closed (more operationally relevant since block valve is right below PORV). Part 2 changed as recommended.

As an example, question 7 from the 2014 RO exam appears to be a good fit for this KA.

JAT 07/01/2015 S SR made the corrections annotated in red, above.

Updated question appears to match the KA. Updated question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 10 C 2 X M E 008AA1.04 Question appears to meet the KA.

Recommend adding the word first or initially to the second question so that there are not possibly two correct answers. Comment incorporated.

Is a temperature under Initial Conditions needed to ensure one and only one correct answer? (i.e., to make certain that Tave was not low enough to cause a FWI before an SI). Added normal Tave indication in Initial conditions.

Since this is a modified question, I need to see a copy of the original question for comparison. Original question is in References section of question 10 (Commanche Peak 2005, Q1-75).

JAT 07/01/2015 SR made the corrections annotated in red, above.

S Verified question is significantly modified. Updated question appears to be SAT.

JAT 08/04/2015 11 F 2 N S 008K1.01 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/01/2015 Question unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 12 C 2 N E 009EK3.11 The question appears to meet the KA.

The reason for tripping the RCPs should be neutral (as in, not imply that the correct answer to the first question is met) and should be tied to a procedure. For example, the second question could read, the reason RCP trip criteria are established in 1-E-1, in accordance with the basis for 1-E-1, is _____. Comment incorporated.

Also - please ensure that knowledge of EOP basis and reasons is RO knowledge at Surry. Yes, this is required knowledge.

JAT 07/01/2015 SR made the corrections annotated in red, above.

Updated question appears to match the KA. Updated question appears to S be SAT.

JAT 08/04/2015 13 C 2 N S 010K6.02 The question appears to meet the KA.

The question appears to be SAT.

JAT 07/01/2015 Question unchanged. JAT 08/04/2015 14 C 2 X B U 011EK2.02 The question does not appear to meet the KA. The question requires the applicant to demonstrate knowledge of pump operation and transfer to the RMT mode, rather than the reason for the pump response, as required by the KA.

To fix, I would leave the first of the two proposed questions, and then ask a question related to reasoning for the second question. This would change the question from a bank question to a modified question.

Changed part 2 to Per ES-1.3 CAP, after step 5, the operator will monitor for oscillating LHSI pump amps and flow to check for _____ .[ sump blockage, RWST < 3%]. Original Bank question was included in submittal.

JAT 07/01/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only M S SR made the corrections annotated in red, above.

Distractor B2 needs the . removed.

Question meets criteria for significantly modified. Updated question appears to be SAT.

JAT 08/04/2015 15 C 2 N S 011K2.02 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/02/2015 Question is unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 16 H 2 X M U/E 012K5.02 (NA14)

Question appears to match the KA.

I would recommend breaking the stem information into two pieces: Initial Conditions, Current Conditions. The part about N44 failing low can then go as a bullet point under current conditions.

Im not sure how plausible the IR High Flux trip is as a distractor when it is placed against the correct answer of PR high flux high setpoint trip. At

~35% power, the unit is well into the power range. Additionally, the justification that the setpoint for the IR trip may be confused with the block point seems to be a stretch, and with a trip setpoint at 35%, the unit would trip given the conditions in the stem.

Instead, it would be more plausible to ask whether or not the IR High Flux trip is currently protecting the reactor against a positive reactivity excursion (i.e., Given the current conditions, the Intermediate Range High Flux Trip [is/is not] protecting the core against a positive reactivity excursion.)

The plausibility of is lies in someone confusing the block setpoint with the trip setpoint. If someone thinks its not blocked until 35%, then N44 failing low would unblock it, and may be led to select the wrong answer.

There would be no cue from the other answer choice that the distractor is wrong.

Question 62 appears to contain information that would also render two of the existing distractors non-plausible, and could also overlap with the proposed modification.

Since this is a modified question, I need to see a copy of the original question for comparison. Rewrote the question entirely to better meet the K/A and eliminate fouling with question 62. Focused on Reactor protection against a high power density. This is now a NEW question.

JAT 07/02/2015 S

N SR made the corrections annotated in red, above.

New question appears to meet the KA. New question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 17 C 4 X X N U 013G2.2.36 The question appears to meet the KA. This question appears to be at the SRO-only job level, in that it asks the applicant to evaluate whether the application of 3.02 was correct for the current plant conditions.

At time 2130, I would recommend putting the entire position in quotations. The description seems to indicate that it reads >10 rather than spelling out the words greater than. If this is the case, I would use the >10 nomenclature. Otherwise, I would include the words greater than inside the quotation marks.

Is the EDG being inoperable associated with a TS action that is less than 1h? If so, that would be RO knowledge and a good place to take this question. Rewrote entire question to better conform to K/A.

JAT 07/02/2015 SR made the corrections annotated in red, above.

The new question does not appear to match the KA, in that the PORV block valve does not really relate to ESF actuation. However, the idea of the first question is good for the part of the KA relating to analyzing the effect of maintenance activities on LCOs. It just needs to be a maintenance activity on an ESF component or the actuation signals themselves.

The second question appears to be a tack-on. The TS definition of hot shutdown is not related to the first question at all, other than they both show up in TS.

JAT 08/04/2015 S The new question appears to match the KA.

Do you need to give the LCO number?

Otherwise, question appears to be SAT.

JAT 08/06/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 18 C 2  ? N E 013K4.10 The question appears to meet the KA.

Recommend the following minor edit to the first question: 1) After depressing both CLS RESET buttons, CLS [is/is not] reset. Made change as recommended.

(These do not need to be operated simultaneously, correct?)

The plausibility of the combination of CLS not resetting with the valve opening is shaky. However, I am accepting that SI is reset as an ok justification of plausibility.

What is the plausibility of 1-IA-TV-100 remaining closed if the applicant mistakenly thinks CLS does reset in the first half question? There may be some enhancement to the stem required to make choice A plausible.

Added to stem CTMT IA indicates 25 psig. This makes choice A plausible because IA-TV-100 requires IA pressure to be > 30 psig to open, therefore even if candidate thought CLS was reset the valve still will not open. This does not affect plausibility of B and D because IA Comp suction valves will still be able to be opened if SI was reset. Also updated plausibility statements and added references.

JAT 07/02/2015 S SR made the corrections annotated in red, above.

Need a comma in the second question: open 1-IA-TV-100, the valve Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 19 C X N E 014K5.01 This question appears to meet the KA: The KA talks about operational implication, which usually suggests a procedural direction of some sort. In this case, the diagnosis of the cabinet with the failure is the operational implication. Although the question does not address a reason for the difference between the RPIS and step counter, which is specifically addressed in the KA, the reason for the difference would be considered when developing operating procedures, and by appropriately diagnosing the event, the appropriate procedural actions could be implemented, and thus knowledge of the operational implication of the reason will be demonstrated by the applicant.

The second question appears to be a tack-on. I would recommend replacing the second question with a question that asks something from the ARP or possibly which abnormal procedure will require entry to mitigate the failure. Part 2 replaced with question asking for AP that will be used to mitigate failure.

JAT 07/02/2015 SR made the corrections annotated in red, above.

S A2 and C2 need to say Rod rather than Ron in the procedure titles.

Question appears to be SAT.

JAT 08/04/2015 20 H 2 B S 015AK2.10 The question appears to meet the KA.

The question appears to be SAT.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. This question has not been used on either the 2014 or the 2012 exam. Question form annotated.

JAT 07/02/2015 Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 21 F 2 N S 017A1.01 The question appears to meet the KA.

The question appears to be SAT.

JAT 07/06/2015 Question remains unchanged. JAT 08/04/2015 22 H 2 N S 022A4.03 The question appears to meet the KA.

The question appears to be SAT.

JAT 07/06/2015 Question remains unchanged. JAT 08/04/2015 23 H 2 N S 022AK1.04 The question appears to meet the KA. Based on the KA match description provided on the 401-5, which specifically addresses the reason portion of the KA, application of the knowledge that CH flow is limited by design while in automatic control is required to answer the question. The operational implications aspect of the KA is demonstrated by recognizing the low flow setpoint and the pzr level setpoint that the unit would be maintaining while in automatic.

JAT 07/06/2015 Question remains unchanged. JAT 08/04/2015 24 H 2 M S 022K1.02 The question appears to meet the KA.

The question appears to be SAT.

Since this is a modified question, I need to see a copy of the original question for comparison. Original question was included in submittal, relabeled as Original question to better annotate question.

JAT 07/06/2015 Question is significantly modified. Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 25 H 2 N S 025AA1.04 The question appears to meet the KA.

The question appears to be SAT.

JAT 07/06/2015 Question remains unchanged. JAT 08/04/2015 26 F 2 M S 026A3.02 The question appears to meet the KA.

The question appears to be SAT.

Since this is a modified question, I need to see a copy of the original question for comparison. Original question was included in submittal, relabeled as Original question to better annotate question.

JAT 07/06/2015 Question is significantly modified. Question remains unchanged. JAT 08/04/2015 27 F 2 N E 027A4.01 The question appears to meet the KA.

Instead of asking what positions there are for the switch to the Iodine filter fans, I would ask how the fans would be started. For example, since the stem of the question has the applicant start the fan per the request of HP, the first question can be rephrased to ask:

• The Iodine Filter Fans [can/cannot] be started by placing the control switch in the Auto position.

Or

• The operator will place the control switch for the Iodine Filter Fans to

[Auto/On] to start the fans.

The second option would also be more plausible if the third bullet of the stem information were revised to state: HP has requested the Unit 1 Iodine filter fans, 1-VS-F-3A and 3B, be placed in service.

(rather than started). Made changes to P1 question using second option. Also reworded P2 to fill in the blank to be consistent with P1.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S JAT 07/06/2015 SR made the corrections annotated in red, above.

The second question needs a 2 in front.

The question appears to be SAT.

JAT 08/04/2015 28 H 2 X N U 027AA2.02 This question does not appear to match the KA. The KA is specific to a pressurizer Pressure Control System malfunction, and the proposed question relates to normal operation of the pressurizer pressure control system.

To meet the KA, a similar question can be asked by having a POT setting in the stem initial conditions, and then stating a pressure under current conditions, and asking about whether a malfunction occurred or what type of malfunction has occurred. (There are other ways to meet this KA, that is just one option). Modified question as follows. New question provides conditions of a failed A spray valve controller. Part 1 asks for POT setting for 2235 psig, and Part 2 asks for operator action to correct. P2 requires candidate to diagnose current conditions and determine what has failed.

This has also changed the COG level from Low to High.

JAT 07/06/2015 S SR made the corrections annotated in red, above.

The new question is a much closer match to the KA. I consider it a match because the applicant must be able to identify what POT setting is associated with normal pressure and determine the failure that occurred in the given conditions.

You can use the pictures of the controller indications for the current conditions if you prefer.

Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 29 H 2 X M E 028AG2.4.47 The question appears to meet the KA.

I would either state that Channel I is selected for control OR that Channel I is the upper channel. The way it is written now, you are explaining in the stem of the question that selected for control means upper channel.

(teaching in the stem).

All options for the second half question include, place CH flow in manual. The only difference between the two options is the addition of and isolate letdown to the second answer choice option. This creates a subset issue, because an applicant can choose either answer and be correct since the stem only states, the expected operator action. Thus, there are two correct answers. To fix, change the second question to: In accordance with 0-AP-53.00, the operator [is/is not] required to isolate letdown for this failure. Made changes as recommended.

Since this is a modified question, I need to see a copy of the original question for comparison. Original question was included in submittal in Reference section of question.

JAT 07/06/2015 S SR made the corrections annotated in red, above.

Need to add a comma to the first question: ..leg leak, the level Also change indication to indicated.

Question is significantly modified.

Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 30 H 2 X X M U 029EA2.01 The question appears to meet the KA. This question appears to be at the SRO-only job level, since it asks the applicant to make a determination on a procedure transition that is not solely related to knowing the entry conditions for the EOP in question. The relevant information is not contained within a procedural note, nor is it contained in a foldout page.

Therefore, this question does not appear to be at the appropriate job level.

The 4th bullet (Power range instruments are all indicating 8%) needs a period at the end.

Since this is a modified question, I need to see a copy of the original question for comparison. This question has been rewritten. Previous Q 30 has been moved to Q 78. New question has both parts relating to nuclear instrumentation and ATWS. Note this is now a NEW question.

N JAT 07/06/2015 S SR made the corrections annotated in red, above.

The question appears to meet the KA at the RO level.

The first question needs a comma: ..less than 5%, the reactor There may be an overlap issue with current question 78 (i.e., old question 30 that this question replaced). However, Q78 asks what is required to return to 1-E-0 from FR-S.1. This question asks what 1-E-0 considers tripped. These may have similar answers, but since they are two distinct pieces of knowledge, Im going to say that the questions are not overlapped; if an applicant misses one of the questions, that lack of knowledge will not necessarily cause them to miss the other question.

Similarly, although the answers contain similar information, knowing one answer does not give you the answer to the other.

Question appears to be SAT.

JAT 08/04/2015 31 F 2 N S 029G2.1.25 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/06/2015 Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 32 H 2  ? N E? 035A2.05 The question appears to meet the KA.

If the entry condition for AP-21.00 states, The reactor is in the power range and a reduction in Main FW flow occurs as indicated by one or more of the following: 1H-G6, STM GEN 1B ERROR, would this also not be a correct answer? Need to provide a justification for why this answer is definitely wrong to avoid the possibility of two correct answers. Changed the stem of the question by adding B Feed Reg valve demand begins to slowly lower due to an instrument failure.

JAT 07/06/2015 S SR made the corrections annotated in red, above.

The question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 33 F 1 X B U 038EK1.02 The question appears to meet the KA.

This question is a direct lookup, LOD=1. All that is required is to find the PZR level stated in the stem and the SG level trend stated in the stem and look up the answer, rendering all distractors not plausible. If there is some plausibility allowed for =35% vs >35%, there are still 2 non-plausible distractors.

The stem question asks should, which is not a definitive question and leaves some room for interpretation.

To fix: 1) Have the applicant diagnose which SG has the tube leak; 2) instead of stating affected SG NR level is 68% and lower, give a level and trend for each SG; 3) change the stem question to Which ONE of the following is the MINIMUM actions required to be taken by the operating team in accordance with step 36 of E-3?. This will allow your answer choices to remain as-is. Made changes as recommended.

The first sentence has a minor grammatical error. It should read: The Operating Team is in E-3, Steam Generator Tube Rupture, and has completed RCS cooldown and depressurization. Made change as recommended.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. Believe this is now modified enough to no longer be a Bank question. Also Note: Found similar question on 2012 exam and have included the question in Reference section. We believe it is different enough to be considered a Modified question but we request Chief Examiner to also review. New (Modified) question is NOT a 2-part question like the 2012 exam question, and stem is markedly different.

S JAT 07/06/2015 H 2 M SR made the corrections annotated in red, above.

Question is significantly modified (and higher cog now).

Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 34 H 2 N S 039K3.04 The question appears to meet the KA.

The question appears to be SAT.

JAT 07/06/2015 Question remains unchanged. JAT 08/04/2015 35 F 2 N E 040AG2.1.30 The question appears to meet the KA.

The second question should end in a . rather than a ?. Change made as recommended.

Otherwise, the question appears to be SAT.

JAT 07/06/2015 SR made the corrections annotated in red, above.

S Question remains SAT. JAT 08/04/2015 36 H 2 M S 045K3.01 The question appears to meet the KA.

The question appears to be SAT.

Since this is a modified question, I need to see a copy of the original question for comparison. Original question was included in Question references section.

JAT 07/10/2015 Question is significantly modified. Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 37 H 2 X X M U 054AK3.05 This question does not appear to meet the KA. The question requires knowledge of PORV cycling to eliminate a distractor rather than to arrive at the correct answer. An applicant can arrive at the correct answer without demonstrating knowledge of the KA the way the question is currently proposed.

This question does appear to require RO knowledge to ANSWER, in that the determination for whether RCS bleed and feed is required appears on the continuous actions page. However, two of the distractors (B and D) appear to require SRO-only knowledge to eliminate in that they involve detailed procedure knowledge and are not associated with a procedural note, caution, or foldout page and are more specific than the overall mitigation strategy. Thus, two of the distractors for this question are not plausible on the RO portion of the exam.

Since this is a modified question, I need to see a copy of the original question for comparison. Changed question to more closely match K/A. P1 requires knowledge of CAP Bleed and Feed due to PORV open. P2 requires understanding of reason for PORV remaining open vs. cycling during loss of heat sink. Significantly modified question therefore this is now a NEW question.

JAT 07/10/2015 SR made the corrections annotated in red, above.

S Per the FAQs, if you based your question, even loosely, on another question, even it looks almost nothing like the previous version, it is still significantly modified rather than new. The only way for a question to be considered new is if you developed it without reference to another question.

Question appears to be SAT.

JAT 08/04/2015 38 H 2 N S 055A3.03 The question appears to meet the KA.

The question appears to be SAT.

JAT 07/10/2015 Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 39 F 2 M E 055EK3.02 The question appears to meet the KA. The distractors are contained within the notes and maintaining SG pressures greater than 200 psig is referenced in a caution of the procedure, indicating that this knowledge is RO level. Is preventing nitrogen injection also referenced in the background document for the EOP, and is EOP background information also RO-level knowledge at Surry? (Asking because the caution references 200 psig and the step references 300 psig. My assumption is that 300 psig allows you to be greater than the 200 psig called out in the caution - Im wondering if there is a specific tie between those two numbers in the background document.)

In ECA-0.0 Rev 39, which was provided as part of the reference package, it is step 24 rather than step 21 (which is referenced in the question) which directs SG depressurization to 300 psig. Changed to step 24.

Since this is a modified question, I need to see a copy of the original question for comparison. Original question was included with submittal and is in reference section of question.

JAT 07/10/2015 SR made the corrections annotated in red, above.

S Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 40 X M U/E 057AG2.4.50 The question appears to meet the KA.

There is a potential subset issue. The way the question is worded, if there is no failure, the SG could be controlled using either method presented.

The applicant could use logic to determine that there cannot be more than one correct answer, and since both methods will result in controlling SG level, then there must not be power to the controller to allow that choice as an option. This results in two non-plausible distractors. Thus, the first question needs to be more specific - you either need to specify the preferred method or the method IAW the applicable AP (which is answered in part 2), etc. This can be avoided entirely by simply asking:

B SG [can/cannot] be controlled using the pushbuttons on the FRV controller. Made changes as recommended.

Since this is a modified question, I need to see a copy of the original question for comparison. Question was submitted and is included in Question reference section.

JAT 07/10/2015 S SR made the corrections annotated in red, above.

This question is significantly modified. This question appears to be SAT.

JAT 08/04/2015 41 F 2 N E 058AA1.02 The question appears to meet the KA.

Should the question stem ask for the alternate source, rather than the alternant source? Changes made as recommended.

Answer choice D is the only one with the word breaker in it, and the stem question specifically calls out that the choices are breakers.

Recommend removing the word breaker from choice D so that it is not an outlier. Changes made as recommended.

JAT 07/10/2015 SR made the corrections annotated in red, above.

S Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 42 F 2 X X X N U 059A1.07 (pre)

Question appears to match the KA.

Question appears to be at the RO level.

• Credible Distractors:

o There is a subset issue that renders two distractors implausible. If Recirc flow is <3000gpm, it is clearly <3600gpm, so an applicant that knows there is one and only one correct answer can eliminate distractor B. Similarly, if feed flow is <3000gpm, it is clearly

<3600gpm, so D can be eliminated.

• Partial:

o If the recirc valve is not fully open, it could give a flow of <3000gpm.

Therefore, an applicant may be able to argue A as a partially correct answer.

• Recommend including valve number - recirc valve appears to be non-specific.

• Backward: The question is written in a backward logic way (observation only)

• Possible way to fix: Set up a timeline with feed flows, recirc flows, and recirc valve positions, and query on MFP status at two different points in time.

JAT 2/4/2015

? E Question appears to match the KA. Question appears to be at the RO level.

(pre2) Comments have been addressed.

Recommend definitively specifying which MFP trips.

Would lowering the feed pump suction pressure make D more plausible?

Otherwise, question appears SAT.

JAT 2/13/2015 Comments have been address. The question still appears to match the KA.

S The question appears to be SAT.

JAT 07/10/2015 Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 43 F 2 X B E 059A4.08 The question appears to meet the KA.

The answer choices do not appear analgous - options C and D offer an effect on the controller rather the control available. Recommend slightly altering the question to ask: 1) the effect on the output of the controller (shift to manual and remain the same as prior to the loss of power/raise to 100%), 2: where control can be from (in MCR/cannot be controlled from control room). Made changes as recommended.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. With the changes made, this question has changed from Bank to Modified Bank. Original question added to Question reference. Also checked the last 2 NRC exams and this question was not asked in either exam.

JAT 07/10/2015 SR made the corrections annotated in red, above.

S I would still lean towards calling this a bank question. There werent really any changes made to the stem, and although the question is now in a 2x2 format that was mostly to enhance plausibility of the distractors since the answer to the original question is still the same two pieces of information.

You are well within limits of Bank questions on the exam, so erring on the conservative side is best.

Question appears to be SAT.

JAT 08/04/2015 44  ? N E? 060AK1.02 The question appears to meet the KA.

Consideration - is it more likely an applicant will confuse an admin limit with the federal limit, or forget to apply the 85%? (There is no distractor that accounts for: [(2000)-(700)]/ (1000 mr/hr) = 1.3 h). This distractor would mirror the one with the fleet admin limit, and would likely make the distractors together more plausible overall, rather than having the federal limit as an outlying distractor. Changed wording of question, and distractor A to 0.50 hour5.787037e-4 days <br />0.0139 hours <br />8.267196e-5 weeks <br />1.9025e-5 months <br />, based on limit for other Dominion sites.

Recommend these changes to avoid potential for two correct answers.

JAT 07/10/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S SR made the corrections annotated in red, above.

Question appears to be SAT.

JAT 08/04/2015 061K1.04 45 H 2 X M U (pre) Question appears to match the KA (cause-effect relationship: the RCS flowing into one SG that is also faulted has the effect of requiring AFW to be maintained to the other SGs at a certain level)

• Credible Distractors: It is not plausible that an applicant would choose to feed both a faulted and ruptured SG (since they were told that was the case in the stem), so choices A and C are implausible, given that they both have AFW flow to B SG.

• Stating that a SG is faulted and ruptured may be teaching based on the conditions you give them in the stem of the question. Given the containment pressure, PZR level/P, Tave, and B SG parameters, it is fairly obvious that both a fault and an RCS rupture of some type is in progress. The fact that B is both faulted and ruptured does not seem to make a difference to the correct answer.

• Concerned there may not be a correct answer: the procedure states that the minimums are 350gpm normally and 450gpm in adverse containment. None of the answer choices are technically at the minimum flows, per the procedure, but the question asks which of the following states the minimum required flow.

• Recommendations:

o Remove statement about ruptured/faulted SGyou only need to say that the team is performing 1-E-0 and the plant conditions are as follows.

o Recommend putting a trend into the SG parameters to make the rupture obvious.

o Query on the actual minimum required AFW flow (and cite the procedure/step/attachment) in one part of the question.

(350gpm/450gpm) o Query on which SGs should be aligned to receive the flow in the second part of the question. (AandC/A,B,C) o Although one of the answers will still have B receiving flow vs. no flow, it is more plausible after the applicant is no longer specifically told they have a faulted/ruptured SG.

• I will need to see the parent question in the draft submittal since this is listed a modified bank question.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only B JAT 2/4/2015 S

(pre2) Question appears to match the KA. Question appears to be at the RO level. Comments have been addressed.

Ensure that lowering slowly means that level is controllable.

Parent question provided. Question may actually be Bank rather than significantly modified. (OK, bc not near bank limit.)

JAT 2/13/2015 S Question remains SAT.

07/10/2015 Question remains unchanged. JAT 08/04/2015 46 F 2 N S 061K5.05 The question appears to meet the KA.

The question appears to be SAT.

JAT 07/10/2015 Question remains unchanged. JAT 08/04/2015 47 F 2 N E 062A4.04 The question appears to meet the KA.

For the first question, it would be more precise to ask: The red breaker position indicating light [can/cannot] be used by the control room team to identify 1-CS-P-1A has been successfully started. This would avoid any potential confusion with the choice, 1-CS-P-1A amps only, since there may be another indication that can be used to confirm flow (i.e., flow meter, change in temperatures, etc.), which could lead to a condition where an applicant thinks there are no correct answers. The words control room team were used to more clearly tie the question to the KA and to ensure there is no other light that could be confused for the indicating light in question. Made recommended changes to question.

JAT 07/10/2015 S SR made the corrections annotated in red, above.

The question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 48 H 2 X N E 062AA2.04 The question appears to meet the KA.

Need to add the words, a minimum of to question 2 to avoid two potential correct answers. Made recommended changes to question.

JAT 07/10/2015 SR made the corrections annotated in red, above.

S Question appears to be SAT.

JAT 08/04/2015 49 F 2 M E 063K2.01 The question appears to meet the KA.

Question 2 needs to have a slight grammatical revision: It should ask, The Black Battery is supplies the ____ or The black battery is the supplyies for the _____

Otherwise, question appears to be SAT.

Since this is a modified question, I need to see a copy of the original question for comparison. Made recommended changes to question.

Original bank question was included in draft submittal, and is located in question reference section.

JAT 07/14/2015 S SR made the corrections annotated in red, above.

Question is significantly modified. Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 50 F 2 X B U 064K2.03 This question does not appear to meet the KA. The KA requires the applicant to demonstrate knowledge of bus power supplies to EDG control power. The question is set up based on the exciter field circuit breaker for the EDG tripping, and seems to be able to be answered entirely from knowing whether the EDG loads (the failure of the EDG to load is a given condition of the stem). The applicant does not need to know the power supply to the EDG control power to answer the question.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. Revised question to better meet the K/A. This is now a NEW question.

JAT 07/14/2015 S SR made the corrections annotated in red, above.

N The new question appears to meet the KA. The new question appears to be SAT.

JAT 08/04/2015 51 F 2 X N U 067AK3.02 The question does not appear to match the KA. The KA requires the applicant to demonstrate knowledge of the reason for the steps in the procedure. The question simply ask the type of fire protection and whether it is manual or automatic.

To have a better KA match, it might be possible to modify the existing question to ask whether the method of CO2 action is manual or automatic and why. If you go this route, ensure that the combinations of answers are plausible together. - Rewrote question to match K/A.

JAT 07/14/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only E SR made the corrections annotated in red, above.

Need to add the word maximum to the time limit in the first part of the question.

I would add the procedure reference containing the caution with the basis for the time for the second question. Additionally, since the distractor is for an EAL call, it seems implausible that the max time would be 15 minutes when the EAL call must be made within 15 minutes. Recommend changing the first part distractor to something lower (3 minutes, 10 minutes, etc).

Question requires enhancement. JAT 08/04/2015 Question SAT. JAT 9/9/2015 S

52 F 2  ? N U? 068AA2.10 Not clear if this question meets the KA. Can the applicant answer the first question without knowing any information other than it has been 20 minutes after reactor trip? If so, this half of the question does not meet the KA, unless there is a unique type of meter on the remote monitoring panel. A better KA match would be to provide information up to the point where the operator reports that both meters are offscale low at the remote monitoring panel, and then ask a question based on that (i.e.,

what does it mean, what do they do, which procedure must they enter keep it RO level, though). This would meet the interpret the source range count piece of the KA.

On the other hand, if the SR meter at the remote monitoring panel reads differently than in the CR, the query can be what they expect to see and the distractor can be an indication in the control room that is not available at the remote monitoring panel. Rewrote the stem to include a picture of Remote monitoring panel with flawed SR indication to better meet the K/A.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S JAT 07/14/2015 SR made the corrections annotated in red, above.

Question appears to match the KA.

Was the wording of the first question validated? Might be easier to just ask: The NI indication at the Remote Monitoring panel [is/is not]

expected.

Otherwise, question appears to be SAT. Wording choice is up to SRthere is one and only one correct answer with the current wording.

JAT 08/04/2015 53 F 2 N E 068K1.07 The question appears to meet the KA.

It looks like the VCT may go to the HL/LL tanks after going through a few other tanks to tighten up the answer choices, I recommend adding the word directly to the stem question: Which ONE of the following water sources is collected directly into the High/Low Level Liquid Waste Tanks prior to transfer to the SRF?

Otherwise, this question appears to be SAT.

Added Directly Into to stem of question as recommended.

JAT 07/14/2015 SR made the corrections annotated in red, above.

S Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 54 H 2 M E 073A1.01 The question appears to meet the KA.

Recommend adding the word automatic to the stem question: Which ONE of the following describes the automatic plant response?

Otherwise, this question appears to be SAT.

Since this is a modified question, I need to see a copy of the original question for comparison.

Added automatic to Stem of question. Pre-modified question is part of full file.

JAT 07/14/2015 S SR made the corrections annotated in red, above.

Question appears to be significantly modified. Question appears to be SAT.

JAT 08/04/2015 55 H 2 N S 076AK2.01 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/14/2015 Question remains unchanged. JAT 08/04/2015 56 F 2 N S 076G2.1.30 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/14/2015 Question remains unchanged. JAT 08/04/2015 57 F 2 N S 076G2.2.22 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/14/2015 Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 58 H 2 N S 077AK1 .02 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/14/2015 Question remains unchanged. JAT 08/04/2015 59 F 2 B S 078A3.01 The question appears to meet the KA.

This question appears to be SAT.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. Checked last 2 NRC exams (2014, 2012) and this question was not on either of those exams.

JAT 07/14/2015 Question remains unchanged. JAT 08/04/2015 60 H 2 N S 103A1.01 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/14/2015 Question remains unchanged. JAT 08/04/2015 61 F 2 N S G2.1.23 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/15/2015 Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 62 M E G2.1.7 This question appears to meet the KA.

There may be some overlap between this question and question 16. If question 16 remains as is (and additional distractor justification is provided) or the recommended change is implemented, the first half of this question would overlap. One or the other needs to change. Keep in mind that this KA is a much broader topic and can likely accommodate more possibilities.

Since this is a modified question, I need to see a copy of the original question for comparison. Question 16 was modified, therefore this question no longer overlaps question 16. Original question was included in submittal and is located in question reference section.

JAT 07/15/2015 S Question remains unchanged. Question 16 was modified and no longer overlaps with this question.

Question is significantly modified. Question is SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 63 F 2 B E G2.2.6 This question appears to meet the KA.

Recommend stating the procedure that the change is in accordance with in the stem question (AD-AA-100).

Recommend making the following change to simplify the question and leave less room for interpretation in the answer:

Which ONE of the following completes the following statement regarding the actions to be performed in accordance with AD-AA-100 to address the issue?:

After making a pen and ink change, the Operator [is/is not] required to have a supervisor initial and date the correction before proceeding. After completing the OPT, a procedure change [is/is not] required to be immediately processed. Made changes recommended.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. Changes made make this a Modified question. Question included in reference section of question.

Also checked last two NRC exams and did not find this question.

JAT 07/15/2015 M SR made the corrections annotated in red, above.

S Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 64 F 2 X M U G2.2.7 This question does not appear to meet the KA. The KA requires knowledge of the process for conducting special or infrequent tests. The question asks which procedure IS an infrequently conducted test or complex evolution.

To meet the KA, I recommend giving a procedure that governs a special/infrequent test and then soliciting knowledge (at least in half of the question) as to any special briefing or preparatory requirements in OP-AA-106, which governs Infrequently Conducted or Complex Evolutions.

You could ask two different procedures (one of which meets conditions of OP-AA-106 and one that does not) and then ask for each test whether a specific requirement is required. This would meet the process part of the KA.

Since this is a modified question, I need to see a copy of the original question for comparison. Rewrote the question to better meet the K/A.

Question should now be considered a NEW question. Original question in question reference section for review by NRC Chief Examiner.

JAT 07/15/2015 SR made the corrections annotated in red, above.

S Question appears to meet the KA. Need to ensure this question is at the correct license level, as it appears to ask specific details of the admin procedure. Provided ROs are expected to know these two pieces of information at Surry, this question is likely okay. Considerations: ROs performing these tests must know contingency plans/termination criteria.

Need to discuss second question.

JAT 08/04/2015 Surry states these are RO level questions and will attest to that in the 401-5 (ops expectation for ROs to know). JAT 08/05/2015 65 F 2 N S G2.3.13 The question appears to meet the KA.

This question appears to be SAT.

JAT 07/15/2015 Question remains unchanged. JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 66 H 2 M S G2.3.14 The question appears to meet the KA.

Need a better justification for why the two choices in distractor A are plausible together. Why would ventilation remain in normal but containment evacuation be required? The plausibility currently given for maintaining CR ventilation in normal is that no rad alarms are in, so the applicant may think there are no hazards. If they have this misconception, why, then, would they choose to evacuate containment? If an appropriate justification of these two choices together can be provided, the question is likely SAT. Otherwise, this distractor may need to be revised.

This question appears to overlap with SRO question 85, which also discusses a fuel handling incident, escaping bubbles. Question 85 elicits knowledge on establishing containment isolation and the basis for when emergency ventilation is required. The correct answer to this question (RO66) is that containment evacuation is required and that emergency ventilation is required. The way question SRO Q85 is asked implies the correct answer to this question. For that reason, either SRO Q85 requires replacement or this question requires replacement.

Since this is a modified question, I need to see a copy of the original question for comparison. Modified question Part 1), closure of MCR ventilation dampers secures MCR normal supply or exhaust fan to exclude overlap to question 85. Modified question included in Master file.

JAT 07/15/2015 SR made the corrections annotated in red, above.

Question is significantly modified. Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 67 F 2 B E G2.3.4 This question appears to meet the KA.

There are 4 different answer choices for the first question and 4 different answer choices for the second question. Recommend choosing one or the other.

(This is modified from a NAPS question although it still meets the criteria for being a bank question. Thus, it was not on a previous Surry NRC exam.)

Revised to limit to Federal limit for Lens of the Eye.

JAT 07/15/2015 SR made the corrections annotated in red, above.

S Distractor analysis for answer A needs updating (no longer a part 2).

Question appears to be SAT.

JAT 08/04/2015 68 F 2 B E G2.4.21 This question appears to meet the KA.

To make all distractors more plausible, I would recommend changing the stem question to: Which of the following Critical Safety Function RED paths is the highest priority CSF that can be directly caused by procedurally directed operator action?

This would lend credibility to the fact that more than one of them could be caused by a procedurally directed operator action.

(This is modified from a Diablo Canyon question although it still meets the criteria for being a bank question. Thus, it was not on a previous Surry NRC exam.)

Revised per recommendation.

JAT 07/15/2015 S SR made the corrections annotated in red, above.

Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 69 F 2 N S G2.4.29 The question appears to meet the KA.

Please ensure that this question is RO level knowledge at Surry. Otherwise, question appears to be SAT.

JAT 07/15/2015 Question remains unchanged. JAT 08/04/2015 70 F 2 N E/S G2.4.43 The question appears to meet the KA.

Recommend adding a procedural tie to the two questions. I.e., add, In accordance with EPIP-2.01 in front of When obtaining meteorological data from the MET panel Otherwise, question appears to be SAT.

Revised Stem as recommended.

JAT 07/15/2015 S SR made the corrections annotated in red, above.

Question appears to be SAT.

JAT 08/04/2015 71 F 2 M E/S WE04EK2.2 The question appears to meet the KA.

Recommend adding a procedural tie to the two questions. Change the stem question to: In accordance with 2-E-1, which ONE of the following identifies:

Otherwise, question appears to be SAT.

Since this is a modified question, I need to see a copy of the original question for comparison. - located in Master File.

Revised Stem as recommended. Added procedure title.

JAT 07/15/2015 S SR made the corrections annotated in red, above.

Question is significantly modified. Question appears to be SAT.

JAT 08/04/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 72 H 2 N S WE08EK3.3 The question appears to meet the KA.

Need to ensure that this is expected knowledge for an RO at Surry.

Additionally, need to verify that EOP and FR background knowledge is RO level knowledge. EOP and FR background knowledge is RO level knowledge. Specific knowledge included in reference of original submittal.

Otherwise, question appears to be SAT.

JAT 07/15/2015 Question remains unchanged. JAT 08/04/2015 73 H 2 B S WE10EA1.3 Question appears to meet the KA.

Question appears to be SAT.

JAT 07/15/2015 Question remains unchanged. JAT 08/04/2015 74 H 2 X X B U WE11EG2.1 .2 Question appears to meet the KA.

Question does not appear to be at the RO level. The first half of the question asks for the procedure that should be used for the next step, and the options are ECA-1.1 and ES-1.3. This is procedure determination, and is SRO-only.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. (In part, because if this was an approved RO-level question on a previous NRC exam, it would not be considered UNSAT, even though modification would still be required.)

Rewrote question to better meet RO level. One of actions in ES-1.3 is to manually initiate RMT if it fails to actuate automatically. Wrote questions testing the ROs ability to manually actuate RMT. Question is now a NEW question, but left original bank question in the reference section of question 74.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S JAT 07/15/2015 SR made the corrections annotated in red, above.

Question appears to match the KA at the appropriate license level.

Second question needs a . at the end.

Question appears to be SAT.

JAT 08/04/2015 75 H 2 N S WE15EG2.4.8 This question appears to meet the KA.

This question appears to be SAT.

JAT 07/15/2015 Question remains unchanged. JAT 08/04/2015 76 C 2 B S 002G2.2.25 The question appears to meet the KA.

This question appears to be SAT.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. This question was not asked in either the 2014 or the 2012 NRC exam.

JAT 07/16/2015 Question remains unchanged. JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 77 H 3 X N E 004A2.35 (pre)

Question appears to match the KA. Question appears to be at the SRO-only level. KA appears to be met at the SRO-only level.

• Regarding 1-CH-TV-1204A, either tell them that the valve is closed, or tha the green light is lit and the red light is out. Telling them both is teaching in the stem. Also, the valve name should be included in the question stem.

• For question 1), is it common practice to deactivate the OPERABLE trip valve when the INOPERABLE valve has the ability to be verified in the closed position?

• For question 2), remove the (MET / NOT MET) in the stem of the question. Also - typically only the NOT is emphasized. Currently all part 2) answer choices are in all capital letters, whether they contain the word not or not.

• Does deactivating the automatic isolation valve render it inoperable?

The note regarding maintaining at least one isolation valve OPERABLE in each affected penetration is unclear to me as to whether the deactivated valve would be OPERABLE or INOPERABLE.

• Instead of specifically differentiating between the OPERABLE trip valve and the affected trip valve, I would recommend just naming the valve that you intend to close and deactivate, and then the applicant can determine if that was the OPERABLE or INOPERABLE valve. This will also get rid of any potential cues in the stem.

JAT 2/4/2015

? Question appears to match the KA. Question appears to be SRO-only.

(pre2) Question apperas to be met at the SRO-only level. Comments have been addressed.

My only remaining concern is the 4h clock - typically we give a reference for >1h action statements, and in this case, the reference cannot be provided due to being a direct lookup. Is there a learning objective for this that can justify no reference? If not, you could simply ask whether the deactivated valve is OPERABLE or INOPERBLE.

S JAT 2/15/2015 (pre3) Comment incorporated. Question appears to be SAT at this time.

JAT 4/2/2015 S

Question remains SAT. JAT 07/16/2015 Question is unchanged. 08/05/15

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 78 H 2 X N U 007EA2.06 The question does not appear to meet the KA at the SRO-only job level.

Both questions are SRO-only, but neither meet the KA. The KA requires the applicant to demonstrate knowledge of the occurrence of a reactor trip. The question relates to restart following a reactor trip. Additionally, the second question, while at the SRO-only level because of being TS basis knowledge, does not relate to the first part of the question. If the computer-calculated leak rate calculation that is present in the stem of the question related to or complicated the reactor trip, it would be fine. But as-is, these are two unrelated questions and neither appear to relate to the KA. This question was replaced from Q 30. This is a modified question and original came from Harris 2004 exam, q 47. Copy of q included in Reference section of Q 78.

JAT 07/16/2015 B

S SR made the corrections annotated in red, above.

This question appears to be bank. There were some modification made, but it doesnt look significantly modified.

Question appears to meet the KA. Question appears to be at the SRO-only level. Question appears to be SAT.

JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 79 H 2 N 012G2.4.1 The question does not appear to meet the KA at the SRO-only level. The second question appears to be at the SRO-only level, but does not appear to meet the KA. Upon further review, this KA does not appear to lend itself to an SRO question, so I am replacing the KA with randomly selected KA: 012 G2.1.23 Reactor Protection / 7, Ability to perform specific system and integrated plant procedures during all modes of plant operation.

The current proposed question, if you choose to keep it, would need to have a slight modification because the part of the question that meets the KA appears to be able to be answered with RO knowledge. If you modify the question such that SRO-knowledge of procedure transitions is required, the question would likely be SAT.

The current question will not be counted as UNSAT because the KA is being replaced.

Question rewritten based on change to K/A.

JAT 07/16/2015 SR made the corrections annotated in red, above.

X U N The question appears to match the KA. The question does not appear to be at the SRO-only level. The question does not appear to meet the KA at the SRO-only level.

The first question meets the KA but can be answered with systems knowledgeplacing the rods in manual is an immediate action of AP-53.00, and knowing the system configuration that would cause rod movement is RO-level knowledge. The second question also meets the KA, but it is associated with a 1h or less TS required action, and is therefore RO-level knowledge. Additionally, the second question appears to be a direct lookup.

JAT 08/05/2015 Look at 2014 exam. What makes this SRO is the applicability of the 1h S

clock (what makes them have to go there).

Based on discussions with Surry, and the format of their TS, although there is a one-hour action associated with the number of channels less than the minimum, the question does require the applicant to make an operability call, and is therefore SRO-only. This question appears to be acceptable as-is.

JAT 08/06/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 80 H 2 N S 015G2.4.47 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

This question appears to be SAT.

JAT 07/16/2015 Question remains unchanged. JAT 08/05/2015 81 H 2 N S 022A2.04 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level (using procedures aspect of KA is met through TS).

This question appears to be SAT.

JAT 7/16/2015 Question remains unchanged. JAT 08/05/2015 82 H 2 X N U 024AA2.01 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA does NOT appear to be met at the SRO-only level.

The part of the question that meets the KA only requires systems knowledge to answer. Although the second part of the question is TS basis knowledge, it is not related to the KA of determining whether boron flow and/or MOVs are malfunction during emergency boration.

To meet the KA at the SRO-only level, recommend asking the appropriate procedural transitions to mitigate the failure. Changed Part 1) to allow closer match to K/A. Changed Part 2) of question to query Operability of required Boration paths per TS 3.2.

Additionally, does the third bullet point need to spell out Emergency (versus Emrg), or is that how 1-CH-FI-1110 is labelled? Spelled out emergency.

JAT 07/16/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only E SR made the corrections annotated in red, above.

The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

Need to ensure that a failure of 1-CH-FI-1110 could not also give the same indications.

Otherwise, question appears SAT. JAT 08/05/2015 S Tank level would be changing. No additional info needed. Question is SAT.

83 H 2 X N E/U 026A2.04 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

Recommend changing the first question to: The operator [is/is not]

required to locally open the supply breaker after placing A ORS pump in PTL. Otherwise, there is a subset issue because an applicant could use logic to eliminate two of the distractors rather than knowledge of the KA (Placing A ORS in PTL is a subset of Placing A ORS in PTL and opening the supply breaker.) Recommended changes to Part 10 of question made.

JAT 07/16/2015 SR made the corrections annotated in red, above.

S Question appears to be SAT. JAT 08/05/2015 84 H 2 N S 026AG2.2.44 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

This question appears to be SAT.

JAT 07/16/2015 Question remains unchanged. JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 85 F 2 N  ? 036AG2.2.37 This question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

This question appears to overlap with RO question 66, which also discusses a fuel handling incident, escaping bubbles, containment evacuation and whether normal or emergency ventilation is required. The correct answer to RO Q66 is that containment evacuation is required and that emergency ventilation is required. The way this question is asked (SRO Q85) implies what the correct answer to RO Q66 should be. For that reason, either RO Q66 requires replacement or this question requires replacement. Modified question 66 to eliminate overlap.

JAT 07/16/2015 SR made the corrections annotated in red, above.

X This question does not appear to overlap with question 66 (although there are similar stem conditions). This assessment was made because someone missing one of the questions would not necessarily miss the other due to the same lack of knowledge (double jeopardy), nor would knowing the answer to one (or eliminating a distractor from one) allow an applicant to eliminate a distractor from the other (i.e., the answer to one question cannot be used to eliminate a distractor from another).

This question appears to meet the KA. This question appears to be at the SRO-only level. The question does not appear to meet the KA at the SRO-only level. However, this was not indicated in the initial 401-9.

Recommend to put information in the stem of the question regarding the emergency ventilation, and then ask if there are enough operable trains to keep it SRO only and hit the KA.

JAT 08/05/2015 S

Question SAT. JAT 09/09/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 86 H 2 M E 054AG2.4.18 (pre)

Question appears to match the KA. Question appears to be at the SRO-only level. KA appears to be met at the SRO-only level.

• It is a little confusing to state that a loss of all feedwater occurs with the plant at 100%, and THEN state the initial conditions. I would call the events that are listed after the 100% power/LOFW current conditions, subsequent conditions, or given conditions (really, anything other than initial conditions).

• Part 1) question refers to this analysis, without specifying an analysis.

Recommend specifying which analysis the question is referring to.

Similarly, state what basis document you are referring to in the part 2) question: what is the basis, IAW with the FR-S.1 background document, for

• I will need to see the parent question in the draft submittal since this is listed a modified bank question.

JAT 2/4/2015 S

Question appears to match the KA. Question appears to be at the SRO-(pre2) only level. KA appears to be met at the SRO-only level. Comments have been addressed.

Based on parent question, question is significantly modified.

Question appears to be SAT.

JAT 2/13/15 Question remains SAT. JAT 07/16/2015 S

Question remains unchanged. JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 87 H 2 X N U 056AA2.75 The question appears to meet the KA. The question appears to be SRO-only. The KA does NOT appear to be met at the SRO-only level. The KA relates to determining and interpreting CVCS makeup/lineup during a LOOP. The part of the question that addresses the CH pump configuration can be answered with systems knowledge, and is therefore not SRO-only.

The second part of the question regarding the EAL call for a LOOP does not require any knowledge of what happened in the CVCS system, and therefore does not meet the KA at the SRO-only level.

To fix, I would focus on keeping the first half of the question (regarding the CH pump running after the EDG loads on 1J) and then ask a second question involving procedures (beyond EOP and AOP entry conditions) that are based on the situation in the stem of the question. This will help meet the KA at the SRO-only level. Rewrote stem of questions to focus Part 1). Changes Part 2) to query TS 3.01 applicability with CH configuration.

JAT 07/17/2015 SR made the corrections annotated in red, above.

While on the surface, the modification to the second question appears to be at the SRO-only level, it can be answered based on knowledge of a note in AP-10.07, and is therefore not SRO-only. Are there any other TS that would require entry or you could apply TS rules of usage to? Any transitions from AP-10.07?

JAT 08/05/2015 S

The modified question appears to meet the KA. The question appears to be at the SRO-only. The question appears to meet the KA at the SRO-only level. The updated question appears to be SAT.

JAT 08/06/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 88 H 2 N S/E 061AA2.03 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

Is the name of the ARP for annunciator 1-RMA-A1 also just 1-RMA-A1?

Does ARP need to be in the first question somewhere? Added annunciator noun name to Part 1) question.

Otherwise, question appears to be SAT.

JAT 07/16/2015 SR made the corrections annotated in red, above.

Question appears to be SAT. JAT 08/05/2015 89 F 2 B S 064G2.1.32 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

This question appears to be SAT.

The question is identified as a bank question not on the last NRC exam.

However, we track 2 exams back, so we need to know if this question was on either the 2014 or the 2012 exams. This question was not asked on either the 2014 or the 2012 NRC exams.

JAT 07/16/2015 Question remains unchanged. JAT 08/05/2015 90 H 2 M S 065AG2.4.9 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

Since this is a modified question, I need to see a copy of the original question for comparison.

The question appears to be SAT. This question was not asked on either the 2014 or the 2012 NRC exams.

JAT 07/16/2015 Question remains unchanged. JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 91 H 2 X N U 069AG2.4.21 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA does NOT appear to be met at the SRO-only level.

The part of the question that meets the KA requires the applicant to determine if the CSF is red or orange, which is RO-level knowledge. The part of the question that is at the SRO-level, the classification, does not meet the KA of parameters or logic used to assess the status of safety functions for a Loss of CTMT Integrity.

Assessing parameters and determining a yellow path is at the SRO-only level. You can modify the question to give parameters in the stem and have the applicant determine which procedure to enter (FR-Z.3 or FR-Z.4, which would require them to make a determination or a yellow path based on containment radiation or containment pressure). Changed Part

2) of question to query whether FR-Z.1 required re-entry following exit from procedure with steps completed.

JAT 07/17/2015 SR made the corrections annotated in red, above.

Is knowledge of the background document for FRPs (which is needed to answer the second question) SRO-only? EOP background and AP background were previously stated as RO level. It cannot be both ways. If it is SRO-only, then the question appears to be SAT. Otherwise, I would recommend addressing the question as suggested above.

JAT 08/05/2015 Question SAT, JAT 09/09/2015 S

92 H 2 M S 086A2.04 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

Since this is a modified question, I need to see a copy of the original question for comparison.

The question appears to be SAT. Original question was included in draft submittal.

JAT 07/17/2015 Question is significantly modified. Question remains unchanged.

JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 93 F 3 X X N U? G2.1.14 (pre)

Not sure if this question meets the KA - KA is knowledge of conditions/criteria that require plant announcements, rather than WHO makes the announcement.

Question appears to be at the SRO-only level.

• Partial:

o Why is it wrong for the unaffected Unit SRO to make the announcement? May be two correct answers.

• Part 2) needs to refer to the MAXIMUM amount of time allowed to avoid any potential subset issues.

• Is it likely any applicant will need to ask what tab HA-2.1 specifically refers to?

JAT 2/4/2015 U/E Question appears to match the KA. Question appears to be at the SRO-X only level. KA appears to be met at the SRO-only level. Comments have (pre2) been addressed.

Not sure how plausible A1/B1 are: why would an applicant think that station personnel are notified via beeper only when the alternative is a plant announcement, which can reach all employees, whether they have been issued a beeper or not? Could possible solve by asking the FIRST method used to contact plant personnel. If you go for this approach, ensure that you can justify a casualty wherein telephone notification occurs prior to a plant announcement.

JAT 2/13/2015 S Comments have been incorporated. Question appears to be SAT.

JAT 07/16/2015 Question remains unchanged. JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 94 F 3  ? N E G2.1.38 (pre)

Question appears to match the KA. Question appears to be at the SRO-only level. KA appears to be met at the SRO-only level.

• Recommend adding the attachment from which the need for open communication is cited (OP-AA-106, Att 3) so that it cannot be argued that open communication is similar to establishing the lines of communication steering the test, as specified in OP-AA-106, Attachment 2. Also - ensure this is not minutia.

• In question stem, the comma after the quotation the need for open communication is unnecessary.

• The (1) and (2) in the question stem are inconsistent with how other questions were formatted (previously, the stem had 1) or 2) to reflect how the choices were presented). It doesnt matter which way you choose to do them, I would just make them consistent throughout the test.

• Is there any chance the test coordinator could be the second line supervisor? Need to ensure this is not possible so as to avoid two correct answers.

JAT 2/4/2015 Question appears to match the KA. Question appears to be at the SRO-S only level. KA appears to be met at the SRO-only level. Comments have (pre2) been addressed.

Question appears to be SAT.

JAT 2/13/2015 S

Question remains SAT 07/16/2015 Question remains unchanged. JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only G2.2.1 95 H 2 X M U The question appears to meet the KA. The question appears to be at the SRO-only level. The KA does NOT appear to be met at the SRO-only level, in that the first half question requires knowledge of a P&L 1-OP-RX-006 (an operating procedure) which is RO level knowledge, and the second half question is a <1h TS action, which is also RO knowledge.

This KA is very broad, and can encompass any pre-startup procedure. It gives an example of systems that affect reactivity, but is not limited to those systems affecting reactivity. I would focus on asking about a pre-startup requirement contained an administrative procedure, which would be SRO-only knowledge.

Since this is a modified question, I need to see a copy of the original question for comparison. Replaced this question with question 78.

Modified Part 2 to specify minimum authority.

JAT 07/17/2015 S SR made the corrections annotated in red, above.

The replaced question appears to meet the KA. The question appears to be at the SRO-only level. The question appears to meet the KA at the SRO-only level.

The question appears to be SAT.

JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only G2.2.21 96 F 1 X N U (pre) Question appears to match the KA. Question appears to be at the SRO-only level. KA to be met at the SRO-only level.

• Capitalize Sat in the part 1) question to keep it consistent with the previous part of the question.

• Credible Distracters:

o C.1) and D.1) do not make sense rework must be performed Sat followed by PMT or PMT deferred. Why would an applicant choose that as an answer when the alternative is that engineering justifies that the failure does not affect operability? It would be more plausible to query on whether an engineering justification can/can not be provided to allow the diesel to be returned OPERABLE status if no rework is done.

o A.2) and C.2) seem implausible why would you waive PMT after testing is starting or complete? Also, this question doesnt appear to directly tie to the conditions above it regarding the EDG test.

Recommend asking a specific question about the administrative requirements for the failed EDG already referenced in the question (or about whether another EDG can be tested while the work for this EDG is complete but not test post-maintenance).

JAT 2/4/2015 Question appears to match the KA. Question appears to be at the SRO-only level. KA appears to be met at the SRO-only level. Comments have been addressed.

Ensure there is documentation for the second question that states that Operations has the final call on operability.

S Question appears to be SAT.

(pre2) JAT 2/13/2015 S This question remains SAT.

JAT 07/17/2015 Question remains unchanged. JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 97 F 2 X M U G2.3.11 (pre)

Question appears to match the KA. Question does NOT appear to be at the SRO-only level. KA does NOT appear to be met at the SRO-only level.

• Not SRO-Only o Querying on the response of the process monitor and the nuclide of concern in the gas storage tanks are both system-level/RO-level knowledge. Although the specific nuclide is in TS, it appears to be the equivalent of above-the-line information, in that the spec is The quantity of radioactivity contained in each gas storage tank shall be limited to less than or equal to 24,600 curies of noble gases (considered as Xe-133). This would make it RO knowledge.

o Similarly, the state the monitor stays in is systems knowledge.

However, if you ask the process by which the normal range monitor is returned to service, that could be SRO-only knowledge (i.e., query on whether the SRO can direct the return to service using instructions contained in procedure 0-RMA-C7 ONLY, or if I&C required).

o To hit the KA at the SRO-only level, you probably need to ask not only how to return the monitor to service, but also who has permission to authorize recommencing the release. The permission/authorization aspect would more directly hit the control part of the KA.

• I will need to see the parent question in the draft submittal since this is listed a modified bank question.

S/E? JAT 2/4/2015 (pre2)

Question appears to match the KA. Question appears to be at the SRO-only level. KA appears to be met at the SRO-only level. Comments have been addressed.

When does Attachment 2 of the WGDT procedure get used? Need to ensure that O2 is a plausible distractor.

Double check the distractor analysis - it looks like B2 might not have the correct justification (since it is correct, but the analysis says it is incorrect).

Question is modified, rather than new.

Assuming the plausibility of O2 as a distractor can be bolstered, this question appears to be SAT. JAT 2/13/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S Question modified to change first question and distractor.

(pre3) Question appears to be SAT.

JAT 4/2/2015 S

Question remains SAT. JAT 07/16/2015 Question remains unchanged. JAT 08/05/2015 98 H 2 B E G2.4.12 (pre)

Question appears to match the KA. Question appears to be at the SRO-only level. KA appears to be met at the SRO-only level.

Recommend a singular time for removal of the individual from the area.

Recommend changing either the dose rate or the time such that the minimum dose is closer to the 25 Rem threshold. You could also change this to a modified question if you made the time a maximum of 10 minutes and adjusted the dose rate so that the dose received was less than 25 Rem (changing the question so that a distractor becomes the answer is considered significantly modified).

If this question remains a Bank question, I will need to know which NRC exam (if any) it was previously used on.

E JAT 2/4/2015 (pre2) Question appears to match the KA. Question appears to be at the SRO-only level. KA appears to be met at the SRO-only level. Comments have X been addressed.

In order to ensure there is only one correct answer, change the stem to state a maximum of 10 minutes rather than at least 10 minutes.

Leaving it at least 10 minutes allows someone to argue that there is the possibility that the total dose could exceed 25 Rem and therefore volunteers are required.

Question is modified, rather than new.

Question will be SAT with the one minor modification.

JAT 2/13/2015 Comment has been incorporated. Question appears to be SAT.

S JAT 07/17/2015 Question remains unchanged. JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 99 H 2 X M E G2.4.23 The question appears to meet the KA. The question appears to be at the SRO-only level. The question appears to meet the KA at the SRO-only level, in that the applicant must use knowledge of the bases for prioritizing emergency procedure response when choosing the appropriate procedural flowpath.

Recommend rewording distractors A and B to: Go directly to to avoid any possible subset issues, since at least 1-ES-0.3 will be entered eventually. Adding the word directly will eliminate any potential subset issues. Recommended Modifications made, added directly to to distractors A and B.

Since this is a modified question, I need to see a copy of the original question for comparison. Original question was included with draft submittal, and is in the reference section of the question.

With recommended modification, question appears to be SAT.

JAT 07/17/2015 SR made the corrections annotated in red, above.

S There are some modifications to the distractors and at least one difference in stem conditions. The answer is the same, but I would consider this modified.

Question appears to be SAT.

JAT 08/05/2015

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 100 H 2 X N U/E WE05EA2.1 The question appears to meet the KA. The question appears to be at the SRO-only level. The KA appears to be met at the SRO-only level.

It does not seem plausible that an applicant would choose the answer choice that enters FR-H.1 after transitioning through both E-2 and E-2.1. Is there a point in E-2 where the applicant would NOT make a transition to FR-H.1 and instead would remain in E-2? (Two non-plausible distractors, but for the same reason.) Additionally, the first question implies that the appropriate procedure to be in is FR-H.1.

You may be able to meet the KA and have plausible distractors if you added information to the stem stating that the SRO announced entry into E-2 and has gotten to the point where he has read step 2, and then state the STA reports a red-path on Heat Sink. Then the question can ask, which procedure will they transition to from FR-H.1: E-2 or ECA-2.1. Since step 2 was started but not completed, the correct answer will be E-2 (step 2 is where the transition to ECA-2.1 occurs when the step cannot be completed). You will need to ensure that there would only be one correct answer if you pursue this path. Changes made as recommended to changes in Part 2), changes made, to Stem and distractors.

JAT 07/17/2015 S SR made the corrections annotated in red, above.

The question appears to be SAT.

JAT 08/05/2015

Final Sample Plan (ES-401-2/3)

The final sample plan is the combination of the draft sample plan and the Record of Rejected K/As (ES-401-4)

ES-403 Rev 10 Written Examination Grading Form ES-403-1 Quality Checklist Facility: Sç 2OOi Date of Exam11291 Exam Level: RO SRO Item Description Initials a b c

1. Clean answer sheets copied before grading .

2. Answer key changes and question deletions justified and documented

3. Applicants scores checked for addition errors (v ,

(reviewers spot check> 25% of examinations)

4. Grading for all borderline cases (80 +/-2% overall and 70 or 80, as applicable, +/-4% on the SRO-only) reviewed in detail J)

5. All other failing examinations checked to ensure that grades (9.i) are justified

6. Performance on missed questions checked for training deficiencies and wording problems; evaluate validity of questions missed by half or more of the applicants Printed Name/Signature Date

a. Grader M.PoNrcP

b. Facility Reviewer(*) il)Pc

c. NRC Chief Examiner (*) Thi /

d. NRC Supeisor(*) 10! lj

(*) The facility reviewers signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.

i