ML092920137
| ML092920137 | |
| Person / Time | |
|---|---|
| Site: | Surry  |
| Issue date: | 10/17/2009 |
| From: | NRC/RGN-II |
| To: | Virginia Electric & Power Co (VEPCO) |
| References | |
| 50-280/09-301, 50-281/09-301 | |
| Download: ML092920137 (52) | |
Text
{{#Wiki_filter:ES-201 Examination Preparation Checklist Form ES-201-1 Facility: 5(lR~~ Date of Examination: T u!.'1. ..(a:d Developed by: Written - Facility 0 NRC [gJ II Operating - Facility KJ NRCD Target Chief Date* Task Description (Reference) Examiners Initials
-180 1. Examination administration date confirmed (C.1.a; C.2.a and b) m'P
-120 2. NRC examiners and facility contact assigned (C.1.d; C.2.e) mB
-120 3. Facility contact briefed on security and other requirements (C.2.c)
JJffi
!'I'fb -120 4. Corporate notification letter sent (C.2.d) mJ3 fYY/3
[-90] [S. Reference material due (C.1.e; C.3.c; Attachment 3)] 1$ {-7S} 6. Integrated examination outline(s) due, including Forms E5-201-2, ES-201-3, ES-301-1, ES-301-2, ES-301-S, ES-D-1's, ES-401-1/2, ES-401-3, and ES-401-4, as applicable (C.1.e and f; C.3.d) ~ {-70} {7. Examination outline(s) reviewed by NRC and feedback provided to facility licensee (C.2.h; C.3.e)} ~ {-4S} 8. Proposed examinations (including written, walk-through JPMs, and scenarios, as applicable), supporting documentation (including Forms ES-301-3, ES-301-4, E5-301-S, ES-301-6, and ES-401-6, and any Form ES-201-3 updates), and reference materials due (C.1.e, f, g and h; C.3.d) ~
-30 9. Preliminary license applications (NRC Form 398's) due (C.1.!; C.2.g; ES-202) twP hfP
-14 10. Final license applications due and Form ES-201-4 prepared (C.1.1; C.2.1; ES-202) 1l1ti Int3
-14 11. Examination approved by NRC supervisor for facility licensee review (C.2.h; C.3.f)
(1m;
-14 12. Examinations reviewed with facility licensee (C.1.j; C.2.f and h; C.3.g) rY$
-7 13. Written examinations and operating tests approved by NRC supervisor (C.2.i; C.3.h) J11P
-7 14. Final applications reviewed; 1 or 2 (if >10) applications audited to confirm qualifications / eligibility; and examination approval and waiver letters sent (C.2.i; Attachment S; ES-202, C.2.e; ES-204) [YYi:>
-7 1S. Proctoring/written exam administration guidelines reviewed with facility licensee (C.3.k) f{fB
-7 16. Approved scenarios, job performance measures, and questions distributed to NRC examiners (C.3.i) nt6
- Target dates are generally based on facility-prepared examinations and are keyed to the examination date identified in the corporate notification letter. They are for planning purposes and may be adjusted on a case-by-case basis in coordination with the facility licensee.
[Applies only] {Does not apply} to examinations prepared by the NRC. ES-201, Page 25 of 28
ES-201 Examination Outline Quality Checklist Form ES-201-2 Facility: Date of Examination: Initials Item Task Description a b* C#
- 1. a. Verify that the outline(s) fit(s) the appropriate model, in accordance with ES-401.
w Oll) Nk1 /Itr R I
- b. Assess whether the outline was systematically and randomly prepared in accordance with Section D.1 of ES-401 and whether all KIA categories are appropriately samjJIed.
sampled. rY1P eft T T
- c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics.
tWP cJf{( E N d. Assess whether the justifications for deselected or rejected KIA statements are appropriate. I~ ,It
~ cIIC
- 2. a. Using Form ES-301-5, verify that the proposed scenario sets cover the required number of normal evolutions, instrument and component failures, technical specifications, S
I and major transients. N/a Nh1 "'kt M b. Assess whether there are enough scenario sets (and spares) to test the projected number U and mix of applicants in accordance with the expected crew composition and rotation schedule I L without compromising exam integrity, and ensure that each applicant can be tested using A at least one new or significantly modified scenario, that no scenarios are duplicated T test(s), and that scenarios will not be repeated on subsequent days. from the applicants' audit testis), 0 c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative R and quantitative criteria specified on Form ES-301-4 and described in Appendix D.
,v ,1I
,V ~
- 3. a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2:
% f (1 ) the outline(s) contain(s) the required number of control room and in-plant tasks W distributed among the safety functions as specified on the form / (2) task repetition from the last two NRC examinations is within the limits specified on the form T (3) no tasks are duplicated from the applicants' audit testis) test(s)
(4) the number of new or modified tasks meets or exceeds the minimums specified on the form (5) the number of alternate path, low-power, emergency, and RCA tasks meet the criteria on the form.
- b. Verify that the administrative outline meets the criteria specified on Form ES-301-1:
(1) the tasks are distributed among the topics as specified on the form (2) at least one task is new or significantly modified re~eated from the last two NRC licensing examinations (3) no more than one task is repeated
- c. Determine if there are enough different outlines to test the projected number and mix ,it ~ IVr of applicants and ensure that no items are duplicated on subsequent days.
- 4. a. Assess whether plant-specific priorities (including PRA and IPE insights) are covered in the appropriate exam sections. fI1i) N/ti
!YIb M/(
G E
- b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate. ,tyP-,
ttl1) rII c# N E
- c. Ensure that KIA importance ratings (except for plant-specific priorities) are at least 2.5. IJrP LlrP ~
C% R
- d. Check for duplication and overlap among exam sections. lilB J1'b eftr Cftf' A e. Check the entire exam for balance of coverage. IIYiB IrYIB (111(
(111{ L
- f. Assess whether the exam fits the appropriate job level (RO or SRO). .mE> t::V ~ (die
(/lit M.~ 1'+, ~Ctn;;sL !lY7Jt~rRr~~e
!lY7Jt~rRr~_~e Date
- a. Author 7 1'>1"'fAA.?l
- b. Facility Reviewer (*) N/A NA
- c. NRC Chief Examiner (#) &~;'< /(awh
&et/{ /(nwn / ~.-i.J 5<2::.11 . Ir:r;':L:~
- d. NRC Supervisor LlAJ.J llAJf 111 ~ ~ .,.-:
IW 1A.. \\lID! t..u. T: "'/lOL' rJa"/ ~ AAtj?f' lA11.t.i.~ JUII>--- l,//I&.--. -..
~1-I/ f\ )
Note: # Independent NRC reviewer initial items in ColumnY,chief Column v'chief examiner concurrence required.
- Not a.!lQIicable applicable for NRC:Qrepared NRC-prepared examination outlines ES-201, Page 26 of 28
FINAL OPfRAIJIol('Ic: O'1/lo/,;qx,9 ES-201 Examination Outline Quality Checklist Form ES-201-2 Facility: Date of Examination: Initials Item Task Description a b* C# w
- 1. a. Verify that the outline(s) fit(s) the appropriate model, in accordance with ES-401. NIA
- N/A. N/A NiA_
R b. Assess whether the outline was systematically and randomly prepared in accordance with
- I Section 0.1 of ES-401 and whether all KiA categories are appropriately sampled.
T T c- Assess whether the outline over-emphasizes any systems, evolutions, or generic topiCS.
- E N d. Assess whether the justifications for deselected or rejected KiA statements are appropriate. 1/
- v
- 2. a. Using Form ES-301-5, verify that the proposed scenario sets cover the required number S
of normal evolu1ions, instrument and component failures, technical specifICations, and major transients. Co( ~ fy$ I M b. Assess whether there are enough scenario sets (and spares) to test the projected number 7( ~ ('r11J U and mix of applicants in accordance with the expected crew composition and rotation schedule L without compromising exam integrity, and ensure that each applicant can be tested using A at least one new or significantly modified scenario, that no scenarios are duplicated T from the applicants' audit test(s), and that scenarios will not be repeated on subsequent days. 0 R
- c. To the extent possible, assess whether the ou1line(s) conform(s) with the qualitative and quantitative criteria specified on Form ES-301-4 and described in Appendix D.
Co.(
~
f? ()t3
- 3. a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2:
,,{1) the outline(s) contain(s) the required number of control room and in-plant tasks W distributed among the safety functions as specified on the form T
I ...-(2) task repetition from the last two NRC examinations is within the limits specified on the form
.-{3) no tasks are duplicated from the applicants' audit test(s) c.r ;!!>
!!> m15
..-(4) the mxnber of new or modified tasks meets or exceeds the minmums specified on the form
,.....(5) the number of altemate path, low-power, emergency, and RCA tasks meet the criteria on the form.
- b. Verify that the administrative outline meets the criteria specified on Form ES-301-1:
/(1) the tasks are distribu1ed among the topics as specified on the form
..-(2) at least one task is new or significantly modified 0...; @ h¢ v(3) no more than one task is repeated from the last two NRC licensing examinations
- c. Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days.
C--( ~
~
I!f$ q ~
- 4. a. Assess whether plant-specific priorities (including PRA and IPE inSights) are covered in the appropriate exam sections. ftl1)
G E b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate. C-p ~ ~ N E
- c. Ensure that KiA importance ratings (except for plant-specific priorities) are at least 2.5. C!/( ~
C!/{ ~ ~ R d. Check for duplication and overlap among exam sections. t¥ ~ ~ A L e. Check the entire exam for balance of coverage. c( ~J m£, mf, 4.( 4.f
~
DttJ
- a. Author
- f. Assess whether the exam fits the appropriate job level (RO or SRO).
-.UI CqrL- ff_~I-1.J,If~
Pri!!!¢ ~~a _!"!..- __ Pri!!!¢~~a l1.J,if "Jik 7'
-~_'7 __ t-fc{ro~
1-fcrro~
- b. Facility Reviewer (*) 1)~V*t\ H
'))A.v.t\ l-\ WI WILSoN LSotv' / JA J.i. t Ia .IT.ff jq ItIIJ'
..,fq/tIIJ'
- c. NRC Chief Examiner (#) MMI'- A- -gAle> //111-12.t..
n2-12i<. T T~? ~.;;1 1;-)
~ CID1~cdf Cflhol ~ccR
- d. NRC Supervisor liM/, IV ~ L-r.
Ll1J/'1"1 1-1: \iADJ jJ.lJJJ I/ u.J~ iJ.lJJJ l":\ ~.
,6";\ ~- -....
D:Vts*J, D:Vfci" I (~~ --. .
~~
Note:
# Independent NRC reviewer initial items in Column "CO; chief examiner concurrence required.
Not applicable for NRC-prepared examination outlines ou1lines
ES-201 Examination Security Agreement Form ES-201-3
- 1. Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of 7/z1<f1 kifQ 7 as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.
- 2. Post-Examination To the best of my knowle~ge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) ofilt/> 1-In{o'( From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.
PRINTED NAME JOB TITLE / RESPONSIBILITY SIGNATURE (1) DATE SIGNATURE (2) DATE NOTE
\IUmll,"~ [\l)!JjI,\\:;,"lllllllk ~
s.~evv.~_r 1;:t.:t.F !f2Aw'j, b,-IJ<>l
~ ~
~~ 07-?f:y~ ~~
" " ' ,,~--~
B"'i "~~,, C)y:f7iJ<1' ri* Bulh In!! - C;fl1 s>'" 51fh
;6'f~~.~
s,",c~.i"oo",-S""~~. w/ bh.- stffrl;Vf< ftr' te/'
~
~
>~1,
..~
yl#'J'i
}\~
t1. t./il&. 37/7/-;; 9ii . .
.~h.
~
~~
..~
~**o I~ ,
~
-r-
.r*bdY~'~ ------~(~/~r=;-~~--------------~~~~~~------~39~~~~~~~-----
_. . ti'f:" 71lli~
~~17'()P CP>
,,"1 -
5ltft'*u:ct~ ffv:c ES-201, Page 26 of 27 fJ I of'-:5
ES*201 Examination Security Agreement Form ES-201-3
- 1. Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of ;/zit
;'zit ~ ~,jdfas of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been a~thorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.
- 2. Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of '-Ht66 ( 1fi1/o~From the date that I entered into this security agreement until the completion of examination administration, I did not Ht (
instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC. PRINTED NAME JOB TITLE / RESPONSIBILITY SIGNATURE (1) DATE SIGNATURE (2) DATE NOTE
- 1. 5S"Ay 'f!'M$(., b & o t J i "?_-
- 2. _'AM':-? ~U.?TE(? i.e.; 71\ ,_ _
- 3. C~/Q'I A. ("If*m., Bi.' ~~ .
- 4. -fi5t+tJ QUlt,LeY I/ccJ i ? ~
io ~ SRD ,- ~
~no ~~
S'"
\ I' ~- ~
,!'\C,
--= ',(
~
51?()
~~
..2:::..2L.~
.. >' f!..o ~-S/~
n . Q')qr ~~
~ ~"4<L S<~",L ;~'T_,v"' PPJ '~.'
t: eI oNe'1 1" h , ir(,.(cf;..... << 0 ~ pavl 'i, efliL';X:U ' j J I ) ' ; ( y \ c J ._ _ fJ l cP ') ES-201, r 27 of 28
ES*201 Examination Security Agreement Form ES-201-3
- 1. Pre-Examination or?/?vk' I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) Of-l{l,tb 1/q/;,as r(tl/;,as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not b en authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g" acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback), Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee, I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised,
- 2. Post-Examination To the best of my knowledge I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) ofT/~o ~ 1-1n!~'iFrom the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC, PRINTED NAME JOB TITLE I RESPONSIBILITY Q! SIG,NATU~E (1)
SIQNATURE DATE SIGNATURE (2) DATE NOTE 1. 4,
= ~: ;./~ ?~ =:'~ 4!;z~_
!.,'_i({\
- 3,3,2, _ _ _ _,L,
- t+
$('/4'" : -: ":"-. . .r'- '=-=
. . ...,.,..) ) 5:-_r",;;.:::th.tJ?le"c{%,-,-
~v<l_V'5~,\J.\+-LoveP~fJ
{jj)~
~
1l,b~ £LA-~J=
~~-E:'?~;;~'J:\~76l;~ttf - - - - - - - - - - - - - - - - - - - - - - - -
~~~~~:::e::.3!="'==~--?!a ~~ ~ ~~=
_ _ _ _ _ _ _ _ _ _ _ _ _ _I _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ 11M-
~__
4, _ _______ 5, _
- 5. __ _ __ __ __ ___ __ __ _________ _ __ ____ _ _________ ______ _______ ___ _______ ____ ___ __
6, _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _____________ __ _ _-_-_-
- 7. _
7, ___________________________________________________ 8, _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
- 8. -_ --__ ___________ _ _____
9, ______________________________________________________________ _ _ _ _ _ _ _ _ -_-_-_ _ - _-_ _ 10, _____________________________________________________________ 10,________ _ _ __ __ 11,1 1 _,_ __ __ __ __ __ __ ____ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ __ ___ 12, _ _______________________________________________________________ 13, _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ 13, 14. 14 . _ 15, 15 . _ _ ====== NOTES: -- ()--:> 1/ 5 ~ G't- > ES-201, r 27 of 28
ES-201 Examination Security Agreement Form ES*201*3
- 1. Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.
- 2. Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of . From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.
PRINTED NAME JOB TITLE / RESPONSIBILITY SIGNATURE (1) DATE SIGNATURE (2) DATE NOTE
- 1. _
- 2. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __
2.
- 3. _
________ === ===
- 4. _ __ _ __ ____ __ __ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ___ _ __ __ _ _
- 5. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ __ _ __ _ _
- 6. _
- 7. _
- 8. _
- 9. _
- 10. _
- 11. ___ __ _ __ ____ __ __ _____ __ __ __ _ __ __ __ ____ __ ___ __ __ __ _ __ __ ______ _ _ _____ __ __ _ ____ __ __ __ _ __
- 12. ___ __ _ ____ __ __ __ _____ __ __ __ _ __ __ __ _____ ____ __ __ _ __ __ ____ ___ __ __ ____ __ _ __ __ ____ __ _ _____
- 13. _ __ _ __ ____ __ __ _____ __ __ __ _ __ __ __ ____ ___ __ __ __ __ ___ __ ______ __ _____ __ __ __ _ ____ __ __ _ __
14.
- 14. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ __
15.
- 15. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ __ ___
NOTES: ES-201, r 27 of 28
ES-301 Administrative Topics Outline Form ES-301-1 Facility: Sur!}: Power Station Date of Examination: 7/20109 Examination Level: RO D SRO r;gJ Operating Test Number: 2009-301 Administrative Topic Type Describe activity to be performed (see Note) Code* Calculate Shutdown Margin (at Power) Conduct of Operations RIN
Description:
Calculate shutdown margin for a stuck control G2.l.25 (3.9/4.2) rod. Review O-OSP-ZZ-004 to determine if conditions allow for fuel movement. Conduct of Operations SIN G2.1.40 (2.8/3.9)
Description:
Review procedure and board indication to determine if fuel movement can commence. Respond to a main control room chiller failure in accordance with O-OP-VS-006, Control Room and Relay Room Equipment Control RIN Ventilation System. G2.2.37 (3.6/4.6)
Description:
Determine ifMCR chiller alignment is adequate and complies with technical specifications. Determine the applicability of a RWP for a specific job and determine which personnel may be assigned the task based on Radiation Control RID personal qualifications and dose limitations. G2.3.7 (3.5/3.6)
Description:
Calculate dose received verse limitations given. Classify an Event and Recommend PAR. Emergency ProcedureslPlan RIN
Description:
Given an event, determine the EAL G2.4.4l (2.9/4.6) classification and protective action recommendation, if applicable. NOTE: All items (5 total) are required for SROs. RO applicants require only 4 items unless they are retaking only the administrative topics, when all 5 are required.
- Type Codes & Criteria: (C)ontrol room, (S)imulator, or Class(R)oom (D)irect from bank (:':3 for ROs; ~ for SROs & RO retakes)
(N)ew or (M)odified from bank (:2:1) (P)revious 2 exams ( ::;1; randomly selected)
ES*301 Control Room/ln*Plant Systems Outline Form ES*301*2 Facility: Sur[y Power Station Date of Examination: 07/20/2009 Exam Level: RO D SRO-I D SRO-U [gJ Operating Test No.: 2009-301 Control Room Systems@ (8 for RO); (7 for SRO-I); (2 or 3 for SRO-U, including 1 ESF) System / JPM Title Type Code* Safety Function
- a. Load the #3 EDG on the 1J bus iaw AP-17 .OS. EPEOSSEA 1 .06 D/E/LIS 6- Electrical (4.1/4.S)
- b. Establish Redundant HHSI Flowpaths 006A4.07 (4.4/4.4) S/LID 2- RCS Inventory Control
- c. Configure spray systems in accordance with ECA-1.1 (Loss of A/N/EN/LIS 5- Containment Emergency Recirculation). 026A2.03 (4.1/4.4) Integrity d.
e. f. g. h. In-Plant Systems@ (3 for RO); (3 for SRO-I); (3 or 2 for SRO-U)
- i. Increase RCS boron SOO ppm in accordance with AP-40.00 step AlE/LINIR 1- Reactivity
- 17. 004A2.11 (3.6/4.2) Control
- j. Isolate Service Water to #3 MER During Flooding. 076A2.01 AlDIE 4- Heat Removal (3.S/3.7) from Core k.
@ All RO and SRO-I control room (and in-plant) systems must be different and serve different safety functions; all 5 SRO-U systems must serve different safety functions; in-plant systems and functions may overlap those tested in the control room.
- Type Codes Criteria for RO / SRO-I / SRO-U (A)ltemate path 4-6/4-6/2-3 (C)ontrol room (D)irect from bank 5,9 / 5,8 / 5,4 (E)mergency or abnormal in-plant ~1 / ~1 / ~1 (EN)gineered safety feature - / - / ~ (control room system)
(L)ow-Power / Shutdown ~1 / ~1 / ~1 (N)ew or (M)odified from bank including 1(A) ~/ ~/ ~1 (P)revious 2 exams 5,3/ 5,3/ 5,2 (randomly selected) (R)CA ~1 / ~1 / ~1 (S)imulator
fiNAL t>P£f<.fI.I1J\1b ToT tYl /to J..1.,,, 9 ES-301 Operating Test Quality Checklist- Form ES-301-3 Facility: Surry Power Station Date of Examination: 07120/2009 Operating Test Number: 2009-301 OperatingTest Initials
- 1. General Criteria a b* C#
- a. The operating test conforms with the previously approved outline; changes are consistent with sampling requirements (e.g., 10 CFR 55.45, operational importance, safety fundion distribution).
Cp @ h$ Qi)
- b. There is no day-to-day repetition between this and other operating tests to be administered during this examination. 4 rre
- c. The operating test shall not dupliCate items from the appliCants' audit test(s).
testes). (see Section 0.1.a.) 0/ ~ fIfE>
- d. Overlap with the written examination and between different parts of the operating test is within acceptable limits.
e..f
~ me
- e. It appears that the operating test will differentiate between competent and less-than-competent appliCants at the designated license level.
c-p ~
@ y}1p
- 2. Walk-Through Criteria -- - -
- a. Each JPM includes the following. as applicable:
~ initial conditions
.... initiating cues
.- references and tools, including associated procedures
.-1' reasonable and validated time limits (average time allowed for completion) and specific
. designation if deemed to be time-critical by the facility licensee
~
operationally important specific performance criteria that include: v- detailed expeded actions with exad criteria and nomenclature "1 r-- (fff system response and other examiner cues "- v_ v- statements describing important observations to be made by the applicant v::. criteria for successful completion of the task
...,. identifteation of critiCal steps and their associated performance standards
...:::: restrictions on the sequence of steps, if applicable
-y ~
- b. Ensure that any changes from the previously approved systems and administrative walk-through outlines (Forms ES-301-1 and 2) have not caused the test to deviate from any of the acceptance ro;p> m6 criteria (e.g., item distribution, bank use, repetition from the last 2 NRC examinations) specified '--
on those forms and Form ES-201-2.
- 3. Simulator Criteria - - -
The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES-301-4 and a copy is attached. 4 @ 1M? a. b. Author Facility Reviewer(*) SkiD irwin 1 Skip David Wilson 1 Printed Name 1 Signature
~.;r/
/Y.~
J. 'J5,{;-
.J ,i.o - -
l-l.q
;h.L~q r te OJ
- c. NRC Chief Examiner (#) MA~'f, A. BA'Tc~/ 1tlJ!2 MA~'f, ;tlJl) I7~~ ;t;~ I Q'J/JoJ~bCJ9
- d. NRC Supervisor ~7.~/~
~7.~ / ;fttuJ/~ 69vS--/b'f (Jit;~/""
NOTE:
- The facility signature is not applicable for NRC-developed tests.
l( \ J)
# Independent NRC reviewer initial items in Column *c"; chief examiner concurrence required. re~uired.
AN~L 6P~IJ" TcsT 0'7}o/.t6J1 C;c.eN$w5 t,.(~ 3 ES-301 Simulator Scenario Quality Checklist Form ES-301-4 Facilty: Surry Power Station Date of Exam: 7/20/2009 Scenario Numbers: 1 /2/3 Operating Test No.: 2009-301 QUALITATIVE ATTRIBUTES Initials a b* C#
- 1. The initial conditions are realistic, in that some equipment and/or instrumentation may be out of service, but it does not cue the operators into expected events. c:..( ~ nrB q
- 2. The scenarios consist mostly of related events. ~ lme 3.
....Eachtheevent description consists of point in the scenario when it is to be initiated the malfunction(s) that are entered to initiate the event
~ t'l!45
~
/ the symptoms/cues that will be visible to the crew d
.A the expected operator actions (by shift position)
"" the event termination point (if applicable)
- 4. No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario without a credible preceding incident such as a seismic event. 4. ~ ft1V
- 5. The events are valid with regard to physics and thermodynamics. C/ Ir,;;;/ mfY
- 6. Sequencing and timing of events is reasonable, and allows the examination team to obtain complete evaluation results commensurate with the scenario objectives. ~ ~ i1$
- 7. If time compression techniques are used, the scenario summary clearly so indicates.
Operators have sufficient time to carry out expected activities without undue time constraints. Cues are given. C-/
~ m{;
- 8. The s.imulator modeling is not altered. C/ ~J I~
- 9. The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator performance deficiencies or deviations from the referenced plant have been evaluated to ensure that functional fidelity is maintained while running the planned scenarios.
eft ~ ~
- 10. Every operator will be evaluated using at least one new or significantly modified scenario.
All other scenarios have been altered in accordance with Section 0.5 of ES-301. C£ [i);? ~
- 11. All individual operator competencies can be evaluated, as verified using Form ES-301-6 (submit the form along with the simulator scenarios). ~ I@ ~
- 12. Each applicant will be Significantly involved in the minimum number of transients and events specified on Form ES-301-5 (submit the form with the simulator scenarios).
~ @ inf>
- 13. The level of difficulty is appropriate to support licensing decisions for each crew position. q
~ fY'If; Target Quantitative Attributes (Per Scenario; See Section D.S.d) Actual Attributes - - -
- 1. Total malfunctions (5-8) 7/817 V ~ lme
- 2. Malfunctions after EOP entry (1-2) 2/4/3 c./ ~ In$
- 3. Abnormal events (2-4) 3/2/4 U 6iJ Tl'r17
- 4. Major transients (1-2) 2/3/2 4' .~ me
- 5. EOPs entered/requiring substantive actions (1-2) 1/2/2 C{ I~ t1~
- 6. EOP contingencies requiring substantive actions (0-2) 1 /1 11 c..,( ~ 1M)
- 7. Critical tasks (2-3) 3/2/2 <:{ I~ I~
1:::/ tv (j L 0 Po? ATI AI" Ti"57 o7Jlt/~? C;ce'/IJIWIC '-I ES-301 Simulator Scenario Quality Checklist Form ES-301-4 I Facilty: Surry Power Station Date of Exam: 7/20/2009 Scenario Numbers: 4 1 / Operating Test No.: 2009-301 QUALITATIVE ATIRIBUTES Initials a b* r;# The initial conditions are realistic, in that some equipment and/or instrumentation may be out
~ IL~
1. of service. but it does not cue the operators into expected events. ~ tnt
- 2. The scenarios consist mostly of related events. qr (W ~ ../11$ 111$
- 3. Each event description consists of
- the point in the scenario when it is to be initiated the malfunction(s) that are entered to initiate the event the symptoms/cues that will be visible to the crew
~ ~ Mb
\......---
\,.../
- the expected operator actions (by shift position)
- the event tennination point (if applicable)
- 4. No more than one non-mechanistic failure (e.g .* pipe break) is incorporated into the scenario without a credible preceding incident such as a seismic event.
q I@ Irt$
- 5. The events are valid with regard to physics and thermodynamics. c..., ~ ~
c...r
- 6. Sequencing and timing of events is reasonable, and allows the examination team to obtain complete evaluation results commensurate with the scenario objectives.
q
~~ lY$
- 7. If time compression techniques are used. the scenario summary clearly so indicates.
Operators have sufficient time to carry out expected activities without undue time constraints. ~ (j ~ Cues are given.
- 8. The Simulator modeling is not altered. '-/' w/ 1mt7
- i11 1m!?
- 9. The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator perfonnance deficiencies or deviations from the referenced plant have been evaluated 4 (iI (YlfJ to ensure that functional fidelity is maintained while running the planned scenarios.
CV ~I/) /fJ1/) yr1l7
- 10. Every operator will be evaluated using at least one new or significantly modified scenario.
All other scenarios have been altered in accordance with Section 0.5 of ES-301. .;-
./
- 11. All individual operator competencieS can be evaluated, as verified using Fonn ES-301-6 (submit the fonn along with the simulator scenarios).
'-{'
I,,;
~) ()ltJ
./
./
- 12. Each applicant will be significantly involved in the minimum number of transients and events specified on Fonn ES-301-5 (submit the form with the simulator scenarios). 7' I§?
& of
- 13. The level of difficulty is appropriate to support licensing decisions for each crew position. C/ I~t) fI,,) I~ ({f$
Target Quantitative Attributes (Per Scenario; See Section D.5.d) Actual Attributes -- -
- 1. Total malfunctions (5-8) 6/ 1 W
<./ r';) n1f; n1£;
- 2. Malfunctions after EOP entry (1-2) 2/ / c( ~) {I!$
!TiJ,l}
- 3. Abnormal events (2-4) 4/ / .", ~ Iyye ~
- 4. Major transients (1-2) 2/ / c/ ~ f1'~ h~
- 5. EOPs entered/requiring substantive actions (1-2) 2/ 1 </ ~ Q rtt~
- 6. EOP contingencies requiring substantive actions (0-2) 11 / c/ ~ l115 \~
- 7. Cri1ical tasks (2-3) 3/ / </ I~ Jlf IQ l~
F/NItL OPE;~fr1li<Jl1 "TFS-r O'lf'Ii/ZfXIf ES-301 Transient and Event Checklist Form ES-301-5 Facility: Surry Power Station Date of Exam: 7/2012009 Operating Test No.: 2009-301 A E Scenarios P V 1 2 3 4 T M P E L N 0 I CREW CREW CREW CREW T N I T POSITION POSITION POSITION POSITION A I C S A B S A B S A B S A B L M A T R T 0 R T 0 R T 0 R T 0 U N Y 0 C P 0 C P 0 C P 0 C P M (*) T P R I U E RO RX N/A 1 1 0 0 NOR
- 1 1 1 SRO-I I/C 1,2,3 4 4 2 0
SRO-U 4,5 [gj MAJ 4,5 2 2 1 TS 1,2 0 2 2 RO RX N/A 1 1 0 0 NOR 1 1 1 1 SRO-I I/C 2,3, 4 4 2 0 SRO-U 4,5 [gj MAJ 4,5,6 2 2 1 TS 2,3 0 2 2 RO RX 4 1 1 0 0 NOR
- 1 1 1 SRO-I I/C 1,3, 4 4 2 0
SRO-U 5,6 [gj MAJ 5,6 2 2 1 TS 1,2,3 0 2 2 RO RX N/A 1 1 0 0 NOR 1 1 1 1 SRO-I I/C 2,3, 4 4 2 0 SRO-U 4,5 [gj MAJ 4,5 2 2 1 TS 2,3 0 2 2 Instructions:
- 1) Check the applicant level and enter the operating test number and Form ES-D-1 event numbers for each event type; TS are not applicable for RO applicants. ROs must serve in both the "at-the-controls (ATC)"
and "balance-of-plant (BOP)" positions; Instant SROs must serve in both the SRO and ATC positions, including at least two instrument or component (I/C) malfunctions and one major transient, in the ATC position. If an Instant SRO additionally serves in the BOP position, one I/C malfunction can be credited toward the two I/C malfunctions required for the ATC position.
- 2) Reactivity manipulations may be conducted under normal or controlled abnormal conditions (refer to Section D.5.d) but must be significant per Section C.2.a of Appendix D. (*) Reactivity and normal evolutions may be replaced with additional instrument or component malfunctions on a 1-for-1 basis.
- 3) Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to the applicant's competence count toward the minimum requirements specified for the applicant's license level in the right-hand columns.
- In accordance with Instruction #2, Scenario wlth Instructlon Scenarlo #1 and #3 are using uSlng a component malfunction in place of a normal evolution.
ES-301 Competencies Checklist Form ES-301-6 Facility: J "r".tQr St::ltinn OJ' Surry Power Station Date of Examination: 7/20/2009 Operating Test No.: 2009-301 APPLICANTS RO D BOP D RO D RO D SRO-I D SRO-I D SRO-I D SRO-I D SRO-U [8J SRO-U D SRO-U D SRO-U D Competencies SCENARIO SCENARIO SCENARIO SCENARIO 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 InterpretlDiag nose All 2,3, 1,2, 2,3, Events and Conditions 4,5, 3,5, 4,5 6 6 Comply With and All All All All Use Procedures (1) Operate Control N/A N/A N/A N/A Boards (2) Communicate All All All All and Interact Demonstrate All All 1,3, All Supervisory Ability (3) 4,5, 6 Comply With and 1,2 2, 3 1, 2, 3 Use Tech. Specs. (3) 2, 3 Notes: (1 ) Includes Technical Specification compliance for an RO. (2) Optional for an SRO-U. (3) Only applicable to SROs. Instructions: Check the applicants' license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.
THE FINAL WRITTEN EXAM SAMPLE PLAN IS THE COMBINATION OF THE DRAFT WRITTEN EXAM SAMPLE PLAN AND FORM ES-401-4 (RECORD OF REJECTED KlAs).
ES-401 Record of Rejected KIAs Form ES-401-4 Tier / Randomly Reason for Rejection Group Selected KIA T2/G2 027G2.2.22 Unable to develop a discriminating SRO-Only question that tested an iodine removal function associated with Technical Specifications. Randomly and systematically T2/G21079G2.2.22). selected a new system within T2/G2 (079G2.2.22). T3 PWG-2.1.45 Unable to develop a discriminating question that tested applicants on using diverse indications to validate response could meet the criteria for an SRO-Only question. Randomly and systematically selected a new Conduct of Operations PWG KIA (G2.1.20) T2/G1 013G2.4.49 Unable to develop a discriminating question associated with immediate operator actions associated with ESFAS due to the limited number of immediate operator actions that the applicants are responsible to perform. Randomly and systematically selected a new KIA for the system (013K2.01) T1/G1 008AK1.02 Unable to develop discriminating question. Randomly and systematically selected a new KIA: 008AA2.20 T1/G1 00054G2.4.4 7 Unable to construct a SRO-Ievel question on the ability to diagnose and recognize trends related to a loss of main feedwater casualty. KIA is more suited to JPM or scenario than written exam question. Randomly
&systematically selected new Generic KIA:
0OO54G2.2.25 T2/G1 008G2.1.31 Unable to construct a discriminatory RO-Ievel written exam question dealing with the ability to locate control room switches as it relates to the CCW system. KIA is more suited to JPM or simulator scenario than written exam question. Randomly & systematically selected new Generic KIA: 008G2.1.7 T2/G1 0064K4.04 Unable to construct a discriminatory RO-Ievel written exam question dealing with the EDG system design features and interlocks associated with overload ratings (more of a nameplate feature). Randomly & systematically selected new KIA: 0064K4.10
ES-401 Record of Rejected KIAs Form ES-401-4 T2/G1 0103K4.04 Unable to construct a discriminatory RO-Ievel written exam question dealing with the containment system design features and interlocks associated with access hatches. Topic is non-licensed operator level. Randomly & systematically selected new KIA: 0103K4.06 T2/G2 0055G2.2.25 Unable to construct a discriminatory SRO-Ievel written exam question: the condenser air removal system is not part of Surry Technical Specifications or Technical Requirements Manual. Randomly & systematically selected new Generic KIA: 0055G2.4.6 T2/G2 0086A1.03 Unable to construct a discriminatory RO-Ievel written exam question dealing with the fire protection system and fire doors. This topic is normally focused on administrative requirements and compensatory measures, and is therefore primarily SRO-only level knowledge. Randomly & systematically selected new KIA: 0086A 1.05 T3 SRO G 2.3.6 Unable to construct a discriminatory SRO-Ievel written exam question dealing with SRO ability to approve release permits. At Surry, the HPs approve release permits. Randomly & systematically selected new Generic KIA: 2.3.12 T1/G1 SRO 062 Surry has no procedure/valve line up to perform this AA2.03 evolution. Randomly & systematically selected new KIA: AA2.06 T1/G2 068 G2.4.21 Surry has nor formal process to access the CSF outside of the control room. Randomly & systematically selected new KIA: G2.4.42 T2/G1 012 K1.01 Surry's RPS system is DC with no-power coming from an AC source. Randomly & systematically selected new KIA: K1.05 T2/G1 SRO Aux operator actions outside of the control room was not 059G2.4.35 considered a basis for SRO level of knowledge. Randomly & systematically selected new KIA: G2.4.14
ES-401 Written Examination Quality Checklist Form ES-401-6 Facility: Date of Exam: Exam Level: RO LJ SRO 0 Initial Item Description a b* c*
- 1. Questions and answers are technically accurate and applicable to the facility. .1'6 NjJ ell lell
- 2. a.
b. NRC KlAs are referenced for all questions. Facility leaminQ leaming objectives are referenced as available. \$ CO!(
- 3. SRO questions are appropriate in accordance with Section D.2.d of ES-401 SROquestions rrS rrfB &I(
- 4. The sampling process was random and systematic (If more than 4 RO or 2 SRO questions were repeated from the last 2 NRC licensing exams, consult the NRR OL program office). ~ vP/(
- 5. Question duplication from the license screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate:
_ the audit exam was systematically and randomly developed; or
~ &If
_ the audit exam was completed before the license exam was started; or L. the examinations were developed independently; or _ the licensee certifies that there is no duplication; or _ other (explain)
- 6. Bank use meets limits (no more than 75 percent Bank Modified New from the bank, at least 10 percent new, and the rest new or modified); enter the actual RO / SRO-only ~ 0 6 if lO .1"t / 11D(1t>
<0 , ?t ~ r!1(
question distribution( distribution(s) s) at right. 9% /01. SI3V3'1~
- 7. Between 50 and 60 percent of the questions on the RO Memory CIA
}l\~
exam are written at the comprehension! analysis level; the SRO exam may exceed 60 percent if the randomly 31 #lie l/i( \q C4( selected KlAs support the higher cognitive levels; enter the actual RO / SRO Question question distribution(s) at right. tjl.3% / ;tiJ't-;' 58;7'1, / 'll9 %
- 8. References/handouts provided do not give away answers or aid in the elimination of distractors. f$ cJ(
- 9. Question content conforms with specific KIA statements in the previously approved examination outline and is appropriate for the tier to which they are assigned; Mf; @(
deviations are justified.
- 10. Question psychometric quality and format meet the Quidelines guidelines in ES Appendix B. Trtf:, 'WI LA!(
- 11. The exam contains the required number of one-point, multiple choice items;
~ ,It
,~ cfl!(
the total is correct and aQrees agrees with the value on the cover sheet. Printed Name / Si/:1(;" Date
- a. Author N\~ A, ~ATC:~ L j), . I~]:) IF:::I.Ul j Jcb'?
- b. Facility Reviewer (*) N/A LI:!
/>;;rra N
- c. NRC Chief Examiner (#) r C'M)1 /1()Jo/I7 /~
MJr l1();./f7 / c - ~X-::: ---- ')./' kV!
- d. NRC Regional Supervisor 11.Airnf 11.A.irl'J1l. _-r. Wlt>A1bJJAJ 1.I._f. PA"-. ~
Vtlfr>AlJ:J.jAJ ~A'":'-. ~ ~ I/ l'4:l,!!u'J, 1\1-
.~.~
Note:
- The facility reviewer'S initials/signature are not applicable for NRC-developed examinations.
# Independent NRC reviewer initial items in Column "CO; chief examiner concurrence required.
ES-401, Page 29 of 33
Rev. 9 Written Exa Exall. U lew Worksheet Form Surry 2009-301 Q KJA# B L L Psychometric Flaws Content Flaws U Comment
# M 0 0 Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explauation N K D Focus Cred Cred KJA Only S Dist Dist BIW
GENERAL COMMENT
S This 401-9 was used to track changes from the draft questions submitted to the Chief Examiner from the three NRC exam writers. This 401-9 does not track any changes that were made via consultation with the licensee after the draft exam was provided to the licensee. Changes made in consultation with the licensee are documented in the LXR file. B= Bank / M=Modified / N=New / F=Fundamental Level (I.E. Memory) / H=Higher Cognitive Level (I.E. C/A) For All BANK questions: swap the order of the answer choices so that applicants cannot rely on recall of the correct answer location. Print out LXR categories for reviewer. Without that info the reviewer cannot tell if the question is designated as a higher or lower cog question; bank, mod, or new, etc. KIA Match Analysis should be written to describe the knowledge that is required to answer the question. ROEXAM G2.3.7 N H 2 x x x S Word the question statement so it does not test on a degree of correctness. Also word the statement to solicit the exact information in the answer (RO) choices. For example, RWP requirements are not being tested in answer choices A and D. Consider the following: Given the above conditions, which one ofthe following correctly describes EPIP-4.04 requirements for the voluntary entry into the Radiologically Controlled Area (RCA)? Q replaced. MAB 5/19/2009 Is this question testing SRO-only knowledge? Typically, SEM responsibilites are not required of ROs. Does the site have an RO learning objective for knowing the SEM duties? Q replaced. MAB 5/1912009 C appears to be a subset ofB. The RAD is a radiological control personnel and he likely operates from the TSC. The only real difference between the two distractors is that B states that the SEM is required to sign, whereas C is silent on that detail. Q replaced. MAB 5/19/2009 For purposes of the written exam question, it would be acceptable to state in the question that the RWP contains the following information, rather than have the licensee supply an RWP. Either way is OK, but you may be Page I of33
Rev. 9 Written Worksheet Form Surry 2009-301 Q KJA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues TfF 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KJA Only S Dist Dist B/W able to test the tbe same knowledge and reduce the tbe burden on the tbe licensee.
OK MAS 0512812009 If the tbe previous comment is incorporated, tbenthen the question could ask for the answer choice tbat tbe that minimizes dose. Or tbethe RWP information tbat that you provide could contain tbethe statement similar to what you included in your NOTE TO SURRY. OK MAB 0512812009 Q will be satisfactory after discussion and possible incorporation of above Comments. OK MAB 05128/2009 029EAl.02 N H 2 x S Reword question statement to allow for more efficient wording of answer choices and to avoid testing on degree of correctness which is implied by (RO) the word "most": tbe Which one of thetbe following correctly describes the next step required to align an alternate boration flow patb path in accordance witb with I-FR-S.l, Response to Nuclear Power Generation!ATWS? Q significantly modified. MAB 5/1912009 B and C could successfully be argued as correct. Much weight is placed on the word ONLY when you are attempting to test that tbat there is no order of preference for tbe the actions. Q significantly modified. MAB 5/1912009 B and C contain conditional answers. Placing conditions in tbe the answer choices should only be done as a last resort. Generally, tbe the conditions of the question should be placed in thetbe stem. Q significantly modified, MAB 5/19/2009 Operational validity? If an applicant performs the tbe actions of B would it be incorrect? If an applicant performed the tbe actions of C would it be incorrect? Q Significantly modified. MAB 5119/2009 A possible fix could be to include how the turbine is being operated (stem would require some modification also). For example: A. Manually SI. B. Place switches for CH-MOV-lllSB & D to OPEN and place switches for CH-MOV -lliSC & E to CLOSE. Ramp turbine down using GV close (or whatever Surry has). C. Place switches for CH-MOV -1115B-ll1SB & D to OPEN and place switches for CH-MOV -IIISC
-1115C & E to CLOSE. Ramp turbine down using the tbe limiter.
D. Place the tbe blender mode selector switch to BORATE and start the tbe blender. Close thetbe MSTVs. Page 2 of33
Rev. 9 Written Exam II lew Worksheet Form Surry 2009-301 Q KJA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KJA Only S Dist Dist BIW Q significantly modified. MAB 5/1912009 Distractors "A" and "B": Delete..... to allow increased injection flow."
This information is not needed to make the answer choices Wlique. Incorporated. MAB 0512812009 Consider using 2210 psig in "A" and "B". The reason this value may be more plausible is that it coincides with the low P alarm setpoint, which is a plant specific parameter. The Alarm setpoint would likely be more credible to the applicant because there is a chance that the applicant has not received training on a part of the WOG which is not utilized at Surry. With this change, the question should be satisfactory. Incorporated. MAS 05128/2009 009EK2.03 N H 2 S The reading burden on this question is very high. Is there a method to present the pertinent info with a lower reading burden ~ (RO) Q was replaced. MAB 5/1912009 The question statement asks for the next actions, however the answer choices all contain information that are not describing the next action. For example, stating which SG is faulted does not describe an action. Q was replaced MAS 5/19/2009 Limit the words in the answer choices to only those that are necessary to make the answer choice unique. Q was replaced. MAB 5/1912009 Is knowledge of the KIA required to answer the question? Is knowledge of a SBLOCA and the interrelation with SGs needed to answer the question? Or does an applicant only need to diagnose a faulted SG and Wlderstand that E-2 will address that condition? Q was replaced. MAB 5/1912009 This question is testing an internal procedure transition. Does this fit the criteria of the SRO-only guidance? It is clear that an RO is required to know that there is a faulted SG, but is the RO required to know the procedure path when faced with both a faulted SG and a SBLOCA? Suggest walking through the SRO guidance to determine whether or not the question requires SRO-only knowledge to arrive at the correct answer. Also consider that the information is on the Continuous Action Page - does this help justify RO knowledge? Q was replaced. MAB 5/19/2009 Replacement Q is satisfactory. MAB 5/19/2009 W/E04EA2.2 M H 2 S Second part of A and B: How can transitioning out of ECA-1.2 be logical Page 3 of33
Rev. 9 Written Exam~w Worksheet Form Surry 2009-301 Q KlA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues TfF 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KIA Only S Dist Dist BIW
}.W~;(l4EA~.1 if pressure is still going down, thus the leak is not isolated? The question is sat with the exception of the second part of these two answer choices. We (RO) should attempt to replace these. Would it be possible to have them remain in ECA-I.2 instead of going to E-I? Consider it and discuss if needed.
Q was replaced, MAB 5/1912009 We should discuss the SRO-only aspects with a couple of other CEs. EOP basis information should be OK to test at an RO level, but we should check the SRO guidance as well as other CEs to be sure. Q was replaced. MAB 5119/2009 Compare this question to 026G2.1.7. This question is an RO question, whereas, 026G2.1.7 is an SRO question. They both look like the justifications would be sinJilar either for an RO question or for an SRO question. You could argue that this question is RO via a "mitigating strategy" logic, but then 026G2.1. 7 likely needs to be enhanced from a procedure selection perspective. Qjustifications bolstered for each question. OK MAB 0512812009 0040AA2.05 N F 2 S Can answer choices be simplified to only contain information that makes the answer choices unique? (RO) Q was replaced. MAB 5/19/2009 Make minor changes to question statement. . Q was replaced. MAB 5/1912009 Much of the information is unnecessary. Sometimes CEs will refer to teaching in the stem. This is when information is unnecessarily provided - namely information that the applicant is required to know. I.E. SI termination criteria. Delete the unnecessary info and massage the stem to accommodate the deletion. Q was replaced, MAB 5/19/2009 Does this question require SRO-only knowledge for procedure selection? The second half of the answer choices could require SRO-only procedure selection knowledge? Stopping the charging pump is the same in each choice, the knowledge needed is whether or not a transition to ES-l.l is required. Typically, internal procedure transitions are SRO-only. ill other words, ROs are required to know simple procedure entry conditions for the AOPs and main EOPs. More detailed procedure transitions within the EOP network are usually considered to be SRO-only. We should review SRO-only guidance and discuss with another CEo Q was replaced, MAB 511912009 Replacement Q is satisfactory. MAB 5/1912009 Page 4 of33
Rev. 9 Exam~ew Worksheet Written Exa Form Surry 2009-301 Q KlA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues T/F 1 Non >1 Non Partial Min Q- SRO E Explanation N K D Focus Cred Cred KIA Only S Dist Dist B/W 077AA2.07 N F 2 S Is the operability call an SRO-only function? Is it required for an RO to know the specific setpoints for detennining operability of high voltage (RO) systems. This should be reviewed against the SRO-only guidance and reviewed by second CEo Methods to justifY as RO may include that the voltage requirements are provided in a P&L from an SOP, or that the information is provided in TS above the line.
Qrevised to move away from operability IUld test TS entry, which is at the RO level. MAB 5/19/2009 The KIA match appears to be questionable., No supporting info was supplied that links the voltage requirements to the operability of the ESFs. I would suggest making the second part of the answer choices to test operability of the ESF equipment based on the offsite power supply, thereby better matching the KIA. OK after discussion IUld slight revision. MAS 0512812009 The KIA Match still appears to be questionable. The KIA requires knowledge of how generator voltage or grid disturbance will impact operational status of ESF equipment. OK after discussion and slight revision. MAS 05/2812009 Can the correct answer be supported by the Tech Specs and/or the Tech Spec Bases? OK after discussion and slight revision. MAS 0512812009 Does TS 3.16 require operability of 230 KV or 500 KV electrical distribution? OK after discussion and slight revision. MAB 0512812009 036AA2.03 M F 3 S Is B potentially incorrect? From the provided references, it appears that the FSAR does analyze SFP fuel handling accidents. Therefore, a fuel (SRO) handling accident occurring in the SFP is in fact analyzed in the FSAR, so how could the resultant TEDE at the EAB be higher than the SFP accident analyzed in the FSAR? CQmIl1.ent incorporated. MAB 5/1912009 Consider using 2597 for "D". 2605 is still very plausible, but is an operator required to know a random number chosen for a safety analysis cale? It may be more reasonable for an operator to know that 102% of nominal is a typical methodology for safety analysis cales. Discuss. The question will be sat either way, but I left it designated as "E" to ensure we have this discussion. RTP used. OK MAS 05128/2009 G2.4.30 N H 2 S I agree that the I&C generated signal from testing was invalid. What about the procedurally directed manual initiation? The procedurally directed Page 5 of33
Rev. 9 Written Exall' &1 lew Worksheet Form Surry 2009-301 ~I I~MOO I lI KlA# B L L L L Psychometric Flaws Content Flaws U U Comment Q N 0 0 K D Stem Focus Cues T/F 1 Non Cred
>1 Non Cred Partial Min Min I Q~
Q~ KIA SRO SRO Onlv Only E E S Explanation Dist Dist BIW I (SRO) I manually initiated SI appears to fit the definition of VALID as defmed in VPAP-2802. Incorporated InCOrporated by asking specifically about the AMSAC signal. OK MAB MAE 05/1912009. In the question statement, be specific on to whom the operator must make the report. I.E. to the NRC Operations Center. COUllllent incorporated. MAB 511912009 I agree with providing the stated references to the applicants. Comment incorporated. MAB MAE 5/1912009 Q is satisfactory. MAB 5/1912009 02602.1.7 N H 2 ? S Is there a reason why the question does not state what procedure they are in and what attachments they are performing? Also, is there any way that the (SRO) answer choices could provide specific attachments that they are supposed to transition to? By avoiding specifics in the stem and answer choices, the question becomes less operationally valid. Q modified. MAB 511912009 Is there a way to include the actions in the answer choices, vice stating that they will perform the RNO actions? Comment incorporated. MAB 5l1912oo9 Ensure the wording of "A" is terminology used in the supporting references. "Forced feed and bleed cooling" Wording OK MAD 0512812009 The SRO justification does not appear to be included in the notebook. or I overlooked it. Added. MABMAE 0512812009 How does the SRO justification for this SRO question compare to an SRO justification for RO Q W/E04EA2.2? justification bolstered, MAB Justification MAE 05/2812009 G2.2.22 3 S Consider deleted unnecessary parts of the stem and answer choices. Comment incorporated. MAB 5/1912009 (SRO) Q is satisfactory. MAB 511912009
", "" " ~~,y !"~ ~ ~:/~ ' .... :v, ",' ;** ,', '}- :"\,~.,
,~ . . , ";" :' ';:', ~
- .: ." Y".""~ol:'::' ',' '; ,,,/,,~~,, f:' " ~~
\ , " , ,
- 1 ',. " " , ~ <" '.".'..... :.:, " :
- ~~~~~~~~~~~~~~~ .. " : . '.' " ~." .' ':; " :.: .. '~
Page 6 of33
Rev. 9 Written Exam~ew Worksheet Form Surry 2009-301 Q KJA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues TIF 1 Non >1 Non Partial Min Q- SRO E Explanation N K D Focus Cred Cred KJA Only S Dist Dlst BIW (RO) Suggest writing "DECREASING" in lower case since it is the same in all answer choices. I would reconunend only capitalizing it if you were trying to emphasize between decrease in one distractor and increase in another distractor.
Incorporated. OK MAB 5/7/09 Suggest adding "approximately" prior to "I" in each answer choice. Incorporated. OK MAB 5/7/09 Will each tank lower at the same rate? There may be different flow resistance in the path for each tank. Suggest adding a statement that states for the applicant to assume that both tanks lower at the same rate. Incorporated. OK MAB 5/7/09 055G2.4.45 N H 2 S GoodQ idea. (RO) Question is higher cog (C/ A). Corrected. MAB 05129/2009 Consider adding "Inunediately" at the beginning of "B". Incorporated. MAB 0512912009 First Bullet: consider stating the IF-B6 has just alarmed. This will remove any doubt that vacuum has been lower than 26.5 inches for five minutes already. Now states alanned 5 minutes ago, which makes the other choice correct and allows KIA. to me met. OK MAB 05/28/2009 K/ A match needs to be reviewed with revised question. OK MAB 0512812009 0041A3.03 N H 2- S If steam dumps are open, would turbine load remain stable? Or, would 3 turbine load decrease? (RO) Last bullet deleted. OK MAB 05126/2009 Would Q be better if the plant was at 75%? This may improve plausibility of"C" because Trefwill top out at 100%, so rods would not move out.. Q remains at 100%. OK MAB 05/2912009 Does Surry have any logic that would cause the Tc failure to not affect Tave? I.E. Medium select or minimum Tc select? OK. Verified. MAB 05127/2009 0\1K6.06 B F 2 S A subset issue may exist with "C" and "D": If "0" was correct, then "c" would also be correct. Suggest wording them as follows: Page 7 of33
Rev. 9 Exam~ew Worksheet Written Exa Form Surry 2009-301 Q KlA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KIA Only S Dlst Dlst BIW (RO) C. The flow limit summator will limit charging flow to a minimum of25 gpm.
D. The flow limit summator will limit charging flow to a minimum of 30 gpm. htcorporated. OK MAB 05/07/09 G2.2A2 M H 2 S Question may be higher cog (C/A). Discuss. Q changed to CIA. OK MAS 05/07/09 (RO) Suggest changing the end of the question statement to: " ... .. .as as listed in Technical Specification 3.7, Instrumentation htstrumentation Systems?" Incorporated. OK MAB 05107/09 05107109 Is "D" actually correct? Do they actually have 49 hours before they are required to open the RTBs? Corrected. OK MAB 05/07/09 Suggest rewording "D" to: IMMEDIATELY suspend reactivity changes that are more positive than necessary to meet the required shutdown margin or refueling boron concentration limit and restore the inoperable channel to OPERABLE status within 48 hours or open the reactor trip breakers within the next hour." Incorporated. OK MAD 05/07/09 Minor enhancements suggested for "A". Incorporated. OK MAD 05/1812009 054G2.2.25 N F 3 S GoodQ. (SRO) 055G2A.6 N H 2 S The wording in "D" is a bit confusing: " ... steps/the .... " Corrected. MAB 0512712009 (SRO) APs can be used simultaneously with EOPs. By stating that the RO is concerned with degrading vacuum, it implies that the rate of degradation is more than the expected rate of degradation. In this case, why would it be incorrect to perform the AP in conjunction with the EOP? Do they have a procedure on procedure usage that would disallow it? Corrected. MAD 0512712009 Answer Choice Analysis may need to be updated. Done. MAD 05127/2009 Help explain how ES-3.1 is plausible when considering FASTEST methods to cooldown. Discussed plausibility. OK MAS 05127/2009 Page 8 of33
Rev. 9 Written Exam.....rew Exa Worksheet Form Surry 2009-301 Q KlA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues TfF 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KIA Only S Dist Dist BIW 007Al.03 N F 2- S Is 125F plausible? Tsat at atm pressure is 212F.
3 Q modified. OK MAB 0512612009 (RO) A stronger KIA match would be to place the temp at 125F and the level at 80%. Consider this change if it does not harm the question. Q modified. OK MAB 0$126/2009 Is the PRT design temp an operationally valid knowledge item for the RO? Would knowing this information allow an RO to perform his duties better or differently than ifhe did not have this knowledge? Is the PRT design temp minutia? Q modifi<<l. OK MAB 05/2612009 Is there an RO learning objective for the PRT design temp? Q modified. OK MAS 05126/2009 Most sites will have a normal level, temp, and pressure band for their PRT which is usually stated in their system op procedure. One option may be to test knowledge of the normal op band for temp, and/or pressure, and/or level. This may allow for a more operationally valid question. Qmodified. OK MAB 0512612009 008A2.04 M H 2 S KIA match: Can the diagnosis be made without the PRM info in the stem? If the applicant knows that pzr level is decreasing, the Th Barrier alarm is (RO) in, and the CCW surge tank level is lowering, is this enough to decipher the correct answer? If needed, suggest changing info in the stem to require the PRM info to answer the question. Q replaced. MAB 0512812009 KIA Match: The NUREG requires that the (b) part of the KIA is tested. The question must require procedure knowledge in order to meet the KIA. Q replaced. MAB 05/28/2009 Consider changing the second part of each answer choice to test procedure actions. I.E. A. RCP Th Barrier leak and l-CC-TV-120B failed to close. [action A] B. [same] [action B] C. CVCS letdown HX tube leak and LCV -460 failed to close. [action C] D. [same] [action D] Q replaced. MAS 0512812009 039Al.05 N H 2- S How could SG pressure reach 1530 psig when the SGs are equipped with
- 3 safety valves and SG PORVs?
(RO] Q replaced. MAS 05128/2009 Page 9 of33
Rev. 9 Written Exam~w Worksheet Form Surry 2009-301 Q KJA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues TIF 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KJA Only S Dist Dist B/W Suggest rewording the end of the question statement to: " ... (b) the target S/G pressure that corresponds to the RCS no-load Tsat setpoint?" This is a little more precise language because it takes coefficients of heat transfer, etc off the table as potential arguments for no correct answer, etc.
Q replaced, MAB 0512812009 The stem states that RCS temp is high. This is a statement of judgment. It would be better to state what the RCS temp is indicating and make the applicant recognize if that is high. It is usually more operationally valid to provide the indications that the applicant would have in the control room. Q replaced. MAB 05128/2009 Is the FSAR design limit for temp required knowledge for an RO? Does the facility have a learning objective? Would the RO take any actions based on knowing that 650F is the design limit? Q replaced. MAB 0512812009 One idea would be to test CET temp needed to place them in a red path and then test the SG pressure needed to get the RCS back to no-load values. This would be operationally oriented and it would also test a design value that definitely is RO required knowledge from the SFSTs. Q replaced. MAB 05/2812009 Enhancement to answer choices suggested on hard copy. Done. MAB 0512812009 061K5.02 N H I- S Q is likely higher cog (CIA) because the applicant has to use the thumb 2 rule to calculate the power ten minutes after trip and analysis is required to (RO) know how AFW demands will change over time. Granted ~ the Q is easy, but I think a mem level may be hard to justify. Changed to CIA. MAB 0512812009 This question is only testing GFE knowledge. Can the question be infonnation, such as plant-specific AFW modified to test site-specific information, control? Q modified to site specific info. MAB 0512812009 The second part of each answer choice tests the same knowledge that is being tested in the first part of each answer choice. I.E. The applicant uses his thumb rule to decide how much power has lowered at the ten minute point and then essentially uses the same knowledge that power and heat load are going down to understand that AFW flow requirements are also going down. Q modified to site specific info. MAB 0512812009 WIE06EK2.l N F 2- S GoodQ. Page 10 of33
Rev. 9 Written Exam Exam""ew Worksheet Form Surry 2009-301 Q KJA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues TIF 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KJA Only S Dist Dist BfW 3
(RO) W/EI6EALI N H 2 S Question is higher cog (CIA) for analyzing which procedure to use. Incorporated. MAB 05108109 RO Suggest rewording the end ofthe question statement to: .." ... and FR-Z.3 actions ensure that recirculation sump pH remains slightly basic?" Incorporated. MAll 05108/09 Answer choices contain a large amount of unnecessary reading burden. Consider: A (1) Enter I-FR-Z.I, RESPONSE TO CONTAINMENT HIGH PRESSURE." (2) To ensure the maximum cladding oxidation will not exceed 17% of nominal clad thickness, as required by 10CFR50.46, "ECCS Acceptance Criteria." Etc. Incorporated. MAB 05/08/09 What procedure is currently being performed by the operators? Ifa If a procedure transition is being tested, should we state in the stem from which procedure they will be transitioning? Incorporated. MAB 05/08/09 A parameter or two needs to be changed in the stem to make the correct answer a red or orange path. This is because rev I page 7 of the SRO guidance states that SRO-only knowledge cannot be claimed for red or orange path knowledge. Q replaced. OK MAB 05/2812009 036AA2.01 N H 2 S This information is from a 1985 SOER where an engineering calc was performed at Connecticut Yankee for the top 4-1/2 feet of a fuel assembly RO in the upender being exposed and about 3 feet of active fuel in the SFP being exposed. Is this operationally valid information on which to test an RO? Q modified. OK MAB 05/26/2009 Does Surry have an ARM that would indicate up to 50,000 Remlhr? Q modified. OK MAB 05126/2009 Does the facility have an RO learning objective that requires them to know this information? Q modified. OK MAB 0512612009 OS/2612009 Would knowing (or not knowing) this information discrimate between a competent and less than competent operator? Would an RO take different actions ifhe know the does rates were 500 Remlhr vs 500,000 Remlhr? Page II of33
Rev. 9 Written Exam ""ew Worksheet Form Surry 2009-301 Q KlA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KIA Only S Dist Dist BIW Q modified. OK MAB 05}2612009 If answers to the above questions justifY leaving the question on the exam, would it be better to simplifY the question as follows and reduce the reading burden?:
Q modified. OK MAB 05}26}2009 Which one of the following correctly states the expected ARM indication 30 feet from a newly discharge irradiated fuel assembly? A. 500 RemlHr B. 5,000 RemlHr C. 50,000 RemlHr D. 500,000 RemlHr Q modified. OK MAB 05}26/2009 The procedure states to evacuate the affected area (SFP or Containment). With rising rad indications in both locations, could it successfully be argued that both areas are affected? Would operators be wrong to evacuate both areas? Q modified. OK MAB 05/28}2009 To address the above concern, should the answer choices state either (I) Fuel Building evacuation is required, OR (2) Fuel Building evacuation is NOT required? Q modified. OK MAB 05}28/2009 Is the evacuation piece a Refueling SRO knowledge item? Q modified. OK MAB 05/28}2009 060AK3.02 N H 2- x S Does a VCT level and trend need to be provided in the stem to ensure the 3 rupture is in the gas space of the tank? Or is the Vent Stack #2 alarm (RO) enough? Q replaced. OK MAB OS/28}2009 The supporting documentation states a ground level puff vs a stack release. Does this impact the question? Q replaced. OK MAS 05}28}2oo9 The question statement should state "in accordance with a procedure whenever possible." Q replaced. OK. MAB 05/28/2009 It appears that AP-22 time critical actions are being tested a well as the basis for AP-5.20 action. Is this correct? I may need some help understanding the supporting documentation. Discuss with me when you get a chance. Page 12 of33
Rev. 9 Written Exam U 'ew Worksheet Form Surry 2009-301 Q KlA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues TIF 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KIA Only S Dist Dist BfW Q replaced. OK MAB 0512812009 SRO-only: I have concerns that going too deep in the FSAR safety analysis is not something for which an RO is required to know. FSAR safety analysis is not something that is defmed in Bruno's SRO guidance, and maybe it should be. One idea may be to modifY the question slightly to accommodate something like the following:
A. Iso UD within 25 min ...... ; Prevent exceeding 10CFRI00 limits ... B. Iso un within 25 min ...... ; Prevent exceeding 10 CFR20 limits .... C. Iso MCR verntilation vemtilation ....... ; Prevent exceeding 10 CFRIOO limits .. D. Iso MCR verntilation vemtilation ....... ; Prevent exceeding 10 CFR20 limits ... Q replaced. OK MAB 05128/2009 056AAI.29 N H 2- S A reference is provided from AP-12.0 I. I understand that the AP is being 3 directed by ECA-0.2. It may be a good idea to add the following to the (RO) first bullet in the current conditions: " ... SI REQUIRED" and performing the step to restore SW to CC HXs in accordance with 0-AP-12.01, "LOSS OF INTAKE CANAL." Incorporated. MAB 05108/09 In the question statement suggest placing "0-AP-12.01" between "the" and "restriction". Incorporated. MAB 05/08109 Suggest stating "Crosstie CC." prior to the second part of the answer choices to be more complete with what the procedure states. This will help take a "no correct answer" argument off the table. Incorporated. MAB 05108109 I was having trouble seeing how Part (2) was linked to greater than 8 hours after time zero. I need help understanding this. Explanation sat. MAD 05108109 Question is borderline memory /evel. I understand the basis being that the applicant only really needs to have the caution memorize and the other piece is open book and very little higher cog thought is used to answer the question (although some is required). Leave as memory for now, but we should discuss this when we review my comments. OK with. memory level after discussion. MAB 05/08/09 058G2.1.23 N F 2 S Whenever possible state iaw a procedure in the question statement. This goes a long way in "tightening" the question because it eliminates other (RO) procedures that the author may not even have considered when writing the question. Sometimes incorporating this suggestion may do more harm than good, but whenever possible it is a good idea to state iaw a procedure. MAB 05/08/09 Incorporated, MA13. Page 13 of33
I:;>-"'U~. Rev. 9 Written Exam ~ew Worksheet Form Surry 2009-301 Q Q KlA# B L L Psychometric Flaws Content Flaws V Comment
# M 0 0 Stem Cues T/F 1I Non >1 Non Partial Min Q-Q= SRO E Explanation
- N K D Focus Cred Cred KIA Only S Dist Dist 8IW Answer choices should generally only contain information that makes them
- unique from each other. I.E. These answer choices can be boiled down to:
A. (1) Generator output breakers will open as expected following a turbine trip, (2) 'A' RCP will stop, 'B' and 'C' Reps will remain running. ETC. for the other choices. Incorporated. MAB 05/08/09 KIA Match: The KIA requires testing knowledge of the ability to perform procedures. If the applicant understands how the plant systems work, he does not need to know anything about performing plant procedures. I.E. If the applicant knows that Gen bkrs will not auto open and that 'B' and 'C' RCPs will stop, then the question can be answered correctly. Q Q modified slightly to better match KJA. MAB 05108109 I think the question should be designated as higher cog (CIA). After discussion, QQ was designated as memory level. OK MAB 05108109. I am not sure if the Q was BanklMod/or BanklModIor New? IncotpOrated. MAB 05108109 073K5.02 N H 2 S Is all the information in the stem necessary? Can some of it be deleted to reduce the reading burden. (RO) Some deleted. OK MAB 05/2612009 Why is the equation provided to the applicants? Deleted. OK MAB 05/2612009 The question is only testing GFE knowledge for a IIr-squared calc. Is there a way to write a site-specific question? Struggled with plant-specific aspects, therefore question remains pure GPE. OK MAB 05126/2009 Is the correct answer the reading on the rad monitor or is it the factor by which the rad monitor reading would change from one distance to the other? Without providing what the rad monitor was reading at the first distance, how can the rad reading at the second distance be determined? Corrected. MAB 0512912009 064K4.l0 B H 2 ? S This question does appear to be higher cog (CIA). How does this question 064K4.04 compare to some of the other questions that you designated as memory level? (RO) Original Q Q Replaced. MAB 05/07/09 Is the question technically accurate? Why should the CC pump not be started? With the conditions provided, the CC pump can be started Page 14 of33
Rev. 9 Written Exam a lew Worksheet Form Surry 2009-301 Q KJA# B L L Psychometric Flaws Content Flaws V Comment
- M 0 0 Stem Cues TfF 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KJA Only S Dist Dist BIW because no Hi-Hi CLS is in progress.
Original Q Replaced. MAB (}5/07/09 I wrote a suggestion below the question. I was just brain storming, so I am not sure it makes sense, but it may spark an idea. Original Q Replaced. MAB 05107109 The question is testing on what an operator "should" do. It is always better to test on what an operator is "required" to do, unless the procedure step specifically states "should." Original Q Replaced. MAB 05/07109 It is usually better to tie the question statement directly to the procedure that is being tested. Suggestion adding iaw the procedure. Original Q Replaced. MAD 05/07/09 If it is not necessary to significantly revise the question, consider wording the question statement to precisely what is reflected in the answer choices. I.E.:
"Based on the current conditions, which one of the following correctly identifies the reason the 'A' component cooling (CC) pump cannot be started in accordance with I-AP-IO.07, LOSS OF UNIT I POWER?"
Original Q Replaced. MAB 05107/09 Replacement Q does not meet the K/A. The K/ A requires testing knowledge of EDG system design features and/or interlocks associated with overload. The proposed question tests knowledge of overloading the EDG, but it does not test knowledge of a design feature or interlock, rather it tests knowledge of a procedural limitation in the form of a caution. KIA changed. Q replaced again. MAD 5/1912009 An idea would be to test something with the EDG sequencer, which would be testing a design feature that is intended to prevent an overload condition of the EDG. KIA changed. Q replaced again. MAB 5/1912009 Would it be possible to change "A" and "D' to: A. F-58B I "A" pzr Htrs I P-3B D. P-3B I "A" pzr Htrs I F-58B This would eliminate the need for using the terminology "ONLY". It would also allow the applicant to nuke out the answer based on relative importance of the equipment, which would be a good idea. Discuss when you get back to the office. With my suggested changes, I would consider this question to be satisfactory. Third items added to other choices. OK MAB 0512812009 Page 15 of33
I:~-"U.L. Rev. 9 Written Exam lW.IrI'ew Worksheet Form Surry 2009-301 Q KlA# B L L Psychometric Flaws Content Flaws U Comment
# M 0 0 Stem Cues TIF 1 Non >1 Non Partial Min Q-Q= SRO E Explanation N K D Focus Cred Cred KIA Only S Dist Dist B/W lO3Al.Ol M F 2 S Are points on Figure 3.8-1 a memory item? Is a reference required to be provided to the applicant? If a reference is provided, would the question be (RO) a direct lookup?
Q modified MAS 0512712009 Is the design information important knowledge for an RO? Qmodified. MAS 0512712009 I think a more operationally valid and more discriminating question could be written if you were to test at what pressure and logic will contain isolate and at what pressure and logic can the isolation signal be reset. This would be very operationally oriented and it would move the question away from any minutia, SRO-only, or provided reference concerns. IncOrporated, MAB 0512712009 Are there any overlap issues with lO3K4.06? There may not be a contlict, but it should be explored and discussed to be sure. OK. MAB 05127/2009 Small editorial changes should be considered to simplifY the question presentation and increase the precision of the language in the answer choices. OK MAB 05128/2009 062K2.01 B F 2 S Good Q. Only minor enhancements needed. OK MAB 05108/09 (RO) The stem states that steam dump control is not affected. Is there a more operationally valid way to state this? What indications would tell the operator that it is not affected? If there is no clean way to provide plant indications, I am OK leaving it the way it is. Discussed and decided not to change. OK MAB 05/08/09 The stem states that there is a loss ofCC to all RCP thermal barriers. Does this occur because a valve fails closed? If so, there may be a more operationally valid way to provide this information. Discussed and decided not to change. OK MAB 05108/09 Should the last bullet be more specific? What is meant by other RCP loads? The supporting documentation provides some info like lube oil and stator coolers. It may be better to specifically state these. Incorporated. MAB 05/08109 Good question construction allows the applicant to read the stem, then read the correct answer and know, without, reading the distractors, that the "only" in the answer answer is correct. The issue with using the word "onlY" Page 16 of33
Rev. 9 Written Exam """""ew Worksheet Form Surry 2009-301 Q KJA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KJA Only S Dis! Dis! BIW choices is that it makes interpreting the answer choices dependent on the other answer choices. In other words, the applicant must ready the distractors to defme the word "only" that is used in the correct answer. It may take a few more words, but the answer choices can be written more precisely. Suggest rewording answer choices as follows:
A. Vital Bus I is de-energized. Vital Buses II, III, and IV are energized. B. Vital Buses I and III are de-energized. Vital Buses II and IV are energized. ETC. Incorporated. MAB 05/08/09 G2.1.40 N F 2 S Good Q. Minor enhancements only. OK MAB 05108109 (RO) Add to the question statement: " .. required operator action( s) in accordance with the Precautions and Limitations of I-OP-FH-OOI, CONTROLLING PROCEDURE FOR REFUELING?" Incorporated. MAD 05/08/09 039A2.03 N H 2 S With a rapidly lowering pzr P and L, would SI be required? Can this be
- presented to make it clear that an SI is not required?
(SRO) Incorporated. MAB 05108109 076AA2.02 N F 2 S Is the designated answer actually correctry The TS states that the actions are based on time of detection. The condition is actually detected when the (SRO) results are available, not when the sample is taken. At the time the sample is taken, they have not detected anything. This would be akin to an operability call and associated TS required action. Regardless of how long a piece of equipment may have been unable to perform its safety function, the TS actions are in effect beginning from the time of discovery. Corrected. MAB 05/2712009 Greater than one hour tech specs are not permitted to be tested in a closed book format. An easy fix would be to test when the condition of the LCO is no longer met. I.E.: A. LCO not met at 10:05 hours. [Second parts can remain the same1 B. LCO not met at 10:45 hours. Etc. Incorporated. MAB 05/27/2009 022KI.04 B H 2 S Sat bank Q from 2004 Surry exam. (RO) 103K4.06 B H 2 S Question is CIA. Applicant must apply TS knowledge to answer the Q. Changed to CIA. MAB 0512812009 (RO) 00802.1.7 B H 2 S Is it plausible for forrods rods to move out? Would it be more plausible for rods Page 17 of33
ES-401, Rev. 9 Written Exam Review Worksheet Form Surry 2009-301 u Q KlA# I!B M N L 0 K L 0 D Stem Focus Cues Psychometric Flaws TIF 1 Non Cred
>1 Non Cred Partial Min Content Flaws Q=
KIA SRO Only U E S Comment Explanation J Dlst Dlst I BIW BIW I I not to move at all? Discuss and tl)' to detennine if the question is best as it (RO) is or ifthere is a better, more plausible, choice of distractor. Surge Tank Level change and EOL added. This creates plausibility in that a determination must be made as to whether the rate circuit or the tavg tn.vg will induce rod movement. OK MAD 05128/2009 073A2.02 M H 3- S Are there technical references from the facility to support the technical 4 accuracy of the rad monitor setpoints? (SRO) Note to licensee added. OK MAB 05/2712009 0512712009 Minor changes being made. OK MAB 05128/2009 039K4.04 M H 2 S I Make minor enhancements as noted on hard copy. Done. MAB 05/2812009 (RO) Enhance the plausibility justification in the answer choice analysis. Done. MAB 0512812009 061K6.01 N H 3 S I Qissat. (RO) G2.3.12 N H 3 S I Q is sat. SRO ~~~~~~~~~~~~~~~~
- " ."..., ~; < " ,~,- ' ' ' " '"
, , , ;:; ~
'. -'C- . * * * . * :
. " ~ ¢<~
~ '" ... ,Y ;- Y. ,
"." .' -' <':' ~ yN, ." <\" ,,~,Y~
015G2.2.22 N H 2 x x S Why are they required to shutdown as directed by Tech Specs? Tech Specs do not pennit aux sprays to be used, but I did not see where Tech (SRO) Specs required a shutdown. Are pressurizers spray valves required to be operable? Is there an LeO for spray valves to be operable? Could pressurizer heaters be cycled to control pressure? Can PORVs be used to control pressure? There is still not enough documentation to support which answer is correct. The question was revised in such a manner that one and only one answer is correct and a note to the licensee has been added for them to help provide supporting dooumentation. MAB 05108/09 It is better to test on what is required. Suggest wording the lead-in statement to read: "Unit 2 is required to be ... " Incorporated. MAB 05108109 Page 18 of33
ES-401, Rev. 9 Written Exam Review Worksheet Form Surry 2009-301 Q KlA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues TIF 1 Non >1 Non Partial Min Q- SRO E Explanation N K D Focus Cred Cred KIA Only S Dis! Dis! BfW Oistractor "0": It would be better to test a different time to HSO, rather than test in one answer choice when they need to be in HSO and the other answer choice when they need to be in CSO. Try to test two different times when they would need to be in HSO (or CSO).
Q revision made this connnent mOOt, MAD 05/08109 There are subset issues between the answer choices which causes multiple correct answers and distractor plausibility issues. For example, if "C" is correct, then "A" would also be correct. If"C" is correct, then "B" is correct. If"A" is correct then "B" is also correct. Q revision made this comment moot. MAD 05/08/09 A note to the licensee still needs to be added to ask them to provide supporting documentation for what constitutes pressurizer operability. Note was added. MAD 05/1812009 Consider making a small wording change to the question statement. LCOs always apply, therefore, the more precise terminology is to ask whether or not the conditions of the LCO are met or whether the actions of the LCO are required to be performed. Incorporated. MAD 05/1812009 To be safe, the status of all the RCPs should be stated in the initial plant conditions. Incorporaj;ed. MAD 05/18/2009 015AA2.09 N H 2 S Suggest changing the question statement to more precisely ask for exactly Ol5Kl.02 what is being tested. I.E.: "Given the above temperature readings, which one of the following correctly states the RCP, if any, that exceeds an (RO) ACTION LEVEL limit in Attachment 2, RCP Parameters, of l-AP-9.00, RCP ABNORMAL CONDITIONS? lricorporated. OK MAD 05/08/09 003A3.04 N H 2 S Are the last four words of "c" necessary? Q replaced. MAD 0512812009 Bates (RO) Is it necessary to state that P-7 is not lie and pol 0 is lit? Q replaced. MAB 05/2812009 Will an RCP trip if the high P tap ruptures? Walk me through the supporting documentation? Q replaced. MAB 05/2812009 Replacement Q is sat. MAB 0512812009 Page 19 of33
ES-401, Rev. 9 Written Exam Reviewew Worksheet Form Surry 2009-301 Q KlA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues TIF 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KIA Only S Dist Dis! BIW 006KI.05 N H 2- S How can distractor "8" be ruled out as a correct answer choice with the 3 information provided in the stem?
(RO) 1381 does not get opened until after Ch and LD re-established. MAB 05108/09 The stem states that actions have been completed to establish normal charging and letdown. Therefore normal charging and letdown are in service. If this were the case, would I-CH-MOV-138I still be in the closed position. 1381 does not get opened until after Ch and LD re-established. MAB 05108/09 Distractor analysis for "8" was not completed. Completed. MAB 05/27(2009 What step of ES-I.I are the operators perfonning at the time the question is asked? What info in the stem would indicate that 1381 has not yet been opened? At the point where charging and UD are re-established, 1381 would be closed. MAB 0512712009 003A4.08 N F 2 S It is preferable to not state that the applicants should assume something in the stem because App E provides specific instruction not to assume. Can (RO) "Assume pressure slowly bleeds off the valve actuators" be added as a condition in the bulleted list?
- Containment instrument air pressure is slowly dropping.
Corrected. MAB 05127/2009 Generic comment to be applied to all questions: Generally component names should not be capitalized. The NRC is fairly consistent with this practice for inspection reports, etc. We will do an admin review of the completed exam to check for consistency for various items. OK. MAB 0512712009 004A4.08 N H 2 S Question should be designated CIA. Conditions in the stem must be analyzed to arrive at answer. FCV-1122 in auto must be linked to flow (RO) limits and then they need to understand that flow limits are not in play while in manual. Changed to CIA. MAB 0512812009 Minor enhancements noted on hard copy. Incorporated. MAB 0512812009 022G2.4.1I N H 2 x S Make small editorial enhancements as marked on hard copy. Comoleted. OK MAB 0512612009 Completed. Page 20 of33
ES-401, Rev. 9 Q KlA# B L L Psychometric Flaws Written Exam Review Worksheet Surry 2009-301 Content Flaws U Comment Form
- M 0 0 Stem Cues T/F 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KIA Only S Dlst Dlst BIW (RO)
"D" is not plausible. It is not credible to allow the plant to run when the unit is entirely without charging flow.
Corrected: MAS 0512612009 005A2.01 N F 2 S "A", "B", and "c" are not plausible because they are essentially the same answer. If anyone of them is correct, then the other two would be correct. Q replaced. MAS 05/1812009 (RO) LOD=l. Q replaced. MAB 05/18/2009 What information is provided in the stem for an operator to determine if a rod is stuck? If the information needed to make that decision is not present, then "A" and "B" cannot be plausible. Q replaced. MAS 0512812009 Could a rod be both dropped and stuck? What indications in the stem allow for this differentiation to be made? I.E. Could a rod partially fall into the core prior to being stuck? Q replaced. MAB 05128/2009 Are rods required to be within 12 steps of either Group I or Group 2, OR within 12 steps of either Group I or Group 2? Q replaced. MAB 0512812009 The stem states that rod B-8 is located at 182 steps. The instruments that provide this information should be stated in the stem. I.E. IRPI or DRPI position is .... 05128/2009 Q replaced. MAB 05/28/2009 Operability can be considered an SRO function unless it is either at a very basic level or it is Tech Spec entry criteria. Can Tech Spec entry criteria be used as justification to ask this as an RO question? If so, a small statement in the analysis section may be a good idea. Q replaced. MAB 0512812009 KIA Match: How are the excore NI readings indicative of a dropped rod? Q replaced. MAB 05/2812009 Replacement Q is sat. MAS 05128/09 05128109 003G2.4.31 M H 2 S The question asks for the withdrawal rate limit. According to the question analysis, this limit is 2.78. The CAUTION in the AP clearly states that the calculated rate must be rounded down; thereby making 3 stepslhour much Page 21 of33
ES-401, Rev. 9 Written Exam Review Worksheet Form Surry 2009-301 Q KlA# B L L Psychometric Flaws Content Flaws U Comment
- M N
0 K 0 D Stem Focus Cues TIF 1 Non Cred Dist
>1 Non Cred Dist Partial Min BIW Q-KIA SRO Only E
S Explanation I I i I (RO) more plausible than I step per hour. Qreplaced. MAB OS/28/2009 Question has to be higher cog (CIA) because they are performing a math equation. Q replaced. MAB OS/28/2009 All SRO questions need to contain a statement for why SRO-only knowledge is required to arrive at the correct answer. This needs to be added. Q replaced. MAB OS/28/2009 Replacement Q is sat MAS 05/28/2009 001K2.05 N F 2 S Delete the Current Condition portion of the stem. Deleted. MAB 05/27/2009 (RO) The Answer Analysis states that" A" is the correct answer, but the supporting documentation supports "B" as being the correct answer. Corrected. MAB ()S/27/2oo9 Further explain the plausibility of"C" and supply supporting documentation that supports the plausibility. Author explained that plausibility comes from similarity in nomenclature. OK MAB OS/27/2009 G2.1.29 N F 2 S 2ua part of "B" and "0" do not appear to be plausible because if a valve wrench is permitted to initially move the valve, then it would make sense (RO) that it would be permitted to further move the valve. Replaced. MAB OS/27/2009 Suggestion provided that changes the wording to make question more memory level and replaces the 2nd part of"B" and "0". Incorporated. MAS OS/27/2009 G2.2.40 M F 2 S Sat with minor enhancements. (RO) G2.1.20 N H 2 x S It appears that the fIrst part of "A" and "C" are correct answer choices. (SRO) Q revised to address plausibility and eilSure one and only one correct answer. OK MAB 07/10/2009. 006A2.12 B H 2 S Question meets KIA in that it addresses required actions that are a result of the impacts of conditions that required an SI. Page 22 of33
ES-401, Rev. 9 Written Exam Review Worksheet Surry 2009-301 Iv Form Q KJA# I!B M N L 0 K 0 D L Stem Focus Cues Psychometric Flaws TfF 1 Non Cred
>1 Non Cred Partial Min Min Content Flaws I Q=
Q= I SRO SRO Onlv Only U I EES Comment Explanation KJA Dist Dist BfW I (SRO) I 0ISG2.2.22 I I I I S I Qissat. Q is sat. 027G2.2.22 ();!;ZG;!.;!.;!;! Capehart (SRO) 013K2.01 N F 2 S Q Q is sat. (RO) 063K.3.02 N F 2 S QQ is sat. (RO) 017KS.02 B H 2 S Q Q is sat. Bates (ROJ 014A4.0l N F 2 S I QQ is sat. is sat. Bates RO) (RO) 00SAA2.20 008AKI.02 ()()8AKI.();! Capehart '~"'ri' ,~, ";\.. ,l>:t,~ .~'!:.;:,\~,~,~, """\,,,,'.,<,,~ ~(~~:).\/~'~'~:(>':.('.:}.t}.>': ~,"'~~"~', ".y. . . "~~.-'*"w;~y,';*':, .
'-:"'~:)""f:::: }P.~"";1
~ '... .. .-',; > .. <; ',,~ . :', ~, ,..... ,," ' '~~, _': ,\ y ~;...- ,; ',':.:, ,~ ... ~y ~ ~ y .. ~' ,,~~r.:: . .' ~:, :. ~\::{
. ~ y' ",,/:"'~:j~\:,:'Y:');< ~':"~"Y:-~:'.'~i""~i~':,,' ._ ', ... ,! ; ""f( ~" " ..... ,':,'!~...:~\*,';u'f;y:~~) ," ,~,{.~ "",,~', "':. " "',,~, ",'~":' :,,,,'~ . . >: ... :,<;,~,,,'~ 01OG2.4.20 N H 2 ? S The basis information infonnation from page 37 of the LP states that the reason for Shaaf minimizing PORV cycling is to prevent excess release to the PRT (AKA (SRO) rupture disk blowing) and to reduce the chance of the valve failing. Would concerns over valve reseating be the same as concerns over valve failure? Could "D" be successfully argued as correct if valve not reseating is a method of valve failure? Addressed. OK MAB 0512112009 SRO-only? The SRO-only guidance is silent on EOP basis information. infonnation. I was hoping that each of you would flow chart the SRO-only questions through that guidance document so that you could successfully justifY each of your SRO-only questions. Does the facility have an SRO-only learning objective? Can we accept EOP basis infonnation information for an SRO-only question? The IOCFRS0.43 states that SROs are responsible for procedure selection, so this is the more standard way of hitting the SRO-only piece. We should discuss this and maybe we can get enough buy-in from other examiners that we can accept EOP basis as SRO-only. SRO-onlv. We have already Page 23 of33
ES-401, Rev. 9 Q KiA# B L L Psychometric Flaws Written Exam Review Worksheet Surry 2009-301 Content Flaws U Comment Form
- M 0 0 Stem Cues TfF 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KiA Only S Dist Dist BfW been discussing this aspect for improving the SRO-only guidance.
Procedure selection now being tested. OK MAS OSI2112009 Add dashes or bullets to the list of conditions in the stem. At the end, we will just ensure that we are presenting the questions in the same way. 025AA2.05 N H 2- S Add "Loss ofRHR" to the KIA statement. Shaaf 3 Addressed. OK MAD OSI2112009 (SRO) For what Unit does the question pertain? Also, the designated AP is a unit specific AP (I.E. l-AP-27.0). Addressed. OK MAD 0512112009 SRO-only? How does this question screen in the SRO-only guidance? We need to make sure we can justifY SRO-only questions iaw the guidance because we are holding the licensee's to this same standard. We need to be careful with the message that we could send if we do not hold ourselves to that same standard. I am not sure we have the latitude on this question as we may have on 025AA2.05. Addressed. OK MAD 05121/2009 Is there any limitation on LPI flow or temperature during a loss of RHR that would impact the E-plan? lust trying to think of ideas that may be SROonly. Addressed. OK. MAB 0512112009 027AA2.15 N H 3 S Your answer choice analysis should include why each distractor is wrong. Shaaf I.E. P-445 has not failed high because that would cause PORV-1456 to fail (SRO) open, not PORV-1455C. Addressed. OK MAD 05121/2009 Does the stem need to state that a rx trip is imminent? Moot. Q modified. MAD 0512112009 Are we testing an important concept to decipher between "A" and "8"? If an operator thought the answer was "A" he would perform the exact same actions as ifhe thought the answer was "8". Ifthis is important to test, then you may be able to use the procedure selection piece of this to justifY the question as SRO-only. I do have some doubts as to whether this is an important concept to test, but I am open to discussion. Moot. Q modified. MAD 0512112009 One idea may be to test on operability of the PORV due to the P-444 failure. The PORV may not be impacted by the P-444 failure because the ReS is still hot and auto operation may not be required for operability, but if that were the case the correct answer could state that the PORV remains Page 24 of33
ES-401, Rev. 9 Q KlA# B L L Psychometric Flaws Written Exam Review Worksheet Surry 2009-301 Content Flaws U Comment Form
- M 0 0 Stem Cues TIF 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KIA Only S Dist Dist BIW operable. You could almost use everything already I your question and word it something like this:
A. P-444 failed high. PORV operability has been affected. B. P-444 failed high. PORV operability has NOT been affected. C. P-445 failed high. PORV operability has been affected. D. P-445 failed high. PORV operability has NOT been affected. Incorporated. OK MAB 0512112009 035A2.01 N H 2 S Answer choice analysis should include why an answer is correct and why a Shaaf distractor is plausible as well as correct. Each of these components should (SRO) be present in the answer choice analysis for each question. Incorporated. OK MAS 0512112009 Question may be satisfactory. Nonnally Normally analyzing plant parameters requiring entry into E-2 and E-3 are required abilities for an RO. This perfonnance of question is a little different in that conditions requiring performance both E-2 and E3 exist. Q significantly modified. OK MAB 05/21/2009 Should "A" and "C" SG pressures be higher than what is stated. Is it clear that a high steam flow did not exist on more than one steam line? Q significantly modified. OK MAB OS/2112009 051G2.4.l1 N H 2 S Question overlaps with a condenser vacuum question. This question is not Shaaf SRO-only and the other Q is a pretty good question, so it may make sense (SRO) to try to change your question. Sorry. Q replaced. MAS 05128/2009 Question is not SRO-only. Reactor Trip and Turbine Trip criteria are typically knowledge items required of ROs. Q replaced. MAS 05128/2009 "B" may not be plausible. Is it reasonable to have criteria to trip the reactor but not have criteria to trip the turbine? Q replaced. MAB 05128/2009 Current Condition time should be changed to 1506 to ensure that 5 minutes have elapsed, or place time in seconds on both the initial and current conditions. Q replaced. MAB 05/28/2009 Idea: The KIA requires writing a question requiring knowledge pf abnonnal abnormal procedures. A liberal interpretation of abnonnal abnormal procedure would apply to the e-plan because EPIPs are only used for abnormal abnonnal circumstances. One idea may be to have a SGTR and a LOOP, where the cooldown must be Page 25 of33
ES-401, Rev. 9 Written Exam Review Worksheet Form Surry 2009-301 Q KlA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues T/F 1 Non >1 Non Partial Min Q- SRO E Explanation N K D Focus Cred Cred KIA Only S Dist Dist BIW perfonned with the SG PORYs. It may be worth checking the e-plan to see if there is any fruit for a question. It may be possible to test the overall topic because the loss of condenser vacuum would create the conditions to require the direct release from the SG PORYs. (Maybe through in some fuel failure to get them somewhere in the e-plan)
Q replaced. MAB 0.5128/20.0.9 o.59G2.4.35 N F 3 S I am not sure that this question can make it through the sceening criteria as Shaaf set forth in the SRO-only guidance. We can discuss how the question (SRO) screens through the guidance document. Q replaced. MAB 0.512812009 One idea for the second part of the question may be to test on what operators are supposed to do if all of their actions, including feed and bleed have not been successful. I think at some point they go to Attachment 4 and verify E-O actions. You could test whether they go to Attachment 4 or somewhere else, thereby testing procedure selection which is supported by the CFR and the SRO-only guidance as being SRO-only knowledge. Q replaced. MAB 0.5/28/20.0.9 062AA2.06 N H 2 ~ S How does this question screen through the SRO-only guidance? Tech Shaaf Spec entry is typically considered RO knowledge. Operation of safety (SRO) significant equipment within specified limits is usually considered RO knOWledge. Q replaced. MAB 0.512812009 Distractor analysis states that "A" is correct, yet "C is checked as the correct answer. Q replaced. MAB 0.512812009 The following may be moot if we can't justify the Q at the SRO-only level. Will bearing temperatures rise in a linear fashion? Is this an operationally valid assumption? I understand that you went in this direction because of the K.I A. I would be a little more comfortable with the following idea: Q replaced. MAB 0.5/28/20.0.9 Add some times and temps 1420 hours Charging Pump Bearing Temp = 160F 1430 170 1440 180 1450 185 1500 190 Q replaced. MAB 0.5128120.09 Given the above bearing temperatures, WOOTF correctly states how the charging pump should be operated iaw 0-AP-12.00? Page 26 of33
ES-401, Rev. 9 Written Exam Review Worksheet Form Surry 2009-301 Q KlA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues T/F 1 Non >1 Non Partial Min Q- SRO E Explanation N K D Focus Cred Cred KIA Only S Dist Dist B/W Then answers could be similar to:
A. Preps to shift charging pumps should begin at 1440 hours. Charging pumps should be secured at 1450 hours. B. Preps to shift charging pumps should begin at 1440 hours. Charging pumps should be secured at 1500 hours. e. C. Preps to shift charging pumps should begin at 1430 hours. Charging pumps should be secured at 1440 hours. D. Preps to shift charging pumps should begin at 1430 hours. Charging pumps should be secured at 1450 hours. Q replaced. MAB 0512812009 G2.2.14 N F 2- S Discuss the incorrectness of"8". Shaaf 3 (SRO) Discuss the clarity of the "next 72 hours" as stated in the stem. 02.3.4 N H 2 S Some minor formatting suggestions that should be applied to this Q and
- Shaaf other similar questions. See notes on hard copy.
(SRO) Whenever possible, always state "iaw a procedure" in the question statement. There is likely an Admin procedure that states the EDLs for saving equipment. If you think this injures the plausibility, then adding the words "in order to save valuable equipment" may be enough if added to the last bullet (it should be added even if we add "iaw procedure"). Is there any method that could lead the operators to I-ECA-3.1 besides not being able to isolate the SG? I other words, is there any way "8" could be correct? G2.4.9 N H 2- S Add unit numbers to procedures in this question, as well as other questions. Shaaf 3 Q replaced, MAB 0512812009 (SRO) Should the condition in the question statement be added to the list of conditions?> Q replaced, MAB 05128/2009 Can> 125 gpm be achieved with on charging pump? Q replaced. MAB 05/2812009 Why is "0" incorrect? Why would aligning and starting another charging pump not be a method for meeting the conditions of I-AP-16.01, step 6? Q replaced. MAE 05128/2009 005G2.4.9 N H 2 S Q is satisfactory as long as 12.5 feet is a level at which vortexing could Shaaf occur. (RO) Page 27 of33
ES-401, Rev. 9 Q KJA# B L L Psychometric Flaws Written Exam Reviewew Worksheet Surry 2009-301 Content Flaws U Comment Form
- M 0 0 Stem Cues TIF 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KJA Only S Dlst Dist B/W 01OA3.02 N H 2 S Is there supporting documentation for the basis of the maximum Shaaf pressurizer spray valve that could be added to the package?
(RO) Added. OK MAB 0512112009 I believe the Q is actually CIA The conditions in the stem must be analyzed in order to determine the status of the spray valve. Discuss. Incorporated. MAB 05121/2009 01lEK3.l2 N H 2 S The reason supplied in "A" is not the reason for securing the RCPs. "A" Shaaf explains why there is no harm in securing the RCPs. The reasons for (RO) securing the RCPs is due to SBLOCA, as well as potentially needed them later in FR-C.l (I think) to force coolant through the loops. This should be an easy fix if you look at their EOP basis document, and/or the WOG. Incorporated. MAB 05/21/2009 All other aspects of the Q are satisfactory, so if you can place a correct reason for securing the RCPs in "A", the Q should be OK. OK MAB 0512112009 012A4.07 N F 2 S Should the status of all the RTBs and bypass bkrs be stated in the stem? Shaaf This may be necessary to ensure one and only one correct answer. The (RO) other way to accomplish the same thing would be to state that they are at a certain point in a PT when the operator attempts to close RTB B, but that would walk the Q away from arguably being memory leveL Incorporated. MAB 0512112009 012Kl.05 N H 2 S KIA: Will Rx Trip directly on low pzr pressure with the conditions in the Shaaf stem? If if so, does the applicant require knowledge of the Rx Trip on SI? If (RO) the RPS does not directly trip on pzr low pressure, then the KIA is met. RPSdoesnottriponpztlowP. OK MAB 05/2112009 015K5.1O N H 2 S Q is pure GFE. If operator actions are tested in conjunction with NI Shaaf indication then site-specific information may be required to answer the (RO) question. Q modified OK MAB 0512812009 Analysis states that Tc will get colder as power is raised. Is this true? Or will Th primarily get hotter with Tc remaining almost stable? Q modified. OK MAB OS/28/2009 Operators will be pulling rods as power is raised. Will this pull flux to the top of the core? Is enough info provided to ensure one and only one correct answer? Q modified OK MAB 0512812009 025AKLOI N H 2 -
? S Is it credible to maintain RCS temp 30F lower than the temp stated in the Page 28 of33
ES-401, Rev. 9 Written Exam Review Worksheet Form Surry 2009-301 Q KlA# B L L Psychometric Flaws Content Flaws U Comment
- M 0 0 Stem Cues TIF 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KIA Only S Dist Dist BIW Shaaf stem in the presence of a total loss ofRHR? Is "c" plausible?
(RO) Words added to Q. OK MAB 0512112009 Is a requirement to start all 3 RCPs plausible when the reactor is just producing decay heat? Is "A" plausible? Changed A. MAS 05/2112009 What is the status of the SGs when RCS Temp is 320F? Are the SGs required to have level maintained within a certain band during these conditions? Yes, OK MAB 05128/2009 Are there any subset concerns between "A" and "O"? It may be OK, but we should discuss to be sure, Changed A. MAS 05/2112009 026K3,02 N H 2 S Exactly what fails to align? Does the stem need to be more specific? Shaaf Incorporated. MAS 0512112009 (RO) Is this SRO-only? I think the question can be justified as RO because the system purpose is RO knowledge and their LP states the correct answer as part of the system purpose. LP states part 2 of question as "system purpose", OK MAS 05/2112009 027AK3.03 N F 2 ? S Plausibility? The level increase is a direct result of the pressure decrease, Shaaf Now here's the hypothetical analogy: If level decrease was the correct (RO) answer, then pressure decrease would also be the correct answer. In other words, these two answers are not independent and unique items, Corrected. OK MAD 05121/2009 035A2,01 N F 2 S Add lAW a procedure if possible, Shaaf Incorporated. MAD 05/2112009 (RO) Consider modifying Q statement as written on hard copy, Incorporated. MAD 0512112009 037G2,4.4 N H 2 S See wording suggestions on hard copy, Shaaf Incorporated. MAB 05121/2009 (RO) 038EA1.2l N H 2 ? S Concerns over plausibility of "Oecreasing," Discuss, Shaaf Discussed. OK MAD OSI2l12009 (RO) 055EKl.02 N F 2 S Which do you think is more plausible? Shaaf TOAFW pump runout OR Brittle fracture concerns? (RO) Deleted in modification. MAD 05/2112009 Page 29 of33
ES-401, Rev. 9 Q KlA# B L L Psychometric Psxchometric Flaws Written Exam Review Worksheet Surry 2009-301 Content Flaws U Comment Form I
- M 0 0 Stem Cues T/F 1 Non >1 Non Partial Min Q-Q= SRO E E~planation N K D Focus Cred Cred KIA Only S Dist Dist BIW Is this Q SRO-only? Do they have an RO learning objective? Discuss.
- Knowledge tested is located in a procedure note, therefore it is RO knowledge. OK. MAB 0512112009 057AA2.20 N H 2 S Minor enhancements.
Shaaf OK MAB 0512112009 (RO) 059K4.16 N F 2 S Minor enhancements. Shaaf OK MAB 0512112009 (RO) 062K.3.03 N F 2 S Discuss plausibility of "A". Shaaf Modified MAB 0512112009 (RO) 065AAl.03 N H 2 S Q is sat. Shaaf (RO)
.iRO) 068G2.4.42 N F 2 S No comments.
Shaaf (RO) 072K4.0l N F 2 S Add component IDs to stem and answer choices to raise the precision of Shaaf the question. (RO) OK MAB 0512812009 Distractor analysis;
- "C"; Is the action correct?
- "0"; Is the action correct?
Enhanced, OK MAB 0512112009 076K4.02 N H 2 S Does the status of both ofthe CH SW pumps need to be provided in the Shaaf stem? (RO) Incorporated, MAB 0512112009 078K.3.03 N F 2 S Minor enhancements. Shaaf OK MAB 0512812009 (RO) Plausibility of stable U2 pressure? Do Unit headers ever auto isolate? Are they ever split out? Discussed. OK MAB 0512812009 WIE05EAl.l N H 2 S Why is depress a second SG prohibited by procedure? See step 8 of Shaaf procedure. (RO) Modifications made to address. OK MAB OS/2112009 WlE08 EK1.3 N F 2 S With the above conditions, how can the applicant determine that the "B" Shaaf RCS 1.oo.p Loop Th is the highest Th? The applicant can determine that the Tc Page 30 of33
ES-401, Rev. 9 Q Q KlA# B L L Psychometric Flaws Written Exam Review Worksheet Surry 2009-301 Content Flaws U Comment Form
- M 0 0 Stem Cues T/F 1 Non >1 Non Partial Min Q-Q= SRO E Explanation N K 0 Focus Cred Cred KIA Only S Oist Dist BIW (RO) from the "8" SG is the highest Tc, but I'm not sure that the stem contains enough info to determine the highest Th. Discuss.
Modified. MAB 0512112009 8e consistent with how KIA is stated at top of question. Westinghouse KlAs should be displayed as W/E08 EK1.3.EKI.3. Incorporate into any other Westinghouse specific KI As. Modified. MAB OS/2112009 G2.L44 N H 2 S KIA Match: Are the actions being tested associated with fuel handling? Shaaf Or are the actions associated with a loss of RHR, with or without fuel (RO) handling in progress? Q replaced. MAB 0512812009 Would Surry conduct refueling activities with an RCS Temp of 185 F? Q replaced. MAB 05128/2009 Wording enhancements noted on hard copy. Q replaced. MAB 05128/2009 Question appears to be CIA. Discuss. Q replaced. MAB 0512812009 How is "C" plausible? What conditions have been provided in the stem that indicates that vortexing is occurring? I understand that level is going down, but reducing flow will not combat the event. Q replaced. MAB 05/28/2009 How is "D" plausibel? The upender is typically a good safe place for a fuel assembly. Q replaced. MAB 05128/2009 G2.3.5 N F 2 S Minor enhancements noted on hard copy. Shaaf OK MAB 05128/2009 (RO) Will a DAD detect beta? Ifso, then wording in "A" and "8" will need to be revised to not use the word ONLY. OK MAB 05128/2009 Does supplied documentation support the incorrectness of the" the " ...
. .. 75% of scale" piece of"8" and "D"?
Modified. MAB 05128/2009 Is neutron detection plausible? Would beta be a better option than neutron? OK MAB 05128/2009 Page 31 of33
ES-401, Rev. 9 Q KlA# B L L Psychometric Flaws Written Exam Reviewew Worksheet Surry 2009-301 Content Flaws U Comment Form
- M 0 0 Stem Cues TIF 1 Non >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KIA Only S Dist Dist BIW G2.4.14 N F 2 S No comments.
Shaaf (RO) G2.4.18 N H 2 S First part of each answer choice contains a subset issue. Shaaf Resolved, MAB 0512112009 (RO) Should parameters be added to stem for determination of adverse numbers? OK MAB 05128/2009 Are "C" and "0" plausible? In the presence of a SBLOCA, would it be credible to allow an RCP to stay running? Resolved. MAB 0512112009 In FR-H.I, RCPs are secured to minimize heat input into the RCS. Minimizing heat input will lengthen the time to PORV lift. Can this successfully be argued as correct? ResolVed. MAB 05/2112009 Resolved. G2.4.49 N H 2 S Minor enhancements noted on hard copy. Shaaf ResolVed. MAS 05/2112009 (RO) Can "immediately" be deleted from the question statement? Having the word in the question statement may not be needed to hit the KIA. If there is a technicality that the actions are not immediate actions as defmed by brackets, then the question would still stand up in the post -exam environment because it would still be technically accurate. Deleted. MAB 0512112009 Does the status of all the VI charging pumps need to be stated in the stem? Do conditions in the stem preclude "A" from being correct? Does distractor "A" overlap with another question? Not to our knowledge. MAS 05/2112009 Define ONLY in "B". Resolved: MAB 05/2112009 W/EI3 EK2.2 N H 2 S Be consistent with how KIA is stated at top of question. Shaaf Westinghouse KlAs should be displayed as W/E08 EKI.3. Incorporate (RO) into any other Westinghouse specific KlAs. Resolved. MAS 05/2112009 _L- Minor enhancements noted on hard copy. Page 32 of33
ES-401, Rev. 9 Q KlA# B L L Psychometric Flaws Written Exam Review Worksheet Surry 2009-301 Content Flaws U Comment Form
- M 0 0 Stem Cues T/F 1 NOD >1 Non Partial Min Q= SRO E Explanation N K D Focus Cred Cred KIA Only S Dist Dist BIW Resolved. MAB 0512112009 Is 535 F plausible? When would operators control at 535F in post trip environment? Would it be better to choose a credible temp corresponding to one of the steam dump controllers?
OK after discussion. MAB 0512112009 Page 33 of33
ES-403 Written Examination Grading Form ES-403-1 Quality Checklist II Facility: OvIIlLY: SU-R.R'-/ :rUl~ ~o~ Exam Level: R@ Date of Exam: ;;1 :rul~ SRO[EJ R()j'"'KJ SROi1lJ Initials Description Item Descrir>tion a b c
- 1. Clean answer sheets copied before grading (It
{It
, ~ mt3
- 2. Answer key changes and question deletions justified and documented /r;t mP
- 3. Applicants' scores checked for addition errors
_(reviewers s~ot check> 25% of examinations) (reviewers spot
.yt nf)
- 4. Grading for all borderline cases (80 +/-2% overall and 70 or 80, as applicable, +/-4% on the SRO-only) reviewed in detail /Yrt ft1P
- 5. All other failing examinations checked to ensure that grades are justified :;;l
/
1/L..r yttP
- 6. Performance on missed questions checked for training deficiencies and wording problems; evaluate validity /.:/[ }1r1S of questions missed by half or more of the applicants if Printed Name/Signature Date
- a. Grader r:::l--O""~ -s.
-So a . .\t ctt~ Y<f://
a<,-\t(H~Y<r://:-cV l 1'~ gA--!~'L
- b. Facility Reviewer(*) N/A / NIA
- c. NRC Chief Examiner (*) MARKA.13A1csllll~.(:~ 05 AU G;J:tIl GiJ:CIl
- d. NRC Supervisor (*) /)Al.l.DUJ -r.'MD~kJ~ /
I'!J,ItJIJ!J).. c:: rW,i I .,, C$0S/ua,
\:'f. ) I
(*) The facility reviewer's signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required. ES-403, Page 6 of 6}}