ML100470452
| ML100470452 | |
| Person / Time | |
|---|---|
| Site: | Catawba  |
| Issue date: | 02/02/2010 |
| From: | NRC/RGN-II |
| To: | |
| References | |
| Download: ML100470452 (100) | |
Text
{{#Wiki_filter:w§Q7,Qr9-sJlpJ w§0?,Or9-su p l -fjnaJ-forms-ms
-fjnal-foDUs-IDS doc
( Form ES-201-1 Facility: CATAWBA Date of Examination: 12/7-18/09 1217-18/09 Developed by: Written - Facility X NRCO NRC D II Operating - Facility X NRC D 0 Target Chief Date* Task Description (Reference) Examiner's Initials
-180 1. Examination administration date confirmed (C.1.a; C.2.a and b) FJE
-120 2. NRC examiners and facility contact assigned (C.1.d; C.2.e) FJE
-120 3. Facility contact briefed on security and other requirements (C.2.c) FJE
-120 4. Corporate notification letter sent (C.2.d) FJE
[-90] [5. Reference material due (C.1.e; C.3.c; Attachment 3)] FJE {-75} 6. Integrated examination outline(s) due, including Forms ES-201-2, ES-201-3, ES- FJE 301-1, ES-301-2, ES-301-5, ES-D-1's, ES-401-1/2, ES-401-3, and ES-401-4, as applicable (C.1.e and f; C.3.d) {-70} {7. Examination outline(s) reviewed by NRC and feedback provided to facility FJE licensee (C.2.h; C.3.e)} {-45} 8. Proposed examinations (including written, walk-through JPMs, and scenarios, FJE as applicable), supporting documentation (including Forms ES-301-3, ES-( 301-4, ES-301-5, ES-301-6, and ES-401-6, and any Form ES-201-3 updates), and reference materials due (C.1.e, f, g and h; C.3.d)
-30 9. Preliminary license applications (NRC Form 398's) due (C.1.1; C.2.g; ES- FJE 202)
-14 10. Final license applications due and Form ES-201-4 prepared (C.1.1; C.2.i; ES- FJE 202)
-14 11. Examination approved by NRC supervisor for facility licensee review FJE (C.2.h; C.3.f)
-14 12. Examinations reviewed with facility licensee (C.1.j; C.2.f and h; C.3.g) FJE
-7 13. Written examinations and operating tests approved by NRC supervisor FJE (C.2.i; C.3.h)
-7 14. (if >10)
Final applications reviewed; 1 or 2 (if> 10) applications audited to confirm FJE qualifications I/ eligibility; and examination approval and waiver letters sent (C.2.i; Attachment 5; ES-202, C.2.e; ES-204)
-7 15. Proctoring/written exam administration guidelines reviewed FJE with facility licensee (C.3.k)
-7 16. Approved scenarios, job performance measures, and questions distributed to FJE NRC examiners (C.3.i)
** Target dates are generally based on facility-prepared examinations and are keyed to the examination date identified in the corporate notification letter. They are for planning purposes and may be adjusted on a case-by-case basis in coordination with the facility licensee.
[Applies only] {Does not apply} to examinations prepared by the NRC. (
ES-201 Examination Outline Quality Checklist Form ES-201-2 ( Facility: Catawba Nuclear Station Date of Examination: 12/7/2009 Initials Item Task Description a b* c# 1. w W
- a. Verify that the outline{s) outline(s) fit(s) the appropriate model.
model, in accordance with ES-401. Iv ~ ~t R Assess whether the outline was systematically and randomly prepared in accordance with T I b. Section D.1 of ES-401 and whether all KIA categories are appropriately sampled. V ~ Yt T c. Assess whether the outline over-emphasizes any systems, systems. evolutions. evolutions, or generic topics. IV' ~ /r1 E N d. Assess whether the justifications for deselected or rejected KIA statements are appropriate. ~ ~ fA
- 2. a. Using Form ES-301-5.
ES-301-5, verify that the proposed scenario sets cover the required number of normal evolutions, failures, technical specifications. evolutions. instrument and component failures. specifications, S and major transients. I
/II M b. Assess whether there are enough scenario sets (and spares) to test the projected number U and mix of applicants in accordance with the expected crew composition and rotation schedule L without compromising exam integrity. integrity, and ensure that each applicant can be tested using A at least one new or significantly modified scenario,scenario. that no scenarios are duplicated T from the applicants' audit test(s).test(s), and that scenarios will not be repeated on subsequent days.
o 0 R
- c. To the extent possible.
possible, assess whether the outline(s) conform(s) with the qualitative and quantitative criteria specified on Form ES-301-4 and described in Appendix D. 1/
- 3. a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2:
(1) (1 ) the outline{s) outline(s) contain(s) the required number of control room and in-plant tasks W distributed among the safety functions as specified on the form ~ 1 (2) task repetition from the last two NRC examinations is within the limits specified on the form T (3) no tasks are duplicated from the applicants' audit test(s) (4) the number of new or modified tasks meets or exceeds the minimums specified on the form (5) the number of alternate path. low-power. emergency, path, low-power, emergency. and RCA tasks meet the criteria on the form.
/II/
- b. Verify that the administrative outline meets the criteria specified on Form ES-301-1:
(1) (1 ) the tasks are distributed among the topics as specified on the form (2) at least one task is new or significantly modified (3) no more than one task is repeated from the last two NRC licensing examinations
- c. Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days.
- 4. a. Assess whether plant-speCific plant-specific priorities (including PRA and IPE insights) are covered in the appropriate exam sections. V ~ Pi G
E b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate. ~ ~ '/£ N E
- c. Ensure that KIA importance ratings (except for plant-specific priorities) are at least 2.5. o..a I~ ~L R
A
- d. Check for duplication and overlap among exam sections. ~ ~ .-Pft L e. Check the entire exam for balance of coverage. V ~~
- f. Assess whether the exam fits the appropriate job level (RO or SRO). dV" ~ ~
J~~ ~. '$u~rin7~~~
- a. Author
- b. Facility Reviewer (*) Ik* 'f. f5l.A-:cR .'3"'e. '/U/~~ /l.-.
t:'.").~~~~",,,R.-b' -C/._tf'~1
~
,2, n 1Z,,-1'1. .
t)
- c. NRC Chief Examiner (#) IZ./'ff/~
- d. NRC Supervisor .llALWLU., . W!t).uAN~ I~ .f !z/r7;/.:J9
( / ~(qlu1 - Note: # Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.
- Not applicable for NRC-prepared examination outlines
~ e-..-...
'vU ...... ((~e....-... eye:L~
e~~
~ e-
..,.. e- C1. ~'" '"'-7 ..
tf a.. ~,\...J/
ES-201 Examination Outline Quality Checklist Form ES-201-2 ( Facility: Catawba Nuclear Station Date of Examination: 12/7/2009 Initials Item Task Description a b* c# 1. W
- a. Verify that the outline(s) fit(s) the appropriate model, in accordance with ES-401.
/
R b. Assess whether the outline was systematically and randomly prepared in accordance with V/
~~
fD+T Section 0.1 of ES-401 and whether all KIA categories me arB appropriately sampled. /
- c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics. V/
E N d. Assess whether the justifications for deselected or rejected KIA statements are appropriate. 1/
./
- a. Using Form ES-301-5, verify that the proposed scenario sets cover the required number
~} ~
~)
2. S I of normal evolutions, instrument and component failures, technical specifications, and major transients. if-fi M b. Assess whether there are enough scenario sets (and spares) to test the projected number
# ~ fitft U and mix of applicants in accordance with the expected crew composition and rotation schedule L without compromising exam integrity, and ensure that each applicant can be tested using A at least one new or significantly modified scenario, scenario, that no scenarios are duplicated T from the applicants' audit test(s), and that scenarios will not be repeated on subsequent days.
0 R
- c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative and quantitative criteria specified on Form ES-301-4 and described in Appendix D. /17 ~ fy£ til
- 3. a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2:
contain{s) the required number of control room and in-plant tasks (1 ) the outline(s) contain(s)
'7f)
It'> ~ W distributed among the safety functions as specified on the form ( 1
/
T (2) task repetition from the last two NRC examinations is within the limits specified on the form (3) no tasks are duplicated from the applicants' audit test{s) test(s) (4) the number of new or modified tasks meets or exceeds the minimums specified on the form (5) the number of alternate path, low-power, emergency, and RCA tasks meet the criteria on the form. it
- b. Verify that the administrative outline meets the criteria specified on Form ES-301-1:
*~ ~ 14-
~"> /f{
(1 ) the tasks are distributed among the topics as specified on the form (2) at least one task is new or significantly modified (3) no more than one task is repeated from the last two NRC licensing examinations
- c. Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days. MP ~ 12 liVP 11
~~ ~
tfo
- 4. a. Assess whether plant-specific priorities (including PRA and IPE insights) are covered in the appropriate exam sections.
G E b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate. ¥ Itm
)-/ tm '7 ~
N E
- c. Ensure that KIA importance ratings (except for plant-specific priorities) are at least 2.5. ~ !~ ~ I~ ¥ R
A
- d. Check for duplication and overlap among exam sections. I~
~ I~ ~ '/r/t
/If L e. Check the entire exam for balance of coverage. V I~ ~ 11 ~
- f. Assess whether the exam fits the appropriate job level (RO or SRO). V1 ~ r1t NA r1t-
-loM~ tS~
"k...J'~S r~tJ,~ure O~ r~AN~~ure
- a. Author
- b. Facility Reviewer (*)
-Jo
'-"\.1=. BLA:cR ~:VI
~. 1=. 8LA:cR
.\,\1\
~~I ~1t.tlA
~ll11A (V..
~.
~
1l-2~-Di 1l*2~-Di 3 uy il . :3 II Py
- c. NRC Chief Examiner (#) r::- _~. E r::-_J. ~ e-,-, ~~ .-::
E~e-,-,Af:....~V' .-::; L. dd~ ~ j'e/I/oy j'e/I/Or d, NRC Supervisor
- d. 1 LAJ./'nL.LL.t.
11A.Li"'DL.LL. vJtD~.( AI. I. VIlA: -.,.
-r. WtrUAAJdAjl'/fJ'.;: I~~!/)'
1~<1/)1 7)
/I VIp flIP, *.'., ~ ~- ~r T
/ ........ .,.. CI' Note: # Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.
- Not applicable for NRC-prepared examination outlines
ES-201 Examination Security Agreement Form ES-201-3
- 1. Pre-Examination 12/07/
/2/ /0'1 (J7 o'l 1tuu.
'1Ic...u I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of IZ/3tl.lflj IZ/.1tl./llj as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not no't been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.
- 2. Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of . From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.
PRINTED NAME JOB TITLE / RESPONSIBILITY DATE NOTE 1. 2. 3.
- 4. _'_'_Y,. ':.. ~ ~Ji"~
5. 6. NOTES,:
ES-201 Examination Security Agreement Form ES-201-3
- 1. Pre-Examination Ie 12/07 /0 '9 'floJ'fA
'fA-rIA I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of 1;t.~1' Iz.~r as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and does not select the training content or authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.
- 2. Post-Examination Tothe best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered TO,the during the week(s) of . From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.
PRINTED NAME JOB TITLE / RESPONSIBILITY SIGNATURE (1) SIGNATURE (2) DATE NOTE 1.
'pvor
~~'~
2. 3.
- 4. .., -~!\~ ~Z-:W--
5.
- 6. or/ofQ, _ _
7.
~"-
- 8. ~r
~O'i~~~ d~ ~ 11.' 12. 13. sac. et;",,'"'
~
-n
~pr.p cI.
~~~ o ~
14. 15. OJ. "'--~ Tl/A ,THY SH~"i4Il.I/v vA- L I f)/}Tlo/V l2.t'i'-- I:J. fl J1. D'1 - - NOTES:
ES-201 Examination Security Agreement Form ES-201-3
- 1. Pre-Examination Iz/O
., J. 7/ut I~07(ojo'\ -----6-=
2 . .JNn~~ PWlftll'lvt ~= 7"'~--::;'rif-~!'ird!"'~-::I-:.,4---
i 110Ci . 3. tV\ICI-III."E1 ~I><Mf:;S ,u * ~ jl.
- 4. tON .4LIC~ Q"'JoSS G~ (?h£~
~: !;:~:d ~o¥t,iM~+
~:~~~ ::~~'t1:~r~V'~..- '~I~<VJ~'~~
10.~f:. ~n:=A"\ ~;.cv> *P?e7p.e[.... ~ 11.~w O'\t..fM~\f ji:~~ NOTES: ES-201, Page 27 of 28
;J71?.J >1-el/& 1r/ i 00 V1 l:.
V1 00 ex! U1
<AI ES-201 Examination Security Agreement Fonn Form ES-201-3 ES*201-3
- 1. Pre-Examination . /'L/4')}"
1'1../4 ') J" I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of ,,"/JId!i 1tz./JII!f as of the date '" m
~
n of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the m NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered '"
~
these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC z o (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect -f
;;0 feedback). Furthermore, Furthermore. I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and ~
z understand that violation of the conditions of this agreement may result in cancellation of the examinations andlor and/or an enforcement action against me or 2: G"l the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.
- 2. Post-Examination To the best of my knowledge, knowledge. I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered week(s) of during the weekes) . From the date that I entered into this security agreement until the completion of examination administration. I did not instruct. evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted be/ow and authorized by the NRC.
below PRINTED NAME JOB TITLE I RESPONSIBILITY DATE NOTE
-ZP
~i6.~ D1fiiifo.;':x ,~ ~;p; ~ ~ '01! 6t j d) (0
~hl
~
~,q V"""'L="V';:P rh'('V,.,UI'\.. ':?L~"
,~'
f~/U/I?t '?7"'~/ ;~'C>" o
'"...,o
--- <AI l:.
--------- ------ -0 3
--------- - ----- --- - o to ID
.-------------~---====
- - - - - - - --------------------~------- -----------
r[ oa
/T-----
[r
------ '"ooa
./
/T '"co
./
/ ::::
.*~y ES-201, PW27 or28 oti8 ViI
ES-201 Examination Security Agreement Form ES-201-3
- 1. Pre-Examination
/Z/o}/M 1/uu
/2/0'?/M 1lu(.t.
I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of 12~fI? 12~p? as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or
~uggestions that examination security may have been compromised.
r~uggestions
- 2. Post-Examination knOWlzd9~' I d.NJl9t To the best of my knOWlr9~' d.i5j.!Wt div~lge d)v~lge to any unauthorized persons any information concerning the NRC licensing examinations administered li~cq ~"MO'm/tfiEfctJt~'hat 1ZP4Cfl -Y'rrO'm tflE1"<tJt~'hat I entered into this security agreement until the completion of examination administration, I did not during the week(s) of It.
instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC. PRINTED NAME JOB TITLE / RESPONSIBILITY DATE SIGNATURE (2) DATE NOTE
- 1. SIM~:'/.)lb{t, sv::t!,::ftC~"!; I-V-/O
- 2. 'S l v'\A.. )!~jJi L ~/4 'lii1J (-5"-10 3.
4. tJl>:2r~aJ C;'V s. . S Roo
- C~:.f . vfJLJt,t~&-
77 .- q,.~~ J()-f):'"
\ ~jIH( U'l'=:'=V -
tZ.-- Z2-Qtj
- 5. 1 S b ':;f?ir~
/~~
G,.JS IZ.. ,~. /~
/" .-07 / --,set:?
1
~- - -----,--
C-NS / lLo Jr;?-Zf-lf , -< "'j' (;[;>slJ'i
~JJ<.. Fr1JuA/vJ.. ---
8. rr\NS77Qt)s /0, I tl It! !h 0
~ t '""_I< Ij'-
~
/,..
I
'.I'!'"
oIZ-Z2-dj VIlA \1llio.,,).t. AlAJ,Topr
~i~'il lU~~4; ~it~it;: p"o,~
':1); '7J:ru. :h/-.. i i'i r 7~j( M/' ~?
/9'
.... ~.'~/r[".
R !:A;.n.vt.Y..~ "~f ,(]. __
/..., "f~~ <e- t1 I _
NOTES:
.-t o Q o ES-201 Examination Security~greement Agreement Form ES-201-3 §J
- 1. Pre-Examination I acknowledge that It have acquired acqui(ed specialized knowledge about the NRC licensing examinations sCheduled scheduled fOf the weekes) week(s) of as of the date of my signature. I agree Ihat that I will not knowingly divulge any information aboutlhese aboullhese examinations fo any persons who have not been authorized by the examiner. I understand [hat NRC chief e)(aminer. that r am not to instruct, evaluate, or provide performance feedback to those applicants scheduled 10 to be adminislered these licensing examinations examinalions from this date dale until completion of examination adminislration, administration, except as specifically noted below and authorized aulhorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceplable if
{e.g., illhe the individual does not select the (raining [raining conlent content or provide direct or indirect feedback). Furthermore, I am aware 01 of the physical security measures and requirements (as [as documented in the facility licensee's procedures) procedures} and understand that violation of the conditions or of this agreement may result in cancellation of the examinations and/or andfor an enforcemenl enforcement action against me 01 Of the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination securily security may have been compromised. CQmpromised. III 2. Post-Examination
~ ~
- II:
To the best ot or my knowledge. knowledge, I did not divulge to any ullauthorizeii unauthorized persons any information concerning the NRC licenSing licensing examinations administered
~ during the week(s) of . From the dale that I entered inta into this security agreement unlil 0' until the completion of examination administration, I did not ~ below and aLlthorized perfonnance feedback to those applicants who were administered lhese instruct, evaluate, or provide performance authorized by the NRC.
these licensing examinatiolls, examinations, except as specifically noted 3L1
~
~RINTED NAME PRINTED NAME JOB T/T~~'
TITLE f RESPONSIBILITY RESPONSIBILITY ~)
<~~ ~(S~.G.
L1 JOB DATE SIGNATURE (2) DATE NOTE
~
~v4iJ~
~~~/~S-~/O TURE .. J
!l
~
- 1. ~~'4IA~tt6~
~~\~tf6~ ()fJ~""hLI () fJ ~ ""t/lot.,:'i... S-,&t....~""
S-.'J..~-
- 2. ___________________________________________________________________
- 2. *
~.
~ __
/. S-'J410
- 3. _________________________________________________________________________
4.
- 5. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ____
- 6. ________________________________________________________________________
- 7. _________________________________________________________________________
B.
- 8. _____________________________________________
- 9. _______________________________________ ___ ------------------- - - - --- ---
10 . _______________________________________ 11 ... ________________________________________ 12., 12 . ___________________________________________ 13.________________________________________ 14._________________________________________
- 15. ____________________________________________________________________ ---------------- ------ ---- ---
* -::t 15.
NO-T-ES-:------- NOTES: (># fv"IIA,)~ -", l~~~ r.J.IJ~~&J,
-.~.~:.:~;:::. "~'
* =:- "';
ES-201,."Page ES-201 J;. :" Page 27 of 28
~~.~ ..
*"'L"*
Afi~ 1rtve f<<.'f/ F& # ,/'o3'*iVl"'3ZtJ + ~ U! U1 A-l>o U! U1 00
<II OCI 00 U!
U1
\AI ES*201 examination Securlty_~greeme~~
Examination Security Agreement . __. _______ . Fonn ES-201-3 Form ES*201.J
;-;;?/oz;
/7;?IZ 1*
- 1. Pre-Ex!mlnltion P!!=Examlnation VI m
I acknowledge that I haVe specialized knowledge about the NRC licensing examinations scheduled for the week(s) of have acquired Specialized of Iz/z f Al as of the date ~ m of my signature. I agree that I will not knowingly divulge any i~ information about these examinations to any persons who have not been authorized by the been VI instruct. evaluate. NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered >> z administration, except as specifically noted below and authorized by the NRC these licensing examinations from this date until completion of examination administration. o
~
(e.g., (e.g .* acting as a simulator booth operator or communicator Is acceptable if the individual does not not select the training content or provide direct or indirect ~ feedback). Furthermore, II am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and z Z understand that violation of the conditions of this agreement may resuit result in cancellation of the examinations and/or an enforcement action against me or G'I the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.
- 2. Ppst-Epmlnatlon Post-Examlnatlon To the best of my kn~J!9~'}
knO~i') I_did I.did not divulge to any unauthorized persons any information concerning the NRC li<lenslnglicensing examinations administered of~. From the date that tI entered into this security agreement until the completion of examination administration, I did not week(S) of~: during the weekes) evaluate, or proVide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted instruct, evaluate. below and authorized by the NRC. PRINTED NAME JOB TITLE I RESPONSIBILITY SIGNATURE (1) DATE~NATURE DATE SIGNATURE (2) (2) DATE NOTE 1.~~esf 1.:h7t7.).pes f 2. 2. 1&~/6CAJ1; fZ
/&YI6wr;R,.
________________________________________________________ _ ~BhxOynuf
. ~vtf /~L-6J ~L-6J
- 3. __________________________________________________________
3.
- 4. _______________________________________________________
4.
- 5. _____________________________________________________
5.
- 6. _________________________________________________________
6. 7 ____________________________________________________________
.~
0 7.
- 8. __________________________________________________ --------------------- ....
- 8. !?
- 9. _____________________________________________________ .......
9. 10.________________________________________________________ --------- -------- I>> 10. 11._________________________________________________ -------------- - - - 3 11. 12.__________________________________________________________ -------------- ---- 12.
- 13. ____________________________________________________ ------------- ---- ~
13. 14.________________________________________________________
---------------------- ---- 0o I
14. 15,___________________________________________________ ------------ ------ U1 VI I 15. NOTES:
------- ~
0o
@ /,A/,J~fJ~#~
tf) ~,A/."j/~,u~ II.A,;;
".~I 7ffd/) I';(~"iy ,£~j/(:.aJt;.,~
71'{/1l1J 1'"ftt.'rY '£YI:ct.A5.{ LO(!d.,{.~ /J A'~/'~ S. ES-201, P 27 of 28
( ES-301 Administrative Topics Outline Form ES-301-1 Facility: Catawba Nuclear Station Date of Examination 12/7/2009 Exam Level: RO M ~ SRO D 0 Operating Test Number 2009-301 Administrative Topic Type Describe activity to be performed (see Note) Code* Conduct of Operations R M Had Venting Time {G 2.1.7) IRIS Calculat-e R-eactor Vessel Head Conduct of Operations RM R 2R2S Determine the amount of boric acid required to get Control Bank D above the Rod insertion limits (G {G 2.1.25) Equipment Control R D 3R Classify a DIG Start and Complete a Logbook Entry (G 2.2.12) ( Radiation Control Emergency Procedures/Plan S D S 5R Notify the HazMat 'Emergency Response Team (G 2.4.11) SR NOTE: All items (5 total) are required for SROs. RO applicants require only 4 items unless they are retaking only the administrative topics, when all 5 are required.
*Type Codes & & Criteria: (C)ontrol room, {S)imulator, or Class(R)oom (D)irect from bank (~3 for ROs; ~ 4 for SROs & & RO retakes)
(N)ew or (M)odified from bank (?: (~ 1) (P)revious 2 exams (~ 1; randomly selected) (
ES-301 Administrative Topics Outline Form ES-301-1 Facility: Catawba Nuclear Station Date of Examination 12/7/2009 Exam Level: RO 0 SRO ~ Operating Test Number 2009-301 Administrative Topic Type Describe activity to be performed (see Note) Code* R M 1R15 lR15 Calculate Reactor Vessel Head Venting lime Time (G 2.1.7) Conduct of Operations Conduct of Operations R M 2R25 Determine the amount of boric acid required to get<:ontrol get <:ontrol Bank D above the Rod insertion limits (G 2.1.25) Equipment Control R D P 35 Determine 5LC requirements and complete a Unit Vent Flow Manual ( Calculation per PT/l/A/4450/017 PT/1/A/4450/017 (G 2.2.12) Radiation Control R N 45 Determine Reporting per RP/0/B/5000/013 (NRC NotifICation Notification Requirements) (G 2.3.14) Emergency Procedures/Plan R D 55 Make Emergency Classification and Complete the Initial Emergency Notification Form (G 2.4.40) NOTE: All items (5 total) are required for SROs. RO applicants require only 4 items unless they are retaking only the administrative topics, when all 5 are required.
*Type Codes & Criteria: (C)ontrol room, (S)imulator, or Class(R)oom (D)irect from bank (~3 for ROs; ~ 4 for SROs & RO retakes)
(N)ew or (M)odified from bank (~ 1) (P)revious 2 exams (~ 1; randomly selected) (
( ES-301 Control Roomlln-Plant Systems Outline Form ES-301-2 Facility: Catawba Nuclear Station Date of Examination 12/7/2009 Exam Level: RO ~ SRO-ID SRO-U 0 Operating Test No.: 2009-301 Control Room Systems@(8 for RO); (7 for SRO-I); (2 or 3 for SRO-U, including 1 :ESF) System I JPM Title TypeCode* Type Code* Safety Function
- a. Pressurize Cold Leg Accumulator 1A {SYS006 (SYS006 A1.13) D S5 2
- b. Align the NS System to Cold Leg Recirculation (SYS026
{SYS02{i A4.01) A DEN 5 S 5
- c. Restore Offsite Power to 6.9 kV Busses per AP/07 (Loss of Normal Power),
Power) , D P 5 S 6 Enclosure 21 (Black Restart Procedure) (EPE055 EA1.07)
- d. Verify Adequate Shutdown Margin Following a Reactor Trip (EPE007
{EPE007 EA1.09) A N 5 S 1
- e. Place Steam Dumps in Automatic Operation per OP/1/N6100/001 OP/1/A/6100/001 (SY5041 (SYS041 A L NS ALN"S 45 A3.02)
- f. Transfer Emergency Core Cooling System to Hot Leg Recirculation {SYS006
{SYSOO6 A D EN S 5 3 A4.07)
- g. Start Reactor Coolant Pump 1B (SYS003 A1.01) A M S 5 4P
(
- h. Shift Operating RC Pumps {SYS075 2.1.23) A D P 5 ADPS 8 In-Plant Systems@(3 for RO); (3 for SRO-I); SRO-f); (3 or 2 for SRO-U)
- i. Transfer Control to the Standby Standby'Shutdown "Shutdown System (APE068 M1.21)
AA1.21) D R 8
- j. Locally Operate Steam Generator PORVs During a SGTR (EPE038 EA1.41) D E 3
- k. Locally Trip the Reactor (EPE029 EA1.12) D E 1
@ All RO and SRO-I control room (and in-plant) systems must be different and serve different safety functions; all 5 SRO-U systems must serve different safety functions; in-plant systems and functions may overlap those tested in the control room room..
** Type Codes Criteria for RO ISRO-II SRO-U (A)ltemate path 4-6 1 4-6 4-5 1 2-3 (C)ontrol room (D)irect from bank ;::9/;::8/;::4
- ,9/::,8/::,4 (E)mergency or abnormal in-plant ~1/~1/~1
?,1/?,1/?,1 (EN)gineered safety feature - I?, 1 ~ 1 {control room system)
(L)ow-Power 1 Shutdown ~1/~1/~1
?,1/?,1/?,1 (N)ew or (M)odified from bank including 1 (A) 1(A) >2/>2/>1 (P)revious 2 exams ;~ 3 1 ;~ 3 1 ;~ 2(randomly 2 {randomly selected)
(R)CA ~1/~1/~1
?,1/?,1/?,1 (S)imulator
(
( ES-301 Control Room/In-Plant Roomlln-Plant Systems Outline Form ES-301-2 Facility: Catawba Nuclear Station Date of Examination 12/7/2009 Exam Level: RO D 0 SRO-I ~ SRO-U D Operating Test No.: 2009-301 System~ (8 for RO); (7 for SRO-I); (2 or 3 for SRO-U, including 1 ESF) Control Room SystemS9! System I JPM Title Type Code* Safety
'Function Function a.
- b. Align the NS System to Cold Leg Recirculation (SYS026 A4.01) A D EN S 5
- c. Restore Offsite Power to 6.9 kV Busses per AP/07 (Loss of Normal Power) , D P 5S 6 Enclosure 21 (Black Restart Procedure) (EPE055 EA1.07)
- d. Verify Adequate Shutdown Margin Following a Reactor Trip (EPE007 EA1.09) A N 5S 1
- e. Place Steam Dumps in Automatic Operation per OP/1/A/6100/001 OP/1/A/61001001 (SYS041 A L N 5 S 45 4S A3.02)
- f. Transfer Emergency Core Cooling System to Hot Leg Recirculation (SYSOO6 A D EN 5 S 3 A4.07)
- g. Start Reactor Coolant Pump 1B (SYS003 A1.01) AM A M 5S 4P
(
- h. Shift Operating RC Pumps (SYS0752.1.23)
(SYS075 2.1.23) A D P 5 S 8 In-Plant Systems@(3 for RO); (3 for SRO-I); (3 or 2 for SRO-U)
- i. Transfer Control to the Standby Shutdown System (APE068 AA1.21) D R 8
- j. Locally Operate Steam Generator PORVs During a SGTR (EPE038 EA1.41) D E 3
- k. Locally Trip the Reactor (EPE029 EA1.12) D E 1
@ All RO and SRO-I control room (and in-plant) systems must be different and serve different safety functions; all 5 SRO-U systems must serve different safety functions; in-plant systems and functions may overlap those tested in the control room .
- Type Codes Criteria for RO I'SRO-I' SRO-II SRO-U (A)lternate path 4-6 I 4-6 ,I 2-3 (C)ontrol room (D)irect from bank ::9
~9/~8'~4 1::8 1::4 (E)mergency or abnormal in-plant ~1/~1/~1 (EN)gineered safety feature - I - I,~ ~ 1 (control room system)
(L)ow-Power I Shutdown ~1/~1/~1
~1'~1/~1 (N)ew or (M)odified from bank including 1(A) ~2/~2'~1
~2/~2/~1 (P)revious 2 exams ~ 3 I ::
- ~ 3 I ::
~ 2 (randomly selected)
(R)CA ~1/~1/~1 (S)imulator (
ES-301 Control Room/In-Plant Systems Outline Form ES-301-2 Facility: Catawba Nuclear Station Date of Examination 12/7/2009 Exam Level: RO D SRO-I D SRO-U ~ Operating Test No.: 2009-301 System~ (8 for RO); (7 for SRO-I); (2 or 3 for SRO-U, including 1 ESF) Control Room System& System / JPM Title Type Code* Safety Function a.
- b. Align the NS System to Cold Leg Recirculation (SYS026 A4.01) A D EN S 5 c.
d.
- e. Place Steam Dumps in Automatic Operation per OP/1/A/6100/001 (SYS041 A L N S 4S A3.02) f.
- g. Start Reactor Coolant Pump 1B (SYS003 A1.01)
Al.01) A MMS 4P h. In-Plant Systems@(3 for RO); (3 for SRO-I); (3 or 2 for SRO-U)
- i. Transfer Control to the Standby Shutdown System (APE068 M1.21)
AAl.21) D R 8 j.
- k. Locally Trip the Reactor (EPE029 EA1.12) D E 1
@ All RO and SRO-I control room (and in-plant) systems must be different and serve different safety functions; all 5 SRO-U systems must serve different safety functions; in-plant systems and functions may room .
overlap those tested in the control room.
** Type Codes Criteria for RO !I SRO-I!
SRO-II SRO-U (A)lternate path 4-6 !I 4-6 !I 2-3 (C)ontrol room (D)irect from bank ~9!~8!~4 S9/S8/S4 (E)mergency or abnormal in-plant ~1/~1/~1
?1!?1!?1 (EN)gineered safety feature -
- !I - !? I ~ 1 (control room system)
(L)ow-Power I Shutdown (L)ow-Power! ~1/~1/~1
?1!?1!?1 (N)ew or (M)odified from bank including 1(A) ~2/~2/~1
?2! ?2!?1 (P)revious 2 exams ~3 I S S ~3 I S ~ 2 (randomly selected)
(R)CA ~1/~1/~1
?1/?1/?1 (S)imulator
(
ES-301 Operating Test Quality Checklist Form ES-301-3 Facility: Catawba Nuclear Station Date of Examination: 12/7/2009 Operating Test Number: Zw~ 2-W~ -sof
-SCII Initials
- 1. General Criteria a b* c#
- a. The operating test conforms with the previously approved outline; changes are consistent with sampling requirements (e.g., 10 CFR 55.45, operational importance, safety function distribution).
- b. There is no day-to-day repetition between this and other operating tests to be administered during this examination.
- c. The operating test shall not duplicate items from the applicants' audit testes).
test(s). {see Section D.1.a.)
- d. Overlap with the written examination and between different parts of the operating test is within acceptable limits.
- e. It appears that the operating test will differentiate between competent and less-than-competent applicants at the designated license level.
- 2. Walk-Through Criteria
- a. Each JPM includes the following, as applicable:
- initial conditions
- initiating cues
- references and tools, including associated procedures
- reasonable and validated time limits (average time allowed for completion) and specifIC specific designation if deemed to be time-critical by the facility licensee
- operationally important specific performance criteria that include:
- detailed expected actions with exact criteria and nomenclature
- system response and other examiner cues
- statements describing important observations to be made by the applicant
- criteria for successful completion of the task
- identification of critical steps and their associated performance standards
- restrictions on the sequence of steps, if applicable h.
- b. Ensure that any changes from the previously approved systems and administrative walk-through outlines (Forms ES-301-1 and 2) have not caused the test to deviate from any of the acceptance criteria (e.g., item distribution, bank use, repetition from the last 2 NRC examinations) specified
;(I J..
")V/
\.&J
~J\.- f on those forms and Form ES-201-2.
- 3. Simulator Criteria The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES-301-4 and a copy is attached.
Printei.Name I/ Signature Signature Date
- a. Author .,..l~
~I.'l~ \L ~\J'fE'A
<;\J9rt:--.A ~,t J'0vJ.-~ ~ If b; 1t"2
"&~ /'-'2
- b. Facility Reviewer(*) 'J- '2.J -CO:;
"-Zj'.Dt:;
- c. NRC Chief Examiner (#) IiZ.~/CO?,
Z-/( / aJ 9"
- d. NRC Supervisor - (z,I61!a9 IZ/6l!a9
/ r ~
f NOTE:
- The facility signature is not applicable for NRC-developed tests.
# Independent NRC reviewer initial items in Column "c "c";
n ; chief examiner concurrence required.
ES-301 Simulator Scenario Quality Checklist Form ES-301-4 ( Scenario Numbers: 1/ Z /3 l/l/3 Operating Test No.: ZItJ~;; No.:Ul~;-! Facilty: Catawba Nuclear Station Date of Exam: 1217/2009 QUALITATIVE ATTRIBUTES Initials a b* c#
~ (I
- 1. The initial conditions are realistic, in that some equipment and/or instrumentation may be out of service, but it does not cue the operators into expected events. ~
U> (/
- 2. The scenarios consist mostly of related events. ~
<f'-- ~ 1£
?£
- 3. Each event description consists of ry-Ir- ~
- the point in the scenario when it is to be initiated
- the malfunction(s) that are entered to initiate the event ~
- the symptoms/cues that will be visible to the crew
- the expected operator actions (by shift position)
- the event termination point (if applicable) ft~
~
~ fl
- 4. No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario without a credible preceding incident such as a seismic event. f£
- 5. The events are valid with regard to physics and thermodynamics. ~ I~ ~
Jv- yt yV ~ yt
- 6. Sequencing and timing of events is reasonable, and allows the examination team to obtain complete evaluation results commensurate with the scenario objectives. rt
- 7. If time compression techniques are used, the scenario summary clearly so indicates.
Jb-- ~ Jt-- Operators have sufficient time to carry out expected activities without undue time constraints. Cues are given. ;t fi 8. S. The simulator modeling is not altered. V ~ _ft Ir- ft If
- 9. The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator
( V performance deficiencies or deviations from the referenced plant have been evaluated to ensure that functional fidelity is maintained while running the planned scenarios. ~ ~ Jz--- J2--"
~
- 10. Every operator will be evaluated using at least one new or signifICantly significantly modified scenario.
All other scenarios have been altered in accordance with Section D.5 0.5 of ES-301. it
~~
~
- 11. All individual operator competencies can be evaluated, as verified using Form ES-301-6 (submit the form along with the simulator scenarios).
- 12. Each applicant will be significantly involved in the minimum number of transients and events specified on Form ES-301-5 (submit the form with the simulator scenarios).
V ~ W' 1</
- 13. The level of difficulty is appropriate to support licensing decisions for each crew position. ~ ~
t-- -a
~
Target Quantitative Attributes (Per Scenario; See Section D.S.d) D.5.d) Actual Attributes -- -- --
- 1. (5-S)
Total malfunctions (5-8) yf /'B/g
/g/g y--'
Y ~ ~
- 2. Malfunctions after EOP entry (1-2) Z / 3/
/.3/ 3 ~
dt- \Wfi lJt- ~ tv- ~.
/'i/'-I
/LJ/4 V ~ ~
3. 4. Abnormal events (2-4) Major transients (1-2) S S
, I i / It
/ J ~~ ~
V~
- 5. EOPs entered/requiring substantive actions (1-2) 2- // I /2 1- d-' I~
~ i\tJct /-tif
/~
- 6. EOP contingencies requiring substantive actions (0-2) 0 /0/
/0/ I rY' ~ ~
rY
- 7. Critical tasks (2-3) :3
.3 / Z / 2- (/
f'" ~ Yi o/i
ES-301 Transient and Event Checklist Form ES-301-5 Facility: CATAWBA NUCLEAR STATION Date of Exam: 12/07/2009 Operating Test No.: 2009-301 A E Scenarios P V 1 2 3 4 T M P E 0 I L N CREW CREW CREW CREW T N I T POSITION POSITION POSITION POSITION A I M C S A B S A B S A B S A B L U A T R T 0 R T 0 R T 0 R T 0 M(*) M{*) N Y 0 C P 0 C P 0 C P 0 C P R I U T P E RO RX 1 2 2 1 1 0 [gJ [8] NOR 1 1 1 1 1 SRO-I I/C IIC 26 345 56 47 0 46 35 4 4 ]2 SRO-U MAJ 7 7 8 8 7 7 2 2 1 0 TS 0 2 2 RO RX 1 1 2 2 2 2 1 1 0 0 NOR 1 1 1 1 1 SRO-I [gJ [8] I/C IIC 23456 26 4567 56 3456 46 4 4 2 SRO-U MAJ 7 7 8 8 7 7 2 2 1 0 TS 2345 347 356 0 2 2 ( RO RX 1 2 2 1 1 0 0 NOR 1 1 1 1 1 SRO-I I/C IIC 23456 4567 3456 4 0 4 2 SRO-U MAJ 7 8 7 2 2 1 [gJ [8] TS 2345 347 356 0 2 2 RO RX 1 1 0 0 NOR 1 1 1 SRO-I I/C IIC 0 4 4 2 SRO-U MAJ 2 2 1 0 TS 0 2 2 Instructions:
- 1. Check the applicant level and enter the operating test number and Form ES-D-1 event numbers for each event type; TS are not applicable for RO applicants. ROs must serve in both the "at-the-controls (ATC)"
and "balance-of-plant (BOP)" positions; Instant SROs must serve in both the SRO and the ATC positions, including at least two instrument or component (I/C) malfunctions and one major transient, in the ATC position. If an Instant SRO additional/y additionally serves in the BOP position, one I/C malfunction can be credited I/C malfunctions required for the ATC position. toward the two IIC
- 2. Reactivity manipulations may be conducted under normal or control/ed controlled abnormal conditions (refer to Section D.5.d) but must be significant per Section C.2.a of Appendix D. (*) Reactivity and normal evolutions may be replaced with additional instrument or component malfunctions on a 1-for-1 basis.
- 3. Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to the applicant's competence count toward the minimum requirements specified for the applicant's license level in the right-hand columns.
( ES-301 Competencies Checklist Form ES-301-6 Facility: CATAWBA NUCLEAR Date of Examination: 12/07/2009 Operating Test No.: 2009-301 STATION APPLICANTS RO SRO-I 0D RO SRO-I 0D RO SRO-I 0D 0D RO SRO-I 0D 0D SRO-U 0D SRO-U 0D SRO-U
- SRO-U 0D Competencies SCENARIO SCENARIO SCENARIO SCENARIO 1 2 3 4 1 2 21 3 4 1 2 3 4 1 2 3 4 InterpretiDiag nose 2-7 3-8 3-7 2-7 3-8 3-7 2-7 3-8 3-7 Events and Conditions Comply With and 1-7 1,2 1-7 1-7 1 11,2 1-7 ,2 1-7 1-7 1,2 1-7 Use Procedures (1) 4-8 4-8 4-8 Operate Control 1-7 1,2 1-7 1-7 1,2 1-7 1-7 1,2 1-7 Boards (2) 4-8 4-8 4-8 Communicate 1-7 1-8 1-7 1-7 1-8 1-7 1-7 1-8 1-7
( and Interact Demonstrate 1-7 1-8 1-7 1-7 1-8 1-7 Supervisory Ability (3) Comply With and 23 34 35 3"5 23 34 35 Use Tech. Specs. (3) 45 7 6 45 7 6 Notes: (1 ) Includes Technical Specification compliance for an RO. (2) Optional for an SRO-U. (3) Only applicable to SROs. Instructions: Check the applicants'license applicants' license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.
The Final Written Examination Sample Plan consists of the Draft Written Examination Sample Plan (ES-401-1 /2/3 in the Draft Administrative Documents) together with the Final Record of Rejected KAs (ES-401-4)
ES-401 Record of Rejected KI K1As As Form ES-401-4 Tier /I Randomly Reason for Rejection Group Selected KA 1/1 WEll EA2.1 (Q18) cannot write a question at the RO level regarding selection of procedures for this procedure. 2/2 SYS002 A3.02 Q(57) Not enough information to write an operationally valid question. Replaced with SYS002 K1.07 by Frank E. 5/13/09 2/2 SYS016 K3.04 Q(59) Q(S9) KIA compromised. Replaced with KIA SYS015 SYS01S K6.02 2/2 SYS028 K6.01 Q(60) KIA compromised. Replaced with KIA SYS016 K3.09 2/2 SYS033 K4.04 Q(61) KIA compromised. Replaced with KIA SYS028 KS.03 2/2 SYS035 SYS03S A1.02 Q(62) KIA compromised. Replaced with KIA SYS017 A3.01 2/2 SYS041 A4.07 Q(63) KIA compromised. Replaced with KIA SYS035 A4.02 2/2 SYS045 SYS04S K5.01 KS.01 Q(64) KIA compromised. Replaced with K/A KIA SYS041 A1.02 ( 2/2 SYS015 A3.03 Q(65) KIA compromised. Replaced with K/A KIA SYS045 K4.37 2/2 SYS071 K1.05 Q(65) No physical conenction or cause effect to write operationally valid question. Replaced with SYS015 A3.03 by Frank E. 7/13/09 3/0 GEN2.12.1.3 GEN2.1 2.1.3 Q(66) KIA compromised. Replaced with KIA G 2.1.4 3/0 GEN2.1 2.1.5 Q(67) KIA compromised. Replaced with KIA G 2.1.15 3/0 GEN2.1 2.1.7 Q(68) KIA compromised. Replaced with KIA G 2.1.40 3/0 GEN2.2 2.2.1 Q(69) KIA compromised. Replaced with KIA 2.2.3 3/0 GEN2.2 2.2.25 Q(70) KIA compromised. Replaced with KIA 2.2.38 3/0 GEN2.3 2.3.6 Q(72) KIA Importance Rating is less than 2.5 for the RO position. Replaced with KIA 2.3.13 per Frank Ehrhardt via phone 3/4/09. 1/1 EPEOll EA2.02 Q(77) Cannot write an SRO level question to this KIA. Replaced with (APE022 AA2.02) 1/1 APE026 2.4.9 Q(78) This KIA is very Similar similar to Q87 KIA also dealing with the KC system and having the same G 2.4.9. Replaced due to oversampling of this topic since both are on the SRO exam. Replaced with EPE038 G2.4.18 per Frank E. 9/8/09
ES-401 Record of Rejected KlAs Form ES-401-4 Tier / Randomly Reason for Rejection i
\ Group Selected KA 1/1 EPE009 EA2.31 (Q3) eNS does not operate with idle loops. Replaced with EPf009 EPE009 'EA2.34 by Frank E.
5/13/09 (
ES-401 Written Examination Quality Checklist Form ES-401-6 Facility: Catawba Nuclear Station Date of Exam: 12/7/2009 Exam Level: RO ~ SRO D ( Initial Item Description a b* c*
- 1. Questions and answers are technically accurate and applicable to the facility. 0' ~ VYf
- 2. a.
b. NRC KlAs KiAs are referenced for all questions. Facility learning objectives are referenced as available. dl ~ li
- 3. SRO questions are appropriate anrlrOllriate in accordance with Section D.2.d of ES-401 l' \W) ~
II
- 4. The sampling process was random and systematic (If more than 4 RO or or*2 repeated from the last 2 NRC licensing exams, consult the NRR OL program office).
2 SRO questions were f£
- 5. Question duplication from the license screening/audit exam was -controlled as indicated below < (check check the item that applies) and appears appropriate:
_ the audit exam was systematically and randomly developed; or _ the audit exam was completed before the license exam was started; or _ the examinations were developed independently; or /J-
~
~e licensee certifies that there is no duplication;
....-{he
_ other (explain) duplk;ation; or fi
- 6. Bank use meets limits (no more than 75 per-cent percent Bank Modified New from the bank, at least 10 percent new, and the rest JJ new or modified); enter the actual RO / SRO-only
~
question distribution(s) distribution{s) at right. 16 / - 5 // - 54 / - it ( 7. Between 50 and 60 percent of the questions on the RO exam are written at the comprehension/ analysis level; Memory CIA CIA the SRO exam may exceed 60 percent if the randomly selected KlAs KiAs support the higher cognitive levels; enter 36 // - 39 // - dl ~ 1 the actual RO / SRO question distribution(s) at right.
- 8. References/handouts provided do not give away answers or aid in the elimination of distractors. d' t!1b If{
- 9. Question content conforms with specific KIA KiA statements in the previously approved examination outline and is appropriate for the tier to which they are assigned; deviations are justified.
V ~ If{
- 10. Question psychometric quality and format meet the guidelines in ES Appendix B. cY ~ ~r
- 11. The exam contains the required number of one-point, multiple choice items; the total is correct and agrees with the value on the cover sheet. ~ ~~
Printed Name / Signature Date J~ t.. Sdf'r&v't / ~ yv..VJ------
~
- a. Author
- b. Facility Reviewer (*) H. F, t; ('Pr"f.e . j£.AJ J.k-1?., v. . lit-. l"2~17-
- c. NRC Chief Examiner (#) r:.~_ E\.q:~.~","~,T </.a~ IZ/(6'!O'?
- d. NRC Regional Supervisor IJ..At.UJI...Ar-r. WtDUM.lII..Y/ J'i;Z.~I. Ilil f1i.D~
/ Xl!tttU lC1 (1£J()
\....-/
Note:
- The facility reviewer's initials/signature are not applicable for NRC-developed examinations.
# Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.
(
ES-401 Written Examination Quality Checklist Form ES-401-6 Facility: Catawba Nuclear Station Date of Exam: 12/7/2009 Exam Level: RO ~ SRO 0D Initial Item Description a b* c*
- 1. Questions and answers are technically accurate and applicable to the facility. V M {;t r
- 2. a.
b. NRC KlAs are referenced for all questions. Facility learning objectives are referenced as available.
'V ~ f4 IOJI~lll1ft
- 3. SRO questions are appropriate in accordance with Section D.2.d of ES-401 of'ES-401 OJ ~ Yl
- 4. The sampling process was random and systematic (If more than 4 RO or2SRO or 2 SRO questions were repeated from the last 2 NRC licensing exams, consult the NRR OL program office). jt
~------------------------------------------------------------------------i r---
- 5. Question duplication from the license screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate:
_ the audit exam was systematically and randomly developed; or _ the audit exam was completed before the license exam was started; or
~
_~
~he tJl,e examinations were developed independently; or
_ the licensee certifies that there is no duplication; or # _ other (explain) I~
- 6. Bank use meets limits (no more than 75 percent Bank Modified New tJ; ~
from the bank, at least 10 percent new, and the rest new or modified); enter the actual RO / SRO-only question distribution{ distribution(s) s) at right. 16 / - 6 // - 53 / - ft ( 7. Between 50 and 60 percent of the questions on the RO Memory CIA exam are written at the comprehension/ analysis level; the SRO exam may exceed 60 percent if the randomly J1l ~ selected KlAs support the higher cognitive levels; enter the actual RO / SRO question distribution(s) at right. 36 // - 39 // - ~
- 8. References/handouts provided do not give away answers )tJ ~
or aid in the elimination of distractors. 11
- 9. Question content conforms with specific KIA statements in the previously approved examination outline and is appropriate for the tier to which they are assigned; deviations are justified.
'I!J ~ 1£
- 10. Question psychometric quality and format meet the guidelines in ES Appendix B. ~ ~ yf.-
- 11. The exam contains the required number of one-point, multiple choice items; the total is correct and agrees with the value on the cover sheet. tJ ~ yt Printed Name / Signature Date
- a. Author JD~ "-~~toA/,QOLAV~ '11/01
- b. Facility Reviewer (*) I-t
- F fZLlJrW _~*.V~~ lL " -rr-0f
- c. NRC Chief Examiner (#) t=='.::> - EO\-.!. f?-.""A,4,* n:/ .:£f,J!L- \Z/,d/if"
- d. NRC Regional Supervisor LlAulX uT. \AlIl1:1iJ. 1.1\1/ A *r (~I!Il./~ '1
/. rJiJIJil1)fl1ftr--____ .
~
~ IU Note: ** The facility reviewer's initials/signature are not applicable for NRC-developed examinations.
# Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.
20o~ R.. ES-401 Written Examination Quality Checklist Form ES-401-6 Facility: Catawba Nuclear Station Date of Exam: 12/7/2009 Exam Level: RO 0 SRO ~ Initial Item Description a b* c*
- 1. Questions and answers are technically accurate and applicable to the facility. {y' ~ Y1 II
- 2. a.
b. NRC KlAs are referenced for all questions. Facility leaming learning objectives are referenced as available. ¥ ~ 1ft
- 3. SRO questions are appropriate in accordance with Section D.2.d of ES-401 Y ~ yt
- 4. The sampling process was random and systematic (If more than 4 RO or Q 2 SRO questions were repeated from the last 2 NRC licensing exams, consult the NRR OL program offICe).office). ):'
- 5. Question duplication from the license screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate:
_ the audit exam was systematically and randomly developed; or ~
~
_ the audit exam was completed before the license exam was started; or _ the examinations were developed independently; or
~e licensee certifies that there is no duplication; or
;1iie liZ
_ other (explain)
- 6. Bank use meets limits (no more than 75 percent from the bank, at least 10 percent new, and the rest t -__ Bank I Modified Ba_n_k_+_M_O_d_ifi_le_d_+-__ New N_eW_--t ~
~ ,\AJl a new or modified); enter the actual RO / SRO-only distribution(s) at right.
question distribution{s) 16 / 1 6 / 3 S3 53 / 21 ~
'f\"" ft 7v
( 7. Between 50 and 60 percent of the questions on the RO CIA exam are written at the comprehension/ analysis level; the SRO exam may exceed 60 percent if the randomly. randomly Memory CIA V selected KlAs support the higher cognitive levels; enter the actual RO / SRO question distribution(s) at right. 36 // 12 39 / 13 ¥h ft-lr
~~
- 8. References/handouts provided do not give away answers or aid in the elimination of distractors.
~
- 9. Question content conforms with specific KIA statements in the previously approved examination outline and is appropriate for the tier to which they are assigned; deviations are justified. ~ 1ft
- 10. Question psychometric quality and format meet the guidelines in ES Appendix B. dI ~ Wi
- 11. The exam exam* contains the required number of one-point, multiple choice items; the total is correct and agrees with the value on the cover sheet.
dI ~ ~ Printed Name / Signature Date J~ ~C:;U,~~ 1,('\.01 l V [l -- ,,,'.,1,,,
~
- a. Author
- b. Facility Reviewer (*) j{.F. f?LA~ :fi.'/'
H. F. PLA:U:: I~~ 1/,
~./ I 1.f::1..1f>~ 4,. II~ -
- c. NRC Chief Examiner (#) i=". >- 6\l--R-I,kA-~T ~Ld:LlL 1~/fl4!<!'
IZ/ft/~
- d. NRC Regional Supervisor iUAUol..l£;t, T. WIOMMJAJ /' Ii-:... ~ z-hi2ltJCl.
r?/all!) 1 ,...... .
! (~!C{U)I(
Note:
- The facility reviewer's initials/signature are not applicable for NRC-developed examinations.
# Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.
(
ZO()? ZOO? ES-401 Written Examination Quality Checklist Form ES-401-6 Facility: Catawba Nuclear Station Date of Exam: 12/7/2009 Exam Level: RO 0D SRO ~ M Initial Item Description a b* c*
- 1. Questions and answers are technically accurate and applicable to the facility. ~
V ~ 11
- 2. a. KJAs are referenced for all questions.
NRC KlAs
- b. Facility leaming learning objectives are referenced as available. V~ ~ ~
- 3. SRO questions are appropriate in accordance with Section D.2.d of ES-401 ~ ~ '7f 7t 4.
5. The sampling process was random and systematic (If more than 4 RO or:2 repeated from the last 2 NRC licensing exams, consult the NRR OL program office). Question duplication from the license screening/audit exam was controlled or 2. SRO questions were Jlf~ .f~ as indicated below (check the item that applies) and appears appropriate: _ the audit exam was systematically and randomly developed; or V #J
~ /f
_ the audit exam was completed before the license exam was started; or
~e~e examinations were developed independently; or
_ he e licensee certifies that there is no duplication; or
<explain)
_ other {explain) ft
- 6. Bank use meets limits (no more than 75 percent Bank Modified New
~
~ pt f*
from the bank, at least 10 percent new, and the rest new or modified); enter the actual RO / SRO-only question distribution(s) at right. 16 / 1 5 / 3 54/21 ({ ( 7. Between 50 and 60 percent of the questions on the RO Memory CIA exam are written atthe selected KlAs arthe comprehension/ analysis level; the SRO exam may exceed 60 percent if the randomly KJAs support the higher cognitive levels; enter 36 / 12 39 / 13 V V~ t?7 1'7 the actual RO / SRO question distribution(s) at right.
- 8. References/handouts provided do not give away answers or aid in the elimination of distractors. t' ~ ITt V
d'" ~
- 9. Question content conforms with specific KIA KJA statements in the previously approved tr examination outline and is appropriate for the tier to which they are assigned; deviations are justified.
t'y ~ If{ lfi: ft
- 10. Question psychometric quality and format meet the guidelines in ES Appendix B. I~
- 11. The exam contains the required number of one-point, multiple choice items; the total is correct and agrees with the value on the cover sheet.
~ ~ LYz 1r' (PZ Printed Name / Signature Date
- a. Author
- b. Facility Reviewer (*)
Jv~ "- u-,f-I.h J($~"..1.01:f,'~
'F, p;,
>t.{I nM L('y~d1..1..
FSlJ)r'f:.R.-:5r!.r 61'2 .~r!.rl:~.
};...L-:}?i£.1.V1. II",
/l... I~
l~h)'Y, (2-n ('2..""'"...
- c. NRC Chief Examiner (#) c:::;)
~.) ._e:~~\S.:cA-~-
~~~\kA-~¢- .-:::/ ~/RALL- IfALl=- 'V~!:
'~/~f:
- d. NRC Regional Supervisor ~i.W{D~"f
;UAL.c.9l.M....\ .WlDMAAJAJ' Iff.:. J ~ ~i ',1 ll/lti6 S
[¥is/lIS
/!(\~qlUl
(~lq l.L.(IIr Note:
- The facility reviewer's initials/signature are not applicable for NRC-developed examinations.
# Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.
(
Catawba Nuclear Station 2009-301 ES-401 Written Examination Review Worksheet Form ES-401-9 Instructions infonnation regarding each of the following concepts.] [Refer to Section 0 of ES-401 and Appendix B for additional information concepts.)
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOO) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more infonnation information is needed, or too much needless infonnation). information). (i.e., clues, specific detenniners, The stem or distractors contain cues (Le., determiners, phrasing, length, etc). The answer choices are a collection of unrelated truelfalse statements. The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable. One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (Le.,(i.e., the question has a valid KIA but, as written, is not operational in content). The question requires the recall of knowledge that is too specific for the closed reference test mode (Le., (i.e., it is not required to be known from memory).
- The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for confonnance conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section O.2.f. 0.2.1.
- 7. Based on the reviewer'S judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
reviewer's jUdgment,
- 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).
General Comments & & Notes: 14 sample test items submitted 9/8/09 per ES-201: RO questions 2, 2,22,30,32,33,39,42,59. 22,30,32,33,39,42,59. SRO questions 76, 76,77,82,99,100. 77, 82, 99,100. 30 questions from initial submittal sampled as follows: Started review of 25 questions at RO question #1, excluding questions in initial sample of 14 and excluding bank questions from previous NRC examinations. Started review of 5 SRO questions at SRO Q#76 excluding questions in initial sample of 14 and excluding bank questions from previous NRC examinations. 0#76 Examiner Note: The 2009 written examination is also being used as a retake examination for applicants who failed the 2008 written examination. The 2009 examination is written for unit 1 (reference unit), which will have a Distributed Control System (DCS) installed. This DCS replaces a 7300 system and was not installed in 2008. As a result, questions 7,29,36, and 58 for the 2008 class are written to the 'old' 7300 system vs. the 'new' DCS system. 12/18/2009 1 of 73
Catawba Nuclear Station 2009-301 ES-401 2 Form ES-401-9 Q# Q# I 1. LbK LOK (F/H) IL~D I 2. (1-5)
- 3. Psychometric Flaws 3.
I Psychometric Flaws
- 4. .Inh
.II. Job r.nntent Content Flaws 5. Other LOD I---,---':""....--r---,---+-....,---r---,,....--+--.....---l I I I
- 6. I 7. 8.
Explanation (F/H) (1-5) UlEtS U/E/S 007EG2.4.50
~ 2 x N e H S What is the basis for the higher cognitive level? It would appear that to recognize the answer, the applicant must only recall the P-9 power level (69%) and vibration threshold (12 mils). See App. e, 8, C.1.d.
Enhancement for stem focus (see two items below). The question should cite the number and title of the procedure containing the required actions in order to limit the possibility of an alternative correct answer. See App. e, 8, C.1.c, last paragraph. Distractor C and answer 0 D do not follow from the question. The question asks for the required response. The first part of options C and 0D contain a statement (vibration limits have been exceeded) and a response (manually trip ...). FJE 10/20109 10/20/09 On 11/3109 11/3/09 the facility visited the RII All office to discuss selected questions from those reviewed as part of the sample of 30 as well as additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The examiner agreed that the question is written at a higher COG level. FJE 11/3/09 1113109 The facility revised the question to address the above comments. The revised question is satisfactory. FJE 1212109 12/18/2009 20f73 2 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
S. Q# 0# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #/ 8ack- Q= sRO Back- 0= B/MIN UlE/s SRO 8/MIN U/EIS Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only 008AK1.02 00SAK1.02 2 H 2 X N e 5 S The answer options contain subjective language, i.e. "approximately" "approximately" and "slightly," which makes the answer and distractors somewhat ambiguous. Additionally, since no values for pressure are given in the Initial and Final Conditions, distractor A could be correct. Remove subjective terms and provide pressure values. Enhancement - concern is multiple correct answers. Stem Focus/Editorial What is the purpose of the 2nd bullet in the Initial Conditions (pzr level was manually stabilized ... ) and the 4th bullet in the Final Conditions (the crew has diagnosed ...). If not necessary to answer the question or support distractors, consider deleting. See Appendix 8, B, Attachment 1, item 3. The second part of the question refers to an "event." What is the event? 1NV-294 in manual, pressure decreasing, or the PRT leak? State the questions more directly, e.g: How does charging flow and PRT leakage change for the plant conditions given above? A. Charging flow does not change (is stable). PRT leakage increases. FJE 9/9/09 Facility revised question to address the above comments. Revised question is SAT. No comments. FJE 10/20/09 009EA2.34 3 H 3 X 8 B e All distractor options do not flow grammatically from the stem 5 S (specific determiner/cue - see App. 8, B, C.2.m). The stem asks if 51SI can be terminated, and, if not, why not. If 51SI can not be terminated, a reason could be, for example, that pressurizer level is too low. The corresponding distractor answers the question 'what is necessary to terminate 51?' SI?' Reword the question and/or distractors such that the answer options flow from the wording in the question. FJE 10120/09 10/20/09 The facility revised the question to address the above comment. The revised question is satisfactory. FJE 1212109 12/18/2009 3 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
B. Q# LOK LOO LOD (F/H) (1-5) Credo Partial Job- Minutia #/ Back- Q= SRO B/MIN UlElS U/E/S Explanation Stem Cues TlFT/F Focus Oist. Dist. Unk Link units ward KIA Only 011 EG2.4.35 4 F 2 X X N e Ii H S Enhancement for psychometric flaws below: Option B does not appear plausible because the option is not internally consistent, i.e. if I know the sump pump starts on a Hi level and not a Hi Hi level (system response only), I can eliminate this option without knowing if this is the correct switch position or reason. Option C is the only option that does not contain an automatic system response, potentially making this option more or less likely to be chosen solely based on psychometrics vs. applicant knowledge. The answer options contain a mixture of 7 elements:
- sump pump switch position pOSition (AUTO, STBY)
- automatic system response (pump start Hi, pump start Hi Hi, alarm on Hi Hi)
- Overall reason (protect from flooding, indicate seal leakage)
Note that the KIA requires knowledge of the task and the operational effects, which is two elements, for example switch position and overall reason. Rather than writing the stem in a complete the sentence format (acceptable), consider a more focused question with fewer elements, e.g: Which one of the following correctly describes the required position of the local control switches for the NO ND & NS sump pumps and the reason the control switches are placed in this position? A. Auto. To protect the NO ND and NS pump rooms from flooding. B. Stby. To provide indication of possible NO ND and/or NS pump seal leakage. C. Auto. To provide indication ... D. Stby. To protect ...... Editorial: The second and third bullets of the stem can be stated more concisely to provide the same information given, for example:
- The crew is perform ing ES-1.3, step 13b to verify NO performing ND & NS rooms sump pump status lights DARK and observes that all lights are LIT.
FJE 10/20109 10/20/09 12/18/2009 4 of 73 40f
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD Explanation (FIH) (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back- Q= SRO B/MiN U/E/S Creel. U/EtS Focus Dist. Link units ward KIA Only The facility rewrote the question. The question is satisfactory. FJE 4 1212109 cont 015AK2.10 5 F 2 !I-
~ B e E
H S What is "full temperature and pressure?" Is this the same as NOP/NOT? Enhancement to preclude potential for multiple correct answers: What procedure requires securing of the pump? Tie the question to a specific plant procedure that requires securing the pump. See App. B, C.1.c, last paragraph. Examiner Note: 2003 NRC exam. FJE 10/21/09 The facility revised the question to address the above comments. Editorial: Consider removing the assumption from the question and making it a 5th1t1 bulleted statement to better stand out to test takers, e.g.:
- All temperatures are increasing at 1F per minute.
What is the .... The question is satisfactory as is. FJE 1212109 The facility made editorial changes and the question is still satisfactory. FJE 12116/09. 12/18/2009 12118/2009 50f73 5 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
0# Q# LOK LOO (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- 0= B/MIN UlElS Q= SRO BIMIN U/E/S Explanation Focus Oist. Link units ward KIA Only I 025AK1.01 6 .p: F 2 X X Ii Y. Y Unsatisfactory due to cue in stem and multiple correct answers (PIC H ~ 1- ~ 1- M e GL). S The stem contains a cue for the applicant to recall a caution that contains the answer to the question being asked. Remove the cue from the stem. Why is "A" incorrect? The distractor analysis for distractor A states that a cooldown will cause a change in volume. However, the stem of the question does not bound the timing or magnitude of the level change. "Rapidly" is a subjective term and no initial or final NC system levels are provided. Why is "0" incorrect? The caution states "NC System level may decrease rapidly when an NO pump is started due to the collapse of system voids." What causes the voids (heatup or air entrainment?) and what causes them to collapse on NO pump start (increased pressure or cooldown?)? I Editorial: The second bulleted sentence is missing a period at the end of the sentence. Examiner Note: 2003 NRC exam. FJE 10/21/09 On 11/3109 the facility visited the RII office to discuss selected questions from those reviewed as part of the sample of 30 as well as additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The facility stated that they were still working on this question. FJE 11/3109. 11/3/09. The facility replaced the question. Enhancement to resolve the following:
- The question type states Modified. Please provide a copy of the original question .
- Two curves for 18% NC level are provided. The stem states NC level is 21 %. What other curves for NC level are available? Two possible issues -1)
- 1) accuracy of answer, 2) cueing - should applicant be required to select correct graph from all available?
FJE 1212109 The facility provided a copy of the original question. The question is modified in that the condition in the stem and all answer choices have been changed in the new question. FJE 1214/091214109 12118/2009 12/18/2009 6of73 60f73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. B.
8. Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #1 Back- Q= SRO BIMIN U/E/S Explanation Stem Cues T/FTlF Focus Dist. Link units ward KIA Only The facility revised the question to address the second bullet above. 6 The revised question is satisfactory. FJE 12116109 cont 027AK3.03 - DeS question 7a F 2 ~ ~ X X N Y. 1J
~ 3 e Ii Unsatisfactory due to not meeting the KIA (PIC BC).
H S The KIA requires knowledge of the reasons for actions for a PZR pressure control malfunction. The first part of the question requires recall of whether a step (provided in the stem) is an immediate action or not. The second part of the question requires system knowledge of inputs to the PZR Pressure Master. Neither part of the question requires selection of an action or a reason for an action. Did not evaluate other aspects of this question due to not meeting the KIA. FJE 10/21/09 On 11/3/09 11/3109 the facility visited the RII office to discuss selected questions from those reviewed as part of the sample of 30 as well as additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The stated that they will rewrite the question. FJE 11/3/09. The facility rewrote the question. Enhancement to verify sufficient information in stem for correct answer. How does the applicant diagnose SPP-1 vs. SPP-2 alternate action? Please include a copy of 1AD-2, F/9 (condition in the stem). FJE 1212109 The facility rewrote the question to address the above comment. The question is satisfactory. FJE 12116/09 027AK3.03 -7300
- 7300 question 7b H 3 N S The question is satisfactory. FJE 1212109 12118/2009 12/18/2009 7 of 73 70t
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
0# Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 B/MiN U/E/S
#/ Back- Q= SRO B/MIN Explanation Focus Dist. Link units ward KIA Only 029EK3.10 8 F 3d X X X X N e Stem Focus The first part of the question contains subjective language, i.e.
"should," which implies that the answer is subject to interpretation or that there are multiple correct answers. Tie the question to the procedure, e.g. *Per "Per 1FR-S.1, which one of the following correctly describes whether or not the 1B NC pump may be tripped?"
Yes, the 1 B NC pump may be tripped 1B No, the 1 B NC pumJ? 1B pumJ;l may NOT be tripped See App. B, C.1.c, 4 paragraph. Cues The first part of the question does not match the answer options provided. The question asks if the 1B NC pump should be tripped, which would elicit a simple. simple Yes/No response. The answer options consist of Yes/No and associated reasons. The question does not ask for the reason for tripping the pump pum p or leaving the pump pum p running and these reasons provide cues for answering the question. Delete the reasons. See suggested wording above. Enhancement for stem focus/cues. Editorial 200 and 3rd bullets are redundant The 2'"' 4th bullet: who reports the parameter isn't important Consider the following (see App. B, Att. 1, item 3): Given the following:
- Unit 1 was operating at 100% power
- The main turbine tripped on low condenser vacuum
- Attempts to manually trip the reactor from the control room failed
- The crew is performing 1FR-S.1, Response to Nuclear Power
- Generation! AlWS GenerationlAlWS
- Reactor power is 8% and decreasing
- 1B NC pump vibration is 22 mils on the shaft Other Question references provided included background document for 1FR-S.1 and 1E-O, Reactor Trip or Safety Injection procedure. How is 1E-O relevant to the question?
FJE 9/9109 9/9/09 12118/2009 12/18/2009 8 of 73 80f73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOO LOD (FIH) (F/H) (1-5) Credo Partial Job- Minutia #/ Back- Q= SRO B/MiN U/E/S SAO B/MIN U/EiS Explanation Stem Cues T/F Focus Oist. Dist. Link units ward KIA Only Facility rewrote/replaced the question. New question comments 8 F 2 X X N e Ii below: cont S Enhancement to improve the plausibility of choices A and O. D. Is there a Tech Spec or other limit on "total core flux"? If this is not a monitored parameter, then it detracts from the plausibility of option A. OptieA ~Iablsieility eesablse Gl'ltisR g lasks plablsi9i1ity 9ssablss feFfSF seme A+WS seAElitieAs, ssms ,O'+wS SSRGitisRs, e~eAiA!! Sl'lsRiR! ef sf SG safety '~alvss .,tJGblIEI safsty ',ral¥es 'NsbliG ee9S ellpesteEl S*flsGtSG (e.!!. AeA safety (s.!. ReR safsty seAEleAseF SSRGSRSSF RSt Aet /i',railaele availa91s aAEI aRG SG PORVs PGRVs aAEI aRG safety safsty wl¥es valvss aF8 aFS t/:le tRS melheEl metRSG ef stsam Elblm~). sf steam Gblml'l). FJE 10/21/09 On 10/29 the facility exam team contacted the NRC NAC reviewer and explained why they thought choice 0D was plausible based on plant design. The reviewer agreed that choice 0D is plausible. FJE 10/29/09 The facility rewrote the question. The question is satisfactory. FJE 1212109 054AK3.04 9 F 2 X X N tl Y S Unsatisfactory for multiple implausible distractors. (P/C GL) The second half of options C and 0D are not plausible at these power levels. The distractor analysis states that this is the basis for tripping the reactor on a turbine trip above P-9, which is 69% power, i.e. about 7 times the power level of the highest in the answer options. Additionally, C would not appear to be plausible if tripping the reactor at 5% power is directed by the normal shutdown procedure. FJE 10/21/09 On 11/3/09 the facility visited the RII All office to discuss selected questions from those reviewed as part of the sample of 30 as well as additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The facility stated that they were still working on this question. FJE 11/3/09. The facility revised the question. The revised question is satisfactory. FJE 1212109. 12118/2009 12/18/2009 90f 9 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. B.
8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1#/ Back- Q= SRO B/MIN U/E/S Explanation Focus Dist. Link units ward KIA Only 055EK3.02 10 F 3 ~ S B Adm inistrative: Administrative: Development References list ECA-O.O step 23, but applicable depressurization step in ECA-O.O reference provided is step 30. The step in the ECA-O.O background document is step 23. Question source indicates uncertainty with respect to new or modified. If modified, please provide the original question. If new, please revise question source notes. Question is SAT. FJE 10/21/09 Facility corrected the items listed above. FJE 1212109 I 12/1812009 12/18/2009 10 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
B. Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #1 B/MiN U/EtS
#/ Back- Q= SRO BlMlN U/E/S Explanation Stem Cues T/F Focus Dist. Unk Link units ward KIA Only 056G2.1.28 056G2.1.2B 11 H 2 ~ B e Ii S Enhancement for KIA match.
The KIA statement is knowledge of purpose and function of major system components and controls. How does the question test this concept? What purpose or function of what control or component is being tested? The question seems to be integrated plant response vs. knowledge or purpose of components/controls. Please explain or modify. Please provide a specific technical reference for this question that shows that the answer is correct. Examiner Note: 2003 NRC question. FJE 10/21/09 On 10/29 the facility exam team contacted the NRC reviewer to explain why they thought the question matched the KIA. The facility exam team agreed to provide a written summary of the justification. The reviewer agreed to provide this written summary, and the question, to a second qualified NRC reviewer. FJE 10/29/09 The question meets the KIA (PIC (P/C BC) and the answer is supported by references. The original question is satisfactory without revision. FJE 1212109 12/18/2009 11 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #/ B/MIN U/E/S
#1 Back- Q= SRO BIMIN U/EIS Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only g 057AA1.06 12 H ~ ~ ~ N @.
2 S Examiner Note: Requires knowledge of DCS modification (AOP-28). Enhancement to resolve the questions below. Since both the action and procedure contained in the second half of all answer options is unique, this question appears to require ROs to make a procedure selection. What is the justification for asking this on an RO exam? Please explain or mOdify. modify. What is the governing procedure after 1ERPB de-energizes? Will the crew necessarily enter AP-028 with 1SB-9 throttled (slightly?) (Slightly?) open and NC temps *slowly" "slowly" decreasing? Concern is no correct answer if crews would be expected to address the loss of bus first or would not be 'required' to enter AP-28 first. Please explain the answer C discussion in more detail. I could not find these actions referenced in E-O. Will a reactor trip signal result from these plant conditions? FJE 10/21/09 The facility replaced the question. The question is satisfactory. FJE 1212109 1211812009 12/18/2009 12 of 73
Catawba Nuclear Station 2009-301 1-
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #/ Back- Q= SRO BlMlN B/MiN U/EtS U/EiS Explanation Stem Cues TlFT/F Focus Dist. Link units ward KIA Only 058AK1.01 13 ~ 2 N .e Ie Enhancement to resolve the items below: H S What is the basis for the higher cognitive level? The question seems to boil down to: 1EMXA is unavailable to 1ECS What other MCC can be aligned to 1ECS? What plant mode(s) allow 1ECS to be tied to bus 1EDC? Answering this question appears to merely require recall of the other available MCC to 1ECS and recall of a caution that trains must be separated in all modes. Since "No Mode" is not a "Mode" per Tech Spec section 1, asking "what plant conditions allow..... ." would be a more correct (and ft potentially less misleading) way of asking the question. FJE 10/21/09 The facility revised the question and explained the higher cognitive level. Editorial: The second bullet appears to be missing a word, i.e.... unavailable to power spare charger.... charger .... or just state that 1EMXA is unavailable. The question is satisfactory. FJE 1212109 The facility made minor editorial changes and the question is still satisfactory. FJE 12116109 12116/09 12118/2009 12/18/2009 13 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
B. Q# LOK LOD (FIH) (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back-8ack- Q= SRO B/MiN 8/MIN U/E/S Explanation Focus Dist. Unk Link units ward KIA Only g 062AA1.02 14 F 2 ~ N ~ Enhancement to resolve KIA match and COG level. S KIA is for Loss of Nuclear Service Water. What constitutes the loss of service water in this question? Please explain how this meets the KIA. What is the basis for the higher cognitive level? The question appears to merely require recall of RN realignment on an emergency low pit level actuation. Editorial:
- What caused the RN pumps to start? If the Emergency Lo Pit Level, then it might be more logical to list this condition (pit level) first.
- The size of the type font in the two bulleted statements appears different. Please make them consistent.
FJE 10/21/09 The facility explained the KIA match and addressed the other comments above. The question is satisfactory. FJE 1212109 12/18/2009 14 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #/ Back- 8/MiN U/EtS 8ack- Q= SRO B/MiN U/E/S Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only 065AA1.03 15 F 2 N E Ii S Enhancement to resolve COG level. What is the basis for the higher COG level? The question appears to merely require recall of which valve, from a list of four, will reposition without operator action. Editorial The question appears to contain unnecessary information infonnation and is also potentially confusing because the question states that VI pressure recovers but then next lists valve positions before the loss of VI. Would the following question be equivalent? Given the following table of valves and valve positions: (table) The VI system subsequently depressurizes completely. Which one of the following valves will return retum to its previous position, with no operator action, when the VI system is repressurized? Administrative I could not find information infonnation to validate the correct answer in the developmental reference listed (AP-22). Please list a specific specifiC document and step or paragraph that shows the answer indicated is the correct answer. FJE 10/21109 10/21'/09 The facility revised the question to address the above comments. The question is satisfactory. FJE 1212109 12118/2009 12/18/2009 15 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO BlMIN B/MIN U/EtS U/EIS Explanation Focus Dist. Link units ward KIA Only WE04EA2.2 16 F 2 X B E Ii S The question appears to contain unnecessary details that are not required to answer the question. See App. B, Att. 1, item 3. Additionally, this information potentially makes answer option A less plausible. If I know the leak is on the ND system injection header and that ND pressure is increasing, I know I have isolated the ND system from the leak and I have not isolated the RCS system from the leak. Consider the following: Which one of the following indications is used in EP/1/A/5000/ECA-1.2, LOCA Outside Containment, to verify that the leak has been isolated? Examiner Note: 2005 NRC exam. Enhancement to remove unnecessary information that also affects distractor plausibility. FJE 10/21/09 The facility revised the question to address the comment above. The question is satisfactory. FJE 1212109 12118/2009 12/18/2009 16 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #1 Back- Q= SRO B/MIN U/E/S Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only WE05EK2.1 17 F +
.:t. X B V U
2 S The question is unsatisfactory for multiple implausible distractors. (PIC GL) The first part of answer options A and B are not plausible if CNS Unit 1 and Unit 2 do not have, and have never had, dedicated air tanks. The second half of A and C are phrased such that the outcome of opening the PORV is stated vs testing the basis, which is not discriminating. Opening pressurizer PORVs in FR-H.1 is commonly referred to (and referred to by FR-H.1 step 5) as "bleed and feed." The high level action required per FR-H.1 step 22 is to "Establish NC system bleed path ...
.. ."* and step 22.b opens the PORVs. So equating Open PORV = bleed path requires no knowledge of the basis for the step, which is to provide for adequate RCS heat removal until a secondary heat sink can be restored.
The distractor analysis concerning the second half of options Band 0 D state that SG tube failure could become a risk with high DIP if the S/G is required to be depressurized to introduce a low pressure water source. Although tube failure caused by high DIP is plausible, there are no conditions in the stem that suggest that an SG has been or will be depressurized to inject water, making these options less plausible. Examiner Note: 2004 NRC exam. FJE 10/21/09 On 11/3/09 11/3109 the facility visited the RII office to discuss selected questions from those reviewed as part of the sample of 30 as well as additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The facility rewrote the question to address the above comments and will resubmit the question for review. FJE 11/3/09 The facility rewrote the question. The question is satisfactory. FJE 1212109 12118/2009 12/18/2009 17 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD SRO B/MiN U/E/S Explanation (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back- Q= Focus Dist. Link units ward KIA Only We11 E;\.2.1 We11EA2.1 18 H 2 X X +
~ 8 E N S Enhancement for distractor A plausibility.
FR-C.2 is entered directly for an orange condition on the core cooling CSF status tree (RO knowledge). No information is provided in the stem that would suggest that core cooling is currently jeopardized. Additionally, most (all?) EOPs only direct entry (directly) to functional recovery procedures associated with red CSF status trees. A more plausible option might be ES-O.O (Rediagnosis). Examiner Note: 2003 NRC exam. FJE 10/22/09 The facility revised the question to address the above comment. Enhancement to ensure question is at the RO level. The revised question appears to be written at the SRO license level. Please verify with Operations management that this type of knowledge is expected of ROs AND provide an associated learning objective supporting this as RO knowledge. FJE 1212109 WE11EA2.2 The facility requested a new KIA and wrote a new question. The new question is satisfactory. FJE 12116/09. 12118/2009 12/18/2009 18 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. B.
Q# LOK LOD B/MIN U/E/S SAO B/MiN Explanation (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1#/ Back- Q= SRO Focus Dist. Link units ward KIA Only 005AK2.02 19 F 2 X X N eE S Enhancement for multiple correct answers. If the misaligned rod is stuck and will not move. move, the lift coil of the affected rod will be disconnected (utilized) per section 2.15 of the referenced OP. Since no information is provided in the stem regarding whether this is the initial attempt at recovering the rod. rod, or whether the initial attempt was successful, successful. both C and D could be correct. Additionally the second question would be clearer if it elicited the answer more directly and avoided the word "utilize" which is not used in the associated OP for this action. Consider the following: Which control rod lift coil(s) will be disconnected. disconnected, per OP/1/A/6150100B, OP/1/A/6150/00B. when initially attempting to recover control rod H-B? FJE 10/22109 The facility revised the question to address the above comment. The question is satisfactory. FJE 1212109 12/18/2009 19 of 73
Catawba Nuclear Station 2009-301
- 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. B.
- 1. 2.
Q# LOK LOD BIMIN UlEtS SRO B/MiN U/EtS Explanation (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 #/ Back- Q= Focus Dist. Link units ward KIA Only 033AG2.1.19 033AG2.1 .19 X X M lIJj 20 H 3 S The question is unsatisfactory because it does not meet the KIA. (PIC (P/C BC) The generic portion of the KIA is the ability to use the plant computer to evaluate system or component status. The question does not test the ability to use the computer; it merely substitutes a computer point for a failed meter. The first half of the question requires determining which IRNI is reading wrong and meets the first half of the KIA, which is "Loss of Intermediate Range NI" Does the OAC computer display change (color, font. Etc) to indicate a change in system, component, or data status (e.g. "bad data)? If so, this might be used to meet the generic portion of the KIA. The question type and source indicate that this question was modified from a 2004 NRC examination. Please include a copy of the original question if applicable for your changes. FJE 10/22109 On 1113/09 11/3/09 the facility visited the RII office to discuss selected questions from those reviewed as part of the sample of 30 as well as additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The facility rewrote the question to address most of the above comments and stated that they will revise the question and resubmit the question. FJE 11/3/09 The facility revised the question to address the above comments. The question is satisfactory. FJE 1212109 12/18/2009 20 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (1-5) Back- SRO B/MiN U/EtS S/MiN U/EiS Explanation (F/H) Stem Cues T/F Credo Partial Job- Minutia #/ Sack- Q= Focus Dist. Link units ward KIA Only I 037AK1.01 21 H ag xX x X ~
!"I- ~
!"l- N e lIJj Enhancement to resolve the issues discussed below:
4 S What document defines whether a release is considered monitored or unmonitored? Does CNS expect ROs to know this? Concern is that these terms may have specific meaning with respect to EALs and that this may not be RO knowledge. Also, does CNS have radiation monitors downstream of the S/G PORVS? Discussion regarding distractor C states that this option is plausible if the applicant believes the S/G PORV is isolated. Since no information is provided in the stem regarding whether or not equipment worked as expected, this could be a reasonable assumption, since isolating the PORV is directed by procedure if it will not reseat. Concern is multiple correct answers. FJE 10/22109 10/22/09 The question does not appear to be operationally relevant (PIC (P/C RB). RS). How would an operator directly know SG bulk fluid temperature and what would he/she do with this information? Additionally, the question appears to simplify to 'What is the S/G PORV lift saturation temperature corresponding to the SlG setpoiint?' - i.e. no knowledge of SGTL is required unless the S/G PORV setpoint is changed in this procedure. Suggest writing a question using AP-01 0 step 28, 36.g, or Enclosures containing subcooling steps, e.g. subcooling meter is broke. FJE 1212109 The facility wrote a new question. The new question is satisfactory. FJE 12116109 12118/2009 21 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD Q= SRO B/MiN U/EtS U/EiS Explanation (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back-Focus Dist. Link units ward KIA Only 067AK1.01 22 F 2 '+~ '+~ X X N e S Enhancement for job content and potential for multiple correct answers. What plant reference document states that foam is the correct extinguishing agent and that water cannot or should not be used? References provided do not appear to directly support the question. Concern is multiple correct answers. Referencing the General Discussion session, I also encouraged you to make the question operationally relevant to a licensed operator. How is this question (fire in a warehouse) operational in content and related to licensed duties? See App. BAtt. 1, item 1. Testing a different location, or fire exposures (risk to safety related equipment) based on knowledge of plant layout might be ways to make the question more operationally relevant. FJE 9/9/09 The facility revised the question based on the comments above. Enhancement to resolve the comments below. The stem does not state the condition of the VF system before the fire. Does this allow applicants to make assumptions that could lead to no correct answer or multiple correct answers? The first half of options A and B contain the word "only." How is "only" being used to distinguish these statements from the first half of answer options C and D? Does both the supply and exhaust fans trip I when swapping to filter mode? Please provide a technical reference that supports the correct answer for the second part of the question (fire classification). I FJE 10/22109 I The facility revised the question to address the above comments. The question is satisfactory. FJE 1212109. 12/18/2009 22 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
0# Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #1 Q= SRO B/MIN U/E/S
#/ Back- 0= U/EIS Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only WE03EK2.2 23 F -+
~ X X +
!1- 3 Ii Y tl The first half of the KIA is LOCA cooldown and depressurization.
2 N S The second half of the KIA is interrelationship between this and the facility's heat removal systems. The question asks for a preferred order of depressurization. While depressurization is not a heat removal mechanism, it is necessary to place the decay heat removal system in operation. For this reason, the question meets the KIA. Based on the reviewer's judgment, the question as written is Unsatisfactory due to multiple flaws and requires repair or replacement. (P/C(PIC BC). Credible Distractors/LOD Because the order of priority of each answer option is unique, an applicant only needs to know that pressurizer spray is the preferred method in order to answer the question. No knowledge of the priority of less-preferred methods is required. The order of priority of each answer option is unique. If an applicant knows the correct order of priority, then no knowledge of the preference description (e.g. "altemate")
"alternate") or reason is required.
The question is essentially, 'which one of the following is the preferred method of depressurizing the NC system?' In the absence of plant conditions that would suggest otherwise, use of auxiliary spray as a preferred method is not credible for a minimally trained operator. Stem Focus The question sentence is confusing. Ask the question directly. The answer options contain multiple elements and are confusing. The answer options consist of the following elements:
- A number/priority
- A method of depressurization
- A description of the degree of preference (e.g. "preferred, altemate, alternate, last resort." corresponds to the number?)
- A reason for the preference/priority Three of the four options containing a reason for the preference of pressurizer spray contain statements contingent on whether NC pumps are running or not. Additionally, 0.2 contains a double negative. This is potentially confusing. Consider making a definitive statement about the status of the NC pumps in the stem and removing this information from the answer options.
12118/2009 12/1812009 23 of 73
Catawba Nuclear Station 2009-301 1. 1- 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. B. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back- Q= SRO SAO B/MIN B/MiN U/E/S Explanation Focus Dist. Link units ward KIA Only Option B.2: What does "better" "better" mean? This is subjective and is not a 23 reason without additional explanation. cont Options B.3 and C.3: What does "too too slow" slow" mean? Too slow for what? This is subjective and is not a reason without additional explanation. Options B.3 and C.3 are different in that B.3 contains the additional statement concerning degradation of the regenerative HX. I did not find a reference to this HX in the development references listed. This information does not appear to be necessary to answer the question and, if incorrect, could result in the answer being deemed incorrect. What is the reason for including this additional information? What PORV POAV is intended in each of the answer options? The PZR PZA PORV POAV or the S/G PORV? POAV? A cooldown using a SG PORV POAV will result in a depressurization. Examiner Note: 2003 NRC NAC exam. FJE 10/22/09 10/22109 On 11/3/09 the facility visited the RII All office to discuss selected questions from those reviewed as part of the sample of 30 as well as additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The facility rewrote the question to address the above comments and will resubmit the question for review. FJE 11/3/09 The facility revised the question. The question is satisfactory. FJE 1212109 12118/2009 12/18/2009 24 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. S.
B. Q# LOK LOD (F/H) (1-5) PartialI Job- Minutia #1 Credo Partia #/ Back- Q= SRO B/MIN B/MiN U/EIS U/EiS Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only WEOSEK1.3 WEOBEK1.3 24 F + 4- N Ii IS 2 S The question requires enhancement due to low operational validity H and level of difficulty. This examination is intended to examine site-specific information. No plant specific knowledge is required to answer the question. This question examines generic fundamentals, Thermodynamics topic 193010, Brittle FractureNessel Thermal Stress. For a site-specific examination, this question has minimal operational validity in that the test item does not address a meaningful activity that would be performed on the job. See App. A, para. C.2. Also, the question has a low LOD for an operator who has passed the GFE. Stem Focus/Editorial All four answer options contain the words ".... because the temperature gradient across the reactor vessel wall and the pressure stress is additive ...
.. ."* which is not necessary to answer the question.
Additionally, a temperature gradient would not be additive with a pressure stress. It would be more correct to say that the stress due to the temperature gradient and the pressure stress is additive. The stem is incomplete in that it states the crew entered FR-P.1 due to a failure of all MSIVs to close. They would have entered P.1 due to the resulting cooldown. The answer options reduce to 1) most susceptible location (inner or outer wall), and 2) the condition causing the highest stress (heatup or cooldown) An equivalent and more direct question would be; Which part of the RV is most susceptible to failure for this accident sequence and when is it most susceptible? Inner RV wall during initial C/D C/O Inner RV wall during subsequent HlU H/U Outer RV wall during initial C/D C/O Outer RV wall during subsequent HlU H/U FJE FJ E 10122109 10/22/09 The facility revised the question to address the above comments. The question is satisfactory. FJE 1212109 211 ~/200~ ~2/1 25 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
0# Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #/ Back- 0= B/MIN U/EiS Q= SRO B/MiN U/EIS Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only e; WE13EK3.1 25 +
~ 2 X X N Ii H S The question requires enhancement to improve focus and eliminate potential cues.
The answer options contain redundant information not necessary to answer the question and provides a cue to answer the question [opening a PORV (and decreasing pressure) causes increased voiding]. The knowledge necessary to distinguish between the answer options is 1) initial level response, and 2) where the change in voiding occurs on a pressure decrease. Consider the following:
- 1) In what region of the SG does the amount of voiding change when the PORV is opened?
- 2) How will 1A SG indicated level.. .....
will1A FJE 10/22/09 10/22109 The facility revised the question to address the above comments. The question is satisfactory. FJE 1212109 e; WE14EA2.1 26 F 2 X X N Ii H S The question requires enhancement to eliminate backward logiC.logic. The question is written using backward logic - they are given a selected procedure and then asked to determine what plant conditions would result in entry into this procedure. The job task would more likely be that the applicant would be presented with a set of plant parameters and be required to select (or concur with) a procedure. This is also implied by the KIA - determine/interpret conditions and (then) select a procedure. Rewrite the question to eliminate backward logic. The answer must test selection of a procedure related to high containment pressure. Distractor A is not plausible. 3 psig is the phase B setpoint. At this pressure, automatic actions designed to mitigate the higher pressure in containment, and prevent challenging the CSF, will occur. FJE 10/22109 The facility rewrote the question. The question is satisfactory. FJE 1213109 1213/09 12118/2009 26 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (FIH) (F/H) (1-5) PartialI Job- Minutia #1 Credo Partia #/ Back- Q= SRO B/MiN U/E/S SAO BlMlN U/EiS Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only WE15EA2.1 27 F 2 X N lYj H S The question is unsatisfactory because it does not meet the intent of the KIA (PIC (P/C MB) The KIA is for operating characteristics/behavior during containment flooding. The plant conditions in the KIA are for a LOCA (KIA 011 or E03) vs. containment flooding. The concept being tested by the question appears to be containment isolation/reset (KIA 103). The question does not require knowledge of containment flooding or associated procedures. The examiner did not evaluate other aspects of the question due to KIA not met. FJE 10/26109 On 11/3/09 11/3109 the facility visited the RII All office to discuss selected questions from those reviewed as part of the sample of 30 as well as additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The facility rewrote the question to address the above comments. The examiner expressed concern that the revised question may be written at the SRO SAO vs. RO AO level. The facility agreed to evaluate whether the question was written at the RO AO level. FJE 11/3/09 The facility rewrote the question. The question is satisfactory. FJE 12/3/09 1213109 003K4.04 28 F 2 X N e S Enhancement for grammar and possibly distractor plausibility. The question is not grammatically correct. The question should read
'Which "Which of the following ..
...."" or 'What "What system provides ..."
Answer B discussion states that some containment components KC cooling are isolated on a Phase A signal, which would appear to be a more plausible choice than a Safety Injection signal. Does YV isolate on a Phase A signal? Low LOD, but acceptable. FJE 10/26109 10/26/09 The facility revised the question to address the above comments. The question is satisfactory. FJE 12/3/09 1213109 12118/2009 12/18/2009 27 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
0# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #1#/ Back- 0= SRO BlMlN B/MiN U/EIS U/E/S Explanation Stem Cues T/F Focus Dist. Unk Link units ward KIA KJA Only 004A4.08 - DCSDeS Question 29a H 2 ~ X X N e Examiner Note: Requires knowledge of DCS modification. S Enhancement to resolve questions regarding plausibility and language in distractors. Distractors A and C do not appear plausible. A.2: Why would manipulating the SLIM station change the method of control of another controller? How is this similar to restoring a FRV? C.1 and C.2 seem inconsistent. If action is required to restore the master controller, why would valve position be changing? Additionally, if pressurizer level is above program level, C.1 is less plausible than 0.1. What procedure (technical reference) specifies "normal" "nonnal" pressurizer level control and specifies the operator actions needed? Tie the question to a reference to limit the possibility of alternative answers (App. B, C.1.c) What is intended by 'valve position increases/decreases'? Would an operator be more likely to say that 1NV-294 would be more open or closed? FJE 10/26109 10/26/09 The facility revised the question. Enhancement to resolve the following: What is program level at 85% power and how will applicants compute expected level? Please include in general discussion. The 4th bullet refers to placing 1NV-294 in "manual." The general discussion refers to placing 1NV-294 in "local. " Are local and manual equivalent? The questions are referenced to different points in time (0#1 to the third bullet, 0#5 implicitly to the 5thth bullet.) Consider making the second question clearer by moving the last bullet to the second question, e.g. What operator actions ...... after 1NV-294 is repaired? FJE 1213109 12118/2009 28 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. B.
Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #1 Back- Q= SRO BIMIN B/MIN U/E/S Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only The facility revised the question to address the above comments. 29a The revised question is satisfactory. FJE 12116109 12116/09 cont 004A4.0B - 7300 Question 29b H 3 M E S Enhancement to resolve question source and distractor C style. Please check the question source and include a copy of the original question. The 2007 exam Q#31 in the master exam file is written to a different topic. Distractor C is less complete than the other distractors. Is it the output signal from the master that has failed high? Editorial: The question asks for single failures that would cause the symptoms. The answers are a single failure that caused the symptoms. Question/Answers should be either Which single failure would cause ... ? Xxx failing low Which single failure caused .... ? The ... signal failed ... FJE 1213109 The facility provided a copy of the original question. The question is modified in that all answer choices have been changed in the new question. The facility revised the question to address the comments listed above. The revised question is satisfactory. FJE 12116109 12116/09 12/18/2009 12118/2009 29 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws s. Other
- s. 6. 7. 8.
Q# LOK LOD LOO (FIH) (F/H) (1-S) Credo Partial Job- Minutia #1 Back- Q= SRO B/MIN U/EIS Creel. UlEtS Explanation Stem Cues T/F Focus Dist. Oist. Link units ward KIA Only 00SA4.0S 30 F 2 ~
!?- N e E
S Could not find a reference to 1ND-33 NO-33 in the references provided (1-E-o and 1-E-0 background document). Please provide a technical reference that supports the correct answer. Enhancement until correct answer is verified using supporting references. Examiner Note: Meets the KIA because NO pump miniflow can be aligned to return (recirculate) to the RWST. FJE 9/9/09 Facility provided technical reference validating answer. Question is satisfactory. FJE 10126109 10/26/09 Editorial: Typo in general discussion. PT///30 PTI1I30 is listed vs. reference PT////40. PT1I1I40. FJE 1213/09. 1213109. Typo corrected. FJE 12116109. 12116/09. 00SK1.13 31 F +
-+ X X N Y.
lJ 2 S The question is unsatisfactory for multiple implausible distractors. (PIC PC) Distractors Oistractors A.2 and C.2 are not plausible for a trained operator with no operator action or other complicating factors in the stem. Although, as pointed out in the answer discussions, hot leg recirculation utilizes only two hot legs, this only occurs after significant time has elapsed and after operator action. FJE 10/26109 On 1113/09 the facility visited the RII office to discuss selected questions from those reviewed as part of the sample of 30 as well as additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The facility rewrote the question to address the above comments and will resubmit the question for review. FJE 11/3/09 11/3109 The facility revised the question. The question is satisfactory. FJE 1213/09. 1213109. 12118/2009 12/18/2009 30 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD B/MIN U/EiS SRO B/MiN U/E/S Explanation (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1#/ Back- Q= Focus Dist. Link units ward KIA Only 006A1.07 32 W M ad xX N tJ U S KIA statement is "Ability to predict and/or monitor changes in F 2 parameters (to prevent exceeding design limits) associated with operating the ECCS controls including: Pressure, high and low." The question does not test an associated ECCS control (or design limits) and does not meet the KIA. The question tests whether the applicant can correctly predict a change in pressure for a letdown malfunction, and, for answer C and distractor 0, the components that terminate the pressure increase (PORVs or NO suction relief valves). Unsat due to not meeting the KIA (peer checked - PC). Did not evaluate other aspects (items 3-5) of the question. FJE 9/9/09 1113/09 the facility visited the RII office to discuss selected On 11/3/09 questions from those reviewed as part of the sample of 30 as well as additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The facility rewrote the question to address the KIA match and will resubmit the question for review. The examiner agreed that the revised question meets the KIA. FJE 11/3/09 The facility wrote a new question. The question is satisfactory. FJE 1213/09 12/3/09 12118/2009 12/18/2009 31 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
0# Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back- 0= Creel. Q= SROSAO B/MIN U/EtS U/E/S Explanation Focus Dist. Link units ward KIA Only 006K5.06 33 H 2 X X ~
!?- X X ~
!?- N U 1J S Unsatisfactory due to multiple implausible distractors.
The answer/distractor analysis does not provide any basis for distractor plausibility. The second half of distractors A and B are not plausible (peer check - MB). During this time period, pressure is changing slowly. Additionally, the resolution of the graph is such that the total pressure change during this time period is approximately the width of the trend line and cannot be determined (unrealistic level of accuracy). "Between 8 and 9 minutes* minutes" would be more plausible. If the applicant is not expected to interpret pressure, then a lower cognitive question regarding recall of NI and ND shutoff head would be a more appropriate format for the question. Who reports parameters and performs actions isn't important in order to answer the question. Including this piece of information merely adds words. Additionally, where the NI and ND pumps discharge (into the cold legs - as stated in the question) is not necessary to answer this question and could provide cues for other questions. See App. B, Att. 1, item 3. Consider the following: Given the following:
- Unit 2 was operating at 100% power when a LOCA occurred at TimeO.
- The NC system pressure trend is shown below.
What are the earliest times that the NI pumps and ND pumps will discharge ECCS flow to the NC system? Consider formatting the answers such that they are easier for the applicant to compare, e.g.: NI Pumps ND Pumps A. Between 0 and 1 minute Between 7 and 8 minutes Could not find shutoff head of NI pumps (1536 psi) mentioned in the reference provided. Please provide a reference that supports this portion of the answer. Concern is no correct answer. FJE 9/9/09 12118/2009 32 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #1 B/MiN UlEtS
#/ Back- Q= SRO B/MIN UJElS Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only On 11/3109 11/3/09 the facility visited the RII office to discuss selected 33 questions from those reviewed as part of the sample of 30 as well as Cont. additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The facility rewrote the question to address the above comments. The examiner determined that the distractors in the revised question are plausible.
The facility will resubmit the question for review. FJE 11/3109 11/3/09 The facility revised the question. The revised question is satisfactory. FJE 1213109 1213/09 007K5.02 34 F 3 X X X X N e S The SLC heatup and cooldown limits are stated as degrees in any 1-hour period. The answer and distractors are stated as degrees per hour. Change the answer and distractor options to be consistent with the technical reference. Concern is for no correct answers. Tie the second question to a procedure in order to preclude the possibility that the distractor could be viewed as correct. FJE 10/26109 The facility revised the question to address the above comments. The question is satisfactory. FJE 1213109 1213/09 008A1.03 35 H 3 N S The question is satisfactory. FJEFJ E 10/26/09 10/26/09 12/18/2009 33 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #/ Back- Q= SRO SAO B/MiN U/EiS U/E/S Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only 010K6.01 - DeS Question
~ X N ltJj 36a H 2!?-
3 e The question is unsatisfactory for multiple implausible distractors (P/C PC). S With one pressure instrument failing High and the other failing Low, with no operator action, A.2 and B.2 are not plausible. Note that the effect of the failure on the reactor is not required per the KIA, merely the effect on the pressurizer pressure control system. Consider limiting the question to the effect(s) on the pressure control system. Eliminate the negative (not) and ambiguity (affected) associated with A.1 and D.1. Additionally, since A.2 states that the reactor tripped, this would have an effect on the pressurizer master, but not on the mode of operation. Consider "The Pressurizer Pressure master remains in automatic.' automatic." FJE 10/26109 10/26/09 On 11/3/09 the facility visited the RII All office to discuss selected questions from those reviewed as part of the sample of 30 as well as additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The facility rewrote the question to address the above comments and will resubmit the question for review. FJE 11/3/09 The facility revised the question. Enhancement to resolve the following comments: General Discussion - what Alternate Action conditions occur for this set of conditions? AA 1 and AA2 both? Why won't a PORV POAV open? Distractor A contains the word "only" "only" but none of the other distractors contain the words in the first part of the distractor. Why include "only"1 "only"? Distractor D lacks plausibility because it does not specify which PORV(s) POAV(s) open as explained in the distractor analysis. The style of AlB are different than C/D. In A and B it is implicit that the master remains in auto. Consider rewording A&B, e.g. The Pressurizer Pressure Master remains in automatic and pressure is controlled based on the .... FJE 12/3/09 12/18/2009 34 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
0# Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #1 Back- 0= UlEiS B/MIN U/E/S Q= SRO BIMIN Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only The facility revised the question to address the above comments. 36a The revised question is satisfactory. FJE 12116109 12116/09 cont 010K6.01 -7300
- 7300 Question 36b H 3 +
~ N e Ii S Enhancement to resolve the below comments:
Replace the word *oscillate" "oscillate" in C and D with *vary" "vary" or "cycle". The reviewer felt that use of "oscillate" made C and D less favorable as distractors (PIC BC). BG). Option B different in style than the other options because the question asks for a pressure response and option B starts with component response vs. pressure response. Consider rewording, e.g. Pressure will be controlled at approximately 2235 psig by modulation of the spray valves. FJE 1213109 The facility revised the question to address the above comments. The revised question is satisfactory. FJE 12116109 12118/2009 35 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LaD U/E/S Q= SRO B/MIN U/EIS Explanation (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back-Focus Dist. Link units ward KIA Only 012K6.10 37 F 2 X X N tl U S The question is unsatisfactory because it does not meet the intent of H 3 the KIA. (PIC PC). The KIA is for the effect of a loss or malfunction of a permissive circuit on the RPS. The question is written to test the RPS setpoint for loss of flow (012K4.02) and does not incorporate testing knowledge of the effect of a loss or malfunction of a permissive circuit. The knowledge required for the second half of the question is
'will a reactor trip occur for the conditions given above?' and the malfunction is a statement in the stem that the trip breakers are shut.
FJE 10/26/09 On 10/29/09 the facility exam team contacted the NRC reviewer to explain why they thought the question met the KIA. The reviewer explained why the question did not meet the KIA and the facility agreed to rewrite the question. FJE 10/29/09 On 11/3/09 the facility visited the RII office to discuss selected questions from those reviewed as part of the sample of 30 as well as additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The facility made minor revisions to the question to address the KIA match. The examiner concluded that the original and revised question met the KIA, but expressed concern regarding the plausibility and level of difficulty of the question. The examiner stated that he would re-review whichever question the facility submitted. FJE 11/3/09 The facility revised the question. The question is satisfactory. FJE 1213109 1213/09 12118/2009 36 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5.
S. Other 6. 7. 8. Q# LOK LOD LCD (FIH) (F/H) (1-5) (1-S) Credo Partial Job- Minutia #1 Back- Q= SRO B/MIN U/EtS U/EIS Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only 013K5.02 013KS.02 38 F 2 N e S Enhancement to resolve the items described below. Please clarify whether or not any operator action is taken for pressure channel 2 failing "as is." Concern is that applicant may make assumptions resulting in multiple or no correct answers. All four answer options contain redundant words - "that input P-11" P-11"-- at the end of the sentence. Please remove these words or explain why they are necessary. FJE 10/26109 10/26/09 I The facility revised the question. Enhancement to resolve the following (editorial) items: Missing a word (into or to) after "input." The words while "while the other channels ... as it decreases" in the first bullet are redundant based on the information in the second bullet and 3 sub bullets. Does underlining "remaining" help to clarify or confuse? The important part of the sentence seems to be that "that input into the P-11 circuit." FJE 1213/09 1213109 The facility made editorial changes and addressed the above comments. The question is satisfactory. FJE 12116109 12116/09 12118/2009 12/18/2009 37 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
B. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back- Q= SRO B/MIN B/MiN U/E/S Explanation Focus Disl. Dist. Link units ward KIA Only 022A2.01 39 F -+4- X X -e-
-e-- U V The question is unsatisfactory due to multiple psychometric 2 ~ N eE flaws.
S The question is a direct lookup question (LOD=1) (peer check - GL). Little mental activity is involved in answering the question other than transcribing an answer that is available in the supplied reference. Initial license testing should emphasize areas of knowledge where procedures do not need to be used (See OL Program Feedback 401.42 and 401.53). The question asks for the first action that needs to be taken." Although a reference is provided, the question does not tie the action to the reference. If there are Annunciator Response Procedures associated with these plant conditions, then the "first action" would presumably be directed by the ARP and not the provided operating procedure. Additionally, per the supplied procedure, the first applicable action for the plant conditions given is to monitor OAC temperature points per step 2.2, which is not the answer (the question asks for the "first action that needs to be taken by the crew"). The stem provides temperature information for these paints, points, but does not state that they are being monitored. Is this implied and the question is looking for the next applicable action? Distractors Band C are not plausible given that a reference is provided. The action of shifting LCVUs to high speed is not mentioned anywhere in the procedure provided. Distractor D (generate a work request) is not plausible as a "first action" to stabilize the plant following an eqUipment equipment malfunction. Additionally, this is the only distractor that consists of an administrative task vs. equipment manipulation. Only answer A directs manipulation of a specific component (PTBF 1A). Distractors Band C do not direct manipulation of specific components, making A different in style - a clue to the correct answer. See App. B, C.2.k.( specific determiner). The answer B discussion mentions an "NRC exam review" which is confusing, as no exam review was conducted up to the point of question submittal. Examiner Note: Although loss of CRDM cooling is covered under KIA System 001, Control Rod Drive System, the licensee's lesson plans show that the CRDM system is treated as a subset of the Containment Ventilation (VV) System. FJE 9/10109 9/10/09 FJE 9/9/09 12118/2009 38 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #/#1 Back- Q= SRO B/MIN U/E/S U/EIS Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only On 11/3109 11/3/09 the facility visited the RII office to discuss selected 39 questions from those reviewed as part of the sample of 30 as well as cont. additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The facility rewrote the question to address the above comments. However, the examiner expressed concern that the revised question was testing an unreasonable level of detail. The facility agreed to reevaluate the question. FJE 11/3109. 11/3/09. The facility revised the question. Enhancement to resolve the following: Is it possible to start the 1D LCVU in HIGH speed either by interlock or procedure? If not, then distractor D lacks plaUSibility. plausibility. FJE1213109 FJE1213/09 The facility explained the plausibility of D. The revised question is satisfactory as written. FJE 12116/09 022K3.02 40 H 2 !l-
'+ N e Ii i S Enhancement to resolve the following:
At 100% 100% power, how much of a level change would be seen in the MCR for a 13F change in temperature (if charging was in manual)? Concem Concern is for no correct answer if the magnitude of the temperature change would not produce any measurable change in pressurizer level. Do you have any (plant specific) operating procedures (lower modes?) that would support the answer for the second question vs. vS. GFE knowledge? FJE 1213109 The facility rewrote the question. The new question is satisfactory. FJE 12116109 12116/09 022A3.02 41 F 2 N S The question is satisfactory. FJE 1213109 1213/09 12118/2009 39 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #/ #1 8ack- B/MIN U/EtS Back- Q= SRO 8/MIN UlEiS Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only 026A2.02 42 !1-
~ !1-
~ X X ~
!1- B 8 E Ie Per the General Discussion, to arrive at the correct answer, the applicant, from memory, must recall Steps 3 RNO, 4, 6 RNO and then step 33. This requires the recall of knowledge that is too specific for the closed reference test mode (i.e.,
(Le., it is not required to be known from memory). On the other hand, since both ES-1.3 and ECA-1.1 contain a Foldout Page step "If FWST level decreases to less than 5%, then stop all pumps taking suction from the FWST" FWST' the question appears to only require memory level recall of the FWST level at which ECCS pumps must be secured. The question is vague and not consistent with the answers provided. The stem states that ECA-1.1 is in progress. The question asks
Which
which one of the following is procedurally required" but does not reference a procedure. What is implied? Required per rules of use or EOP users guide? Three of the answer options contain unique actions related to ECA-1.1. Distractor D 0 references FR-Z.3. ECA-1.1 would not direct a transition to FR-Z.3, but FR-Z.3 is applicable for the given conditions (but a relatively low priority).
Is 27R containment radiation in the stem a realistic and achievable value for the conditions given? Answer A states to "secure all pumps* pumps" (no system(s) or components specified) while distractors 8 and D Band 0 specify which system pumps to align. As such, Answer A differs in style from distractors 8 and D Band 0 and may be more likely to be selected (see App. 8, B, C.2.m, specific determiners) Additionally, Answer A closely paraphrases the procedure step, which is not desirable (see App. 8, B, C.1.g). Do not understand why distractor 8 B is plausible. The stem states that automatic and manual swap to CLR failed. If both auto and manual swap to CLR has failed, how can NS pumps be aligned for recirculation? Distractor C is vague. What does it mean to "address high containment radiation"? Is this the same as 'go to FR-Z.3'? If so, assessment of plant conditions and selection of procedures (including to rule out a distractor) would appear to be SRO-Ievel knowledge. Additionally, Distractor D 0 is implausible in that addressing Yellow path FRs are discretional and would not take priority over an ECA procedure. continued 12118/2009 12/18/2009 40 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back- Q= SRO B/MIN B/MiN U/E/S Explanation Focus Dist. Link units ward KIA Only 42 F 2 ~
!?- M eE Examiner Note: The KIA merely requires the ability to use cont N S procedures with respect to a failure of automatic recirculation transfer. The question involves failure of both automatic and manual recirculation transfer. However, because failure of automatic transfer is a subset of a failure of automatic and manual transfer, the question meets the KIA (peer check - BC).
FJE 9/10109 9/10/09 On 11/3109 11/3/09 the facility visited the RII office to discuss selected questions from those reviewed as part of the sample of 30 as well as additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The facility rewrote the question to address the above comments. The examiner expressed concern that some distractors do not appear plausible. FJE 11/3109 11/3/09 The facility revised the question. Enhancement to resolve the following: The question type and source indicate that this question was modified from a 2004 NRC examination. Please include a copy of the original question. The answer options consist of three parts. Only the NS portion is required to meet the KIA. Only one additional ECCS pump would be needed to provide a sufficient number of plausible distractors. If for some reason an applicant knows that no NI and no NV pumps are required, but does not know how many NS pumps are required, then he/she could arrive at the correct answer without knowing anything about the NS portion (KIA) of the question. What is the reason for providing a three part answer? FJE 1213109 12/3/09 The facility rewrote the question to address the above comments. The question is satisfactory. FJE 12116/09 12/16/09 12118/2009 12/18/2009 41 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
B. Q# LOK LOD (1-5) #/ Back- Q= SRO B/MIN U/E/S U/EtS Explanation (F/H) Stem Cues T/F Credo Partial Job- Minutia #1 Focus Dist. Link units ward KIA Only 039A1.09 43 H ~ ~ N eE S Enhancement to resolve the following: 3 Do you have a technical reference that supports the first part of the question, i.e. that shows or states that for a stable leak rate counts will decrease with power? Do you expect RO applicants to answer the second question by knowing the contents of NSD 513? Would they be able to derive the answer from systems knowledge or from knowledge of AP////1 APIII11 O? FJE 1213/09 1213109 The facility addressed the above comments. Per Ops (B. Blair), NSD 513 is RO knowledge and the knowledge required of the question is contained in the associated lesson plan. The question is satisfactory. 12116109 FJE 12116/09 039G2.2.37 44 ~ 2 N eE F S What is the basis for the higher COG level? FJE 1213/09 1213109 The facility classified the COG level as "memory" (LOK = F). FJE 12116/09 059A4.08 059A4.0B 45 H 2 B eE S Enhancement (placeholder) to resolve the following regarding the exam (vs. this question). Does the Unit 1 DCS mod install different controllers for feedwater control? If so, why is there not a different question for the 200B 2008 retake applicants? Examiner Note: 2005 NRC Exam The question is otherwise satisfactory. FJE 1213/09 The facility explained that, although DCS uses a different controller, the control response, which is what is being tested, is the same for 12116109 both the DCS and "old" systems. FJE 12116/09 12118/2009 12/18/2009 42 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD Q= SAO B/MIN U/E/S SRO BIMIN Explanation (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back-Crect. Focus Dist. Link units ward KIA Only 061G2.4.20 46 H 3 B S i Examiner Note: 2003 NAC NRC Exam I The question is satisfactory. FJE 1213109 062A3.01 47 H 3 M S Examiner Note: The question was modified such that one of the three distractors in the original question is now the correct answer in this question. The question is satisfactory. FJE 1213109 062K2.01 48 F 2 N S Editorial: Consider moving the second half of the question, a plant condition, into the stem in order to simplify the question. 1213109 The question is satisfactory. FJE 1213/09 The facility made minor editorial changes and the question remains satisfactory. FJE 12116/09 063K2.01 49 F 2 N S Editorial: The question is missing the word "the"
the before "250V."
The question is satisfactory. FJE 1213109 063K3.01 50 F +
-+ X N 1.1 U
H 3 S The question is not discriminating for a trained operator (LOD=1). (PIC BC) It appears that to answer the question the applicant only has to recall that loss of DC control power (DIG 1A E/5) = no auto or manual start. FJE 1213109 1213/09 The facility rewrote the question. The question is satisfactory. FJE 12116109 12116/09 064K4.01 51 F 2 N S 1214/09 The question is satisfactory. FJE 1214109 073A2.01 52 F 2 N S 1214109 The question is satisfactory. FJE 1214/09 12118/2009 43 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
I Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #1 Creel. #/ Back- Q= SROSAO B/MIN B/MiN U/EIS U/EtS Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only 076K2.01 53 F -+ X M tJ U N S The question appears to have an LOD=1 for a trained operator. (PIC (P/C H 2 CK) What is the significance of the word "normally" in the stem? Can the 2B RNAN pump be powered from another supply other than 2ETB? Note that the KIA does not limit the topic to the pump power supply. Consider a 2x2 question using C and D and also testing power to another major RN AN component or set of components. Examiner Note: Modified from 2008 NRC NAC exam. FJE 1214/09 1214109 The facility rewrote the question. The question is satisfactory. FJE 12116109 078K3.02 54 H 2 X N Ie S Enhancement to improve stem focus/clarity of answer options. The question is incomplete - affected by what? Consider adding ....." ... affected by the loss of VI pressure?" The answer options are confusing because they are not complete sentences. Revise Aevise to make the answer options clearly follow from the question, e.g. I I Control of the CA flow control valves is immediately lost and CA flow will immediately decrease. Control of the CA flow control valves is maintained for at least 1 hour, and then CA flow will decrease. FJE 1214109 The facility revised the question to address the above comments. The revised question is satisfactory. FJE 12116/09 12116109 12/18/2009 12118/2009 44 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
B. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back-8ack- Q= SRO B/MIN 8/MiN U/EIS U/EiS Explanation Focus Dist. Link units ward KIA Only 103K1.01 55 H 2 N S Editorial: If the stated assumption is important to answer the question, then consider placing it before the question. The question is satisfactory. FJE 1214/09 1214109 The facility made minor editorial changes and the question remains satisfactory. FJE 12116109 12116/09 001K2.01 56 H ~
!1- M e 3 S Examiner Note: The question stem was modified and the answers were modified such that one of the three distractors in the original question is now the correct answer in this question.
Enhancement for stem focus. The question background indicates that the intent of the question is to recognize that the 1C NC pump pum p will trip and that, with the unit in Mode 3 and no control banks withdrawn, a reactor trip is not required. However, it is possible to read the intent of the question as 'what is the status of the shutdown banks (as a result of the bus fault)?' If an applicant looks at the question this way, answer B 8 is still correct and option C appear implausible. FJE 1214/09 1214109 The facility revised the question to address the above comment. The question is satisfactory. FJE 12116109 12116/09 002K1.07 57 F 2 B 8 S Examiner Note: 2003 NRC Exam question. The question is satisfactory. FJE 1214109 1214/09 011 A2.11 - DCSDeS Question 58a 5Ba H 3 X N e S Enhancement for distractor plausibility. Change the word "ruptured" in B.1 8.1 and 0.1 to a word implying a less drastic response in order to improve distractor plausibility, e.g. 'is leaking'. "Ruptured" seems inconsistent with a slow decrease in pressure, as stated in the second bullet. FJE 1214109 1214/09 The facility made minor editorial changes to address the above comment. The question is satisfactory. FJE 12118/2009 45 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/MIN U/EIS U/E/S Explanation Focus Dist. Link units ward KIA KJA Only 011 A2.11 - 7300 Question 58b H 3 N S The question is satisfactory. FJE 1214/09 12118/2009 46 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #/ Back- Q= SRO B/MiN U/EiS U/E/S Explanation Stem Cues T/F Crecl. Focus Dist. Link units ward KIA Only 01lKd.04 018K3.04 59 F 2 X ~
!?- W e
~
Enhancement to resolve non-editorial items listed below: How does the General Discussion and answer/distractor discussions explain the question and distractors? The question and distractors concern loss of a pressure signal and controllers swapping to manual. The General Discussion is for a drop in pressure ending in a reactor trip and the answer/distractor discussion concerns delta pressure. The stem states that the "DFCS Unit Feedwater Header Pressure signal fails" but does not state how the signal fails. Can the signal fail in different ways (e.g. high, low, or 'as is') and is this important to answer the question. Concern is unstated assumption with potential for multiple correct answers. In order for the equipment to swap to manual, it would have to start out in automatic. Is all of this equipment normally in automatic at 60% power per plant procedures? Development Reference listed with the question is "IFE." "I FE." Reference provided with question is the Steam Generator Level and Feedwater Pump Speed Control Lesson Plan. Is this the same as "I FE"? What is the basis for a higher COG level? This would appear to be recall vs. knowledge of the design of the control system and failure analysis. Editorial The format of the question forces the applicant to read the answer options and then refer to the list of four items to determine the corresponding component. Additionally, with use of the word 'only" "only" the applicant is compelled to compare answer options against each other. Acceptable as is. However, consider stand-alone answers, e.g: A. CF Main Reg valves and CF Bypass Reg valves swap to manual CFPT Master controller and CFPT slave controllers remain in auto FJE 9/11/09 KIA replaced for exam security. New KIA 015K6.02 12118/2009 12/18/2009 47 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
B. Q# LOK LOD Q= SRO B/MiN 8/MIN U/E/S U/EJS Explanation (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back-8ack-Focus Dist. Link units ward KIA Only 015K6.02 59 H 2 B 8 S cont The question is satisfactory. FJE 1214/09 016K3.09 60 F 2 B 8 S Examiner Note: McGuire 2008200B Retake Exam The question is satisfactory. FJE 1214/09 02BK5.03 028K5.03 61 F 2 ~
!1- B 8 eIi S Enhancement to ensure single correct answer. Tie the second question to a procedure to ensure a single correct answer and provide a frame of reference for the applicants.
FJE 1214109 The facility made a minor editorial revision to address the above comment. FJ E 12116109 com m ent. The question is satisfactory. FJE 12116/09 017A3.01 62 H 2 N S 1214109 The question is satisfactory. FJE 1214/09 035A4.02 63 F 2 N S Editorial: Consider moving the plant conditions embedded in the question (CA available, CETC decreasing) up to the stem as current conditions. Portions of ECA-2.1 were highlighted as a developmental reference. How was this used in the distractors or questions or was it leftover from earlier development? The question is satisfactory. FJE 1214109 The facility revised the question to address the above comments. The question remains satisfactory. FJE 12116109 12116/09 12118/2009 12/18/2009 48 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD U/EIS Q= SRO B/MIN UlElS Explanation (F/H) (H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back-Focus Dist. Link units ward KIA Only 041A1.02 64 H 2 N S Editorial - redundant words in stem or first question. Stem states "For these conditions:" Q#1 asks ..... for current conditions." There is
" .. Jor only one set of conditions given.
1214/09 The question is satisfactory. FJE 1214109 The facility made minor editorial revisions and the question remains satisfactory. FJE 12116/09 045K4.37 65 F 2 !1-
~ !1-
~ B Ii S Examiner Note: 2006 NRC exam Enhancement 1 explanation to ensure a single correct answer and ensure stem focus.
Please explain which lockout(s) occur for the given conditions. Zone 1Al1 B, or both. If a Zone 1A&1B G only, Zones 1A11B, 1A& 1B lockouts would occur, would they be related to the turbine trip? FJE 1214109 The facility provided an explanation for the above questions. The question is satisfactory without revision. FJE 12116109 12116/09 G2.1.4 66 F 2 !1-
~ N Ii S Enhancement - tie the question to a procedure to ensure a single correct answer (concern is the respirator requirement).
FJE 1214109 The facility revised the question to address the above comment. The question is satisfactory. FJE 12116/09 12118/2009 12/18/2009 49 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #1 #/ Back- Q= SROSAO B/MIN B/MiN UlElS U/E/S Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only I G2.1.15 67 F 2 !I-
~ !l-
~ N e Ii S Enhancement to resolve the following:
"Appropriate" is a subjective word. Change the wording to remove subjectivity, e.g. "allowed,"
"allowed."
Please verify that Operations expects ROsAOs to know this process at this level of detail. The procedure did not specifically list any RO AO responsibilities, only SRO. SAO. FJE 1214/09 1214109 The facility wrote a new question at the ROAO level. The question is satisfactory. FJE 12116/09 12116109 G2.1.40 68 F 2 X B IJ U S Unsatisfactory due to non-plausible distractors. (PIC (P/C BC) Since the Fuel Handling SRO SAO is responsible for fuel movement in the RB, AB, he/she would be involved in any decision regarding interlocks. It is plausible that permission from others might also be required, but not plausible for a trained operator that the SRO SAO would be excluded. Although the question could be rewritten/reformatted for this topic (e.g. minimum requirement to bypass with some answersldistractors answers/distractors of 'SRO
'SAO only, SRO SAO AND Rx Ax Eng' etc., a more relevant question for ROs AOs might be based on control room duties.
I Examiner Note: 2006 NRC NAC exam. I FJE 1214109 1214/09 The facility revised the question to address the above comment. The revised question is satisfactory. FJE 12116109 12116/09 G2.2.3 69 F 2 B S I The question is satisfactory. FJE 1214109 1214/09 I 12118/2009 12/18/2009 50 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (1-5) #1 Back- Q= SRO B/MIN U/EIS Explanation (F/H) Stem Cues T/F Credo Partial Job- Minutia #/ Creel. Focus Dist. Link units ward KIA Only G2.2.38 70 H 2 :x X ~
!:!- N e Ii S Enhancement for stem focus. The order of the two questions seems backwards. It seems that one would first ask if the FWST were required to be operable for the given NC conditions and, if so, then take action to restore operability. The logic (plausibility) of distractors A and B does not make sense the way the question is worded.
These answer options are 'take an action, but the action isn't required.' So, since the only choices for the first part are one of two required actions, then the FWST must be required to be operable (cue) One option might be to revise the question along the lines of 1.For the stated conditions, is the .... 2.when the FWST IS required to be operable, and the FWST is below.... FJE 1214109 The facility revised the question to address the above comments. The revised question is satisfactory. FJE 12116109 G2.3.14 71 H 2 B S The question is satisfactory. FJE 1214/09 1214109 G2.3.13 72 F 2 N S The question is satisfactory. FJ# 1214/09 12118/2009 12/18/2009 51 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/MIN U/E/S U/EtS Explanation Focus Dist. Link units ward KIA Only G2.3.7 73 F 2 N Ie E S Enhancement for stem focus. These appear to be two separate questions that do not necessarily need to imply that there is a problem with the RWP since you are not testing whether to enter the area or not. Consider removing potentia:! potential confusion or suspicion associated with RWP accuracy by rewording the question. For example, In accordance with NSD-507, an area with a dose rate of 325 mRlhr .... is required to be posted as a , and a flashing yellow light at the entry path to a Radiation Control Area signifies FJE 1214109 1214/09 The facility revised the question to address the above comments. The revised question is satisfactory. FJE 12116109 12116/09 G2.4.21 74 H 3 N S Editorial: A LOCA occurred as a result of the transient would be more grammatically correct. The question asks for the highest priority CSF at this time, but the stem does not reference times. Consider changing the question to
" .... For the given plant conditions."
The question is satisfactory. FJE 1214109 1214/09 The facility made minor editorial changes to address the above comments. The question remains satisfactory. FJE 12116/09 12118/2009 52 of 73
Catawba Nuclear Station 2009-301
- 1. 2. I 3. Psychometric Flaws 4. Job Content Flaws LOD r---.-~'---.----r----~~r----r---r--~~~--~
- 5. Other 6. 7. 8.
Q# LOK (F/H) (1-5) Stem Icuesl Focus Cues T/F Credo Partial Job- Minutia #1 T/F Icred.lpartiaIIJOb-IMinutial Dist. Link
#1 IBack- Q= ISROIB/MlNIUlE/S Back-I units ward KIA Only SRO B/MIN U/EIS Explanation G2.4.29 75 F 2 x ~ M M
S Examiner Note: 2008 NRC exam. The question is modified in that the stem, and a portion of all answer choices, is different in the new question. Enhancement to ensure single correct answer. Distractor A could be considered correct because, as stated in the distractor analysis, some SAEs (but not all) require evacuation and a SAE is the minimum level of the choices given. Reword the question to ensure a single correct answer, e.g. consider adding the word "always" before "requires" in the question. FJE 1214/09 The facility made minor editorial changes to address the above comment. The revised auestion is satisfactorv. FJE 12116/09 12/18/2009 53 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. B.
Q# LOK LOD (F/H) (1-5) PartialI Job- Minutia #1 Back-Credo Partia Q= U/EIS SRO B/MIN UlElS Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only 76 H 2 X X X X ~ N V U 008AA2.18 00BAA2.1B E S Unsat due to multiple flaws. KIA match is marginal. KIA statement is the ability to determine and interpret computer indications for RCS temperature and pressure as they apply to the pressurizer vapor space accident. The stem of the question provides single values for core exit temperature and NC pressure and states that these values are from the OAC (computer). So, the applicant must use the only data given (computer data) in order to arrive at the answer, which meets a portion of the KIA. However, the applicant does not need to determine or interpret these indications, e.g. determine which of multiple potential data points or parameters are required to be used when the ICCM is unavailable. Consider providing multiple data points (with titles) or a picture of one or more OAC screens so that the applicant both determines and interprets which OAC datapoints must be used to derive this portion of the answer. As is, the question appears to involve interpreting a graph of subcooling using the data provided, which happens to be computer data. The Student References Provided section specifies both Steam Tables and Databook Figure 57 be provided to applicants. The General Discussion states that the CCM uses a 20 degree margin to saturation. Does obtaining the correct answer depend on which reference is used? The General Discussion states that some students may confuse the 7 day action as being retroactive to the first inoperable channel or read the wrong action. However, the question is testing a time requirement vs. selecting an action. The reference provided contains completion times of "immediately, 30 days, 7 days, 6 hours, and 12 hours." Both of the answer options for this part of the question contain completion times derived from 7 days, so even if an applicant applies the wrong action, he or she would be able to select a time different from 7 days. The only possibility for choosing an incorrect answer is that the applicant applies the 7 day action to the first failure. This is not plausible for a trained SRO, especially since a reference is provided. The question could be written to test at the SRO level, without a reference, (specifically, test how many ICCM channels must be inoperable before TS 3.3.3 entry is required) by asking the earliest time LCO 3.3.3 is NOT met (1211(12/1 or 1215). 12/5). Although this is testing an entry condition, it is SRO- only because the information necessary to determine whether the LCO is met or not (number of required channels) is found 'below the line' in Table 3.3.3-1. h2/18/2009 2/18/2009 54 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues TIF Credo Partial Job- Minutia #1 Back- Q= SRO BIMIN B/MIN U/E/S Explanation Focus Dist. Link KIA Only units ward KJA 76 The second part of the question does not match the answer options cont provided. The question asks if NC pumps should have been tripped, which would elicit a simple Yes/No YeslNo response. The answer options consist of Yes/No YeslNo and associated reasons. The question does not ask for the reason and these reasons provide cues for answering the question. Delete the reasons. See suggested wording below. Editorial The stem of the question contains details not required to answer the question. For example, who performs actions specified in the stem is not relevant to answering the question. See App. B, Att. 1, item 3. Consider the following: Given the following plant conditions:
- On 12101/09 ...
- On 12105109 12105/09 ....
- The crew has manually tripped, safety injected, and entered 1-E-0 (title) due to a rapid decrease in RCS pressure.
- OAC parameters are (list or provide picture) 3.Which one of the following correctly states whether or not NC 3.which pumps must be tripped per 1-E-0, 1-E-O, Enclosure 1 (Foldout Page) 4.Tech spec question
- 1. NC pumps must be tripped - or - NC pumps are NOT required to be tripped Other There is no justification for the correct answer, incorrect distractors, or a distractor analysis (not required, but facilitates the review process - see ES-401, D.2.g). Additionally, it does not appear that this question was approved by operations training or operations before being sent to the NRC for review.
FJE 9/11/09 12118/2009 12/18/2009 55 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/MIN U/EIS Explanation Focus Dist. Link units ward KIA Only 76 The facility rewrote the question. cont Enhancement - please tie the question to a procedure (or group of procedures) in order to preclude the possibility of multiple correct answers. See App. B, pg. 4. For example: 5.Which NC overpressure transient establishes the required relief capacity of the pressurizer safety valves per Technical Specification Bases? Because the picture of the OAC is gray shaded, please ensure that applicants are provided with good quality copies - otherwise the numbers are hard to read. The question is otherwise satisfactory. FJE 11/30109 The facility revised the question to address the first comment above. The revised question is satisfactory. FJE 12116109 12118/2009 56 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. !
Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #/ Crect #1 Back- Q= SRO B/MIN U/EtS U/E/S Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only 77 ~ 14 2 X X X X N U Q11EA2.Q2 Q11E/\2.Q2 The question is unsatisfactory because it does not meet the KIA and is not written at the SRO level. The KIA statement is "...consequences to RHR of not resetting safety injection." The answer options are either consequences to the core (loss of injection) or containment (flooding), not the RHRlLPSI system. The question does not meet the KIA (peer check - MB). The question does not test any of the seven items listed in 10CFR55.43(b) and is not written at the SRO level. Knowledge of actions to secure the ND pump is systems knowledge expected of ROs. Knowledge of the consequence of not being able to secure an ND pump operating in the injection mode is integrated plant knowledge expected of an RO (peer check - MB). The question asks for the "minimum" actions that must be completed before the 1A ND pump can be secured. If the answer omits any step necessary to secure the pump, e.g. placing the control switch to stop, then there is no correct answer. The knowledge required to answer the second half of this question (If ND pumps can't be secured when they are injecting from the FWST then injection flow will be lost) overlaps with the knowledge required to answer RO question 42 (If NO ND pumps are injecting from the FWST and cannot be aligned to the sumps then they must be secured). The second half of the question uses subjective language subject to interpretation - "potential consequence" and "timely manner." Consequence in what context or per what procedure? With a large break LOCA, the containment receives a large volume of water. This could be construed as flooding, even though the level of water in containment may not (will not) meet the level of flooding
"flooding" as defined by the Functional Recovery procedures for the containment. The General Discussion states that containment flooding is not an issue, but does not state why containment flooding is incorrect.
The General Discussion provides a description of system operation, but does not explicitly state which answer option is the correct answer. FJE 9/10/09 9/10109 12118/2009 12/18/2009 57 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD LOO (F/H) Back- Q= SAOSRO B/MiN 8/MiN U/EtS u/EtS Explanation (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ 8ack-Focus Dist. Oist. Link units ward KIA Only 77 F 3 N S The facility was unable to write a satisfactory question to previous cont KIA. Replaced Aeplaced KIA with 022AA2.02. The new question is satisfactory. FJE 11/30/09 78 H 2 X N Y 038G2.4.18 S The question is unsatisfactory due to two implausible distractors. (P/C MB) The first half of A and B are not plausible because 1) for the plant conditions given (LOOP) the condenser is not available and the cooldown must be performed with SG PORVS, POAVS, even if the MSIVs are open, 2) E-3 contains instruction to attempt to avoid a MSIV isolation during C/O C/D and provisions for plant C/O C/D if MSIVs are shut. FJE 10/26/09 On 11/3/09 the facility visited the RII All office to discuss selected questions from those reviewed as part of the sample of 30 as well as additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The facility rewrote the question to address the above comments and explained why they thought distractors A and B in the original question were plausible. The facility stated that their verbal explanation of plausibility was not included in the distractor analysis. Based on this explanation, the examiner concluded that distractors A and B were plausible. The facility stated that they may resubmit the original question for review (with a more thorough distractor analysis). FJE 11/3/09 I The facility revised the question. The revised question is satisfactory. FJE 11/30/09 12/18/2009 58 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 8ack- B/MIN U/E/S Back- Q= SAO 8/M1N U/EIS Explanation Focus Dist. Link units ward KIA Only 79 H 3 X X N 1J 054G2.1.30 Y. S The question is unsatisfactory for multiple implausible distractors-the answer to the second part of the question is a 'direct lookup.' (PIC M8). MB). The first part of the question appears to be SROSAO only in that the applicant must 1) know which part (Case Ilor or II) of the AP applies to the conditions in the stem, 2) know which evacuation location the procedure specifies for that case. FJE 10/26109 10/26/09 On 10/29/09 the facility exam team contacted the NRC NAC reviewer to understand 'direct lookup' in the context of the EAL classification matrix. The reviewer explained the specific logic and philosophy used to arrive at the conclusion that a portion of the answer is a
'direct lookup' and explained how a question could be constructed to test EAL classification, with the reference provided, without being a
'direct lookup.'
FJE 10/29/09 On 11/3/09 the facility visited the RII All office to discuss selected questions from those reviewed as part of the sample of 30 as well as additional questions reviewed by the examiner. The facility and examiner discussed comments on this question. The facility rewrote the question to address the above comments and will resubmit the question for review. FJE 11/3/09. The revised question is satisfactory. FJE 1211/09 12118/2009 59 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (FIH) (F/H) (1-5) Stem Cues T/FTIF Credo Partial Job- Minutia #1 Back- Q= SRO B/MIN U/EIS Explanation Focus Dist. Link units ward KIA Only 80 +
~ +
~ +
~ X N e OnG2.4.18 077G2.4.18 F 2 S Enhancement to resolve the questions and issues described below.
Applicants should recognize that the RN, VC, YC, and VA systems are shared systems. Ask the first question more directly without copying text directly from the reference material, e.g. 'Which "Which one of the following describes the operability of the VC system at 11:1 O?" See App. B, C.1.g. Why are A.2 and C.2 plausible? Would initiating a blackout cause a DG start and realignment of systems? Can this be eliminated using systems knowledge? If so, the question may not be testing the KIA at the SRO level. (PIC MB) What is the basis for the higher cognitive level? This question appears to require 1) recall of whether or not shared systems are operable when RTCA indicates voltage would be inadequate on a unit trip, 2) recall of. the purpose of installing jumpers. ofthe FJE 10/27/09 10/27109 Facility revised the question. Enhancement to eliminate possible ambiguous language. Is "double sequencing" terminology that will be understood the same way by all applicants? Is this same terminology used in operating procedures or training material? Editorial: Tense is not consistent. 1100 and 1300 use past tenst. 1102 uses present tense, even though the actions occurred in the past. Operable and inoperable look and sound similar. Consider "operable" and "NOT operable." FJE 1211/09 12/1/09 The facility revised the question to address the above comments. The revised question is satisfactory. FJE 12116109 12/16109 12118/2009 60 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #1 Back- Q= BIMIN U/E/S SRO B/MIN UlEtS Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only 81 H ~ ~ ~ N e Ii WE05EA2.1 2 S Enhancement to resolve the items below. Please provide the Unit 2 procedures that show plausibility of A.2 and 0.2. Development references list TS 3.8.1 and basis. What part of these references are relevant to the answer or distractors? Is the UST level given in the initial conditions necessary to answer the question? FJE 10/26109 10/26/09 The facility addressed the above comments. Editorial- Consider adding "Based on the Current Conditions ..." to question 2 in order to focus the applicant on the information under Current Conditions for the second question. The question is satisfactory. FJE 1211/09 The facility made minor editorial changes and the question remains satisfactory. FJE 12116109 12116/09 12118/2009 12/18/2009 61 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
0# Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back- 0= Q= SRO B/MiN U/EiS U/EtS Explanation Focus Dist. Link units ward KIA Only 82 H 2 X X X X N U 005AA2.04 S The question is unsatisfactory due to not being 'SRO-only' and may have two correct answers. Enhancement for KIA match. KIA 005 is "Inoperable/Stuck Control Rod." AA2.04 is "Interpretation of computer in-core TC map for dropped rod location." Per 10LB IOLB (DM (OM 13:509/10/09), KlA005AA2.04 is incorrect and should read "Interpretation of computer in-core TC map for inoperable/stuck control rod." Per DM, OM, suspect 'cut and paste' carried forward from 003AA2.05. Licensee should rewrite question for a stuck control rod per APE005. The question is not written to the SRO licensed level (peer check check- - MB). Determining the location of a dropped rod based on incore thermocouple temperatures is RO knowledge. Knowledge that a dropped rod does not affect shutdown margin, but does affect core peaking, is GFE knowledge. Neither technical specifications nor the bases are required to determine that a dropped rod does not impact SDM. The question does not ask the SRO applicant to use technical specifications to determine actions or completion time limits or use the basis to determine technical specification applicability or equipment operability. The second half of the question asks what concern is being addressed by LCO 3.1.4. Although, per the General Discussion, shutdown margin does not change for a dropped rod, Condition B of 3.1.4 would still require action to verify or restore SDM and these actions could be applicable for the plant conditions given (if the rod were not or could not be restored within 1 hour). Additionally, the reviewer could not find an explicit statement in the Bases for 3.1.4 that said that SDM was NOT a concern for a dropped rod (as opposed to a misaligned or stuck rod). The question appears to have two correct answers. FJE 9/10/09 The facility revised the question. The revised question is satisfactory. FJE 1211/09 12118/2009 62 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #1 Back- Q= SRO BIMIN U/EIS B/MIN U/E/S Explanation Stem Cues TIFT/F Focus Dist. Link units ward KIA Only 83 H 2 X X !:l-
!:f N Ie E 032AG2.4.4 S
Enhancement to resolve the items discussed below. The question may not be written at the SRO level. The first half of the question requires diagnosing plant conditions to arrive at AOP entry - RO knowledge. Although the answer for the second half of the question can be found in 3.9.2 bases, it is also in OP/11A161001002, OP/11A161 001002, Enclosure 4.3, Unit Shutdown from Mode 5 to Mode 6 ... Additionally, KIA 2.1.44 requires ROs to know RO duties in the control room during fuel handling, such as supporting instrumentation. Please provide evidence that only SROs perform the associated bulleted steps in Enclosure 4.3? (PIC MB) Tie the second part of the procedure to technical specifications in order to preclude a second correct answer. FJE 10/27109 The facility satisfactorily addressed the above comments by providing additional explanation in the distractor analysis and comments. The question is satisfactory. FJE 1211/09 I 12118/2009 63 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. B.
8. Q# LOK LOD U/EtS Q= SRO B/MIN U/E/S Explanation (F/H) (1-5) Stem Cues TIF Credo Partial Job- Minutia #1 Back-Focus Dist. Link units ward KIA Only B4 84 M t4 3 X !1-
!?- N N. e 033G2.1.30 F S Please provide evidence (e.g. a learning objective) that SROs are required to know greater than one hour tech spec action statements and completion times from memory. Recent IOLB 10LB guidance is that applicants, including SROs, are not required to know greater than one hour tech spec completion times from memory. For this question, the applicant must recognize that the 30 hours to repair N-35 exceeds the 24 hours allowed. Additionally, the applicant needs to recognize that the time taken to increase power to greater than P-10 is within the completion time limit of the technical specification.
Note that if a reference is provided for this exact question, it would be considered 'direct lookup." (PIC PC) 10/27109 FJE 10/27/09 Enhancement: The second half of the question, regardless of which SRO at the facility performs the function, is 'Is N-35 operable or inoperable?' Asking the applicant if they concur or not just asks the question indirectly and has the potential to cause confusion if an applicant thinks that they are being asked to assess whether all other functions will operate as designed after the SUR circuitry is disabled. Revise the question to be more direct, e.g.:
- Engineering has evaluated the repair and determined that all other functions will operate as designed.
- 2. For the conditions given above, is N-35 operable or inoperable?
The tense of the answer options does not match the tense of the first part of the question. The first part of the question is written in past tense (" ... ~ removed?") The answer is written in present tense (no fused are removed). Make the tense consistent so that the answer follows from the question without any grammatical errors. Editorial - Initial conditions are in the past and should use past tense. Editorial-FJE 1211/09. The facility revised the question to address the above comments. 12116109 The revised question is satisfactory. FJE 12116/09 12118/2009 12/18/2009 64 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Credo Partial Job- Minutia #/ Back-Creel. Q= SRO B/MiN B/MIN U/E/S Explanation Stem Cues T/F Focus Dist. Link units ward KIA Only 85 F ~ X N e 067AA2.02 3 S Enhancement for distractor plausibility. It does not seem plausible that automatic actions occurring as designed would cause a system to become inoperable (distractors B.2 and 0.2). Does this situation occur on the VC system also? Consider emergency plan classification or reportability for SRO portion? FJE 1211/09 The facility revised the question to address the above comment. The revised question is satisfactory. FJE 12116/09 86 F 2 ~
!?- M e 003A2.02 S
Examiner Note: The question was modified such that one of the three distractors in the original question is now the correct answer in this question. Enhancement: The crew might eventually be directed to enter the controlling procedure for unit shutdown after the pump is tripped. Step 13 of AP 008 directs entry into AP 004 WHEN the NCP is tripped. Concern is the potential for two correct answers. Change "after" in the question to "when." FJE 1211/09 The facility made minor editorial changes and addressed the above comment. The question is satisfactory. FJE 12116109 12116/09 12118/2009 12/18/2009 65 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 8ack-Back- Q= SRO B/MIN U/E/S U/EIS Explanation Focus Dist. Link units ward KIA Only I 87 H 3 !1-
~ ~
!1- N eE 008G2.4.9 S
Enhancement for stem focus. Concern is for single complete correct answer. Given the conditions in the stem (KC has not been restored) will the crewe able to maintain NC pump seal injection? The stem does not state the scope of the NC problem and the answer is, in part, to realign KC to the 1A NV pump. Concern is that the answer (procedure direction) may not result in maintaining seal injection. It is not clear from the current conditions and second half of the question that the author is asking for the procedure transition from AP021 step 8 vs. direct entry into one of the procedures. Does the crew enter AP 005 directly because of the LOCA (RO knowledge) or due to the loss of KC? FJE 1211/09 The facility provided additional explaination and also revised he question to address the above comments. The revised question is satisfactory. FJE 12116/09 88 H 2 M S 039A2.02 Examiner Note: The question was modified such that one of the three distractors in the original question is now the correct answer in this question. FJE 1211/09 89 H 3 ~
!1- N g g? 063G2.2.40 S
Enhancement to consider the order in which the two part question is asked. The logical order of the two questions seems reversed.
- 1) The surveillance was not done, when do we need to do it?
- 2) The surveillance was then done and a bad cell was found.
When do we cascade? Also, question 2 is phrased as past tense (was the latest time), but the options are in the future (after 12115 @ 1800) FJE 1211/09 The facility revised the question to address the above comment. The revised question is satisfactory. FJE 12116/09 12/18/2009 66 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
B. Q# LOK LOD (FIH) (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back- Q= SRO B/MIN U/E/S Explanation Focus Dist. Link units ward KIA Only 90 H 3 N S 076A2.02 No comments. FJE 1211/09 91 F 3 X X N Ii
~ 016A2.02 S
Enhancement - remove the underlining of required. It appears to cue the applicant to the answer. Editorial- consider moving the first sentence (Unit 1 is performing ... ) to the first bullet, since it is a given condition. FJE 1211/09 The facility revised the question to address the above comments. The revised question is satisfactory. FJE 12116109 12116/09 I 92 F 2 +
~ +
~ N Ii
~ 029G2.1.28 029G2.1.2B S
Enhancement: I do not understand why if per the TS Basis, The "The heaters do not affect OPERABILITY of the Containment Purge Exhaust System filter trains" the TS contains a condition requiring the heaters (part of the CPESNP system) to be operable and a related I surveillance requirement. Please explain and possibly reword the question. Concern is no or multiple correct answers. TS 3.9.3 uses CPES as an acronym for the system vs. VP in the question. Are they equivalent? i I FJE 1211/09 The facility explained the first item above and made minor revisions to the question. The revised question is satisfactory. FJE 12116109 12116/09 12118/2009 12/18/2009 67 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. B.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1#/ Back- Q= SRO B/MIN UlEtS U/E/S Explanation Focus Dist. Link units ward KIA Only 93 F~ 2 X X ~ N e 033A2.03 S Enhancement for question focus. The term "mitigative actions" is vague. Ask the question more directly. directly, e.g. What procedure will direct movement of the fuel in the manipulator crane to a safe location? Where must the fuel assembly in the manipulator crane be placed? Seems like a lower COG level If the applicant recalls that AP041 contains actions for the SFP side and AP026 for the canal side. Editorial: Is the RP tech in the SFP (!) or at the SFP? FJE 1211/09 The facility revised the question to address the above comments. The revised question is satisfactory. FJE 12116/09 94 F 2 N S G2.1.26 FJE 1211/09 12/18/2009 68 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOO (F/H) (1-5) Credo Partial Job- Minutia #1#/ Back- B/MiN U/EtS Q= SRO BIMIN U/EiS Explanation Stem Cues T/F Focus Oist. Link units ward KIA Only 95 H 3 N e G2.2.41 S Enhancement to resolve the issues listed below: The answer analysis discusses closing 2 NO-65B for a Unit 1 question. Is this a typo? If the closest valves in the piping are used to isolate the leak, what valve(s) must be closed? Please include this in the general discussion. Editorial: Consider if the question would be clearer if worded differently, e.g:
- 1. How many trains of NO are inoperable?
- 2. When shutting down, what is the next operational mode in which TS 3.5.2 will no longer apply?
A. 1. Both trains of NO are inoperable
- 2. Mode4 B. 1. Only 1B 1B Train of NO is inoperable
- 2. Mode5 Mode 5 FJE 1211/09 The facility revised the question to address the above comments.
The revised question is satisfactory. FJE 12116109 96 F 2 X X Ii 8 Y tJ G2.2.44 N N Unsatisfactory for two correct answers. B.1 (Modes 1, 2, 3) is a subset of 0.1 (Modes 1, 2, 3, 4). Since 0.1 is correct, so is B.1. Add the word "only" to A.1 and B.1. Examiner Note: 2005 NRC Exam. FJE 1211/09 The facility rewrote the question (now a new question). The question is satisfactory. FJE 12116/09 12/18/2009 69 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. B.
8. Q# LOK LOD (1-5) U/EtS SRO B/MIN U/EIS Explanation (F/H) Stem Cues T/F Credo Partial Job- Minutia #1#/ Back- Q= Focus Dist. Link units ward KIA Only 97 F 2 M S G2.3.4 Examiner Note: The question was modified such that one of the three distractors in the original question is now the correct answer in this question. FJE 1211/09 9B 98 F 2 !I-
!f. B eE G2.3.7 S I Enhancement to resolve the following:
Is NOD 507 the only procedure that governs dose limits and approvals of extensions? If not, tie the question to a procedure in order to prevent the possibility of multiple correct answers. For example, Per NSD 507, (title)
- 2. What is ......
FJE 1211/09 The facility revised the question to address the above comment. The revised question is satisfactory. FJE 12116/09 12/18/2009 70 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
0# Q# LOK LOD LaD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ 8ack- 0= Creel. Q= SRO 8/MiN U/E/S U/EiS Explanation Focus Dist. Link units ward KIA Only 99 H 3 X X N E G2.4.20 S Enhancement for stem focus The first part of the question asks for the current status of the 18 NI pump, but does not provide a context for the question. Tie the first part of the question to a specific procedure in order to provide context and eliminate alternative correct answers. See App. 8, S, C.1.c. The second part of the question is not specific as it merely asks which procedure will provide actions to address the problem and does not bound or define "the problem." The stem contains at least two potential problems: 1) 1NI-1368 will not open, and 2) all ECCS pumps cavitating. Tie the second part of the question to a specific procedure in order to provide context and elim inate alternative eliminate correct answers. See App. 8, C.1.c. The stem states that "Unit 1 i§ at 25% and increasing following a refueling outage" and then states that "1 hour ago, Unit 1 experienced a large break LOCA." These statements are contradictory. FJE 9/14109 The facility revised the question to address the above items. Enhancement for stem focus - first part of question. I am still confused by the first question. Why is it necessary to preface the first question with "based on the status of 1NI-1368"? Is "Per ES-1.3, what is the status of the 18 1S NI pump?" an equivalent question? Another possibility is 'What action, if any, was required per ES-1.3 when 1NI-1368 would not open?" FJE 1211/09 The facility revised the question to address the above comment. The revised question is satisfactory. FJE 12116109 12116/09 12118/2009 12/18/2009 71 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD Q= SRO B/MiN U/EtS U/EiS Explanation (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back-Focus Dist. Link units ward KIA Only 100 H 2 X X X X X X N e G2.4.50 S Enhancement for multiple psychometric flaws. Stem Focus Tie the first part of the question to a specific procedure in order to eliminate alternative correct answers. See App. B, C.1.c. None of the reference material supplied indicates why it is incorrect to trip the reactor for the plant conditions given. The General Discussion states that the reactor "shouldn't be tripped" but not that tripping the reactor is 'wrong' or 'incorrect:
'incorrect.'
The General Discussion states that "it can be assumed that level in one S/G may have gotten below 37% but not 2." Explicitly state assumptions necessary to answer the question in the stem of the question. See App. B, Att. 1, item 5. Concern is multiple or no correct answers. Cues The first part of the question does not match the answer options provided. The question asks if the the reactor should have been tripped, which would elicit a simple Yes/No response. The answer options consist of Yes/No and associated reasons. The question does not ask for the reason for tripping the reactor and these reasons provide cues for answering the question. If the applicant does not know if the decision is correct, he or she only needs to determine whether or not the stem provides indications that a transient was in progress before the reactor was tripped. The 25 page Fission Product Barrier Matrix supplied to applicants as a reference has the potential to provide cues to other questions. No specific concerns have been identified as this review only covers 14 sample questions. Editorial The stem of the question contains details not required to answer the question. For example, who performs actions specified in the stem is not relevant to answering the question. See App. B, Att. 1, item 3. FJE 9/14/09 12/18/2009 72 of 73
Catawba Nuclear Station 2009-301
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOK LOD U/EIS SRO B/MIN U/E/S Explanation (F/H) (1-5) Stem Cues T/FTIF Credo Partial Job- Minutia #1 Back- Q= Focus Dist. Link units ward KIA Only 100 The facility revised the question. cont Editorial: The format fonnat of the stem is potentially confusing. Consider reformatting for clarity, e.g.: Given the following two events:
- 1. Earthquake
- Seismic instrumentation ...
- 1 AD-4, B/8 ...
1AD-4,
- 2. High Auxiliary Building Radiation
- 1 RAD-1, B/3 ...
- EMF41 ...
Which one of the above events requires .... FJE 1211/09 The facility revised the question to address the above comment. The revised question is satisfactory. FJE 12116/09 12118/2009 12/18/2009 73 of 73
( ES-403 ExaminatiOOi Grading ,., Written Examination c"'~ ., ES-403-1 Form ES-403*1 (
, Quality Checklist Checklist .
Facility: Facili c... A "\ A VU '8:. A. Date of Exam: rz. /z:z../ip&k~~
\-z./z:z./tpC; Exarn Level:Ror;>(
Levei:R' SRO!><r Initials Item Descri"tiori' Item Description a b c
- 1. Clean answer sheets copied before;grading Atr~ tJlA- ~~
I
- 2. c Answer key key' changes and question deletions justifi~d Justified and documented and'documented; M.1~ -G'£
/
- 3. Applicants' scores checked for addition errors (reviewers revi,ewers sotspot check> 25% of examinations) examinations. r.tra. ~
- 4. Grading for all borderline cases (80 +/-2% overall and 70 or 80, II as applicable, SRO-onry) reviewed a Iicable, +/-4% on the SRO-onf reviewed in detail Mla V~
/
- 5. All other failing examinations checked to to ensure thatthat grades*
grades. are justified
'ustified M'lR. ~A
- 6. Performance Perjor:mance PO on missed questions checked checkedf9.t for ftainjng training /
(
. deftciehcies deficiencies.and problems; evaluate validity .......
ahd wording problems;.evaluatevaUditY
'Jic,ilrits 1\tJ. (t ¥ ?£ of questions missed b ot'uestionsmissed by halt half or more of the applicants of. thea; (
Name/Signature" . Printed Name/Signature* Date
- a. Grader ~t¥. J.Jw~.t.s/n1JJ.~
l\t¥- J. ~\~~S brJJ.J.,tL.... Ql*Otp gl-:O(Q""O... /O
- b. Facility Reviewer(*) AJ LA AliA ,NIA NIA
- c. NRC Chief Examiner (*) r-hX-\oA~-:S-. E\.AII--~c;L~~/<LtI4~- ~L~/\IO Supervisori'~*)
- d. NRC Supervisof'(*) ~-r. WJDJJ,(NJJ//(ff;m~ '1. ,y4Jo
/ Q
(*) reviewer's signature is not applicable for examinations graded by the NRC; The facility reviewers two independent inde endent NRC reviews are required. re uired. ( ( ES-403~ Page 6 of 6 ES-403,}}