ML093640778

Initial Retake Exam 2009-302 Final Administrative Documents

ML093640778
Person / Time
Site:	McGuire, Mcguire
Issue date:	12/08/2009
From:	Division of Nuclear Materials Safety II
To:	Duke Energy Carolinas
References
50-369/09-302, 50-370/09-302
Download: ML093640778 (29)

Text

FACILITY NAME: McGuire Section 7 REPORT NUMBER: 05000369,370/2009302 FINAL ADMINISTRATIVE DOCUMENTS CONTENTS:

(X) Exam Preparation Checklist (ES-201-1)

(X) Final Exam Outline Quality Checklist (ES-201-2)

(X) Exam Security Agreements (ES-201-3)

(NA) Final Administrative Topics Outline (ES-301-1)

(NA) Final Control Room & Walk-Through Test Outline(ES-301-2)

(NA) Final Operating Test Quality Check Sheet (ES-301-3)

(NA) Final Simulator Scenario Quality Check Sheet (ES-301-4)

(NA) Final Transient and Event Check Sheet (ES-301-5)

(NA) Final Competencies Check Sheet (ES-301-6)

(X) Final Written Exam Sample Plan (ES-401-1/2)'f (X) Final Record of Rejected KAs (ES-401-4)

(X) Final Written Exam Quality Check Sheet (ES-401-6)

(X) Final Written Exam Review Work Sheet (ES-401-9)

(X) Written Exam Grading Quality Check Sheet (ES-403-1)

(with RO computer answer keys)

Location of Electronic Files:

Primary Folder:

All applicable documents are attached Submitted By: _ _ _ ~----,,--.~

~----'_.~

_ _ _. _ _ VerifiedBY~

V~ifiedBY~

ES-201, Rev. 9E Examination Preparation Checklist Fonn ES-201-1 OPNA Facility: McGuire Date of Examination: WR 11119109 Facility NRC Examinations Developed by:

Written Written Exam Outline only Chief Target Task Description (Reference) Examiner's Date

• Initials

-180 1. Examination administration date confirmed (C.1.a; C.2.a and b) rfa

-120 2. NRC examiners and facility contact assigned (C.l.d; C.2.e) rfa

-120 3. Facility contact briefed on security and other requirements (C.2.c) rfa

-120 4. Corporate notification letter sent (C.2.d) rfa rfa

[-90] 1(S.

S. Reference material due (C.1.e; C.3.c; Attachment 2)]

{-7S} 6. Integrated examination outline(s) due, including Forms ES-201-2, ES-201-3, ES-301-1, ES-301-2, ES-301-S, ES-D-l's, ES-40l-1I2, ES-401-1I2, ES-40l-3, ES-401-3, and ES-401-4, as rfa applicable (C. I.e and f; C.3.d)

{-70} {7. Examination outline(s) reviewed by NRC and feedback provided to facility rfa licensee (C.2.h; C.3.e)}

{-4S} 8. Proposed examinations (including written, walk-through JPMs, and scenarios, as applicable), supporting documentation (including Forms ES-301-3, ES-301-4, rfa ES-30l-S, ES-301-S, ES-301-6, and ES-401-6), and reference materials due (C.1.e, f, g and h; C.3.d)

-30 9. Preliminary license applications (NRC Form 398's) due (C. I.!; C.2.g; ES-202) rfa

-14 10. Final license applications due and Form ES-20 1-4 prepared (C. I.!; C.2.i; ES-202) rfa

-14 11. Examination approved by NRC supervisor for facility licensee review rfa (C.2.h; C.3.f)

-14 (C.1.j; C.2.f and h; C.3.g)

12. Examinations reviewed with facility licensee (C.l.j; rfa

-7 13. Written examinations 8ft8 tu18 8fJ@fstmg 8f}@Fsting t88t8 approved by NRC supervisor rfa (C.2.i; C.3.h)

-7 14. Final applications reviewed; 1 or 2 (if> 10) applications audited to confirm qualifications / eligibility; and examination approval and waiver letters sent rfa (C.2.i; Attachment 4; ES-202, C.2.e; ES-204)

-7 IS. Proctoring/written exam administration guidelines reviewed with facility licensee rfa (C.3.k)

-7 16. Approved scenarios, job performance measures, and questions distributed to NA NRC examiners (C.3.i)

• Target dates are generally based on facility-prepared examinations and are keyed to the examination date identified in the corporate notification letter. They are for planning purposes and may be adjusted on a case-by-case basis in coordination with the facility licensee.

[Applies

[AJ>Plies only] {Does not apply} to examinations 2repared prepared bythe the NRC.

JC/NIJL ES-201 Examination Outline Quality Checklist Form ES-201-2 Facility: McGuire Nuclear Station Date of Examination: 11119/2009 Initials Item Task Description a b* c#

1.

W

a. Verify that the outline(s) fit(s) the appropriate model, in accordance with ES-401.

l~

.~ 9JII ~

R b. Assess whether the outline was systematically and randomly prepared in accordance with

~

T T

E N

I d

Section D.1 of ES-401 and whether all KiA categories are appropriately sampled.

c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics.

Assess whether the justifications for deselected or rejected KiA statements are appropriate.

~

~

iJ!1

.V1 0"\

~

\.

2. a. Using Form ES-301-5, verify that the proposed scenario sets cover the required number S

of normal evolutions, instrument and component failures, technical specifications, and major transients.

JfII< ~A <P-I M b. Assess whether there are enough scenario sets (and spares) to test the projected number U and mix of applicants in accordance with the expected crew composition and rotation schedule L without compromising exam integrity, and ensure that each applicant can be tested using at least one new or significantly modified scenario, that no scenarios are duplicated

't/{<< N{t\

A I'-

T from the applicants' audit test(s), and that scenarios will not be repeated on subsequent days.

0 (f~

c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative R

and quantitative criteria specified on Form ES-301-4 and described in Appendix D. tf,f< wi/"

3. a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2: NitJ (1 ) the outline(s) contain(s) the required number of control room and in-plant tasks W distributed among the safety functions as specified on the form 1 (2) task repetition from the last two NRC examinations is within the limits specified on the form T (3) no tasks are duplicated from the applicants' audit testis)

(4) the number of new or modified tasks meets or exceeds the minimums specified on the form (5) the number of alternate path, low-power, emergency, and RCA tasks meet the criteria IJR'

~ tJ/r on the form.

b. Verify that the administrative outline meets the criteria specified on Form ES-301-1:

(1 ) the tasks are distributed among the topics as specified on the form (1)

(2) at least one task is new or significantly modified 1ft J-I~

(3) no more than one task is repeated from the last two NRC licensing examinations

c. Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days. 1ft ~

4. a. Assess whether plant-specific priorities (including PRA and IPE insights) are covered in the appropriate exam sections. 161.. 5Ur\ ~

G E b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate. 1f4( )J.JI\ ~

N E

c. Ensure that KiA importance ratings (except for plant-specific priorities) are at least 2.5. ~ .Sl.M ~

1tlJI, I~ ~~,

R d. Check for duplication and overlap among exam sections.

A L e. Check the entire exam for balance of coverage. t!K. <liA ~fl;

f. Assess whether the exam fits the appropriate job level (RO or SRO). ~ ~ '\

Fred 8KJI'kJr /~m~r~

Date

a. Author 1/-13-09

b. Facility Reviewer (*) .S-k1L~n

~~n~ -.L Mcdelkf l~

Mcdell<<-l~~-;l IhJ-D~

IhJ-dJ

c. NRCChiefExaminer(#)

NRC Chief Examiner (#) ~

~A/~/ A/~/ ~ ///$'or

///$'Or NRC Supervisor A1AI//v 1L lL i.lJJlfJ//AJI.,A(

i.lJJlfWAJI.,A( /{l\ ~ It/truj I t/L1/D.j.

d

// t7JVn,~

( /

,"-, flU(

• II-N-o-t-e-:-----------#--,-nd-e-p-e-n-d-e-nt-N--R-C-r-e-v-ie-w-e-r-in-it-ia-'-it-e-m-s-i-n-c-o-'u-m-n~"c'"~;-Ch-i-ef-e-x~a-m-i-ne-r-c-o-n-c-u-rr-e-n-ce--re-q-U-ir-e-d-.---------41 Note:

n Independent NRC reviewer initial items in Column "c chief examiner concurrence required.

• Not applicable for NRC-prepared examination outlines

ES-201 Examination Security Agreement Form ES*201-3

1. Pre-Examination ackno'oAo1edge that 1 I acknowledge I have acquired specialized knoYJIedge knowledge about the NRC licensing examinations scheduled for the weekes)week(s) of )\ \/'01.,!cfl 1 ('/r:8 of my signature. II agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the as of the date NRC chief examiner. I understand that I am not to instruct, evaluate, evaluate, or provide performance perfonnance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training trainIng content or provide direct or indirect feedback). Furthermore, I am aware of the physical security feedback). securtty measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may resuHresu" in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that thai examination securtty security may have been compromised.

2,

2. Post-Examination kno~Al/Jqid not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered To the best of my knO~ifl:a;qid during the week(s) of \ . From the date that I entered into this security agreement until the completion of examination administration, administration, I did not evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted instruct, evaluate, below and authorized by the NRC.

PRINTED NAME JOB TITLE I RESPONSIBILITY SIGNATURE (1 DATE SIGNATURE (2) DATE NOTE ES-201 , Page 27 ooff 28 ES-201,

ES-201 Examination Security Agreement Form ES-201-3

1. Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of fjA ('/or as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licenSing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2. Post-Examination To the best of my knoM~did , did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of ii7lr:b1!f. . From the date that I entered into this security agreement until the completion of examination administration, I did not p~rformance instruct, evaluate, or previ e performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME JOB TITLE I RESPONSIBILITY " A SIGNATURE (1) DATE SIGNATURE (2) DATE NOTE

!iii~ f/[sf!irre /!f~, ~ l~~

1.~L \)~ lLc IOh ~\tx

5. _______________________________________________________________

~Yl~

~

~.

p~ fIE ----

6. ______________________________________________________________

7. _

8.

8. ___________________________________________________

9. __________________________________________________

10.______________________________________________________________

11._________________________________________________________

12._

13.___________________________________________________________

14.________________________________________

15.___

15._- _- _- _-_ -_ -_ -_ -_ __ __ ___ __ __ __ __ _ ________ ___ ____ __ __ _____ __ __ __

NOTES:

ES-201, Page 27 of 28

FINAL ES-401. Rev. 9 Record of Rejected KJAs Form ES-401-4 Tier / Randomly Reason for Rejection Group Selected KA 1/1 APE022 AK1.04 (Q42) 9/15/09: It is not possible to write a discriminating question for this KIA. This KIA is being replaced by APE 027 AK3.04 which was randomly selected by Lead Examiner.

1/1 APE027 AK3.02 Q(44) 9/15/09: Replacement KIA requested because it was impossible to construct a discriminating question with the origional KIA. APE 027 AK3.04 was randomly selected and provided by the lead examiner.

1/1 APE062 AA1.03 Q(53) 9/15/09: It is not possible to write a question that matches this KA. Replacing with APE 062 AA1.05 which was randomly selected by Lead Examiner.

1/1 WE05 EK2.1 Q(55) Unable to write question tor RO level. Chief Examiner randomly selected new K/A WE05 EK2.2 1/1 WEll 2.4.8 Q(56) Unable to write RO level question. New KIA (WEll 2.1.27) selected by Lead Examiner.

f!'IN4L ES-401 Written Examination Quality Checklist Form ES-401-6 Facility: McGuire Nuclear Station Date of Exam: 11/19/2009 Exam Level: RO ~ SRO Initial Item Description a b*! c*

j

, I

1. Questions and answers are technically accurate and applicable to the facility. :f!KI~;'-\

, I,

2. a. NRC KlAs are referenced for all questions.

b. Facility learning objectives are referenced as available. l~i9P~

,---- - -1 ------1

3. -

SRO .questions are appropriate in accordance---with

--~ --- - - - - - - - -- - - - - - - . - - - - - - , , - - - . - - - - --- --~----- -

Section D.2.d of ES-401 1tf/4-,j--1 I- ,Jlffi "- --- ,

. I ',1

4. The sampling process was random and systematic (If more than 4 RO or 2 SRO questions were repeated from the last 2 NRC licensing exams, consult the NRR OL program office). 11fA< 19J'\! \) :

-~ ----t---I- I i ' , I I 5. Question duplication from the license screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate:

_ the audit exam was systematically and randomly developed; or the audit exam was completed before the license exam was started; or iZ'the examinations were developed independently; or

_ the licensee certifies that there is no duplication; or

_ other (explain) 1'\

I i

d/);/ , Ij..I\. I

71.:7\ I  :

- - r -----

6. Bank use meets limits (no more than 75 percent from the bank, at least 10 percent new, and the rest

~ ___Bankj_~_()dified_

new or modified); enter the actual RO / SRO-only question distribution(s) at right. I 36 / - I 3 / -

1_ L

7. Between 50 and 60 percent of the questions on the RO Memory CIA exam are written at the comprehension/ analysis level; 1- I j

~:gjI\ ~

the SRO exam may exceed 60 percent if the randomly selected KlAs support the higher cognitive levels; enter 31 / 44 /

the actual RO / SRO question distribution(s) at right. I i

8. References/handouts provided do not give away answers or aid in the elimination of distractors.

9. Question content conforms with specific KIA statements in the previously approved examination outline and is appropriate for the tier to which they are assigned; ~ i /, lfI ~

deviations are justified.IP.~  ;':J-F' I

10. - ~:-stl~n psychometric qua~~_~~-f~~n1~: m~et~he~~~d~l~es~~ -~~~~pp:~;~~-~ ~~=~=-------1~,d~~-_t; _1 i 11. The exam contains the required number of one-point, multiple choice items; iii  !

the total is correct and agrees with the value on the cover sheet.

!~ISU'" ~

~ _ _ _J _ ____ _

Date i a. Author i b. Facility Reviewer (*)

! c. NRC Chief Examiner (#)

I d. NRC Regional Supervisor Note:

• The facility reviewer'S initials/signature are not applicable for NRC-developed examinations.

1. Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.

ES-401 Written Examination Form ES-401-9 Review Worksheet McGuire 2009-302 1.

1.

lOK I 2.

2. 3. Psychometric Flaws LOK lODr---,---.--.---.-

4. Job Content Flaws 5. Other 6. 7.

Q#I LOD I

(CIA) (1-5) Stem Cues T/F Credo Partial Job- Minutia I #1 Back- Q=

Q= SRO SRO U/E/S Explanation Focus Dist. link Link units I ward KIA KJA Only Instructions nstructions Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

[Refer

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

• The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• The stem or dlstracters contain cues (i.e., clues, clues. specific determiners, phrasing, length, etc).

• The answer choices are a collection of unrelated true/false statements.

One or more distracters is not credible.

One or more distracters is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

• The question Is not linked to the job requirements (i.e., the question has a valid KIA but, as written, is not operational in content).

• The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e.,(i.e .* it is not required to be known from memory).

• The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).

16.

6. Based on the reviewer's judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

7. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

RO/SRO Combined Question Generic:

CAF = Check at Facility DV = Discriminatory Value NP = Non plausible On the fill in the blank Os: State "Which one of the following completes the statement below" One of many examples is 0 8. Many questions marked green require this statement.

003A1.02

!UUjAl.U~

M 2 x y N E 1Stem: Underline and/or cap the word "minimum" RFA 10/27/09 Ii'. 'I' 003A301 2

M 2 x y N E Swap B with C so values are increasing iSwap

nI 1.

1.

2.

2.

3. Flaws 4. Job Content Flaws 5. Other 6. 7.

)

Q#/ILOK 0#1 LOK LOD~--'---~-.---r-I ward I

~

(CIA) (1':.'5) i~~:

(CIA) I (1-5) I.;ues IIr I~t~~ Partial Partial Job- Minutia I #1 Back-a' Q= ~~?

SRO UIEIS utE/S Explanation Link units ward KIA i vuly Only D: Why would,"ow flow 90 go up given 9"eo!this , failure andIInese these conditions?

RFA 10108/09 10/08/09 Accepted 10/27/09 IA",epled 10127109

':;, i:,

N I '" "

" t ', ", ;,... :',

f"',  ;:,

004K5.49 3 y M 2 N No comment at this time lime RFA 10/27/09 10/27/0 9

/\:~. ;J!~" '. ' t' "' ~" ;. ,1

... .. .. , .-. j .

UUbG2.4.50 4

c 3 x N N u The second, part of distracters, A and 8B do notslale not state any

~

~Wii' manual or automatic actions. That That action action is is not much not much different

, ~ then~~

nb~h ~;

,;.;..;;.; answerr aa-state~

the correct stated in in C.

C. Potentially Potentially A and C

,"" f Aare ;nd bothe.' correct W"'" answers. Modify Modify the the second second I!;;S;~s parts of A and 8.

This Q is U because of two potentially correct answers.

IRFA RFA 10/27/09

..'\ ~,.

,'::\.!;. ::-:

'('

006K2.01 5

cC 3 x N N E IPlease; justify why. 2EMXA .,raN\ and 2ELXA are plausible. plausible.

Neither i the distracter I analysis I i nor the reference material supportss their plausibility. i, I!~is This E is U U until justified due to two NP distracters. distraclers.

~~..\110/27/09 IRFA 0/2 7/09 Added feeder to stem of question.

IA,dd"d 6

M ,

1 x N N 1006K2.04 Thiss Q valve I e to a'has has low DV, 10 each choice:

DV. Increase DV by adding a second choice, A. XZ Xl incorrect 8 . XY correct 8.

C. zy z:f incorrect D .., WZ incorrect This Q a is U because of low DV RFA 10/27109 10/27/09

~ ,-;. **' .. : :. ~ 1-1 ' . -:,

.:. ~

007AT01 7

c C 3 x N N E The containment floor and equipmentI sump I sum plausibility 1 is Is I I19t justified. The distracter analvsis says t~e floor is used

Q#I 0#/

n 1.

1.

LOK LOD

2. 3. Psychometric Flaws 4. Job Conlent Content Flaws n

()

5 _ Oth~

5. Other r---

6. 7.

(CIA) (1-5) Stem Cues TIF (CIA) T/F Credo Partial Job- Minutia #1

1. 1 Back- A'"

Q= SRO U/EIS UIEIS Explanation Focus Dist.

Dis!. link Link units ward KIA Only som~t~me~ .out ogesn some:.t!mesbut doesn't say w when en . conS.I.d~r Consider c an9,n9 A2 changln~ IV.

and C2 to the radwaste facility or something or justify justi plausibility plausibility..

a This Q is E until A2 and C2 are either justified or replaced. replaced .

RFA 10f27{09 10/27/09

,;, .. Sc.-;\ ' '.;'/ -' f:; ,,>_

', ' , :: ~ . :..;. '. : .

'" I

"' j ; '

00BA2.03 008A2.03

>' .' ' i, I

B 8

c C 3 xX N N E Change motor bearing temperatures to 190. This will give C1 and D1 C1 D1 more plausibility.

I' ; . '"

RFA 10127/09 010K6.04 10/27/09

,I ;,' , ': ~

I 9

cC 3 x X N N E It's common knowledge knowled~e th that at the Pzr relief valves discharge dischargel to the PRT. If the PRT PR ruptures\ ruptures, it will result in a lower pressure. This also is common Knowledge. knowledge. Therefore,  !

distracter dislracter C is NP. Change C to increase, increase.

RFA 10/27/09 I I; c':

, : I " ~ '/

I 1100 cC 012K6.03 012K6.03 3 x X N N N E The 31' 3'd0 bullel bullet is not required because they were never I declared inoperable.

RFA 10f27f09 10/27/09 I

013M03 013A4.03

'" I 11 11 cC 3 xX N N E I don't think ~ou you need the word ONLY ONL Y in A because an I auto 5 SI1will NOT OT reoccur until the timer has timed out as in B. You can also rewrite A as follows: After the SI 51 reset I pushbuttons are depressed and the RTBs have been cycled,, an auto SI cycled 51 reinitiation can immediatelK immediatel* occur. I (Note, this is wrong because the timer has no timed out yet) I Distracter A Is is NP as written.

RFA 10/27/09 10f27f09 I r 1 ,, 1 1~'2-+--~~---+--~-+--+---~--~---+---+----+---~-+---+'' '3~K~,n.0<,---------------------------II 013K1.01 12 M 2 Y N S No comment at this lime time I E RFA 10/27/09 Changed question to use Containment Spray logic instead of SI Sllogic logic based on validation feedback resu results lts., This changed the correct answer to '8 ' S'.', This also changed the question from an '5 'S to an 'E'.

'E '.

-~ -- --- -

()

t I

, 1. 2. 3. Psychometric Flaws 4. Job Content Flaws

(())

5.0~

5. Other 6. 7.

U 0#1 LOK 0#1 LOK I ~.u~

LOD  ;---

(CIA) I \1-:)1 (CIA) (1-5) Stem Curu: T/F I~.

Cues TIF ed Partial Credo Job- Minutia #I

1. 1 Back-8ack- Q= SRO UtE/S U/£IS Explanation Focus I Dist.

UISI . Link units ward KIA Only Changed ApprovedI RFA " '. ,11/12/09 " w **

022A203 13 c

C 3 x X N N Uu Under de; these'hese 'conditions, i , I, am not no' su.e 'ha' Swap&ing to sure that '0 i

changing ~~Pn"g"~A~~nd!C" high speed on their normal~ower is plausible. u~est "A" and "cn part (' ) to, swap to slow spee \""'0'; ,peeifon on

~ecause pot:~~~:;

their emergency power.

U because Potentially 2 NP distracters.

Part 2 of the KA is not met (facility identified). Use (faci lity identified),

procedures) to correct, control miti~ate...

con trol or mitigate ... I(you If fr0u cannot

~ae;t meet the entire KA, you have to meel mee at least Ihe he second part.

~~~'Reissue CE: Reissue a new KA if necessary. necessary.

RF 1"'<1- A ..... 10f08/09 10108/09 Revisited i i

!AiA2 needs to read " will remain in slow speed ... ."

i Part p~rt 20f 2 of the KA still not met. CAF RFA 10/27/09 IRFA10/27109 Changed distracters per Chief Examiner's comments.

1;\. I, ';, . '

. , .., :J 025K5.02 14 c

C 3 x X N N Uu Whether the,ice, ice sublimates or.me"s, melts, theIlle .esu" result is a higher highe.

containment i temperature~ which whict' yields ~her pressures.

I 82 B2an~,'~2 Factor in meltin~

and 02 are a.~ ~P. instead'01 P.

of pressure.. KA is is not affected.

nol affec'ed.

IThi;o This Q Is is U due to 2 NP distracters. distraclers.

RF A 10/27/09

'RFA 10127109

' '-:,' ,4 ' , :, .,

026A208 15 cC 3 x X y Y N E r£!A CAF: Do you ~ou expect the ap~licants 185A is? Iff you state the va applicants to know what 1NI-ve nomenclature, valve nomenclature, would that 1NI*

answer?

II give

~':': away a away the answer?

RF A 1 Irtt"A 0108/09 10108/09 Revisited i .

The -valve IThe valve nomenclature was added RFA 10/27/09 10/27 /09

.~ :'.-"::- ' t""- :"-1

"-, ,,,, ' '.\ "

026G2.2.12 16 cC 3 x X y Y N E E I' I am ,th~2~ ~ 'h'e<:ii;~'~

am not sure C2 is ~Iausible with 'hey~~."~

wi'h the valves open. Please re-evaluate this.

I If t e disch I ~ valvs~~I:,<~ are open, open.

are open, a a basic baSIC Please understanding i l of fig 7.1 ofthe of the reference renders 'endelS this'his choice I ,NP. NP.

RFA 10/27/09

~

n (L

1. 2. 3. Psychometric Flaws 4 . Job Content Flaws 4.

(")

5. Other I 5.

5.0~ 6. 7.

Q' II ~~ I LOK lLOD I~~:)

0#1 0#1 OO (CIA) (1-5) Stem Focus 100" I ~i~

TIF Partial Job-Link Unk Minutia units Back-w,rn ward 0=

KIA SRO Only I

UtE/S utEIS I I

Explanation

(;T;.;." ; ., * ;", "::' '.':;;:<;\

, :',) ~'< ~': ~

. 039K3.04 17 I e I I I I I C 3 X N N

.:~;~;~l~

Since aux steam is normally isolated at 100%

change the stem power level to 85%.

C more plausibility.

85% . This will 100% ~owepowerr wil give 8Band and RFA 10/27/09

\ ....,/ ;;r:'; j ' ;",.':: ., ."""'~ ':: '< ~:; It; ~~*, ~:" { .j :'~

,":,j - .,,! '> . .. " .' / . l'

• ' ro'i

,: ::,~ .,

059K302 eC I I I "cn: Chan~e P-11 to 1960 PSi9r 18 I 3 X Y I N I E !Choice i i~~'62~an~e to PSi~ This way they way have have'to I

plausibility to know I  ;"andha wha pressure P-11 IS. his will increase and make the answer less obvious RFA 10/08/09 RFA Revisited i I Additional comment:

i 101!

o is NP for set of events given. For this level of exam

[ and

~~~I~.nOWled~e,knowled~e, failure to recall that P-11 resets is not likely.. There ore 0 is not plausible. I suggest the following:

lA,'

IA. Following the CF isolation reset or when the SG Hi-Hi level cleared. cleare d.

I~

Ie.

B When the SG Hi-Hi level cleared ON C. FOIIOWln~ FOllowin~ the CF isolation reset or when pressure ONLYLY increase Increase above 1955 psig. psig.

10.D. when pressure Increased increased above 1955 psig ONLY RFA 10/27/09 IRFA 10127/09

.:::.~". ,;f ',;: : :,,:. ~ ;.:,;..

• : '; 'f.~,~ i ~.: ~ :

059K3.03 19 I e I C 3 I X I Y I N I E Too much teaching in ':: i all distracters/choices.

Furthermore, the secondj parts parts may give away the correct answer.

Reduce to 10 the following:following:

IA.

IA. The feedwaler feedwater regulating valves on A and 0 SGs only will open.

lB.B. The feedwater feedwaler regulatiG1 regulating valves will remain in the same position for all SGs S s I C. All feedwater regulating (WR or NR) levels r~ulatin5 valves will open to feed all SG leve s to 65%. 6 %.

1D.0. The feedwater fee dwater refhulatin2 regulating valves on Band C SGs will IRF:pen open to 10 increase increase the e (WR (W or NR) levels or NR) levels toto 65%.

RFA 10127/09 10/27/09

• '(.'. ' . I ,

(I J) 0#/

Q#I 1.

LOK LOD 2.

L.~?

J3.. Psychometric Flaws 44,. Job Content Flaws l

5. Other

~-

6.

6. 7.

7.

r:-

0 I ~~:s Ic""

(CIA) (1 -5) TIF Cred o Partial Job- Back- UtE/S UIEIS Explanation Explanation c,;,~1 Q ~

~~~

Stem Cues T/F Minutia #1 8ack- Q= SRO Focus Dist. Link "

units ward KIA t<iA Only I 061Kl.0l

(~.r*:*): !,: ~:{: :'?,,:'. :~~  :  : TT:;:)

20 M 2 Y N No commentco~Vment at this time RFA 10/27109 10/27(09

'1/'" .

062K4.10 21 C 3 X Y N E Need bounda~( for correctt since Suggestion forcD.

I ~_o~c!s_

Loads will I

sinc~_i!

~_" auto for.D.

i it~ takes D:

D. As written, D lakes 30 s or so to return to normal.

auto swap back to the inverter as soon as 0 could be potentially inverter

, over~ :urrent curren is detected to be less than 120%

RFA 10/27109 10127/09

.'. .').'; :;

'. ': {- ':.:  :",.- -: ri :'.:"C } ::.:-:';

063K2.01 22 M 1 X N N U I This

~hi ~ Q has low DV. Suggested fix: fix:

IA.

A. DIG Fuel Oil Booster Pump and FWPT A Emergency Oil Pump(s)

B. FWPT A Emergency Oil Pump(s) and Reactor Trip Switchgear Control Ie.

C. Turbine Emergency Bearing Oil Pump and Turbine Backup Vapor Extractor(s) Extractar(s) 10.

D. Turbine Emergency Bearing Oil Pump and DIG Oil Booster Pump DIG Fuel This Q is U I!his U because of low DV RFA 10/27109 10f27/09 Ii:,;:  :):, ..

064A3.01 23 M 2 X N N E Move the WOOTF statement to the top. lop.

1 ~1 CAF: Look at A1 and C1 C1 and make sure that they are not subsets of  ? 0L~.~ B1 and D1 01 because 235 psig is between both choices.. You ~~~emight consider using an actual value where the limits i have to be applied instead.

RFA 10/27109 10f27f09 l~t8n~~~ '~§Ib Chan~ed the Rressure range for distracters 210 - 25 PSIG.

dislracters 'B' and "D' "0 ' to

' ~' . .'.. ' 11' <;

064K608 24 C 2 X Y N U I~~~;;I I, ~;. ,~nd the'~l~~ 7'A,.'nr ~ ~~I~~n Since ref is provided, and there is no mention of problems with DG starting air, the second part of A and Bare NP.

Why not chan,ge to 7 days and put a particulate bullet in a':~~s;t: ~~~~;~~~,

I the stem that IS within the limits.

Unless I am missing something, this Q is U because of 2 a NP distracters.

RFA 10/27109 10f27/09

nI . ___. _._ __

L~?

3. Psychometric Flaws n

() ~

1.

1. 2. 4. Job Conlent Content F!aws Flaws 5. Other 6. 17..

Q#I I LOK LOD 0#1 r-(CIA) (1 -5)

I~~:s ICues T/F I~i~' Partial Partial Job-Link Minutia units 8ack-Back-ward Q.

Q=

KIA SRO U/EtS Only Explanation

, , ' .. ~ ~ i.': . '.:,:;> , ,';",:'

/. ~;'" .\ ' ;':1'.5 F: i:,'.' ,', ;;"/>

073A403 I ~"~"V' 25 I MI 2 I M I X X I I Y I N I U In accordance~_~11 with~ lhe the reference, it arpears appears that aa source Icheck .

check is required not a selpoin!

correct answer..

setpoint chec check.. Please fe-verify re-verify The stern IThe,- stem ask f~r re~uirement. The word "Ensure" for a reg,uirement.

~stracters "c" and "0

~C* "0')) are NOT "requirements*.

"requirements". These I i Itwoo distracters will be immediately eliminated. eliminated.

I~U _because *C* "c" and 'D" '0" are NP I ~FA RF A 10108/09 10/08/09 Revisited I ;~e_

The stem asks for a requirement. C e and 0 aarere "verify"

~~ A and B are performance steps. If the stem steps.

asked for a requirement, then the applicant w will rule ill rul e out e lout c and 0 because they are NOT performance steps. steps.

RFA 10/27/09 IRFA Added~~~s~~!~cdfstracters I ~~~-~~~;~

distracters 'e' to 1~~i~~~V.~s~:ri AND have RP perform a setpolnt ONLY to distracters 'A' to ""Start Start the

'0' to to "Start the associated sample pump AND ave

'A' and the associated and '8' 'B' and associated sample setpoint check and chan and changed changed sample pumppump chan~ed

~ave ed RP perform a source check"

, ' ',' , ")'

26 I M I 2 I X I N I N I E I';;~;;;~~

With train se8aration believe that ! ;,

1 ~eing is plausible being ,common ~"Ihe! '

knqwle,dge, I ~ since Ithe suction IS knowledge, I do not is common Ifmm from SNSWP.

Replace D. D.

RFA 10/27/09 IRFA 27 I C I 3 I I Y I N I"' comment at this time RFA 10/27/09 28 I 26 M I 2 I X I N IN E If C was correct, 0 would be also.

Modify C or 0 or add an ONLY ONL Y to C.

IRFA RFA 10127/0910/27/09 I.'. .

."'< : ,,: ,"" ~. '" .'. " "~:

002K3.03 29 I C I 2 I X X I N IN I E II,:,n, fans ;inn,:;t "No impact" (distracter B) is not plausible especiaUy high ,peed- speed.

especially with Replace B

( ,

o n

( ) D I

~. . . 2.

. 1. ~_ 33. Psychometric Flaws 4. Job Content Flaws

4. 5.0t~

5. Other 6. 7.

7.

Q#I 0#/ LUK I L LOK LOD UU r--

(CIA) (1-5) 1 Focus

~~: l cues IT/I I~i~'

Stem Cues T/F Cred o Partial Dist.

Partial Job-Unk Link Minutia #1

1. 1 units Back-ward Q~

Q=

KIA

~R?

SRO Only U/E/S UIE/S E~planation Explanation t<JA """

IKI"'A KCA 10/27/09

~: :.: ,", . '., . :';i:: ,  ;~ ':~. "  ;::., "':" -".:,-:. ,-.' ',: ,..: .

I **,

I' -, I ',-

r'. ,/', ,., ,~

I (::;:1 i 1",': .~ ~~ :;'~::<.';:' ~~

014A1.02 1014A1.02 30 M 2 x X N N E I;'

A 'is is a subset of B.

[iiIf C C was correct so would D. O.

jP,;,cho,nellrics Psychometrics need to be addressed.

jRFA RFA 10/27/09 10/27109

..::~ ;' ::. . , '.< ";  ::.: t.

~ ~.' ','!':' ,  : ", c:

017K601 7K601 31 cC 3 xX yY N E E

I do not see where "An is plausible given the I;I~, d:~o,~,":0::'s~e~e~~W:h:ere "A" is plausible given the circumstances.

IfReplace "A" or prove plausibility. plausibility.

I RFA 10/08/09 10108/09 leAF:

CAF: Must demonstrate plausibility RFA 10/27/09

[RFA 10{27/09 I"i.\.,:., :., :,?'t ',':;'::>'.:

\~<:.

',),

It;

~ :;~~

" '- ~/ .

i;:

oj'

.~ .: .. ,"~ ,-' .... . ~ .. ~** I . ::: :: ~ -i !? : ~

'I * 'I ,

';",::;; ":.': :. ;. >~  :';":;:. ';

029A2.03 32 cC 3 x X y Y N E Identify i the procedure that must be used to support the second part of the KA KA .

..... and how can it be stabilized as stated in procedure t'p"""d",.1 RF A 10/28/09 IRFA I: i:':,. ., I " ' ,~ -: ' ::": , r\ ~. " .

, .  ; " ' .lid'* -

/*;**nc*'" ~"  :  ;~: .

: ~ <.: ~ :~ <J 034A303 33 y M 2 Y N Revisited No comment al at this time RFA 10/28/09 10/28109

• 1,,\, (  : : ':: :. \~ :'-' ~ ;.

34 I 045K4.13 Uoo" O."

M 2 y Y N No comment at this time RF A 10/28109 10/28/09

'i.:. .;; -: r.1'- ;*, 'll .~

r . Flaws n

()

5.0~

L~D

1. 2. 3. Psychometric 4. Job Content Flaws 5. Other 6.

6. 7.

7.

0111 Q#I LOK LbK LOD r--

I :~:s Ie,,, I'i;,~

KIA ~~~

(CIA) (1-5) Stem Cues TIF Credo Partial Job- #1 Q" SRO U/E/S Q= Explanation Focus Dist. Link Minutia #I units Back-ward I Only UlEIS 35 35 I IUOOK1.UO M 2 Y Y N NE No comment mmenl at allh" this time lime RFA 10/28/09 10128/09 D'i,,' :'*./'~*~

r.' *,*"";:':: ~"; :; t': ,:,'!.",'~:

068G2.4.50 36 36 M 2 X Y Y N N E ISle~:

Stem: 'D~lele Delete the word "is." "is,"

ITh~'reference The reference provided does not support answer and choices.

Facility demonstrate and provide proper reference during review.  ;

RF A 10/28/09 IRFA I., . ,

l / ,!", I, ,:' '.' ,'.

37 37 071A4.14 C 3 Y Y N N ' The KA says to operate andlor and/or monitor in the CR. Rad alarms are a form of monitoring. KA is met. mel.

No additional comments at this time lime RFA 10128/09 10/28/09

, ,;.~: ~ { ',i ,, ;.~;'.

.:':':. f ',j: ::

38 075K2.03 38 M 2 X X Y N N U I~('TB If 1TB supplies power to 1ETB, then C is also a potentially correct answer. The stem is Is not specific enough.

enough.

I~~! a This Q is U because of potentially two correct answers.

RF A 10/28/09 If---+--+---+-+---r--+--+---+--I---+-+---+---r---I---I'I-Fr .."

i\;};.

,,""r.

39 EPE007G2.4.2 U ITh~KA(~~i~j.:\s 39 M 2 X X Y N N u The KA match is OK for actions that should have occurred for I?~<E~::~~~.

on EOP entry.

"Main feed pump !p turbines are tripped" is not nol plausible without i something . in the stem that mi.9ht might drive the applicant to thinking i I ~ . so. so~ Add something in the stem like a .

Ifa;l, failed annunciatorr that Oial might elude the applicant to '

thinking so. '0.

I This

~~: Q Q. is U because of two NP distracters.

RF A 10/28/09 i I

40 APE008AK1.01 (1.01 i 40 3 Y N I. iQ~ Q9JI~~

I with a ~~1!i~:~~1e. ~opa",y C Y Double jeopardy with Q 9: This Q is not double jeopardy with Q 9 because 1~9w!lh 9 dealt ruptured PRT. This hlsQQ deals with Ideal, w ! the Ihe. usage off~h~ the mollier m0tUei diagram. I am .m researching,-I';; the SRO ONLY aspect a'pecl of this Ih;,..

RFA 10/28/09

~" :~ ):' '~.:' ,

1 41 41 I'APE015/017AK2.08

D W:Q~~#,r:~~K;JII~L02~I.D;}~ 3.~~~~~F~la~W~'~ :;~ 4.~J~O~b~C~Ofo1;*O~I~F;,a~w~'~ f~5~.~~~"~r: 6.:~~~~~~~~~~::7:.::~~~~~~~=-"i~'.i

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

()

~ I ~.ed 0#/1 LOK LODr---.---.--.---.----+---~----~---r-----4 I

(CIA) (1-5) Stem Cues Focus ICuesl T/F Icred.1 Partial Dis!.

I uisl.

Job-Link i

Minutia 1 #!

1. 1 units I Back-ward Q=

Q=

KIA ISRO SRO Only utE/S UIE/S Explanation I x I I T-r lAIl notoolicil not solicit ~uchoi much of Ihe the info;'info Included.. "'f, Ihe slem does M 2 X YIN dloices Involve teaching. AddltJonally, the stem does

-V -W

~j~:e Since the Q Q is comprehensive, comprehensive , redesign it to a two part Q Q..

RFA 10/28/09 tv*,.,.

Changed all distracters for consistency and balance and removed wording that was teaching in the answers.

Approved RFA 11 11/10109

/10109 I APE022AK102 42 M 2 Y N i' Revisited No comment at this lime time RFA 10/28/09 I]APE026AA2 ,.

I ' ,\

.04 43 C 3 X Y N &6 "I th~k-120oF think 120°F is NOT ~Iausible plausible because it is 100 too low.

sS I th~"

Consider ~,~~~~

usinQ a a highigher temperature. AI At what what temp does E

resin . er temperature.

the~. resin bum? Something higher than 138 would also be acceptable.

temp does IRFA'IOI28109 RF A 10/28/09

'I, ~, I I ..

, ;.;\ . , :;", ', ii'  ; . ." ,

question i final I Following final validation on was revisedj to 8add 1

' ~ .11/11/09, 'h.,'em ofthe

~J~'e,mn . the stem of the i

". duel to reactivity effects" after the  ;'1:~

asking forrthe

'.~lblank first blank to clarify ,in Ithe Ii that the question w",

the change in NC system temperature due was I:~:he I i ,,','~~g~O~uf~~~,~h.;

to the release of boron from the demineralizer and not the potential change due to the increase in ;i~' and not letdown/charging\' '5; '0 temperature. . This This changed changed the the i ""ni question from an' an 'S' to an E'. an E',

Changed approved RFA 11112109 11112/09 APE027AK3.04 44 C 2 X X Y N U E I I sie~:-

Stem: 1 Additionally, i

1sl-bullet:

st bullet: Change "has*

Psychometrics:: As written, B is also a correct answer.

II If C was correc!.

to "has" to "have".

correct, so would D.

answer.

I ~~: U This Q is U because of multiple correct answers.

RFA 10/28/09

, ~:' , .: , ': ~ ':;.~" c ' \'

I  ;,;" ;.;, ,

I t"';~'

Following final validation on 11111109, 11/11/09, changed first fi rst two) bullets .to to set iinitial nitial conditions as a Steam Break I ~sas opposed to a Pressu Pressurizer rizer Safety Varve Valve stuck open It; to improve the plausibility plausibitlty of the Initial initial condi conditions.

tion s.

Change approved RFA 11/12/09 t 1/12/09

~ , ()

~ ~

~~~3t*P~'~%~hF,0~m~e~~I<:r," ~F~"JW~'~:t~~4i*~JO~b~C~oi't=e~",~F4"r"~~1

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5.0~

5. Other 6. 7.

Q#I LOK LOD ,--

0#/ 2.

(CIA) (1-5)

(C/A) ( 1-5) Stem !cues Cues T/F ed.

TIf Credo Partial Job*

Job- i Minutia #1 Back Q= i SRO UlElS U/EtS E:.:planation Explanation O:~Eomme~~:t Focus I UISt.

Dis!. Link Li nk units ward KIA Only 1I::I-'I::lI/!-I{';/ A .2" 45 :Y" I N this time lime M 2 No comment at this RFA 10/28/09 10128109

';';';',~~--i:..'-:: ,',~ (':,, }' ,:; "~ :/;/': ":":')

II----I----I---I---l----+--I---l---I---- I - --I-----I---I-----I--I= ""

EPE038EA 1.10 Thl;~~~,~ Ithe .1is'~~e 46 M 2 xX xX y Y N E This E I andlIhe, Q is confusing. NSO-513 states that the best 13 , N-16 method is EMF-33 the seconda'6 method the N-16 determine I that ~'

monitor. The Q stem asks what the est indication is to iii a SG tube, leak ~~~~ be" leak has occurred. '~;;te~

I,dl"The stem

,eem, seems to contradict theHef worded .

ref material,I slightly "Ighlly by the wi;';'",

1e way it is Facility;~Iease RF"i" please re-evaluate. re-evaluate.

RFA 10/28/09

,,'.. ::'l. '.~(>,j.>:~ nr :\ ... ,':!' /:~

APE040AK1 .01 47 cC 3 xX y Y N E Oistracter 0 is NP. Replace 0 with a statement that would cause a CO without stating so. For example openin~ a valve or series in the primary or secondary that woul cause a CD would be acceptable.

RFA 10/28/09 1* \: .) , ( ,.,

Ape054AA2 .08 48 cC 3 y Y N No comment at this time RFA 10/28/09 I ""I " ,:" ~ " -: :', " .

• ' ,"';,\1 ' . ;' ; ~' ;fj. :.':

EPE055EK3.02 49 M 2 xX y Y N U U IA A is NP because there is nothin% nothing in the stem to indicate Itha-that' NC system subcooling has been een lost or challenged challenged..

1 ~~;:'~'::ill'~~~r':~;?~,l~

There is nothinp in1 the reference to support the use or potentialI use 0 CLAs the s in this caseto .... support the use or Oistracters

. A and B do not appear to be plausible plausible..

I ~~~

This Q is U until untit it is proved otherwise.

RF A 10/28/09 It'\rM 10128/09 I, {.-', :,: .. '-:: , . "'~ ~,.~>~ . ; ; ~ ' /:::'  ; ~, .. :';. '.,,>; I'J :~f~ :)'J ;'.< ~~

~ :"

, ", . i .'~ ~

' .

i A n ' ';.: . '; , " '.: p . ; ~ .

I '*"

1,'<> ',"'>.> < ....

APE056AA2.37 37 50 cC 3 y Y N No comment at this time RFA 10/28/09 10{28/09 I"APE057AA1' .06

,, '. 'i' ,', '; . )

51 M 2 xX x X y Y N E I To T~ 'be be consistent, state in B, C, and D 0 the procedure you

n '"

1.

1. 2. 3. Psychometric Flaws 4.. Job Content Flaws 4

~

5 . Other

5. 0~ 6. 7.

7.

Q#I 0#1 LOK I LOD LOK ~."":: ~

P-:>> Stem I Cues (CIA) I (1-5) c,,".~,.IT~ Ie,," UlEtS U/E/S Explanation Foclls Focus T/F Credo Partial Dist.

Dis!.

Job-Link Minutia

1. 1 units Back-ward Q=

KJA KIA SRO Only will go to affer compTeting,.he lflelAs. lAs.

RF A 10/28/09 Ih . *.  !' ... ~ ,.,

52 I APE058AK3.02 M 2 yY N No comment at this time RFA 10/28/09 APE062M 1.05 53 eC 3 xX x X yY N E Reverse the WOOlF WOOTF statement and put it up front.

The*

The discussion (distracler (distracter analysis) does nolline not line up with I the Q Q.. Please re-evaluate. re-evaluate. However, the reference does support

.

:~:-. the correct answer. answer.

RFA 10/28/09 Kr-A 10/28109 I:, ~

J, , , , " ~ . j -: . ': , ~J ;' ~ * ,'. ~ 1' .. :

I*:

,f',,~,';,~:,,~':,,:>:~"'j ;.~;::::" ~ ",~ ': u ; 5 ~~

WE04EK2.2 54 c

C 2 xX yY N U ;' IfIflh~

u the o,"ph graph wasn;. wasn't (0' markedIIhen then 1 f~~~ a"'ee I would agree v.i.hwith the 'h~;" **

.~eg",ph

. distracter analysis for distracters Band D. However, since the graph is 1 is incrementallr marked!,.itis ~;*~§if2~;:~"'--

it is not possible to I

misread, :he it. on the horizon al,I axis.  ;::'31,'0"GP';;:

axis. t herefore, 380 GPM is

• no.

not plausible. 1 IId~rn~t do not ~~ve have a suggested fix for this problem unless you l.c~an~~t~~de can provide a a curve curve that tha t doesn't have the 2-9 incremental markings.

, This

~~~ 0Q is is U because 2 distracters are not plausible.

RFA 10/28/09 KI'"A I .. :. ;y; .:',"* ..' r: f : ~,\ ~': '...~

WE05EK2 .1 55 c

C 1 xX yY N u U '!L~"','

It is common knowledge that an SI 5 1signal isolates FW and

~:~-I that FWcannot FW cannot be regained until SI SI has been reset.

This

~~~ Q Q is U because it has no DV.

RFA 10/28/09

KrA 10/28109

,; ; "; ~

.. /" ,: .;:. ::,L ~ * ~~

r , '  !

'\, ";:" ,.:,:: .. ,,"{ , .' ;.' I L**. "  : :~i WE11G2.4.8 56 cC 3 x X x X xX y Y yY u U Band 0 are not plausible because~ you asked them in the I

stem WOOTF describes the requirements or implementing ...

Therefore Band 0 D are automatically ruled out.

Change~ stem: WOOTF describes whether or not AP-22 AP*22 can be Ican be implemented (something to that affect)

) that 1d~ct)in conjunction with with ECA-1ECA-1 .1. .1.

nI

22. 3. Psychometric Flaws o0 6.

,~LOK

~K,

1. 4. Job Content Flaws 5. Other

5. 6. 7.

Q#/I LODr---~--~~---r-LaO 0#1 (CIA) ((1-5) 1-5) Stem Focus Ic'"' TIF I'i;;~ Partial Pania! Job-Link Minutia I #1

1. 1 units I 8ack-ward ward a~

Q=

KIA I1~

SRO 00';

Only utE/S U/EIS Explanation II~.Y,,,.

agree that tms Q .0 "", ~ "'.may ', bEfSRO ue o~v v,,, ONLY, Replace I the Q Q or request a new KA.

RFA RF A 10128109 10/28/09

~ .~ .  ;

' ., d \.F::':"

• .~*~;' , *: **.~{: ( -:"\ , ::" '\ ,.~~.,V:

~,

:, -, : : . * -  :':-: 1- ,

" t\ . '; (~)

APE003AK1 .07 57 eC 3 xX y Y N E Please re-evaluate .th~ assum~tion J r I the assumption in the guestion guestion.. If they know Xenon) di!;l,nol,' did not change, tthen en power did not Chan~e change

,and I and reactivityf did not Ichange.

answer.

"~" This may key them to tfle t e jean Ihe Can the assumption be removed without affecting the Iquestion?

I IRFA RFA 10/28/09

, "', ' .~'. 'f ~.:. } ~. ,: '<.

,,: ':~ .~ : .' -',:. , : ,., ; ! !

~ ',~ '; ,

" :.' , ,:; " t : ,

" r, '

• ~: .' i! , l * " '~< ';

. fina' Following final validation on 11/11/09, changed " on1"'

, ih~:;,o.;.~~~:" n:e . ' I~;50U'd be distracters A1 and D1 to 'increases ' instead of

'decreases '. This was done since decreasing could be 1~~~I~:i:'~

considered correct if the Operator were to perform a SDM determination as opposed to considering the ,!;~e_ " ~' ',10' ,'he a lacI"ai ' , In SOM;,~}

actual change in SDM. If a ,determination was performedI the the ~D~ SDM ,calculation I I I ,:"ould'would yield

~

~~a:,

ffor-; a decrease decrease lin SOM dU~ IOg~~~:~jr;~'

In SDM due to the large penalty imposed for a dropped

a, Ir!,d dropped (mUi_~o~{9s~,te\lenrod). So, even ~~:~~~a!(?TduAlrts'D~rd~es rod (much larger than the actual rod worth for a though ACTUAL SDM does not not change,-'j the Ir~ed~ calculated S~~~T~~i.

SDM will1I alwaysalways decrease decrease for for a a dropped rod.. Also, in the stem of the question I th~~

changed~ the bullet regarding i rod motion to sa.y say Ithat.

temperature was restored through manual rod *od motion I

'!.O!.

instead of automatic rod motion. Also, added a bullet 0,od . ,~~S~ot~_~:~!aj~J~i' I:~a~~~~h;al 'bo,on: ;'ii~nh: ;!;, 0has to say that boron concentration has not changed so that the Operator will have all necessary parameters to s~ 10 determine" the 'he~ffeci'of effect of the dropped rod on SOM ,;;;"on SDM .

Changed approved RFA 11112/09 11/12/09 I APE005AK2.02 58 eC x x yY 3 X X o is not plausible because the stem stated that there w~W liT' n,olsla:Ie'.t,'~e,~t,i,s;~~~,emtS,a'fvb~~(~ln N

no indication that the rod is mechanically bound in the 4 bullet. To state in 0 that it cannot be moved , Ibe ~oveo !is a Ihe."

contradiction i to the he stem. The distracter i I analysis forDis for 0 is not acceptable.

Modify or replace distracter D.

RF A 10/29/09

'-:- * 'il L. ' :, ,'

o r~

I 0 n~

L~D

1. 2. 3. i Flaws 4. Job Content Flaws 5.0~

5. Other 6.

6. ,I 1.

7.

Q#I I L6K Q#! LOK LOD 11 ' --.,.--,---,---,-

(CIA) (1-5)

I :~:s Focus ie,,, I'i:;~

TIF Partial Job-Link Minutia I #1

1. I units I Back'*

Back-ward

~,d Q.

Q=

KJA SRO KIA .I~~~

Only 1

UlEtS U/E/S I Explanation

,*r' ;"i '-< f~ ~ ;:.  ; ' : ," ~ ,>t;~,,'

59 59 M 2 X Y N E I ~AF~

CAF: The The KA KA i.s is +~;oCi, for low temperature Concentration. This Q references an over-temperature

.~~i

"its limits fo for' Boron Boron temperature i , con ""':n~;'tl;~?

annunciator. Is this for a high temperature or a low condition? i . 'TThe ref does; not expl.in ' 0' tow explain well.

ISte~.;

Stem: Is this a high T emperalure or a low temperature condition? , Temperature Remove the double peri period od from "C' "C" and "D" RF A 10/09/09 IRFA 10109/09 Revisited IAft~*r After- 1revisiting i i thisi Q , I think 80°F would be more more plausible i ~ th~~. then 65 of. 80 Is is the excessive wat~RCS the excessive water temperature in the RWST where 65 Is is the min RCS cold Ie!=!

Icald leg temp. temp. Consider changing and we*wiJi* we will discuss i during~ the the 'on site review.

IRFARFA 10129109 10/29/09

, 5": ,~ . .~., -: ': / ": ' i:: '::) ' "

IAPE061AA2.04 60 C 3 x X Y Y N E hfY~~

Ilf you wanted to write this WID . W /0 it being a NOT Q, Q , consider ith.

the following ,:

I ~~~i WOOTF ~larm(S~ alarm(s) can be used to confirm that the 1EMF-16 alarm I is valid valid?.

IA.A. alarm 1EMF-17 annunciator or SFP level Low computer lB B.. Incore Inst Room Sump Hi Level or 1EMF-51A 1 EMF-51A alarm 12::C. 1EMF-17 1EMF-17 aannunciator nnuncialor ONlY ONLY

10. D. 1EMF-S1A 1EMF-51A alarm ONLY IRFARFA 10129109 10/29/09

. . <, ., . " ~.

'. -: I' ~ "' : ,

1,.,, * ;." k . '. - .

'( : ,

IAPE067AK3.02 61 M 2 Y Y N Uu Accordingl to l~* thelhe reference (OMP 2-2, 2-2 , Page 13,) A and B la,eare both correct..

IfI mat that IS is nolnot so then: then :

nd I;'~Change~ !~: the 2~d 2 part of A to "The fire brigade will respond ."

nd Change the 2nd 2 part of B 10 to "The

The fire brigade will NOT respond ."

IlAsA;:;'riH~n written , Ih. the Question "i' ,i, "h.

is leadinQ the applicant i ,lin to Ih.

the

()

0 0

()

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. 0~

5. Other 6.

6. 7.

0#1 .----

I (~~:)

Q#I LOK lOD LOD (CIA) (1-5) Stem IC,,,

Focus T/F Credo Partial Cues 71F 1'6,:1 Dist.

Job-Link Minutia #1 units Back-Back.-

_,d ward Q~ 1~~~

Q=

KIA KJA SRO Only UlE/S U/EIS Explanation correct answer.

I Currently, I the Q Q is U because of two correct answers.

~FA:

RFA 10/29/09 fl,!', ;' i ~ :" ;1:): , 1' J",: , -,-

' ,\ .~'. ,

i\; -; <<

" . .:.:~ ';:',  ;','

1\, .::,"

. Changed to new question. '~t~ question . Original Ori ginal question w wasas determined i to be SRO level kknowledge nowledge fo following llowing final fi nal validationi on o n 11/11/09.

11 /11 109.

Changed approved RFA 1"1112109 11 112109 APE068G2 N~' .1.28

.1.2.

62 2 yY M N S

$ No comment at this time I

E RF A 10129/09 RFA 10/29/09 r,

1:,< 1,1

. ~. .,'  ;; .  ;':

'j .,: -:~ .- '

I *'" '.

WE03EK3.2 63 M 2 Y N No comment at al this time r.:,

RFA 10/29/09

"'l 10/29/09

,I, WE09EA1 .1 64 C? 2 Y N E i ihi~klhi~ 'i~

I think this is a comprehensive Q since a fair amount of evaluation required..

a I is required IRFA .,

RFA 10129109

( :

10/29/09 WE13G2.2 .37 I No

~i~omm'ent 65 C

C 3 Y N S

$ comment al at thisthis time time E RF A 10/29109 10/29/09

,' ,\ /"': " '.! ""'::,, . ",'

",' ~ .,, '~ ~ ... . ( : ~ : ! I ~.

I> , ., '; ' I * ~~f- ' ~; ~{ I

.': ~ ;': . . .

G2.1.28 66 M 2 Y N No commen commentt at this lime time RFA 10129'09 10/29/09 e., {,r _..... .,.: .

G2 .1.30

c:

'~;'r-IThi~' ,_;~~~ave a"y1hi"~

67 M 2 II N N U This cO' Q does not have anythin~ to to do do with with

~~~P~~

locatin~operating local contro local contro Ss.. II SU9aBst suggest writing writin~ aa Q Q Iwhere where e CR haC! to be evacuated an the opera operator or has l~h~:POft to report to to a a localI station station to to operated operated equipment. equipment.

This a Q is U because the KA is not mel met

1\

or... *I 1

1.

1.

Q#/I LOK ' LOO 0#/

2.

2. J. Psychom etric Flaws LaD ~--r-"":'-.----r--'--

4. Job Content Flaws

()

(J I

5.0~

5. Other I 6.

6. 7.

0 11 I

"I I (CtA)

(CIA) (1.5)

(1 -5) Stem Cues TIF Credo Partial Job- Minutia #1 Back-8ack- Q.

Q= UIE/S SRO I U/E/S Explanation Focus Dist. link Link units ward t<JA KIA Only

~[RFA KrA 1U/£8/U8 Ii{,)',

)

,  : ':,~ {: ~ ,; ; n I****

1, , ,(" (;',

.:(, . ',(.,;

1-*' '\,;-""'::1;:': '1 ~; ': .:";

I ;1':

G2 G2.1 .1.31 .31 68 e C 3 yY N S SIN No

. . O commentcomment at at this this time time E RFA RF A 10/29/09

'" ).~

".1 , s;.-:

e, '.,' , ..

',:~;:<: <.: ">l ~:': ,,;t-:,."'

j

,;~~ " ~ ~:

'-'.,.; .:.~'"  ;.~.

'I- ,, , ',,'

G2.2.21 69 xX yY M 2 N E IRemove Rem ove double period from "A" and *C* "e" Oistracter Distracler analysis: analysis: "D" "0" what does "the exposure will be greater if you you wear the respirator" respirator" have to do with this this Q?

"8"" and "0": remove "to provide ...

"8 .. :" as this is teaching.

Plus it is additional information thai that is not needed.

RFA 10/09/09 10109/09 Revisited There is way too much teaching in all of the choices.

Reduce as follows:

A. 11ND*15B NO-15B is interlocked Interlocked with 1 NO-58A (TRAIN A NO 1ND-58A TO NV & NI).

B.

B. 1ND-158 1 NO-15B would cause one NO train to become inoperable.

C. 1 C. NO-15B is iND*i5S Is interlocked Interlocked with 1iNI*136S. NI-136B.

O.

O. Both trains of NO will become inoperable.

Put all of ofthe the reasons in the distracter analysis.

RFA 10/29/09 Rewrote per Chief Examiner's suggestion suggestion..

... > J ,.':. .-: I .~

70 <32.2.35 G2.2.35 eC 2 yY N S I think e C and 0 will immediately be eliminated but it meets E NUREG requirements. The Q a IS OK as is.

RF A 1 RFA 0/29/09 10/29/09

QNI Q#I

r. 1.

LOK LOD

2. 3. Psychometric Flaws 4.

4. Job Content Flaws C:~

5. Other

-r--

6. 7.

(CIA) ((11-5)

-5) Stem Cues T/F Partial TIF Credo Partial Job- Minutia #1

1. 1 Back- Qo Q= SRO U/E/S Explanation Focus Dis!. link Link units ward t<JA KIA Only

,>:;, ,' '.,,', ',: (:;': . "." .. " .,;T :d:,"'*

'J ' '; :~:.~ L* >  :: ~

~ < ,1'.. (.~

G2.3. 12

,,2.3.12 71 C 2 X Y N E oo is not plausible for this failed fuel event. It is common knowledge to redirect letdown.

Replace D D..

RFA 10/29/09 RFA 10129/09

" . . ~ :' . " : '; ' .. \:.' ,li ' , \: " .' :,,'  !; '.

~!. " . ' f'-,' : :  ;': t' r.~ .:.. "

I.- ,... ,

\' . "i'  :: f : , ! ,j, '

.<,;', ; ;.:. <:.. :  :.~;: , ~

. ~ ..

..r .

, .f ',, '

G2.3.14 72 C 2 X Y N E Distracter A is weak. This is too important of an evolution to be evaluated and approved by an NLO.

Replace A.

Distracter C: State a reason instead of "under ' under these condi tions*

conditions" RF A 10/29/09 RFA I; , , ~

'. ),

--'-:" ; ':::',, :.:.:  : .. ~ . .',.~

G2.3.7 73 C 2 X Y N E oo is not plausible as written.

0: Change to: "should" to "must" and "could" *could" to will "will" RF A 10/29/09 RFA

,,':;,*:i"";:

".  ::', ~ ;: .. "

',' f,: ,: ( '. i 1 .~ (P,,/

I G2.4.16 74 C 2 Y N S

$ No comment at al this time E RF RFA A1 0/29/09 10/29/0 9 Approved RFA 11/10/09 11 /10/09

<.,".;,', . ./'{

',: . ~ " ,

'c' ~1 ' * .~ I(

'\" .

~:; ,; ...,

r.,

I 1.

1 2. 3. Psychometric Flaws 4 . Job Content Flaws 0)

(

~

5. Other I 6. 7.

(J

~

0#1 Q#I LOK LOO LOD (CIA) (1-5) Stem Focus C"" '" I ~i~

Stem Cues T/F Cred o Partial Dis!.

Job-Link i

Minutia

1. 1 units Back-ward I

Qo Q=

t<JA KIA

~

I ;~~

SRO Only U/E/S UIE/S Explanation IG2.4.4b .*.* 0 75 C 2 X Y Y N Is it fair to ask this question without a reference? If you allow II a reference, Will will it become a direct look up (Ol.U). (DLU).

Please re-evaluate RFA 10/29/09

, -. ,. l~i . : '. ".

.,.', ~'. ",~ f , .,:. ; .: .' '.: : -;.:" :. "',: {

"",; .,. : 1'**,

~.  : . ,: ~J: ,"

-~ f ~";

.,J..:>;-:< ". .(.'~ r~~* .".

- " A SRO ONLY N/A 76 77 78 79 "

80 81 81 82 I

83 84 85 86 87 88 89 90 -~. - ~ - - ". - - . --- --

<l Q#I 1.

LOK LOO 2.

LOD

3. Psychometric Flaws 4. Job Content Flaws 0*5. Other r--

~

6. 7.

(CIA) (1-5) Stem Cues T/F (CIA) TIF Cre(1.

Credo Partial Q Q=~ UtE/S UlE/S Explanation Focus Dis!.

Job-Link link Minutia

1. 1 units Back-ward KIA SRO Only 91 92 93 193 94 i 94 95 96 97 98 99 100

---

~ -~ .- . - - ... -.-..------.....-.- - .- -....-. -.- - -............."------ ---

• Independent of whether or not it met the KA, RO/SRO, or required grammar or minor enhancement.

KA , mis-classified WRT RO/SRO, Quell1lon Statu.

Question Status Question is Sat with NO Comments E(42) I Editorial Changes Needed E'

E* I Questionable Editorial U(14)

U (14) I Question is All Hosed Up 81(0) I Questionable riO) I Question and stem sat - editorial elsewhere

$1(0)

N*(O) 11"(0) I Partially met KA

ES*403, Rev. 9 Written Examination Grading Form ES*403*1 Quality Checklist Facility: McGuire Date of Exam: Nov 19, 2009 Exam Level: RO Initials a b c Item Description

1. Clean answer sheets copied before grading rfa blc rfa

2. Answer key changes and question deletions justified and NA NA NA documented

3. Applicants' scores checked for addition errors rfa blc rfa (reviewers spot check> 25% of examinations)

4. Grading for all borderline cases (80 +/-2% overall and 70 or 80, rfa blc rfa as applicable, +/-4% on the SRO-only) reviewed in detail

5. All other failing examinations checked to ensure that grades rfa blc rfa are justified

6. Performance on missed questions checked for training rfa blc rfa deficiencies and wording problems; evaluate validity of questions missed by half or more of the applicants Printed Name/Signature ./ Date

~-:z7

-~

a. Grader Ronald F. Aiello / ~

~ A

/h~<J9

b. Facility/NRC Reviewer(*) Bruno Caballero/ .~C!~

• ~udL 11/?1J/or

~ )

T--~

~~

c. NRC Chief Examiner (*) Ronald F. Aiello/ /J/?tI;h9

, )

d. NRC Supervisor (*) Malcolm T. Widmann/ I~~

~!fj ~l1I---

~lU--- tl/~,,/tYl

(*)

\---7

\---}

The facility reviewer's signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.

--- . KEYID SCORING &

r-------,---------y--------------------------j

~------r---------~-------------------------)

CJ RESCORE

~IJ!rNJIffs~ >-':'.c=i

.. ~-'_C'=_(::')R?_~::cE,::.:(=_7=-_AL>.J,,:-:;S*'::'n~lE=.:Jl='_'-"

>-:'.c""~~:-'_C==_(::':)R='.~='E,"":(=.7=-.:""!:..vs=.'n=lE=.:Jl='_' ~

C MULTIPLE "4NSWE,'R SCORING

__"'g~'!lf1~c1.:.::..l?:.:K:':"\':":.."'_n~~~_":r'='2=T/i::..!.:-,_-:::Q7:N-:L.Y::O}--<

.. ''._-:g~':"lf1=.Cl..c:I}'-=".K:-:"\=:..""_n::=~~_:-:r='2:::L""4!=-,.-=Q""N-:L.Y=}*-<

Pearson NCS Test Sheet 100

©

,-._~_",--:,,--____ \f/lRK

\f/1RK(J/dJ (lid.! ();"};

T F T F Form No. 106173

- -fr fr l.©@ T F (];).@

51 Cf0.@ 1 F

-.., 26@.©@

26.©@ 76@@

76@ Reorder Form No. J(}6J73

-~ 52.@@~

52@.@~ 1-8()()-367-6627 Fax 1-S07-4SJ-,f513

-;:, 77@@@

77@© va Iii ebridge.llcspearso/!.com

-~ 3@©e

-:j 2Se© 78@@

78@©

-- ,S'S -I@.@

-I@. S-I_@©@@

S-l_@©@@

-0 ..... 29.@© 79©@

79© 5.©(l!,)@

5.©(l!,) 55.©@

5S.@ (PERFORMANCE) t

-;:., ("ANSWER ')

~

-::'] 3fJ.©@

3fJ.©@@ so© So@© L@:YINFO. I I ASSESSMENT I

_:::l 6@.COCQ)@

6@.COCQ) 56.@@@

56.@©@ ~FAFYS 1r OF I

-~

cf OF [\Pi POnTS f'(

31 @.(Qj@

@.(l!j 81© TOH/_ ) ,

81@©

' COlt ~ rr I"l" TOTiI_ T: .

, !TEJl ,SCORE SCORE EARSED EAR.\Ui 7.Cf;.'J@

7 .Cf;.J@ 57.

S7.

32.@© 1i2© 1!2© .1""

W ii 5S. [

33.@@

33.@ 83@©@

83@© w ~I

~I~

S9.@@

S9.@© icD CD CD GJI ,ICD CD CD CD CD 34@@.@

34.@ 8-1© 8..f@ I CDwG)  :,!CD wCD WW

.(§)@@@

3.1 .(§)@@

.'IS 60.© S5@©@

S5@@

I GJG)

G)G)

I JG) G)

G) G)IG)

G)G) 11 .©

@.© 61.©@ I I °1G) r

--- 36.(12)@

37.@©CQ)@

37.@©CQ) 62.@

62.

86© f!7@@

f!7© I

I.

CDCDiJl

,t I LJ~ a-;l~1 I

CD I

\

CD CD II

/3 (l',).GD 63.@ ~.-

~-- I 3S.@©@

3S.©@ 88@@

88© 14.CID© 39@©.

39©.

-IO.@©

-IO.©@

M.@

M.

65@.@

65.© 8'1@

8'1© 91J@© 91J© Copyright © 1995. 2001 NCS Pearson, Inc, All rights reserved. Printed in U.S.A.

AI! U ,S,A, Inc.

- 16 GD.©CP)

GD.@CP) 66@@@.

66©.

91 @@

91@

67©e

- 42 (];)@.@@

(];)@.@ 92@

92@© 68.@©@(~)

- -I3.©@ 93@@

93@

- 19.@@

19.@© 69.©

- 44.@@

44.@© 94@

94© 70.@

- -IS@@.',:Q) 95© 96@@

96@

- 46©.

72@.@

72.@

-I7@@.@

-I7@.© 97@© 97©©

- 23@@.@

23@.@ n.©@

4S@@.

4S©. 91!©@

91i©©

- U.,C(Q) 74@.

- 49(£;J. 99©

- 2S .~O*@

- fr

-.., NUMBER CORRECT 50.@© 50.©© COMBINED POINTS I

]{)() (ID@

(ID©©

-~ EARAED

-~

-~

r~

COMBINED PERCENT

-'"' ~ CORRECT

~

V::)

-~

-0

-t;:,

_s;

- -~...

-~

\::

ROSTRR NUMBER SCORE

(;RADE SCORE 1------------------...,

1 l

STUDENT ID NUMBER

--I*-.._*

--I-----

I

-II J

_v';

-~~~~ -~~-

""-1-1--'

'---1-1--'

RESCORE RESCORE , I MARKING i INSTRUCTIONS rNAME 2c::xfl N'<<S ~O 1<E1I\K~

ICD CD CD CD CD CD CD CD CD CD

->t '~~-d~£~~~~~~

,~~-d~£~~~~~~ Iw CD CD CD CD CD W CD W w GJ GJ GJ G) G) G) G) GJ G) G)

UVi'ii /\'(j, 1 Pt'llfi!

I G)G)G)G)G)G)G)G)G)G)

AN~i<.. '<<'C}i

---

lSUBJECT PERIOD DATE )

I

© IICD CD CD CD CD CD CD CD CD CD I

I@..<;v...

I@..<;v.. ..

- I'Aark Reflex' bnns by PC3ison NCS r\i1M106173 -3 3~