ML082730775

NRC Comments on Draft Salem-Developed Operator Licensing Exam (Written & Operating Tests) (Folder 2)

ML082730775
Person / Time
Site:	Salem
Issue date:	07/02/2008
From:	Public Service Enterprise Group
To:	Operations Branch I
Hansell S
Shared Package
ML080030005	List: ML081060248 ML082350047 ML082470469 ML082730736 ML082730775 ML082730779 ML082730784 ML082730786 ML082730790 ML082820486 ML082820496 ML082820517 ML082820534 ML082820540 ML082820562 ML082970764 ML082970767
References
50-272/08-301, 50-311/08-301
Download: ML082730775 (34)

Text

Salem RO Exam

7.

UIEIS U

E S

S -

45-Day Version

8. Explanation Possibly replace B with High PR trip. Good question but it does not test knowledge of the bank step counters. Change the KIA to match the question. Good question -step counters are LoD I Capitalize "one minute" in stem. Choice A - Is 45% too close to 33%. Give actual parameter values for " 6 and "D". Will lower seal injection by 2 gpm. Will lower HX CCW flow by

?? - TBD 45% is sufficiently high to discriminate from 33%.

Do ROs have to know how to make procedure transitions?

Will get back with answer Why is "6" plausible -why would someone think injeciton sources should be isolated during a LBLOCA? This questior was used on the last 2 NRC exams so it should be OK ES 401 -9 form E

E S

E E

S

6.

BIMIN NRC -

N N

N NRC B

N N

B N

N N

Move "RHR HX Bypass" to A. Better yet - rearrange stem anc choices so that both valves are in the stem - or both valves art in the choices - but not split. This tests knowledge of a subsequent action in an attachment to an AOP. Is this fair?

Should the applicant know this based on system knowledge alone? Will verify this is RO level of knowledge - had a high miss rate by ROs during validation. If used -will rearrange distracters to clarify Substantially revised question to test concept - not procedure steps A - why plausible in mode 1 (RHR HX)? Replace "A" - add "by 50% without exceeding limits to last sentence in stem Controllers and positioners not in Q - But isn't this implied?

Can we get controllers or positioners into the stem? Draft KU match analysis Replace "CRS" with "crew." Based upon answer explanation B could be correct. Edit B and D to say "under any conditions or such. I am OK with their wording. Verify this really RO level of knowledge at Salem. This is memory level of knowledge - you have either memorized the steplbases or not High miss rate on validation - will review and get back to us. Decided this was really higher LoK after dicussions with DMS.

Choice A sounds like a good operator practice - possibly technically correct??? Time is wrong - 10 minutes not correct good distracter. Add "required by the FSAR in the stem Will addFSAR Ref needed -

N N

Y

5. Other

3. Psychometric Flaws

2. LOD

4. Job Content Flaws Partial -

Back-ward -

Job-Link -

linuti:

I unit: -

~

L Y

N Y

N Y

Y?

Y N

Y N

Salem SRO Exam Page 1 45day ES 401-9 form.xls

b 5

0 ui c

wI wI W

I Z

z z

m z

z z

z z

z z

z I

I N

L

r-:

ml ml 2

2 m

0

0) a

-I

> H i-t I

I I I I

XI 0,

sy n i -

S I

ILL I

I c u m u

U N N N

0 N

N

Salem RO Exam Partial 45-Day Version t?;

Minuti, X

X X

X X

ES 401-9 form SRO only N

N N

N N

N N

N N

2. LOD (1-5) -

3 -

4 -

4.5 3.5 2.5 -

3.5 -

2 -

2.5 3 -

Ref needed Y

N N

N N

N N

N N

Stem Focus -

lunits

3. Psychometric Flaws I

4. Job Content Flaws Back-ward Cred.

Dist. -

UlElS S

E

8. Explanation Question 31 states that curve book Sl.RE.RA.ZZ-0016 will bt provided as a reference. Does this include the SI pump curve which makes this question pretly easy. SI Pump curve not provided in Q37 Requires operators to have memorized the design bases seal leakage from an RHR pump in the FSAR. This is not generally expected knowledge from memory. Verify that this i expected knowledge (learning objective is not specific). Can use question if delete the 50 gpm and revise distracter " D This requires detailed knowledge of an infrequently performec procedure from memory (step 5.3.5 in SI.OP-SO.RC-002).

BUT there is a learning objective ("major precaution and limitations"). Is this a major precaution? This does not matct the KIA - this is about drawing a bubble in the PRT not the U

E 7

5. Other PZR. Replace question Will get back Delete "Given the following conditions" because there is only one condition. Why is knowing this from memory important??? How does this test the "system implications" part of the KIA? Will get back Revised question to test system implications The KIA is to test how a malfunction in the PZR PCS effects the RCS. Need to add in something about RCS pressure intc each distracter. As it stands, this just tests the PZR PCS response. " B seems implausible - do not expect code safeties to be the first response for an instrument failure.

Consider changing "8" to "PZR Heater turn on and pressure slowly increases" which is correct. This would make "8" the Q=WA -

Y Y

N Y

?

Y Y

Y E

S U

E S

Y correct answer because B happens before D. Will get back Are they required to memorize yellow path entry conditions?

They would also have to have memorized the first step in FRCI-1. Yellow paths may be referred to - can an applicant claim that "8" is not correct because it could be entered and immediately exited. There may be no correct answer. Is this fair question for ROs? question is considered fair and 6 is correct Why are B 8 C plausible if there is no self test feature?

Furthermore, it is not an operator actionslresponsibility. Does not test KIA explicitly. Does not test knowledge of design features or interlocks. Replace question Q40 Typo - desinged vs designed. What does PZR low pressure have to do the power density limits? Can you replace " D with high flux (high power)?

Q41 Why is a 1 second inadvertent SI chosen as the stem condition? Is there anything important about 1 second?

6.

BlMlN NRC -

N N

B N

B N

N M

N

7. I Salem SRO Exam Page 4 45day ES 401-9 form.xls

Salem RO Exam Ref needed N

N N

N N

N N

N N

N N

N N

N

• -Day Version

6.

BlMIN NRC B

M B

M N

B N

0 B

N N

N M

B ES 401 -9 form Stem Focus X

?

X X

X X

Cues TIF X

Cred.

Dist.

A C

A B Ainutia X

1. I unit!

E E

S Q43 Delete the last zero in "100.0%" - possible cue. What is the correct answer C or D as per the key? " A does not appear to be plausible. Why would a high level condition cause a locked in low level alarm? WiN exprain linkage Q44 Is it reasonable for operators to memorize power supplie:

to HSD panel indications??? Narrow range SIG levels not plausible for HSD panel. - Change to PZR pressure - Will revise as discussed Q45 Editorial change to stem - remove "will".

S S

s Q48 Q49 Need better explanation of answerIdistractors. Could " A be correct? A is not correct - Added to answer explanatior m n E

E S

S S

Q51 For distractor C 11SGFP "trips" or loses steam supply. C is incorrect because SGFP will not trip. Lo lo level vs SG leve shrink? Revise distracter "C" to read: "The 11 SGFP will trip and the MDAFW pumps will auto start. The TDAFW pump will start if S/G levels drop to the lo lo level setpoint." Revise stem to make power level higher. No need to chnage C Q52 Capitalize "ONE MINUTE. " 6 - change "interlock" to "isolation" and add "SGFPs will not trip". If " A was correct -

then "B" is a subset answer. " C and "D" not related to feedwater - does not test the WA? almost a TIF question.

See proposed revision. Will make revision Q53 Typo in A & B "following." A & B - Why would some one think that overcooling is a concern when an AFW pump fails tt start? Q is more about operator action than AFW system response? Agree - " A and "B: are not credible distracters Expained why A and B were plausible. Has to do with sih specific orientation of AFW system' Q54 Why does "MODE 11"" have an "*"? Single bus undervoltage Q55 Does the last sentence of the explanation make sense

??? Yes - to me.

2. LOD (1-5) -

3 -

3.5 4.5 3

3 2

2.5 -

2.5 3.5 -

3 2.5 2

3 3

3. Psychometric Flaws

4. Job Content Flaws I

5. Other 4

8. Ex lanation Partial -

Job-Link -

Q=WA Y

Back-ward Y

Y Y

Y Y

Y Y

Y Y

Y Y

Y Y

RO only N -

N N

N N

N N

N N

N N

N N

N Q46 Typo - remove first "will." Distractor C is vague and need:

a reason to be parallel with other choices. Why would D be wrong? The stem implies that the failure of 21 RH4 to close is addressed in the EOPs in the context of containment spray.

Salem SRO Exam Page 5 45day ES 401 -9 form.xls

1 i

C w

w m h

Salem RO Exam 45-Day Version S

E E

ES 401 -9 form Q70 KA seems to be a stretch as the Q deals with a mode 6 condition vs something directly related to refueling. Does not match the WA selected in the sample plan. 2.2.26 was not selected? This question is matched to 2.1.40. OK Q71 Seems like an SRO level Q. Combination of A, B and C together makes A, B and C not plausible. All Tech Specs are on associated equipment and are not required to be invoked (cascaded) under 3.0.4. Can fix by replacing C OK to replace C - not SRO level - this concept is important for ROs to know.

Q72 A is subtle "..MUST be made in person." WA is not correct -the new WA is different but still fits. Is this RO level of knowledge? Has a valid learning objective for RADCON E003? RADCON is a ILT objective - not a RAD Worker objective. ROs are responsible. Revise "A" Q73 Is B a conservative action that could be done and would be correct? This is not required knowledge from memory.

How would someone know this unless they were experienced in this eiiolution? Does not test learnina obiective and KIA is 68 F

69 H

70 F

71 F

72 F

73 F

74 F

75 H

'otal 7:

U S

S S

s

3. Psychometric Flaws

4. Job Content Flaws

5. Other

2. LOD

-~

broad. No reference provided. Will replace Q74 Seems like a stretch for the KA - maybe?? Agree -

radiation releases means controlled releases. Also, knowledge of tech spec bases has traditionally been reserved for SRO level. Will get back to us Question is considered appropriate for ROs Q75 Q76 Q is about DG aligmentlresponse vs knowledge of a fire protection procedures. Need to discuss with licensee? Tests the fire emergency bypass swtiches - fits K/A Q77 Possibly an SRO level Q. - Disagree - ROs actually make this adjustment OK as is F

35 46.7%

H 40 53.3%

100.0%

U B

Bank=

28%

E M Modified=

11%

S N

New=

57%

NRC Exams=

4%

6.

BlMlN NRC -

N N

N N

B N

M N

75 21 8

43 3

75

8. Ex D lanation U L S I Salem SRO Exam Page 7 45day ES 401-9 form.xls

?

WF m

8

_I I

N d

m s I

I X I Z

Z Z

Z z

z t

t t

z t

t t

t 0

m I

l l

d d

d d

N m

I I

I I -

~-~

I I

LL LL U

I 5 e

?

r 0

d v)

W

E e 9.-

0 rk v) w C

L? 8 c

6 x ;

E W

0 cc v)

E W

m v) 2 z

cdp m

z r

0

~

8 a

0 a

m I 0 I m

II E

.2 c?

7 s v)

W E

v)

N m-0 a

a n

0 E

2 W

Salem SRO Exam Q# -

20 -

21 -

22 -

23 24 -

25 Tota F

45-Day Draft Version

6.

BIMIN ES 401 -9 form

7.

UIEIS dinutia X

1

1. I unit:

0

2. LOD (1-5) -

2 -

2 -

2 -

1 -

2 -

3 2.8 48.0%

52.0%

5. Other

3. Psychometric Flaws

4. Job Content Flaws -

Back-ward -

Stem Focus -

Q=WA I SRO only Y 1 Y?

Q21 While I think that "C is the best answer, how do you defend "C" as being the only correct answer using the referenced procedure? Is this really SRO level of knowledge' N

I S?

lTry to make "C" tighter - reference another procedure?

I 1Q22 Distractor analysis does not correspond with the Q. "C" is not plausible -change to "Director of Work Management" N

E Will make change as requested Q23 Where does the value 4775 Mrem come from to make it plausible? Change ankle X-ray to chest X-ray - whole body exposure vs extremity exposure Consider adding 50 Mrem exposure to right forearm or some other 50 Mrem exposure that does not count to make " A plausible. Will make changes as noted to stem and to "A" Q24 Q assumes that the worker will enter the RCA. Instead c nuclear worker use a primary NPO - or describe path of worker in stem (i.e. entered PA, entered RCA, departed RCA, departed PA.) Pretty simplistic - only requires knowing location of monitors, nothing about what the monitors willlwill not detect. Not SRO level of knowledge - replace question Q25 Typo - two periods in choice B. Also, in B, what is "Loss Fire Protection"? Why is "B:" plausible? Choice D states "Notification" whereas answer explanation states " A R - does E

U Will replace question H -

H -

F this matter? Verify title of the superintendent - change AR 25 Sum 25 5

3 UNSAT F

25 12 13 0

3 0

U E

S B

Bank=

M Modified =

3 9

Enhancement required N

New=

17 12 SATISFACTORY Total 25 24 Total Graded 0

1 Number not graded 12.5%

%UNSAT 100.0%

Salem SRO Exam 3

Salem ES 401-9 form 45 day review.xls

b 5

0 m:

a V

v)l V

I U

4

l.

9 cu I

I r

a, m

2%

CO 0

0.

s m

?

" 2 m

P i -

mlv)l m m m

m m

X I I

U I

Y 0

f o r 7 7 v:

YII X

m m

I

=I

z ws m

Y -

F

' 9 %

9 C

L t+

NI N/

• )/

1 I

c 03 0

0 hl

0).

2

SALEM SRO ES 401-9 FINAL SRO EXAM

6.

7.

BIMIN UIEIS

3. Psychometric Flaws

4. Job Content Flaws

5. Other

2. LOD

1.

Q#

LOK Q=WA SROonly (FIH)

Stem Job-Back-Link Minutia #I units ward Cues T/F E.::;

Partial Focus

7. Explanation Previous Q25 Typo -two periods in choice B. Also, in B, what is "Loss Fire Protection"? Why is "B:" plausible? Choice D states "Notification" whereas answer explanation states " A R -

does this matter? Verify title of the superintendent -

24 F

2.5 Y

Y M

S change AR to notification DONE Previous Q26 Could any of the conditions trigger an Alert based upon ED judgement? Are SROs required to memorize the EALs? Providing EALs - note EALs in reference material required. Applicants are told NOT to use ED 25 F

3 Y

Y N

S judgment when making EAL calls - DONE Total 25 3.0 1

0 0

0 0

0 0

0 0

0 1

25 25 Sum F

8 32.0%

H 17 68.0%

100.0%

5 0

UNSAT U

B Bank=

E M

Modified =

3 0

Enhancement required S

N New=

16 25 SATISFACTORY NRC NRC=

1 25 Total Graded Total 25 0

Number not graded 0

0.0%

% UNSAT FINAL 9/4/2008 Page 4

Salem RO Exam RO ES 401-9

6.

BlMlN NRC FINAL Version I

7.

Y units Q=WA Back-ward Y

Y Y

Y Y

Y Y

Y S

S S

S SRO only Possibly replace B with High PR trip. Good question but it does not test knowledge of the bank step counters. Change the KIA to match the question. Good question - step counters are tested in another question. Did not change B? B is considered plausible after further discussion -

high power range trip would be a second correct answer Capitalize "one minute" in stem. Choice A - Is 45% too close to 33%. Give actual parameter values for "8" and "D". Will lower seal injection by 2 gpm. Will lower HX CCW flow b 71 -TED 45% is sufficiently high to discriminate from 33%. Revised B and D as requested - OK Do ROs have to know how to make procedure transitions?

Will get back with answer Yes - ROs are required to know prodcedure transitions Why is " B plausible - why would someone think injection sources should be isolated during a LBLOCA? This questia was used on the last 2 NRC exams so it should be OK N

S B

S I

UlElS 1

8. Explanation I

I Move "RHR HX Bypass" to A. Better yet - rearrange stem ar choices so that both valves are in the stem - or both valves are in the choices - but not split. This tests knowledge of a subsequent action in an attachment to an AOP. Is this fair?

Should the applicant know this based on system knowledge alone? Will verify this is RO level of knowledge - had a high miss rate by ROs during validation. If used -will rearrange distracters to clarify Substantially revised A -why plausible in mode 1 (RHR HX)? Replace "A" - add "by 50% without exceeding limits to last sentence in stem. Revised as directed Controllers and positioners not in Q - But isn't this implied?

Can we get controllers or positioners into the stem? Draft KIA match analysis completed - accept as is N

N N

NRC FINAL 9/3/2008 Page 1

I C

.- e 3 a

z

-I U

9 r

s v)

W 0

[II 6

E W

0 CC v) sy 4-O - I U

C U

t i U

F r

v:

z 2

F 7

r

• I N

a z

U

C L?

a z

9 J

U.

?

r 0

Tt v)

W E

!?

W 2

E v) 2 zl v:

a z

T-r\\l N

m v:

v)l 2

z U.

Salem RO Exam RO ES 401 -9

7.

UlElS FINAL Version

8. Exolanation Of the choices A is very obvious. Other distractors don't seen to be very discriminating. "8" and "D" are not plausible Replace Reselected WA after spending an inordinate amount of time trying to draft a question -replaced question New question - revise c and d to read "Has been S

S S

S S

established" vice occurrin. Chan e made as re uested Delete dashes in 2nd set of bullet.(i.e., "-10 F" indication of superheat????) Reword stem to ask for the alignment that meets minimum heat removal while minimizing RWST depletion. Revise distracters C and D Revisedas I direcfed - state reference rovided in stem - DONE Verify correct answer in TRIP-I vice LOSC-1/2 (E2) OK as is This is different from what would be predicted by the pump curves - bypasss flow is higher. Is this a fair question?

Licensee retained this question Interesting but not operationally oriented. "D" is not plausible for Mode 6. Is this a fair question? Change D to RCDT -this is considered fair Choice A is (over)conservative??? It is the AL3 action statement - therefore is A plausible. Do you hold your ROs responsible for this level of knowledge? Will get back to us Licensee retained this question What is C7? = Letdown isolations valve S

S Reference needed for this question??? C is too ambiguous -

would expect RCS boron to increase a little. Little =

"approximately the same'? The distracter analysis says no reference is provided? The stem points to the exact reference. 1. Do they need a reference? 2. Distracter D is a correct statement - make it wrong. Do the applicants have to know cold shutdown boron concentration at EOL from memory? Learning objective is from LOR? Consider providing the reference and ask for exact change? Or ask what happens and why. Remove reference in stem - DONE provided as a reference. Does this include the SI pump curve which makes this question pretty easy. SI Pump curve not provided in Q37 Resolve LoK - discussed LoK -

convinced me that this is H. Not at either end of the pump curve - have to interpolate mentally using pump curvelaws :

FINAL 9/3/2008 Page 4

L C

2 5

0 ui z

d U

m m

~

L LI ZI ZI I

I I

I x/

x/

c W

0 0

Q!

6

Salem RO Exam RO ES 401 -9 FINAL Version Q#

39 40 41 42 43 45 47 48 49 50 51

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

BIMIN

2. LOD LOK (FIH)

NRC

7.

U/E/S

8. Explanation Stem Cues T/F :E;'..

Partial Job-Link Minutia #/ units Back-ward Focus Q=WA SRO only Q40 Typo - desinged vs designed. What does PZR low pressure have to do the power density limits? Can you replace "D" with high flux (high power)? Replaced C and D Why are they plausible? Discussed at length - allowed tc Q41 Why is a 1 second inadvertent SI chosen as the stem condition? Is there anything important about 1 second?

F 2.5 Y

N M

S remain as is. DONE H

3 Y

N N

S Removed 1 second H

3 Y

N B

S Q42 Q43 Delete the last zero in "100.0%" - possible cue. What is the correct answer C or D as per the key? " A does not appear to be plausible. Why would a high level condition cause a locked in low level alarm? Will explain linkage -

Q44 Is it reasonable for operators to memorize power supplie to HSD panel indications??? Will revise - Revised - answer explanation states CCW hdr pressure is not present in H

3.5 Y

N M

S noted in answer explanation F

4.5 Y

N M

S the CR? True statement 4

4 H

3 Y

N M

S Q45 Editorial change to stem - remove "will".

Q46 Typo - remove first "will." Distractor C is vague and needs a reason to be parallel with other choices. Why would D be wrong? The stem implies that the failure of 21RH4 to close is addressed in the EOPs in the context of containment H

3 Y

N N

S spray. Will this be confusing? No leave as is 4

6 F

2 Y

N B

S Q47 F

2.5 Y

N N

S Q48 Q49 Need better explanation of answer/distractors. Could "A be correct? A is not correct - Added to answer F

2.5 Y

N B

S explanation H

3.5 Y

N B

S Q50 Q51 For distractor C 1 ISGFP "trips" or loses steam supply. (

is incorrect because SGFP will not trip. Lo lo level vs SG levc shrink? Revise distracter "C" to read: "The 11 SGFP will trip and the MDAFW pumps will auto start. The TDAFW pump will start if SIG levels drop to the lo lo level setpoint." Revise stem to make power level higher. No need to change C -

Q52 Capitalize "ONE MINUTE". " 6 - change "interlock" to "isolation" and add "SGFPs will not trip". If " A was correct -

then " 6 is a subset answer. "C" and "D" not related to feedwater - does not test the WA? almost a T/F question.

See proposed revision. Will make revision - add "SGFPs H

3 Y

N N

S discussed H

2.5 Y

N N

S will not trip" to 8 DONE FINAL 9/3/2008 Page 6

c t? 9 1

2 LL 9 s -

v, w

0 0:

5 W

0 Cf E

a, -

d IC U

v,l U

>- I

+

I

+

I P

C

Salem RO Exam

4. Job Content Flaws RO ES 401 -9

5. Other FINAL Version

6.

BlMlN NRC I

7.

3. Psychometric Flaws N

UIEIS I

8. Explanation Q64 Is there a discernable difference between 80 and 85#?

Testing knowledge of setpoint only. At 80psig - the operators have direction to trip the reactor. This is the pressure where valves start moving around. They expect their operators to know this Q65 Why is B plausible if there are no flow indications on S

SRO only -

N N

N N

N N

N N

N S

2RP5? Agree OK as is - no change required - discussed why B is plausible Q66 A is not plausible. Is D testing minutia? Dis a tech spec - we expect them to know. A - replace distracter -

62 63 64 65 66 67 F

2.5 F

3 H

3 F

4 F

2 H

4.5 memory? Is this KA appropriate for a written exam - making verbal reports? This is reallyjust reading a written report.

Either change the KIA or replace the question New question - need to state IAW [procedure] in stem. Need to reference procedure and provide copy to us. Reference doe not provide solid clarity. Discussed at length with the licensee. Agreed to accept as is based on their insistance that they hold their people accountable for B

NRC N

NRC S

(replacedA lU6l c; apitalize KtUUIKtU I mportant to mow trom S

this knowledge S

Q68 Q69 SRO level question. Providing a reference cues the N

N applicants to correct answer. Willget back - Indicate reference is provided in stem without saying what S

reference is. Corrected FINAL S

S S

9/3/2008 of refueling administrative requirements - OK Q71 Seems like an SRO level Q. Combination of A, B and C together makes A, Band C not plausible. All Tech Specs are on associated equipment and are not required to be invoked (cascaded) under 3.0.4.

Can fix by replacing C OK - replacc C - not SRO level - this concept is important for ROs to know.

Q72 A is subtle..MUST be made in person. WA is not correct -the new WA is different but still fits. Is this RO level of knowledge? Has a valid learning objective for RADCON E003? RADCON is a ILT objective - not a RAD Worker objective. ROs are responsible. Revise A OK Page 8

Salem RO Exam

3. Psychometric Flaws RO ES 401-9

4. Job Content Flaws

5. Other FINAL Version Partial

2. LOD (1-5) -

3 -

2.5 2.5 -

3 3.5 -

2.E t'?;

o c

Stem Focus -

TIF 0

a Cred.

Dist.

a 35 46.7% 40-50%

40 53.3% 50-60%

75 100.0%

dinutia #/ unit!

o c

2ues -

S S s S

S c

required to memorize) - made changes as requested to A Q74 Seems like a stretch for the KA - maybe?? Agree -

radiation releases means controlled releases. Also, knowledge of tech spec bases has traditionally been reserved for SRO level. Will get back to us Question is considered appropriate for ROs Q75 Q76 Q is about DG aligmentlresponse vs knowledge of a fire protection procedures. Need to discuss with licensee? Test:

the fire emergency bypass swtiches - fits K/A U I I Possibly an SKU level a. - Disagree - ROs actually make this adjustment lndicate reference is provided DONE

6.

BlMlN NRC -

B B

N M

N 75 -

U B

Bank=

22 E

M Modified=

8 S

N New=

42 NRC Exams=

3 75 UlElS Q73 Is 6 a conservative action that could be done and would be correct? This is not required knowledge from memory.

How would someone know this unless they were experienced in this evolution? Does not test learning objective and KIA is broad. No reference provided. Will replace - New question:

Remove DAC-hrs to ALI to dose thumb rule - even NRC inspectors have to memorize this thumb rule - change A to 412.5 - unlikely that applicant will forget to include TEDE dose

- instead add 3-hr TEDE dose with 5-hr CEDE dose Agreed to allow DAC-ALI thumb rule to remain in question (not FINAL 9/3/2008 Page 9

Comments on 75-day Sample Plan Submittal:

Note - the last NRC exam was held December 11-18, 2006. The class consisted entirely of SROs.

ADAMs did not have the final ES 301-1/2 forms, only the draft forms. There may not have been anychanges?

1. Admin JPMs:

0 1 potential SRO repeat from 2007 exam - Emergency Plan - classification. Will this JPM be performed at the end of scenarios or separately? ANS: JPM will be performed at the end of each scenario.

1 potential RO repeat from 2007 exam - Calculate SDM (vs. review SDM) - not noted as a repeat JPM on sample plan? ANS: Will review.

0

2. Simulator JPMs 0

RO JPM h appears to be very similar to 2007 exam JPM h - reads Start 21 CCW pump IAW APPX-1 vs. Perform CCW pump restoration IAW APPX-1. This JPM is not listed as a repeat on this exam - but was listed as a repeat on the 2007 exam.

ANS: Will review.

JPM e on the sample plan (Perform CFCU operability and service water flow verification) is listed as an alternate path JPM for SROs but not an alternate path JPM for ROs. What is the difference? ANS: The alternate path drives the SRO into making a tech spec determination.

0

3. Simulator Scenarios:

0 The licensee is counting the component failures that occur after the major plant transient on form ES 301-5. ES 301 (page 17) states:

The required instrument and component failures should normally be completed before starting the major transient; those that are initiated after the major transient should be carefully reviewed because they may require little applicant action and provide little insight regarding their perfor man ce.

These events may be counted as component failures if they are carefully reviewed during the exam validation week.

0 The licensee is counting the same normal evolution for reactivity and normal events on the ES 301-5 form. This is not allowable per ES 301 (page 17):

With the exception of the SRO TS evaluations, each event should only be counted once per applicant; for example, a power change can be counted as a normal evolution or as a reactivity manipulation and, similarly, a component failure that immediately results in a major transient counts as one or the other, but not both.

0 The licensee is double counting events - a single component failure is counted for both the RO and PO.

0 After re-evaluating the total number of scenario requirements, the scenario set for RO-2(4) will not get enough component failure unless we count component failures that occur after the major plant transient. All other minimum requirements are met - but some have no margin if the another applicant gets the bean.

0 ESGl(SGTR)

o The RO gets a reactivity manipulation; the PO gets a normal event.

o Will need to look at PZR level channel failure to see if both RO and PO can get credit for one event. This is likely not to happen.

ESG-2 (LBLOCA - low power scenario) o The RO gets a reactivity manipulation; the PO gets a normal event o We will need to look at the sequence of events on the reactor trip. If the reactor is tripped manually due to a loss of auto and manual rod control, then the major transient initiates coincident with the trip and therefore - the MDAFW pump trip occurs after the major transient.

o The component failures that are inserted after the major malfunction (LBLOCA) are generally considered part of the major transient, not as separate component failures.

o The RO gets a reactivity manipulation; the PO gets a normal event o The SGFP oil leak cannot count as both a reactivity event and a component failure - best fits a component failure.

o The RO and PO cannot get "credit" for events 1 and 2 - one bean for each applicant only.

o The feedwater discharge header rupture with MSLl failure -will reduce pressure and level in the S/G -why does the S/G code safety open? Does this occur when pressure is very low?

o Will need to look at malfunction 5 (loss of a single 4KV group bus fails to transfer) in the scenario to see if it truly qualifies as a component failure.

o The sequence of events 2 and 3 needs to be clarified. This appears to be a single component failure and would be credited to either the PO or RO. I can agree to giving the RO credit for the reactivity event and the PO credit for a com ponent fai I ure.

o The loss of the 4'h circulator is a component failure, not a major event.

o The LOCA is the major event, not a component failure.

o The RO and PO cannot get credit for both a component and reactivity event for the event 2. They cannot get credit for another reactivity event in event 3.

ESG-5 (Steam line rupture - Spare) o Cannot credit both the RO and PO for the event 2 and event 3.

o Event 5 is part of the major transient event - not a stand alone component failure. Will need to evaluate in the simulator.

0 0

ESG-3 (Feedwater Rupture)

ESG-4 (IS LOCA)

Salem NRC Initial Written Exam Comments: 7/30/08

1.

2.

3.

4.

5.

6.

7.

8.

The Sample plan ES 401-2 forms did not include question numbers in the extreme right-hand column. Can the written exam sample plan be provided with question numbers in the extreme right hand column? ANS: Will provide. The numbers provided do not match up with the question numbers on the test questions. Need to fix. SRO Exam -

missing record number 26 in ES 401-112 sample plan. RO Exam - Sample plan has correct record numbers Question numbering had a gap in each exam. SRO Test question #I6 is missing from the exam (goes from 15 to 17). RO question #39 was also skipped (goes from 38 to 40) and RO question #60 (numbering goes from 59 to 61). This problem needs to be corrected for the final exam. Still not fixed! SRO Exam - missing record number 26 in ES 401-1/2 sample plan. RO Exam - Sample plan has correct record numbers but record numbers do not match test question numbers. Will need a correlation to be made.

The test questions did not cite references to the level of detail that would have facilitated locating the information. Generally, the reference section cited the applicable procedure but did not include the section or page of the reference. Some references cited in the questions were not included on the reference CD-ROM disk. Need to provide these references. Noted - Salem did not correct this problem. They only provided additional references when references were specifically requested.

The questions did not include a plausibility analysis (not required but helpful). Do you want to prepare this or shall we? They did not prepare -we reviewed each question.

This effort added many hours (-20) to the exam review.

Can the written exam be provided in Word or PDF format for docketing and upload to INPO? ANS: No this is too difficult. We will scan the hard copy.

Written exam references -when references are provided, the question should not do tell the applicant what reference is appropriate to answer the question. Just indicate that a reference is provided. Some questions that require references do not state that references are provided. NOT CORRECTED The ES 401-6 form needs to be filled out for each exam (i.e. 1 form for the RO exam, 1 form for the SRO exam). The 1

blank space in blocks 6 and 7 are for the number of questions. The second blank space is for the percentage. DONE ES 401 -6 block 5 states that the audit exam was developed systematically and randomly. Some WAS were suppressed on the bases that the WA was on the audit exam. This is not necessary if the exams were developed using a systematic and random sample process. Do we have a document that states how it was developed?

Both exam were developed using the WOG Random K/A generator

9.

The submittal did not include a description of the sample plan selection methodology (PWROG Random Sample Pian Generator - p version). DONE

10. For those questions that were modified, no copies of the original bank questions were provided for comparison. NOT DONE

Salem Initial Exam - Operating Test Comments:

Simulator JPMs:

General Comments: The JPMs look good overall. We do not need to replace any JPMs based on our review. JPM E may not be adequately discriminating so we need to look at that one in the simulator.

1. Please consider establishing specific and objective task standards for operator performance.

NUREG 1021 Appendix D states: "The JPM must clearly identifL the task standard (Le., the predetermined qualitative and/or quantitative outcome) against which task performance will be measured. Every procedural step that the examinee must perform correctly (i.e.,

accurately, in the proper sequence, and at the proper time) in order to accomplish the task standard shall be identified as a critical step and shall have an associated performance s t a n d a rd. "

2. Please ensure that the cue sheets match the 1" page of the JPM. Some examiners will read from the 1" page while the applicant is handed the cue sheet. In addition, cue sheets may not be included in the docketed test file.

NRC SIM A Respond to Main Turbine runback with malfunctioning rod control:

The JPM task standard should address when the applicant has to identify the problem with the runback circuit or the rod control circuit.

If the reactor trips, does this constitute a failure of the JPM? Would have to evaluate -

probably. Add task standard.

Does the reactor operator have responsibility for / respond to the main turbine control panel?

The applicant will respond to this panel.

Does the RO have responsibility to determine the applicable procedure? Yes Change rod failure from 8 SPM to failed completely failed. NOT DONE?

NRC SIM B: Place the CVCS make-up control in the manual mode:

The JPM task standard should address the desired completion state - i.e. "Restore VCT level using manual make up and secure alignment" Do we need to tell the applicant that the auto makeup function is INOP? Should this be diagnosed after the VCT high level alarm? Respond to the failure of the level transmitter.

Lower VCT level to auto-makeup setpoint.

Add cue to tell applicant that 61 or 63 GPM is close enough to 62 GPM.

NRC SIM C: TCAF failed open pressurizer spray valve:

The task standard should identify criteria for pass/fail. E.g. - manually trip the reactor prior to 2000 psig? What if SI initiation occurs or Rx trip prior to manual trip / stopping RCPs, etc.

Criteria added in task standard - fails of does not trip before getting auto trip. Fails if does not stop RCPs before SI initiates.

Should we insert auto trip failure (Rx trip required if PZR pressure evaluate how fast pressure drops in simulator. Trips on OTDP at around 2000 psig. If the applicant takes an auto trip - failure occurs. Verjv in simulator.

2000 psig)? Need to The applicants are directed to go to TRIP-I when the reactor is manually tripped. Should they be cued to continue on in AB.PZR-OOI? Yes - this is clear in the AB.

NRC SIM D: Swap operating RHR loops with one loop aligned for ECCS injection:

Task standard - make specific to JPM. DONE Evaluate time to warm-up 22 RHR loop in step 5.6.2.G. Consider using time compression cue once applicant opens 22RH12. Not required.

NRC SIM E: Perform a CFCU operability and service water flow verification:

Make task standard specific to task. There are RO and SRO versions of this JPM. DONE Need to evaluate the degree of difficulty when on site - may be too simplistic. Need to add system number to WA. Will add 022A4.05 - Safety function 5 Typo in step C.6 for both RO and SRO Delete refer to TSAS in initiating cue for SRO Make changes to initial conditions - delete for post maintenance run and replace with for monthly test. Ensure 25 CFCU is added to handout page. DONE NRC SIM F: Failure of a 2C 4KV Vital Bus to transfer to alternate source:

Make task standard specific to task. DONE Add note explaining step 3.3 in procedure (charging pump running) DONE NRC SIM G: Calculate RCS Subcooling during performance of LOCA-I:

Make task standard specific to task. DONE Change this to an ADMIN JPM - run in simulator in freeze. OK - Make RO ADMIN A2 WA cited is for SBLOCA which correlates to SF3 not SF7. Select SF7 K/A on title page. Will do Change to 2.2-44 The JPM does not allow simulator to run dynamically - containment pressure prevented from increasing. No switch manipulation. More like an ADMIN JPM. Recommend replace JPM.

Swapped this JPM to become ADMlN JPM A-2. Replaced with ADMIN JPM A-2 which was more appropriate as a dynamic simulator JPM.

NRC JPM H: Perform actions for CCW restoration (21 CCWE Pump):

Make task standard specific to task. DONE

Verify critical steps -why are steps 5.d, 5.e, and 6 not critical? Steps 5d, e are there to unload the EDG - if not completed, will not prevent starting CCW pump in simulator.

Step 6 is not required to meet task standard - plus it is a not performed step (dispatch operator). OK Note - there are 2 steps labeled as 5.d on page 5 of the JPM. This mirrors what is in 2-EOP-APPX-1 (typo).

NRC In-Plant JPM i: Locally isolate seal injection during a LOPA Make task standard specific to task. DONE Make this a Unit 1 JPM vice Unit 2. Will change - DONE WA cited is EPE 055 -which is correlated with SF-6 not SF-2. Find new WA that correlates to SF-2. Will do - new MA Changed to 003K6.02 This task would require entering a contaminated area. The JPM presumes that the applicant will not actually enter the area. But the applicant should discuss the necessary radiological precautions that would actually be taken to enter the area. Do they need a special RWP? If so

- which one? Requirements for protective clothing? Are these requirements different for performance of an EOP step? Will add discussion. There are no standard or expectations for this action - therefore - no critical task. This will be a post-exam comment.

NRC In-Plant JPM j: TCAF Control Evacuation - Start 23 AFWP and feed SGs.

Make task standard specific to task. DONE WA for APE 068 is correlated to SF8 not SF4 (sec). Find new WA or draft WA match analysis.

Specify valve locations in JPM. Will revise to 061K7.01 DONE Add examiner note regarding entering alarmed panels. DONE Consider changing this JPM from Unit 2 to Unit 1. All in-plant JPMs were written on Unit 2. NOT DONE NRC In-Plant JPM k:

Make task standard specific to task. DONE Designate valve locations for steps 5.3.3, 4, 5 and 5.5.4. DONE Step 5.3.8 - valve 21 SW472 - task standard says locked closed. Should be locked open -

Correct JPM step standard - DONE Change flow < 100 gpm - change in 2 places. 5.5.4 and 5.5.6 (Meter ranges 0-1 00 gpm).

Change to 80 gpm in step 5 5. 4 and 95 gpm in step 5.5.6 - DONE ADMIN JPMs:

NRC RO ADMIN A: Calculate Shutdown Margin Make task standard specific to task.

Add + or - acceptance criteria to calculations on page 4. Will add

1. Need revised ES 301 -5 forms showing applicant rotation and number of malfunctions. This form was provided but revised D-I forms were not included

2. Did you respond to the comments on the 75-day outline? Discussed at validation week Scenario #I Changes:

0 0

0 0

Page 12 - address tech spec AS 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> if they do not get to AL3 hour shutdown.

How do we grade this if they miss the 2-hour shutdown? Not addressed?

Page 13 1. Specify actions for isolation of 23 SG. 2. Add examiner note for variations between SEC B or C. DONE Pages 13 and 15 Add list of valves to be checked for S/G isolation. Added to page 13 but not to page 15 (now 16)

Page 16 Add examiner note: If S/G safety valve lifts -the task is not a critical task. If the crew exceeds the 50 minute FSAR time requirement to isolate the S/G - not critical.

The SRO should direct feeding 21, 22 and 24 S/G - not just 24 S/G. DONE Scenario #2:

0 0

Provide crew with IOP-3 marked up to step 5.4.1. Not done?

Page 11 - note that MSLl may be required or have occurred. DONE Scenario #3:

0 Noted 24 S/G steamflow = 0 but CRT and computer shows 24 S/G = flow - corrected problem - problem was in A 0 with reversed inputs. Fixed problem. Potential simulator fidelity issue Page 11 -Add a step to trip the condensate pumps DONE Page 12 - List valves for verifying 24 SG is isolated. DONE 0

Scenario #4:

0 0

0 0

0 Page 8 -Add several actions (reset SEC, 2CC 131 open) - add note about NOT closing 2SW26 - enter AB.SW0002.

Page 10 - Add Implements attachment 3 - delete Znd entry for rapid boration Critical tasks - clarify minimum complement CT#I - 2 supply and 1 exhaust Took out CT CT#2 - 1 CCW pump Consider shortening scenario by putting the leak in earlier. Ramp back leakage-no need - put in full amount.

Scenario #5 0

0 Page 12 - critical task 1 - state the task standard clearly - how many ESW pumps -

how do we evaluate this CT? DONE Page 13 - Note - add in 2SW-26 to note that says 23SW20 should be closed. - NOT DONE?

Missing SRO ADMlNs A-5s (1-4) classification JPMs from package. Please submit ASAP.

(Called/emailed Gerry Gauding 8/8 -will FEDEX to Regional Office.

RO ADMIN AI-I The answer key was not attached to the JPM. Placed answer key from 45-day package in final exam.

RO ADMIN AI-2 Missing answer key screen shots. Note was added. Will send RO ADMIN A-2 Should we add an error band to the answer? The original error band was + or - 2 degrees. No need - tolerance is 0 - getting numbers off fables RO ADMIN A-4 Added steps to fax NRC data sheet and respond to an in-coming phone call from CNN. Need to assess grading criteria for these steps.

Proposal - The steps should be broken down into 2 separate JPM steps. Each step should reference the applicable step in the procedure.

0 FAX form - Task standard should be locate the correct FAX machine and indicate which button will cause the FAX machine to dial out. This should be a critical step using the same logic as previous steps. ECG attachment 8 step A.7 Obtain completed NRC Data Sheet from the primary communicator and FAX for to Group B.

Respond to CNN call - reference step ECG attachment 8 C.3 include:

0 o CAUTION Communicators are NOT authorized to release any information to the News Media.

o When contacted by any News Media Representative, READ the appropriate message below: Reading A is correct. Reading B will be sufficient for this step -

depending if the ENC is activated (information not provided - should it be provided?) If the applicant should not provide any other relevant information regarding the event. This step should be critical?

Simulator JPMs SIM A NRC had a comment to evaluate failing all rod motion rather than failing rods to 8 steps per minute. We asked you to evaluate this comment. There is no change to the JPM.. We remain concerned that asking the applicant to assess that the rods are stepping in - but not fast enough - during a one person simulator scenario is VERY challenging. Do you think this is a fair challenge to your applicants? The simulator does not have the capability to fail auto rod motion. They can fail rod motion completely - auto and manual-but not one or the other. They expect operators to identify the slow speed of the rods. This is considered a fair JPM.

SIM B OK SIM C OK SIM D

OK - challenging - schedule for 45 minutes SIM E - RO / SRO versions Correct typo in step C.6 page 5. Test results [?I by initialing the SAT... NOT NEEDED SRO version has a mismatch between applicant handout sheet and page 2 initiating cue.

Difference is whether the initiating cue states If required, indentify entry into TSAS Delete this from page 2 - leading.

SIM F Initial conditions do not match between page 2 and handout sheet. Missing item 1 SIM G OK SIM H ES 301-2 for the ROISRO-Is - JPM I is matched to SF 2 for the ROs but SF 4(pri) for SROs?

Same JPM - probably 4(pri) is correct?

In-Plant I This task would require entering a contaminated area. The JPM presumes that the applicant will not actually enter the area. But the applicant should discuss the necessary radiological precautions that would actually be taken to enter the area. Do they need a special RWP? If so

- which one? Requirements for protective clothing? Are these requirements different for performance of an EOP step? Will add discussion. There are no standard or expectations for this action - therefore - no critical task. Removed cue eliciting discussion. This will be a post-exam comment Gerry will check further but will not compromise exam security.

This JPM should also be coded as R on the ES 301-2 form. It is inside the RCA. Yes - will change In-Plant J Typo in Task page 1 - AFWPp Also on page 2 and applicant handout. Is this a typo? No -

the small p means pumps This JPM was not changed from Unit 2 to Unit 1. Why not? We walked this down on Unit 1.

We said that if the only changes were to change unit designations, then we would make the change. What happened? Before we left the site - we said dont bother In-Plant K OK Simulator Scenarios Need revisedhpdated D-I forms for all scenarios - incorporate 75-day scenario comments on all D1 forms.

Need to agree on crew rotation.

ES-301-5 form does not have revised numbers from the D-I forms (bean count). You are counting on post-major transient instrumentkomponent malfunctions to meet minimums.

0 0

RO #2 appears to be missing one scenarios worth of beans on the ES 301-5 form. The totals do not add up.

SRO(I) 1/3 appear to have 3 scenarios. SRO(I) 214 and RO 1 /2 have only 2 scenarios -

ESG 5 is a spare - and the total event numbers do not add up.

Resubmit corrected copy of ES 301-5 Scenario # I 0

Page 12 - address tech spec AS 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> if they do not get to AL3 hour shutdown.

How do we grade this if they miss the 2-hour shutdown? Not addressed? Will add tech spec number Pages 13 and 15 Add list of valves to be checked for S/G isolation. Added to page 13 but not to page 15 (now 16) Will add to page 76 0

Scenario #2 0

Provide crew with IOP-3 marked up to step 5.4.1 and add to turnover sheet. Not added to turnover sheet? Will add Scenario #3 OK Scenario #5 0

Page 13 - Note - add in 2SW-26 to note that says 23SW20 should be closed. - NOT DONE? Not required