ML031840177

Draft - Facility Comments

ML031840177
Person / Time
Site:	Seabrook
Issue date:	06/05/2003
From:	Roy D Florida Power & Light Energy Seabrook
To:	Conte R NRC/RGN-I/DRS/OSB
Conte R
References
50-443/03-301 50-443/03-301
Download: ML031840177 (8)

Text

ES-401 Written Examination Form ES-401-9 Review Worksheet

3. Psychometric Flaw Dist.

4. Job Content Flaws Partial Job- Minutia #/ Back- Q-Link units ward 5*rI SRO U6*NS 7.

Explanation I I s 1

~~

S X U condition in stem made correct answer incorrect; modified condition stated answer was incorrect; further info brought by licensee X I i m o t p o m t ~into mmct answer Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regardingeach of the following concepts.]

I. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)iher cognitive level.

2. - - -

Enter the level of difficulty (LOD) of each question using a 1 5 (easy difficult) rating scale (questions in the 2 4 range are acceptable).

3. Check the appropriate box if a psychometricflaw is i d e n t i :

The stem lacks sufficlent focys to eIiG the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

, sp8cific determiners, phrasing, length, etc).

The stem or distractors contain cues (laclues, The answer choices are a collection of unrelated trudfalse statements.

More than one distractor is not credible.

One or more distracton is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a 'ob content error is identified:

The question is not llnked to the job requirements i.e., the question has a valid WA but, as written, is not operational in content)

The questjon requires the recall of knowled e that(is too specific for the c!osed reference test mode (i+,it ISnot T i r e d t0.b known from memory).

The questpn contains data WIUIan unrealis& level of accuracy or inconslspnt u n h (e.g., panel meter in percent w h question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check auestions that are sampled for conformance with the approved WA and those that are desianated SRO-only (WA and license level mismatches are unacceptable).

6. Based on the reviewer'sjudgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

7. At a minimum, explain any 'U" ratings (e.g., how the Appendix B e_sychometrica t t r i b u w n g met).

ES-401 2 Form ES-401-9 I. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

R# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U N S Explanation Focus Dist. Link units ward KIA Only 11 S editing; conected 12 S 13 S 14 S editorial; conected.

15 S editorial; corrected.

16 s 17 S editorial; comted 18 S

~ ~~ ~~ ~

19 X U question did not reflect realistic situation; question replaced.

20 S 21 S 22 S 23 S 24 S 25 S 26 S 27 X U distractor too close to right answer (minutia related); distractorchanged 28 S 29 S 30 S 45 of 46 NUREG-1 021, Revision 8, Supplement 1

ES-401 2 Form ES-401-9 45 of 46 NUREG-1021, Revision 8, Supplement 1

ES-401 2 Form ES-401-9 45 of 46 NUREG-I021, Revision 8, Supplement I

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO U/WS Explanation Focus Dist. Link units ward KIA Only 71 S 72 S 73 S 74 X answers required detailed knowledge of proceduresteps; refocused answr on broader topic.

75 S editing; comted edits 76 S D distractor unclear; rewonled Ddistractor 77 S 78 X distradors were requiring memorization of MCC loads; disfractors rewritten at a bmader /eve/

79 S 80 X U Distractor D could be construed as correct; rewrote distractor 81 S 82 S a3 S 84 S 85 S 86 S 87 X distractor D was coned answer; reshctumd initid conditions to make D incorrect.

88 X U distractor C was too close to correct answer; changed distractor 89 S 90 , S 45 of 46 NUREG-I021, Revision 8, Supplement 1

ES-401 2 Form ES-401-9 45 of 46 NUREG-I021, Revision 8, Supplement I

ES-401 2 Form ES-401-9 45 of 46 NUREG-I 021, Revision 8, Supplement I

Summary of Facility/NRC Comments on the Operating Examination Section A Administrative No significant comments made.

Section B Plant Walk-through JPMs JPM #I Power IncreaseTo Criticality The facility commented that the JPM was too difficult to complete in a timely manner with the desired effect. The JPMs purpose was to have the reactor operator identify a significant deviation from the expected critical position and take appropriate action. This would require several shimming intervals of the control rods. Each shim would require the operator to wait several minutes until count rate stabilized to record the power level.

The NRC agrees that the same knowledge can be tested in a more contiguous manner by having the applicant perform the I/M plot for the same startup procedure and take appropriate action when the deviation is noticed. The Power Increaseto Criticality JPM was replaced with the JPM entitled, I/M Plot During Approach to Criticality.

JPM 87 Blended Makeup Calculation and Performance The NRC noticed that this JPM selection does not conform with the guidance in NUREG 1021 section 301 D.3.a in that the same safety function was being tested twice in subcategory 8.1. The item was replaced with Service Water System / Transfer Service Water to the Cooling Tower which is not a duplicate safety function item.

Section C Simulator Scenarios No significant comments made.