ML042990553

RB-09-2004 - Draft - Written Exam Comments

ML042990553
Person / Time
Site:	River Bend
Issue date:	09/23/2004
From:	Gody A Operations Branch IV
To:	Hinnenkamp P Entergy Operations
References
50-458/04-301, ES-401, ES-401-9, NUREG-1021, Rev 9 50-458/04-301
Download: ML042990553 (6)

Text

ES-401RB 09/2004 Written ExaminationForm ES-401-9 Review Worksheet Q#1.LOK (F/H)2.LOD (1-5)3. Psychometric Flaws4. Job Content Flaws5. Other6.

U/E/S 7.Explanation StemFocusCuesT/FCred.

Dist.PartialJob-LinkMinutia#/units Back-ward Q=K/A SROOnly 1BH3-S from RB NRC exam 10/2000; 2H*3-E (S)author says LOK=F, but I think LOK=H. (S)-fixed.3H3-S 4H3-E (S)Correct grammar and punctuation errors in stem by changing to: "During a plant startup, after thegenerator has been placed on the grid, a turbine trip occurs. . . and steam line drains stabilizereactor pressure. . ." (S)-fixed.5H3X-E (S)recommend rewording distractors A and B to look more like C (ie: "A. riseas control rod insertion lowers core void content.") (S)-fixed6H3X-E (S)indicate what time the CRS gave direction; (S)-fixed7H3-S 8H3-S 9F2-S10H3-E (S)typo in stem - change to: "... the Reactor Cavity where it strikes..." (S)-fixedInstructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] 1.Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2.Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable). 3.Check the appropriate box if a psychometric flaw is identified:The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much nee dless information).The stem or distractors contain cues (i.e., clues , specific determiners, phrasing, length, etc).The answer choices are a collection of unrelated true/false statements.One or more distractors is (are) not credible.One or more distractors is (are) partially correct (e.g., if t he applicant can make unstated assumptions that are not contradicted by stem). 4.Check the appropriate box if a job content error is identified:The question is not linked to the job require ments (i.e., the question has a valid K/A but, as written, is not operational in c ontent).The question requires the recall of knowledge t hat is too specific for the closed refer ence test mode (i.e., it is not required to be known from memory).The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with quest ion in gallons).The question requires reverse logic or applic ation compared to the job requirements. 5.Check questions that are sampled for conformance with the approved K/

A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6.Based on the reviewer's judgment, is the question as written (U)nacceptable (requi ring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 7.At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attr ibutes are not being met). 33 of 34 NUREG-1021, Draft Revision 9 ES-401 2 Form ES-401-9 Q#1.LOK (F/H)2.LOD (1-5)3. Psychometric Flaws4. Job Content Flaws5. Other6.

U/E/S 7.Explanation StemFocusCuesT/FCred.

Dist.PartialJob-LinkMinutia#/units Back-ward Q=K/A SROOnly 11MF3-Sfrom RB NRC exam 2/2003;12F2-S 13MF2-S from RB NRC exam 2/2003; 14F2-S 15BF2-S from RB NRC exam 1/1993;16H2-S 17F2-S 18F2-E (S)fix typo in stem by: "...below the Power box as follows:"; (S)-fixed19F2-S 20F2-S 21MF2X-U (S)from RB NRC exam 2/2003; distractor A is not credible. (S)-distractorfixed and question converted to "modified" (was originally "bank")22H2-S 23F2-S 24BH3-S from RB NRC exam 2/2003; 25F2-S 26F2-S 27F3-S 28F3-S 29MF3-Sfrom Perry NRC exam 1994; 30MH3-S from RB NRC exam 2/2003; 31H2-S 32F2-S34 of 34NUREG-1021, Draft Revision 9 ES-401 2 Form ES-401-9 Q#1.LOK (F/H)2.LOD (1-5)3. Psychometric Flaws4. Job Content Flaws5. Other6.

U/E/S 7.Explanation StemFocusCuesT/FCred.

Dist.PartialJob-LinkMinutia#/units Back-ward Q=K/A SROOnly33H4-S 34BH2-S from RB NRC exam 7/1997; 35H2-S 36H3-S 37H3-S 38H3-S 39H3-E (S)fix typo in stem to: "Following a plant..." (S)-fixed 40BF2-E (S)from RB NRC exam 7/1997; fix typo in distractor A to: "...hand switch in the control room..." (S)-fixed41H3-S 42F3-S 43H2-S 44H3-S 45H3-S 46F3-S 47F3-S 48MH2-S from RB NRC exam 10/2000; 49F2X-U (S)three distractors can be eliminated by cues from handout. (S)-replacedwith satisfactory NEW question.

50BH3-E (S)from RB NRC exam 10/2000; fix typo in distractor B to: "Due to a..." (S)-fixed 51BH2-E (S)from PB2 NRC exam 9/2002; is "diesel engine emergency shutdown signal" a separate signalfrom energizing the"diesel stop solenoid?" Also, capitalization of "diesel/Diesel" needs to beuniform. (S)-diesel shutdown language clarified/fixed.52F2-E (S)change 'A' and 'B' to "...operates alone to maintain..." (S)-fixed53F2-S 54H3-S34 of 34NUREG-1021, Draft Revision 9 ES-401 2 Form ES-401-9 Q#1.LOK (F/H)2.LOD (1-5)3. Psychometric Flaws4. Job Content Flaws5. Other6.

U/E/S 7.Explanation StemFocusCuesT/FCred.

Dist.PartialJob-LinkMinutia#/units Back-ward Q=K/A SROOnly55F*2-E (S)author says LOK=H, but I think LOK=F. (S)-fixed56H3-S 57F2-S 58F3-S 59H*2-E (S)author says LOK=F, but I think LOK=H. (S)-fixed60F3-S 61H3-S 62MH3-S from RB NRC exam 2/1999; 63BH3-S from RB NRC exam 7/1997; 64F2-S 65BH3-S from RB NRC exam 2/2003; 66MH2X-U (S)from RB NRC exam 2/2003; distractor A is not credible; perhaps eliminating descriptive verbiage in di stractors and just use Mode-2, etc? (S)-fixed; question is now "modified" (was originally "bank")

67MF2-S from BF NRC exam 6/1995; 68BF2-S from RB NRC exam 1/1997; 69BH2XU (S)not RO license level question. (S)-replaced with "bank" question from RBNRC exam 2/2003; question was originally "new".70F2-S 71BH2-S from RB NRC exam 7/1997;72H3-S (S)from RB NRC exam 2/2003; (S)-replaced with "new" question to reducethe number of "bank" questions from previous two NRC exams.73F2-S 74BF2-S from RB NRC exam 1/1995;75H3-S34 of 34NUREG-1021, Draft Revision 9 ES-401 2 Form ES-401-9 Q#1.LOK (F/H)2.LOD (1-5)3. Psychometric Flaws4. Job Content Flaws5. Other6.

U/E/S 7.Explanation StemFocusCuesT/FCred.

Dist.PartialJob-LinkMinutia#/units Back-ward Q=K/A SROOnly 76BF2X-U (S)from RB NRC exam 2/2003; K/A listed on the Question does not match K/A listed on outline, and question does not match outline K/A. (S)-fixed77H3-XU (S)not SRO-only - more like 55.41.b.5 than 55.43. (S)-fixed78H2-XU (S)from RB NRC exam 7/1997; not SRO-only - more like 55.41.b.10 than 55.43. (S)-replaced with "new" question at SRO level (was "bank")79H3--S 80H3-XU (S)fix typo in first sentence of stem by deleting "is"; not SRO-only - more like 55.41.b.10 than 55.43. (S)-replaced with "new" SRO-level question 81MH2--S from RB NRC exam 10/2000; 82MF2--S from RB NRC exam 2/1999; 83H3--S 84H3X--E (S)are SRO's expected to know the PSP curve by memory? (S)-modified to include curves as handouts.85F3X--U (S)use something other than "-" in front of numbers in the stem so that no one can be confused as towhether it is a negative sign; distractor D seems to be also correct even though A is a betteranswer. (S)-fixed and added EP-3 as handout.86H2--S 87BH2--S from RB NRC exam 10/2000; 88F2--S 89H2--S 90F2-XU (S)from RB NRC exam 10/2000; not SRO-only - more like 55.41.b.7 or b.10 than 55.43. (S)-replaced with "new" SRO-only question91H2-XU (S)from Perry NRC exam 3/2002; not SRO-only - more like 55.41.b.10 than 55.43. (S)-replaced with "new" SRO-only question 92BH3--S (S)from RB NRC exam 2/2003; (S)- "modified" this question to reduce thenumber of "bank" questions repeated from previous NRC exams 93MH2--E (S)from FitzPatrick NRC exam 7/2003; fix typo in stem with: "...bundle is being..."; (S)-fixed 94BF2X--U (S)distractor C is not credible - try using SRO with different notification time. (S)-fixed34 of 34NUREG-1021, Draft Revision 9 ES-401 2 Form ES-401-9 Q#1.LOK (F/H)2.LOD (1-5)3. Psychometric Flaws4. Job Content Flaws5. Other6.

U/E/S 7.Explanation StemFocusCuesT/FCred.

Dist.PartialJob-LinkMinutia#/units Back-ward Q=K/A SROOnly 95MF2--S from RB NRC exam 1/1997;96F2X--E (S)change stem to "...modification must always have prior..." (eliminatespossibility of assumptions leading to multiple answers) (S)-fixed97F2-XU (S)not SRO-only - more like 55.41.b.10 than 55.43. (S)-replaced with "new" SRO-only question 98MF2--E (S)from RB NRC exam 2/2003; author says "Modified," but not sufficiently changed from parent. (S)-more changes made to make this a "modified"question (was "bank")

99MH*3--E (S)from RB NRC exam 10/2000; author says LOK=F, but I think LOK=H. (S)-fixed 100MH3-XU (S)from RB NRC exam 2/2003; Not SRO-only. (S)-changed to make itSRO-only and "modified" (was "bank").34 of 34NUREG-1021, Draft Revision 9