ML072320326
| ML072320326 | |
| Person / Time | |
|---|---|
| Site: | Grand Gulf  |
| Issue date: | 05/30/2007 |
| From: | NRC Region 4 |
| To: | Entergy Operations |
| References | |
| 50-416/07-301, TAC X02328 | |
| Download: ML072320326 (8) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 GG-03-2007 INIT EXAM (TAC#: X02328)
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 2 X X U - answer requires knowledge of TS bases; E - answer doesnt match TS bases: FCBB < 1.0
- stem uses past tense (reactor was) and present tense (RFP trips)
The tech spec reference was removed and supporting training material references was added. This question does not require knowledge of the TS basis and is an RO question. The stem was text edited as an enhancement. This is not considered an unsatisfactory question.
2 F 3 S 3 H 3 S 4 H 2 E Consider adding initial condition of reactor water level Comment Incorporated.
5 H 3 E Change a refueling outage to an extended refueling outage Comment incorporated.
6 F 2 X U Question does not address ability to operate or monitor Question was completely rewritten.
7 H 3 X X U - Question does not address effect on component/system operation S - Stem states radial well flow is degrading, not lost as stated in explanation
- Explanation states loss of RWCU flow accentuates cooldown effects Is this true?
After further discussion with licensee, it was determined this question is satisfactory as written.
8 F 2 S Wide variation in time reqmt: 6 hrs/5 days/10 days 9M F 2 E Explanation states RHR B available but stem states it is operating Comment addressed.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
- The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
- The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
- The answer choices are a collection of unrelated true/false statements.
- One or more distractors is (are) not credible.
- One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
- The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
- The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
- The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 7. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401, Page 32 of 33
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 10 H 2 S 11 H 3 S 12 H 3 S 13 H 1 X U Direct look up AND incorrect answer Question was rewritten.
14 H 2 S ES-401, Page 33 of 33
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 15 F 1 U Direct look up with handout Question was rewritten.
16 H 2 S 17 F 2 S 18 H 2 S 19 F 2 S 20 H 2 S 21 H 3 E Consider rewording a-b-c. Example: (a) N21-F014A can be fully opened only after RFP A trip is reset by depressing the RFPT A TRIP RESET Comments were addressed.
22 H 2 E In (b), change has to be to is. Reword (d) to At 100% power, reactor water level falls to +15 inches due to a FW leak in the TB. (matches format of distractor (c).
Comments were addressed.
23 F 2 E Consider reducing DW pressure < 7 psig After further discussion with the licensee, the drywell pressure was left as is.
24 F 2 X U Question is bases, not ability to determine and/or interpret Question was rewritten.
25 F 2 S Explanation states SP temp be maximized - should be SP cooling.
This comment was not addressed because it does not have any bearing on the validity of the question and was therefore left as is.
26 H 3 S 27 H 2 X U SRO knowledge reqd to obtain correct answer (Emergency Plan)
Question distractors were revised to address this comment.
28 H 3 E Chg (d) to match (c ) except system fill in not reqd by plant procedures before manually starting the system.
Distractor d was revised.
ES-401, Page 33 of 33
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 29 H 3 E Delete small from stem Comment was addressed.
30 H 3 E Consider adding only to (b) and (c); change (d) to read must be stopped AND SSW A initiation .
Comments were addressed.
31 H 3 S 32 H 2 S 33 H 3 X U Question is knowledge, not ability to operate and/or monitor Question was rewritten.
34 H 3 S 35 H 3 S 36 F 2 X U Question is tagging process, not control of temp procedure changes Question was rewritten.
37 F 3 S 38 H 3 E Answer key needs to reflect correct answer Comment was addressed.
39 H 3 S 40 F 2 S 41 H 3 X U (b) could potentially be correct Comment was addressed.
42 H 2 S 43 H 3 X U Handout provides answer Handouts will not be provided for this question.
44 H 2 S 45 H 3 S ES-401, Page 33 of 33
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 46 H 2 S 47 H 3 S 48 H 3 S 49B F 2 S 50M H 2 S 51 H 3 S 52 H 3 S 53 H 3 S 54 H 3 E Add to after assumed Comment was addressed.
55 H 2 S 56 F 2 S 57 H 3 S 58 H 4 S 59 F 3 S 60 H 2 S 61 H 2 S 62 F 2 S 63 F 3 S ES-401, Page 33 of 33
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 64 H 3 S 65 F 3 X U Requires TS knowledge E The sources for the correct answer were revised so that they are no longer based in the technical specifications.
66 H 3 S 67 F 2 X U Requires TS knowledge E Question was revised to not require TS knowledge.
68 H 2 E Ref matl not adequate to support answer Question and answer key was revised to address this comment. No reference material was provided.
69 F 2 S 70 H 3 X U Requires TS knowledge This question was rewritten to address this comment.
71 H 2 X U Distractors require TS knowledge E This question was rewritten to address this comment. TS references in the justification were removed.
72 H 2 E Chg stem to Which ONE of the following..
This comment was addressed.
73 H 3 S 74 H 3 S 75 H 2 S 76 H 3 S 77 H 3 X U Wide range level in (d) is not a familiar nominal band - not plausible E The band is plausible in that it is based on the corresponding NR indication. The range was revised to make it more plausible.
ES-401, Page 33 of 33
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 78 H 2 X U KA requires ability to determine and/or interpret reactor water level E This question is believe to meet the KA in that it requires the SRO to interpret the TS for this condition and the RPV level must be a part of this determination. The distractors were revised so they are more consistent.
79M H 4 S 80M H 4 S 81 H 3 S 82 H 2 S 83 H 2 X X U - attachments make answer a simple look up S - could (b) be correct given advance knowledge of 12 hr repair time?
Upon review this is not considered a simple lookup in that the candidate must fit together several bits of information and knowledge to derive the correct answer.
84 H 2 S 85 H 3 S 86 H 3 X U Answer does not req knowledge of TS bases given note on TS 3.4.9 This question was rewritten.
87 H 3 S 88 H 4 S 89 H 3 S 90 F 2 X U Handout makes answer direct look up This question was rewritten.
91 H 3 X U Handout makes answer direct look up This question was rewritten.
92 H 3 S 93 H 4 S 94 F 2 S ES-401, Page 33 of 33
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 95 F 3 S 96 F 2 E Capitalize NOT This comment was incorporated.
97 H 3 S 98 F 2 S 99 H 3 S 100 F 3 S RO 2.48 2 1 0 2 0 0 0 0 0 5 6 B-1 H-54 U - 15 RO TOTALS M-2 F-21 E - 13 N-72 S - 47 SRO 2.84 0 3 0 2 0 0 0 0 0 2 0 B-0 H-19 U-6 SRO TOTALS M-2 F-6 E-1 N-23 S - 18
GENERAL COMMENT
S:
- 1. New questions are not specifically identified; it is assumed that if a question is not noted as Mod or Bank, then it is a New question.
- 2. RO answer breakdown: a-4 b - 13 c - 22 d - 18 (Q55-58 all bs)
- 3. SRO answer breakdown: a-4 b-7 c-7 d-7
NOTES:
- 1. Bank questions are indicated by B
- 2. Modified are indicated by M
- 3. comment resolution is indicated in blue; ES-401, Page 33 of 33