ML052420507
| ML052420507 | |
| Person / Time | |
|---|---|
| Site: | Comanche Peak  |
| Issue date: | 03/28/2005 |
| From: | Operations Branch IV |
| To: | TXU Power |
| References | |
| 50-445/05-301, 50-446/05-301, CP-03-2005, ES-401, ES-401-9, NUREG-1021, Draft Rev 9 50-445/05-301, 50-446/05-301 | |
| Download: ML052420507 (9) | |
Text
ES-401 CP-03-2005 Written Examination Form ES-401-9 Review Worksheet
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 2 X X - U (S) Definitely A and possibly D are not credible; this question can be answered by logic rather than knowledge of the plant.-wrote a new question 2B H 3 - S 3 H 3 X X X U (S) reword stem question to what major factor could require a more rapid cooldown than EOS-0.2A allows? and try changing B to decreasing subcooling margin; this may be an SRO-level question -changed distractors and stem 4B H 3 X - U (S) A and C are weak distractors because the stem points to the Letdown system as the problem and indicates no abnormalities in the RCP Seal system. - changed two distractors 5 H 3 X - U (S) not sure D is credible; - changed D distractor 6 H 3 X - U (S) A and D do not make sense (any leakage into containment ... would be into containment) and D is not credible; reword all 4 distractors; is the first sentence of the stem really necessary? -
removed first sentence of stem, modified distractors 7 H 3 - E (S) what does failed as is mean in the stem? -elaborated on failed as is 8 H 3 X U (S) use the same valve in both C and D; put blanks next to 4, 5, and 6 in the stem and number the three steps in each distractor; doesnt test the knowledge of the REASONS for Emerg. Bor. for ATWS per KA; - changed question stem and 4 selections as commented Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
@ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
@ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
@ The answer choices are a collection of unrelated true/false statements.
@ One or more distractors is (are) not credible.
@ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
@ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
@ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
@ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
@ The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 7. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
33 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 9 H 3 X - U (S) A, C, and possibly D could be considered correct (although the meaning of D is unclear);
replace bad with ruptured and good with non-ruptured; - replaced bad/good with ruptured/non-ruptured. Also created new distractors, modified others.
10 H 3 - E (S) what is the S/G compartment room? Eliminate stem sentence with In the ERGs, ...; number the two answers in each stem and separate them by semi-colons; change stem question to read ...
and the minimum indicated level to be considered intact in the ERG would be__.; change distractors to read either 5% or 26% as needed; - modified stem for use with semicolons (Mark preference), modified stem content to elaborate on selection of normal/adverse containment values (bracket/no bracket) 11 H 2 X - U (S) A and B are not credible since they both include mutually exclusive actions (replace 1" with 8" in both); - replaced as requested.
12B H 2 - E (S) cannot find the reference to support this answer; - reference found, verified.
13 H 3 - E (S) change stem to what is the condition of the TDAFW flow control valve controls in the control room? and change A, C, and D to ...are lost but control may be recovered by... and eliminate no actions required from B; modified stem and distractors as requested.
14B H 3 X U (S) Doesnt test REASONS for EOP ACTIONS for loss of vital AC bus per KA; - found new bank question, F for LOK 15B H 2 - U (S) Also, D is not a component; need to see the parent in order to verify that this is a legitimate modified question; - removed word component from stem, changed to bank from modified (modified criteria not met) 16B F 2 X X - X U (S) procedure selection is specifically SRO-level (55.43.5); picking the correct procedure to blow down a SSW strainer is not a test-worthy topic; D is not credible; - wrote new question 17B H 3 - X U (S) procedure selection is specifically SRO-level (55.43.5); rewrote stem such that question is asking for EOP entry conditions, specific to RO knowledge.
18B H 3 X X - U (S) C doesnt make sense; A is not credible; delete most likely from stem
- changed stem and modified distractors.
19B H 4 - S INPO Cook-1 5/21/2001; 20B F 3 X - E (S) how did the switch fail? - elaborated on switch failure 21 H 3 X U (S) doesnt test the REASONS for start-up termination on loss of SRNI per KA; use consistent nomenclature for NIs (NI-31 or -31); - modified question to meet KA intent, used consistent nomenclature.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 22 H 3 X X X X U (S) distractors dont make sense and there is no correct answer; doesnt test the REASONS for EOP guidance on loss of IRNI per KA; -modified stem and distractors to include reasons for EOP guidance and make more sense.
23 H 4 X - U (S) TSB reference is incorrect; the FSAR seems to indicate that subcriticality can be maintained without boron, but the TSB (3.7.16) indicates that a minimum amount of boron is required; what is the basis for assuming that some rad alarms will come in rather than none? - wrote new question 24 F 4 X - X U (S) knowledge of TS LCO required actions is SRO-level (55.43.2); there is no distinction in the TS between personnel and emergency air locks and no distinction between electrical and mechanical interlocks; -modified stem and removed TS/1 hr aspects 34 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 25 H 3 - S wrote new question per previous comments on K/A match.
26 H 3 - X U (S) procedure selection is SRO-level (55.43.5); would an RO be expected to know the various legs of the CSFST by memory (with no reference)? -
wrote new question.
27 H 3 X - U (S) correct answer needs to be reworded to more clearly agree with basis on page 24 of FRP-0.1A; C and D do not seem credible; - reworded stem and changed two distractors.
28 H 3 X - X U (S) procedure selection is SRO-level (55.43.5); distractors dont seem to correspond to stem question; -modified stem.
29 H 3 X X U (S) add a 1" and 2" to each part of each distractor; doesnt test knowledge of LOSS or MALFUNCTION of demin. for CVCS per KA; - changed format of question, demin malfunction verified with utlity: where boron is changed by more than 10ppm this is a malfunction (ie operator induced, but still a malfunction) 30 H 3 - S 31 F 3 - E (S) change C from Normal: 2-EB4-2" to Normal: 2-EB3-2"; - changed as recommended.
32M H 3 X X U (S) change the Unit Supervisor would: to the required actions are:; procedure selection is SRO-level (55.43.5); doesnt really point towards ECCS per KA; cannot tell if this is truly modified without parent; - modified stem, and showed Mark parent question, agreed it is modified.
33 H 2 - S 34B H 3 - S 35B H 3 - S 36B H 4 - S 37B H 3 - E (S) delete the first sentence of the stem; cannot tell if this is truly modified without parent; deleted first sentence as requested, changed to bank.
38M H 3 - X U (S) this sounds like SRO-level (55.43.2); - wrote new question.
39 F 3 - E (S) delete the first sentence of the stem; - as requested.
40B H 3 - S 41B H 3 X X U (S) I dont think B is correct; doesnt test knowledge of effects of loss of CCS on instrument readings per KA; - modified stem to include CCS, answer is correct, verified with utility.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 42B H 3 X X U (S) change AS to As in stem; this is SRO-only (55.43.5); doesnt test knowledge of EOP TERMS and DEFINITIONS related to CS per KA; -
wrote a new question.
34 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 43B H 3 - S 44 H 3 - S 45 H 3 - S 46B H 1 X - U (S) too easy because A, B, and C do not seem credible; - found new bank question.
47B F 3 - E (S) change stem to ...following a reactor trip from 102% power:; - changed as requested.
48B H 3 - S 49 H 3 - S 50 F 2 - E (S) add only to C; - changed as requested.
51 F 2 - S 52B H 2 - S 53B F 3 - S 54 F 3 X - U (S) stem is misleading by implying that there is a breaker to check when there isnt; -modified stem and selections.
55B F 3 X - U (S) B and D are the same and cannot both be valid distractors; -changed all four selections 56 F* 3 - E (S) I disagree with the authors LOK; - changed LOK to F.
57 H 3 X U (S) doesnt test ability to monitor AUTOMATIC operations of the leakage detection system per KA; - wrote new question.
58B H 3 - S 59 H 2 - ?(S) not sure the answer is correct; fix symmetry with distractors; -modified answer and corrected symmetry as requested 60 H 3 X - U (S) A doesnt address the question; -changed distractor A.
61 H 2 X - U (S) C and D are not credible since the procedures dont exist; - replaced with valid procedures.
62B F 3 X X U (S) this sounds like SRO-level (55.43.2); should they know this by memory?
Doesnt test knowledge of DESIGN FEATURES/INTERLOCKS per KA; -
wrote new question.
34 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 63 H 3 - S 64B F 2 X U (S) doesnt test ability to LOCATE/OPERATE components per KA; - wrote new question.
65 F 3 - E (S) change stem to ...at CPSES and an alarm...; - changed as requested 66 F 2 - S 67B F 3 - E (S) cannot verify this is a true modified question without the parent;-
changed to bank.
68B H 3 - S 69 F 3 - E (S) change stem to ...than Unit 1 because it gives_____; - changed as requested 70 H 3 - E (S) change stem to ...Select all the items below... - changed as requested 71B F 3 - S 72 H 3 - E (S) change stem to 98mrem/hr and answer will be B; changed stem to 96mr/hr, and kept correct answer A as written.
73 F* 3 - E (S) I disagree with the authors LOK; delete first sentence of stem; -
changed LOK and deleted first sentence as requested.
74M F 3 - S 75 H 3 X X U (S) procedure selection is SRO-level (55.43.5); doesnt test ability to PERFORM IMMEDIATE ACTIONS w/o reference per KA; -changed K/A because couldnt write question on the topic and replaced with new bank question for new K/A.
76 H 3 X X U (S) RO-level (55.41.7/8); what is the difference between A and B? Doesnt test ability to determine SIGNIFICANCE of CCP operation; 77 H 4 X X U (S) RO-level (55.41.3-8); doesnt test ability to determine ACTIONS TO BE TAKEN if PZR pressure instrument fails low per KA; - wrote new question.
78 H 3 - X U (S) drop a necessary from the stem; RO-level (55.41.10); - wrote new question.
79 H 2 - - E (S) mention the procedure selection in all 4 distractors; - changed distractors.
80 H 3 - X U (S) RO-level (55.41.5-8); - wrote new question.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 81 F 3 - X U (S) RO-level (55.41.7/8); - wrote new question, new K/A.
82B H 3 - - S 34 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 83 H 3 X - - E (S) any handouts? This shouldnt be from memory, but it may be just a look-up if the reference is provided; - - utility approved as written.
84 F 3 - X U (S) RO-level (55.41.10); - wrote new question.
85 H 3 X - - U (S) D is not credible; - wrote new distractor D.
86 H 1 - - E (S) should RCP Trip/Malfunction be in quotes for A and C? - modified question.
87B H 3 - - E (S) fix typo in stem; - fixed typo.
88 H 3 X X - - U (S) unless MTC is currently positive, only one action is required; both A and C are correct (page 504 or TSB); -- changed wording so more clear, however A and C are not both correct.
89 H 3 X - X U (S) D should not be bolded; stem is unclear (what is the difference between initial load shedding and further load shedding?); not sure any of the distractors are really correct (page 33 of reference); RO-level (55.41.10); - wrote new question, changed K/A.
90 H 3 - - S 91 H 3 X - - U (S) D is not credible with fictitious title; - wrote new question, changed K/A.
92 H 3 X - - U (S) C and D not credible with fictitious titles; - fixed question and outline.
93 H 3 - X U (S) RO-level (55.41.10); establish distractor symmetry with STA-612 and goes up for B; - wrote new question.
94 H 3 - - E (S) use CET vice TC in stem; - changed to CET.
95 H 3 - - S 96 H 4 - - E (S) does a reference need to be provided? -no reference per utility.
97 F 4 - - E (S) delete first sentence of stem; - done.
98 F 4 - - E (S) reword stem to delete the first two sentences; - modified stem.
99B H 3 - - S 100B F 2 X X - U (S) A is not credible; doesnt test knowledge of E-PLAN PARS per KA; --
new K/A and question.
34 of 34 NUREG-1021, Draft Revision 9