ML061950377
ML061950377 | |
Person / Time | |
---|---|
Site: | Arkansas Nuclear |
Issue date: | 01/12/2006 |
From: | Gody A Operations Branch IV |
To: | Forbes J Entergy Operations |
References | |
50-313/06-301, ES-401, ES-401-9, NUREG-1021, Rev 9 50-313/06-301 | |
Download: ML061950377 (5) | |
Text
ES-401 ANO1-12-2005 Written Examination Review Worksheet
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 1B H 2 X - E Need to link post-accident instruments on these panels in Notes section in order to satisfy K/A.
2M H 3 X - S Previous review by Mark Haire - possible GFES question.
3B H 3 X X X E Remove the word best from the stem. For B to be incorrect, need more info in stem to clue operator that RCPs were secured within the 2 minute time frame. Need EOP bases (for ICC?) to validate.
4B H 3 X X - U Remove the words which of the following is true from the stem-true false not allowed.
Restructure distractors so that it isnt a collection of True false statements. D distractor is not credible.
5B F 2 - S 6B H 1 X X - E Substitute question due to error in original question. Reword stem to remove between from the stem.
7 H 3 - S 8B F 2 S Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
@ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
@ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
@ The answer choices are a collection of unrelated true/false statements.
@ One or more distractors is (are) not credible.
@ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
@ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
@ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
@ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
@ The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 7. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
33 of 34
ES-401 2
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 9B H 4 X X - U Need inches for units added to stem. Need reference for SGs for isolation to validate distractor B.
10B F 2 X X - E Modify stem to make distractors more credible, stem more operationally based with alarms.
11 F 4 X - U Startup and low load block valves also close on reactor trip. Remove from stem.
12B H 3 - S Minor edit to stem to increase understanding of battery discharge rate.
13B F 2 - S 14 H 3 X - U C is not credible since instrument does not fail midscale. Remove mid-scale.
15B H 2 X - U Need references to validate distractors. Change B distractor to read...D01" to make it credible.
16 H 2 X - E Minor stem edit to include HPI P-36B is aligned to A4 bus. This is a swing pump that can be aligned to two diff. buses.
17M H 3 X - E Need references for C and D distractors.
18M F 2 - S System Title on exam incorrect (says excessive, should be inadequate),
point value is also wrong on outline, should be 3.4.
19B H 3 X - U D is a poor distractor. Consider Align RPI with API as substitute.
20B H 3 - E Minor distractor edit for D for clarity on stabilizing power.
21B H 3 - S 22B H 3 X - U B is a poor distractor. Consider Reset RE-4830 and Notify Nuclear Chem.
23 H 4 - S 24 F 2 - S 34 of 34
ES-401 2
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 25B H 2 - A distractor-should Tcold have leg after it? Outline is incorrect X E (changed KA from EA1.17 to EK1.03 in exam, not outline).
26 H 3 X - U Modify distractors and answer to include total feed flow because flow is re-ratioed on 1 RCP trip. Also consider removing temps from distractors because too similar to question 52.
27 H 3 - S 28B H 3 - S 29M H 3 - S 30 H 3 X - E Put equipment/valve numbers with noun descriptions.
31B H 3 - E Need description of connectivity of buses, preferential trips, etc. Label pump B P-34B in stem.
32B H 2 X - E reference is weak in supporting only 1 answer.
33B H 3 X - U D distractor is not credible-change from maximize to minimize letdown flow.
34 H 3 - S 35 H 3 X - E Answer D is actually incorrect for plant conditions in stem. Heaters would be off because of the cooldown in progress.
36 H 3 - S 37B F 2 - S 38B H 3 - S 39B F 2 - S 40B F 2 X - E(S) references weak in supporting the correct answer.
41 H 3 - E references weak in supporting the correct answer.
42B F 2 X E Wrong K/A was described for question. Recommend modifying the stem to include the procedure step where MSR stm valves are closed.
34 of 34
ES-401 2
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 43B F 3 - E Minor edit of B and D distractors-remove HI load limit and change setting values in same distractors to increase credibility.
44B H 4 - S 45B H 4 X - E Minor edit to stem to lower temp/pressure to ensure only answer is correct and distractors credible without providing SCM graph.
46M H 3 X - U Edit stem to include degraded power such that RCPs are secured (reqd for thumb rule in question).
47M H 4 X - U Is B6 downstream of A4, no guidance on B6 restoration in the provided reference? Yes, auto pickup of B6 when A4 renergized.
48B H 4 X - E Move if the auto transformer...service to stem for all four selections.
Place is between equipment and operable in the stem.
49 H 3 X - E Need references to confirm all distractors will not cause a trip.
50B F 4 - S 51B H 3 - S 52M F 2 - E Change distractor B since it is also correct. Also need references to confirm distractors.
53B F 3 X - E Need references to tie B62 to answer.
54M H 3 X - E Change stem pressure from 48psig to 50 psig (actual setpoint inprocedure for action). Also need to review entire procedure for distractor credibility.
55B F 2 X - E Outline pt value is wrong at 2.8* (it is actually 3.9).
56B F 2 - S 57B H 3 X - U(S) Too similar to question 26 - modify distractors accordingly.
58B H 3 X - U Need more references for answer completeness and distractors.
59M H 4 - S 60B H 4 - S 61B H 2 - S 62B H 4 - S 34 of 34
ES-401 2
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 63B F 2 - S 64M F 2 - S 65M F 2 - S 66B H 3 - S 67B F 2 X - U Distractors A and D do not seem credible. Are cell phones available for other comm used in CR extension?
68B H 3 - U Substitute question because....Also need references for distractors.
Outline pt value is wrong-should be 3.4, not 2.8. Remove the word for in distractor C.
69B F 2 X - U Distractor A symmetry - It should read verification of the associated trip actions of all logic elements for the channel. B distractor-replace known values with a known standard. For the D distractor - replace proper response with Operability.
70B F 3 - S 71B F 3 - S 72B H 3 - S 73B H 3 X - U Why cant Bdistractor also be correct?
74B H 2 - S 75B F 2 X X X - U Poor question for discriminatory validity. Stem focus on distractor makes it too easy. Outline pt value is wrong-should be 2.9, not 3.3.
34 of 34