ML12285A132
| ML12285A132 | |
| Person / Time | |
|---|---|
| Site: | Dresden  |
| Issue date: | 09/26/2012 |
| From: | Bielby M, David Reeser Operations Branch III |
| To: | |
| Shared Package | |
| ML11354A120 | List: |
| References | |
| Download: ML12285A132 (13) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 DRESDEN 2012
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 3 Y B E Answer explanation could use some enhancement. It appears that not all the isolation valves will operate but that the associated penetrations will be isolated by at least one valve.
Response/Resolution: No changes made. Valve motive power is S not affected, all isolation valves will isolate once the isolation signal is initiated by the loss of power.
2 H 3 Y B S 2008 NRC Exam 3 H 3 Y N S 4 H 2 Y B S 5 H 2 Y B S 6 F(H) 2 Y B S Question is F vice H, only requires knowledge of power supplies.
Response/Resolution: Answer is not just a simple power supply recall; recalls knowledge of chain of power supplies from Bus 36 to H
the RWM. No changes were made.
7 F 3 Y B S 2010 NRC Exam 8 F 3 Y B S 9 H 3 Y B E Lettering of distracter explanations doesnt seem to match that of distracters.
Response/Resolution: Facility reordered the explanations (b swapped with c).
10 F 3 Y B E DAN 902-8 D-1 is cited as the reference for this question but the question appears to be related to circulating currents between the diesel generators and not the main/reserve breakers; explain why the DAN is the appropriate reference.
Response/Resolution: Facility replaced reference with BWR Fundamentals training guide Chapter 5.
Facility also deleted the words output breaker from the distracters.
11 F 2 Y B S 12 H 3 Y N S 1
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 13 H 3 Y B S 14 F(H) 2 Y B E 2006 NRC Exam; Question is F vice H, only requires knowledge of interlocks.
H Response/Resolution: Facility raised LOK by revising question to S
include Reactor Period and indications on all four SRM channels and change IRM ranges to 4/5 15 F 2 Y B S 16 F 2 X Y N E D(1) not plausible, that Target Rock valve will not operate due to bellows leak. Better distractor would be Relief ONLY. NOTE:
Would make question 1 of 2 twice.
S Response/Resolution: Facility corrected reviewer misunderstanding of bellows failure effect on valve operation. During review it was identified that 2nd part of the answer to identify procedure for mitigation provided no discriminatory value (either of the two choices provided could be used to mitigate a relief valve failure. Question rewritten to eliminate 2nd part.
17 F 2 Y B S 18 F(H) 3 Y B S Question is F vice H, only requires knowledge of FWLC operation/interlocks. Rewrite or replace.
H Response/Resolution: Comment withdrawn following discussion with facility. Question in more than simple recall of design features/interlocks. Examinee must evaluate conditions listed in stem in order to answer question.
19 F 2 Y N S 1 of 2 twice 20 H 3 Y N S 21 F(H) 3 Y N S Question is F vice H, only requires memorization of TS immediate action LCO.
Response/Resolution: Comment withdrawn following discussion H
with facility. Question in more than simple recall of Tech Spec requirements. Examinee must evaluate the various subsystem combinations to answer question.
1/2X2; 2007 NRC Exam 22 H 3 Y B S 23 H 3 Y B S 2
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 24 H 2 Y B S 25 H 3 Y N S 26 F 2 X Y N U B and C not plausible. Both Torus and DW are part of primary containment, if there is PCI, it is not plausible that only one or the other would isolate. Rewrite or replace.
Response/Resolution: Drywell and Torus N2 lines are separate penetrations each with an isolation valve, therefore B and C are plausible. Facility revised answer choices to address isolation of vent (exhaust) and supply lines (either and/or both).
E 27 H 2 Y N S 28 H 2 Y N E 1/2X2; A-Delete Then, capitalize t in The in second line to resemble C.
Response/Resolution: Licensee made recommended change.
29 F 2 Y N E Stem question needs to be restated to eliminate possibility of two correct answers; e.g., Given the loss of power to the AEER Halon panel, which of the following.
Response/Resolution: After discussions with facility the comment S was withdrawn and no changes were made to the question.
30 F 2 X X Y B U B not plausible. C is a possible correct answer.
Response/Resolution: Facility contends that B is plausible since applicant needs to know that rod drift alarm is due only to odd reed switch activation without a demand. Facility could not determine conclusively that C was not a possible answer and the question was replaced with a new question testing the same K/A.
30R F 2 Y N S 1/2X2 31 F(H) 2 X X Y B E/U Venting of the over-piston area is accomplished indirectly.
What are SCRAM Dump valves; do they exist?
H S Response/Resolution: Comments withdrawn after discussions with facility. Over-piston area does not vent to the exhaust header (vents to scram discharge volume) and Dump valves do exist.
3
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 32 H 3 Y N S 33 F(H) 1 Y B U Level of difficulty may be enhanced by including H2 pressure in the stem selecting other intersections with the H2 pressure curves.
H Response/Resolution: Facility made the recommended changes to E
question to raise LOK and LOD.
34 H 3 Y N E 1/2X2; Last sentence of stem, replace word initial with IMMEDIATE (in caps).
Response/Resolution: Facility made change.
35 F(H) 2 X Y N U 1) A and B arent plausible. Neither receives trip from either fire or room high temp. Why are they believable? 2) Question is F vice H, requires memorization of interlocks/isolations.
Response/Resolution: Facility contends that A and B are S plausible since examinee must have the knowledge to know that high room temperature or fire are not automatic trips. Facility agreed to change LOK to Fundamental(Memory).
36 F(H) 2 Y N S Question is F vice H, requires memorization of interlocks.
Response/Resolution: Identified control rod is an edge rod which affects determination of the answer.
H 37 H 3 X Y N E Explain why D is not a correct answer.
Response/Resolution: Technically D was correct (procedure specifies sequence), but Facility replaced the 2nd part of choices A and D to eliminate concern.
38 H 3 Y B S 39 F 2 Y B S 4
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 40 F 1 X N N U 1/2X2; Action statements in stem need to be numbered. Simple memory question which can also be answered (without knowing the correct answer) by simply eliminating the one action that is not an immediate action. While communications are important, not sure that this meets the intent of the K/A.
Response/Resolution: Numbering in stem apparently inadvertently deleted, but now restored. Facility contends that if the applicant believes that any of the other possible action combinations is correct that the wrong answer will be selected. The referenced communications/notifications are what are used to assemble the 2 Y S Fire Brigade. Examinee must know the difference in requirements between a fire within or outside the protected area and know that the crib house is inside the protected area. Corrected typo in distracter explanation. No other changes made to the question.
41 F 2 Y B S 42 H 3 Y B S References Required; 2010 NRC Exam; 43 H 3 X Y B U 1/2X2; Are operators expected to know specific parameter numbers that require Torus vs DW venting? If not, replace or rewrite question.
Response/Resolution: Comment withdrawn. Answer is based on S system knowledge of DW design pressure and knowing when torus vent line is covered with water.
44 H 2 Y B S 45 F(H) 2 Y N S Question is F vice H, requires memorization of reason for system response.
Response/Resolution: Question requires evaluation of stem H
conditions. Comment withdrawn.
46 H(F) 2 Y B S 1/2X2; Response/Resolution: Facility changed LOK to High 47 F 2 Y N S 1/2X2; 48 F 2 Y N S 1/2X2; 49 F 2 Y N S 5
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 50 F(H) 4 X Y B U 1) Why is term should used in last sentence of stem? Should implies action may, or may not, be performed. If not a required action, replace or rewrite question. 2) Are applicants expected to memorize these DSSP Attachment B procedural steps (ie, know which breakers to verify open or closed)? 3) Question is F vice H, requires memorization of procedure step.
Response/Resolution: 1) Replaced should with must. 2) & 3)
H S Question is answered based on availability of power sources and not simply memorization of procedure steps.
Inserted the phrase to align the Auxiliary Power system to the last line of the question stem.
51 F 2 Y B S 1/2X2; 52 F 2 X Y B E 1/2X2; insert the word ACTUAL in first line just before RPV water level was Response/Resolution: Facility made change. Changed RPV to reactor.
53 F(H) 2 Y B S 1/2X2; Question is F vice H, requires memorization of power supply.
Response/Resolution: Requires analysis of power supply availability. No changes made to question.
H 2001 NRC Exam; Explain why B and D are plausible (ie, believable).
54 F 2 Y B E If not plausible, then question needs to be rewritten or replaced.
Response/Resolution: Subsequent actions of DOA. No changes S made to question.
55 F 3 Y B S 2008 NRC Exam; 56 H 3 Y N E Dont completely understand answer justification. How does applicant know there is loss of Bus 23-1; hence, loss of both operating SDC pumps? Stem only says 2B SDC pump tripped on overcurrent, doesnt say there was OC fault on bus.
S Response/Resolution: Apparent cut and paste error. Fixed 57 H 3 Y B S 6
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 58 F 3 X Y B E Reduce cuing by rephrasing answer A so that exact words from the DOA are not used, ie, Reduce main generator load to 35% and trip the main turbine.
Response/Resolution: Generator load will be changed to 25%
59 H 3 Y N E Add H-8 to distracter B since it is an individual CRDM action.
Response/Resolution: Facility made change 60 H 2 Y B S 2007 NRC Exam; 61 F 3 Y B S Comment for distracter D could be enhanced for understanding since seal injection does provide some cooling effect and RR pumps can continue to run without RBCCW as long as seal injection is maintained and visa-versa.
Response/Resolution: Comment withdrawn; doesnt apply to question event.
1/2X2; 1) Choices A and B make question low LOD. They contain 62 H 1 Y B U simplistic concept of stopping blowdown to RWCU will mitigate the RWL decrease. This narrows correct answer to either A or B (50-50 chance of guessing correct answer) and makes question low LOD.
- 2) First part of choices C and D defy law of physics in that maintaining blowdown to RWCU would mitigate the RWL decrease.
Response/Resolution: Question is bases on differences in use of 2 S system for level control and pressure control. No change necessary.
Included additional detail to the answer explanation.
63 F(H) 2 Y N E Question is F vice H, requires memorization of pressure setpoint for low set ERVs (B and C ERVs).
Response/Resolution: Must evaluate stem condition to arrive at H S correct answer.
64 H 2 Y N S 65 F 2 Y B S 66 H 2 Y N E Revise question to state Using the OP-DR-104-1001 curves provided, determine the time to boil.
Response/Resolution: Facility made change.
67 F 2 Y B S 7
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only
- 1) Explain why B, C and D are plausible (ie, believable).
68 F 2 X X Y B U Misconception, or justification for another condition? If not plausible, then question needs to be rewritten or replaced.
- 2) Question closely related to question 95.
Response/Resolution: Question replaced.
68R F 3 Y N S 69 F 2 Y N S Under Comments the reference to choices do not match the 70 F 2 Y N E justification. Need to clarify so justification matches respective question choices.
S Response/Resolution: Fixed 71 H 2 Y B S 2010 NRC Exam; 72 F 2 Y N S 73 H 2 Y N S 74 F 1 Y N U Low LOD, as written and without knowing the correct answer, the applicant could get correct answer by process of elimination.
Knowing that starting the 2/3 DFP was not Immediate Action eliminates choices A, B and C. Rewrite or replace.
2 E Response/Resolution: Will reformat to Which of the following is NOT?
75 F 1 Y N U Low LOD, as written and without knowing the correct answer, the applicant could get correct answer by process of elimination.
Knowing that electrical safety coat was not required eliminates choices B, C and D. Rewrite or replace.
2 E Response/Resolution: Answer choices will be restructured.
LOK (F/H, 50-60%H) 35/40=47%/53%
B/M/N (56/?/10) 43/0/32=57%/0%/43%
UNSAT for RO Only (20%=15) 1/75=1.2%
8
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 3 X X YES B U 2009 NRC Exam; 1/2X2; (76) 1) Stem needs to be revised to indicate the time relationship of the bulleted items. (e.g., instead of the following set of conditions: use the following sequence of events:
- 2) There is no correct answer for the question as currently written.
E If the MSIVs are closed then D.(2) is not applicable to the event.
Response/Resolution: 1) Facility made recommended change.
- 2) Licensee argued that verification of the closure of the control valves is an anticipatory action in preparation for reopening the MSIVs. While verifying that closure of the control valves prior to opening the MSIVs would be required, it is not clear that this was the intent of the step in the DOA. The facility proposed swapping distracters B.(2) and D.(2). Agreed with the proposed action and changed question status from U(nsat) to E(nhancement).
2 H 2 NO B U Question can be answer using only RO level of knowledge; specifically knowing HX flow paths can eliminated A and B AND (77) knowing that Pumpback compressors are cooled by U2 RBCCW eliminates C.
Response/Resolution: Distracters revised to eliminate concern 3 F 2 YES B S 1/2X2; (78) 4 F 3 X YES N U As written there could be multiple correct answers. The number of valves required to be operable is dependent on whether a MCPR (79) penalty is applied to the operating limits or not.
Response/Resolution: Facility revised stem to include (without thermal limit penalties applied).
5 H 3 YES N S (80) 6 F(H) 2 YES B S 2010 NRC Exam; 1/2X2; (81) 7 F 3 YES B S 2007 NRC Exam; (82) 9
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 8 H 2 NO B U 2007 NRC Exam; Reactor SCRAM and securing testing are Tech Spec Immediate actions, and D is the only choice that contains (83) both of these actions. Starting all available Torus Cooling is a basic mitigating strategy for high Torus Temperature.
Recommend changing temperature data so that bulk average temperature is > 105°F and < 110°F.
M E Response/Resolution: Licensee modified (similar to suggested change) the question to remove concern associated with immediate actions. Change resulted in new correct answer. Changed status from U(nsat) to E(nhancement) since question had been previously used on a previous RIII NRC exam (2007).
9 H 3 X YES N U 1/2X2; Per the EOP bases, the direction to shutdown the reactor does not preclude SCRAM, just that normal shutdown would be (84) more appropriate. Without change there are two correct answers.
Response: License inserted the phrase at a minimum after Based on these conditions,...
10 H 3 X YES N U Subsequent actions of DOA 3700-01 also directs venting of the Drywell making this a second possible answer.
(85)
Response: Replaced DOA 3700-01 with DEOP 200-1. Also added E
Drywell Temp and rate of change to the stem to make DEOP entry plausible.
11 H 2 NO N U I would expect that the NSO would perform the action of B(2) without requiring direction from the SRO.
(86) YES Question overlaps with Control Room Systems JPM Response: The expectation is that NSO would obtain permission or E
at least concurrence from SRO prior to performing step.
JPM now designated for RO only.
12 H 3 X YES B E Possible double jeopardy with question 93 (87) S Response: Comment withdrawn after discussion cleared up an examiner misunderstanding related to Decay Heat Removal System unique to Dresden/BWR3 design.
10
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 13 H 3 YES B E Is temperature indicator on 923-5 the same instrument loop as the alarm relays? Recommend revising the last bullet to read that local (88) temperature is 230F and rising at 1-2F every 15 minutes. Replace DAN reference with DOS reference.
S Response: Question OK as written; comment withdrawn.
14 H 5 X YES B U 2007 NRC Exam; 1) Operator should not have to memorize pump test acceptance criteria. Recommend providing a pump curve.
(89)
- 2) What is the correct answer? B is listed but explanation implies that A is the correct answer.
Response: 1) Related Tech Spec contains required pump flow data 2 E and is provided as a reference, therefore pump curve not required; LOD changed.
- 2) Corrected answer key.
Status changed from U(NSAT) to E(nhancement) 15 H 3 YES N S (90) 16 H 3 YES B S 2009 Exams; (91) 17 H 2 X YES M U Answer choices A, B and C are subsets of D (92) Look at original question to see how modified.
Response: Distracters were revised to eliminate RPV level bands and reference pressure control bands only. Also reworded stem to reference OP-DR-103-102-1002, Strategies for Successful Transient Mitigation, instead of DEOPs. Aligns question closer to K/A also.
18 H 1 X X NO B U 1/2X2; B(2) and D(2) dont seem to be credible in that RHR would not be used for pool cooling in MODE 1; leaves a one out of two (93) choices which can be answer using system knowledge.
Possible double jeopardy with question 87 Response: Comment withdrawn after discussion cleared up an 3 YES S examiner misunderstanding related to Decay Heat Removal System unique to Dresden/BWR3 design.
11
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 19 F 3 X X YES B E Answer choice A is partially correct in that all control cells containing fuel must have their control rods fully inserted.
(94)
Answer choice C is also true since the stem does not indicate whether a spiral offload is in progress or what the status of the off load is.
Response/Resolution: First comment withdrawn.
Licensee provided clarification to stem related to fuel load status and changed distracter C to state 3 vice 1 SRM.
20 H 3 X YES B U Operator should not have to memorize pump test acceptance criteria.
(95)
Also is the correct procedure listed in the stem? (should it be 1500-12)
Response/Resolution: Required pump flows is a well identified S U2/U3 difference and is required knowledge for NSOs. Procedure number is correct.
21 H 2 X NO N U 1/2X2; Can be answered using only system knowledge.
(96) YES S Response/Resolution: SRO only training objective.
22 H 2 YES B S (97) 23 F 2 YES N S (98) 24 F 2 YES B S 1/2X2 (99) 25 F(H) 2 YES B S 2007 Exam; (100)
LOK (F/H, 50-60%H) 8/17=32%/68%
B/M/N (75%/?/10%) 15/2/8=60%/8%/32%
UNSAT for SRO Only (20%=5) 4/25=16%
12
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
- The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
- The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
- The answer choices are a collection of unrelated true/false statements.
- The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
- One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
- The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
- The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
- The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
Identifies initial sample Identifies enhancements made Identifies UNSAT questions 13