ML11235A637
| ML11235A637 | |
| Person / Time | |
|---|---|
| Site: | Arkansas Nuclear  |
| Issue date: | 08/26/2011 |
| From: | NRC Region 4 |
| To: | Entergy Operations |
| References | |
| 50-313/11-08, 50-368/11-08 | |
| Download: ML11235A637 (13) | |
Text
ES-401 AN1-2011-08 Written Examination Review Worksheet Form ES-401-9 Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met).
Page 1 of 13 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
H 3
B S
A, CR, 41.7 2
H 3
B S
B, CR, 41.7 - Repeat (2008 - Q3) 3 H
3 X
B S
A, CR, 41.8; 41.10 4
H 3
X N
U S
D, CR, 43.5 - The question is written to 000015/17 AK2.10, when the ES-401-2 says the question should be written to AA2.10.
Typo on ES-401-2, it should be AA2.10 (verify) - the Importance Rating is consistent with AK2.10, subject and Importance Rating support it being AK2.10 - will accept question.
5 H
3 B
E S
A, CR, 41.5, 41.10 - Recommend stating in question which actions are procedurally required, vice what the operator should do. This is a common observation throughout the exam.
Agreed. Changes made.
6 F
3 X
B E
A, CR, 41.8; 41.10 - Repeat (2008 - Q7, 2010 - Q6) - Unless there is another way to
ES-401 AN1-2011-08 2
Form ES-401-9 Page 2 of 13 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only S
increase DHR flow in this situation, other than starting the standby DHR pump, then distracters B and D are the same. Resolve.
Modifications were made to tie the question to procedural actions.
7 H
3 X
B E
S A, CR, 41.5; 41.10 - The given question tests what actions need to be taken to ICW cooled components manually based on ICW auto re-alignment on ESFAS. The K/A tests knowledge of the reasons why the ICW re-alignment occurs on ESFAS.
With discussion, it was shown that the question can address this KA. A modification to the stem clarified its tie to the KA selected.
8 F
3 X
N E
S D, CR, 43.5 - The question is written focusing on a time, not a system pressure.
Also, the question is asking at what point a TS action statement is entered, not what a what point a LCO is applicable.
Revision made.
9 H
3 M
S C, CR, 43.5 - Post comment resolution in validation week, the licensee proposed to modify the original Bank question proposed. The change is acceptable. The approved question is now a Modified question.
10 H
2 N
S B, CR, 41.7 (Editorial) - The References refers to RT-5, instead of RT-6.
11 H
3 B
S C, CR, 41.5, 41.10 12 F
2 B
S B, CR, 41.10 13 H
3 B
S C, CR, 41.7 14 F
3 B
S B, CR, 41.10 15 H
3 B
S C, CR, 41.7 16 H
3 B
S A, CR, 41.7 17 F
3 X
B E
S B, CR, 41.7 - Since the tube-to-shell delta Ts are 100 and 60F, justify why 40F is a plausible distracter. No justification for the distracters being plausible is provided.
Revised to use 30F, since it is the set point for SCM temperature requirement.
18 F
3 N
S A, OR, 41.4,.5,.7, and.10 19 F
3 B
S B, CR, 41.8,.10 20 H
3 B
S B, CR, 41.5 21 H
3 N
S B, OR, 43.5 22 H
3 M
S B, CR, 41.7 - Originally, the proposed question was a Bank question. After
ES-401 AN1-2011-08 3
Form ES-401-9 Page 3 of 13 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only discussion with the licensee the week of August 15, 2011, they proposed to change this question to a Modified question. Question reviewed, and it is acceptable.
23 H
3 X
N E
S A, CR, 41.8,.10 - Is it possible to test the applicants knowledge on what lit indication on C13 they expect to change in order to determine that there is a loss of NNI-X, and not just a loss of indication? Step 1 of AOP 1203.047 has them do this.
Just as in the justification for the correct answer, it is recommended to change the question stem to say which loss of power alone could cause the need for a reactor trip. Otherwise, it could be appealed that the others are partially correct too.
Will add change in stem to say alone. Change made.
24 H
3 M
S C, CR, 41.7 25 H
3 B
S C, CR, 41.7 26 H
3 B
S A, CR, 41.5,.10 27 F
3 X
B E
S D, CR, 41.7 - Distracter C will be plausible if the applicant population was trained on knowledge of a SE instrument table notebook. The explanation says it was around in the distant past. Licensed operators in the requal cycle may see this in an exam, but new applicants in the process may have no knowledge of this, making this a distracter they can easily eliminate.
This is typically an area that the SE may address with supplemental materials.
However, a change was made to address them with bracketed values in EOPs the crew uses. Change to question justification explanation made.
28 H
3 X
B U
S B, CR, 41.7 - The K/A selected in the outline is K6.02 - RCP Seals and Seal Water Supply for 003 Reactor Coolant Pump System (RCPS). The question is written to K6.04 - Containment isolation valve affecting RCP Operation. The question needs to be written to the K/A number selected in the outline.
K6.04 was selected in outline process, but error in transposing to outline submitted.
Second occurrence. NEED TO ADDRESS. Database to Word transfer problem.
Due to the fact that the next question addresses RCP seals as well, recommended that the K/A 6.02 be rejected, and re-sampled. K6.04 selected and written to.
Question acceptable.
29 H
3 X
N U
S B, CR, 41.7 - The K/A selected in the outline is A4.01 - Seal Injection for 003 Reactor Coolant Pump System (RCPS). The question is written to A3.01. This needs to be fixed.
A3.01 was selected in outline process, but error in transposing to outline submitted.
Third occurrence. NEED TO ADDRESS.
Discussed, the Importance Rating for A3.01 and A4.01 are the same. They technically can address the same information, so the question will satisfy either one.
ES-401 AN1-2011-08 4
Form ES-401-9 Page 4 of 13 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only This is an outline submission quality issue. No changes to question needed.
30 H
3 M
S D, CR, 41.7 - Originally, the proposed question was a Bank question. After discussion with the licensee the week of August 15, 2011, they proposed to change this question to a Modified question. Question reviewed, and it is acceptable.
31 H
3 X
N S
C, CR, 41.5 32 H
3 X
M E
S A, CR, 41.7 - Recommend changing recircing to recirculating. Also, the justification for distracters B and C refers to the RB spray set point at 30 psia. It is 30 psig according to the attached procedure.
Changes have been made.
Originally, the proposed question was a Bank question. After discussion with the licensee the week of August 15, 2011, they proposed to change this question to a Modified question. Question reviewed, and it is acceptable.
33 F
3 X
B E
S D, CR, 41.2-41.9 - Is there a Makeup Tank Outlet Valve separate from the Makeup Block Valves? Distracter B may also be a correct answer.
Valve number added to distracter B.
34 F
2 B
S C, CR, 41.7 35 F
2 B
S B, CR, 41.7 36 H
3 M
S A, CR, 41.5 37 F
3 B
S C, CR, 41.7 38 F
3 X
B U
S B, CR, 41.7 - The outline says that this question is to be written for K/A K3.01 (CRDS), but it says Sensors and detectors, which is K6.06. The question is written testing knowledge of what happens to RPS when one of its inputs malfunctions. RPS is operating correctly based on its inputs. K/A K3.01 involves a failure of RPS and its effect on CRDS. This is a K/A mismatch. Propose another question to address the K/A selected in the outline.
Issue with why this happened not sure why it happened. Similar to other issues with transposing information from outline database to MS Word outline format (Fourth Occurrence).
The Importance Rating and K/A description match in the outline for K6.06. Will accept that K3.01 was denoted in error. The question addresses K6.06, and the final outline is revised to reflect this. No changes to the question.
39 H
3 B
S D, CR, 41.2-41.9 40 F
3 X
B E
B, CR, 41.2-41.9 - Distracters A, C, and D are written as general statements, while
ES-401 AN1-2011-08 5
Form ES-401-9 Page 5 of 13 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only S
correct answer B is written as a reason, which is what the question is looking for.
Caution should be used in using this kind of text, as it provides cueing on what the correct answer is. See NUREG-1021, Appendix B, Section C.2.m.
Will enhance wording on distracters.
Text revised - question acceptable.
41 F
2 N
S A, CR, 41.7 42 F
3 X
B E
S C, CR, 41.7 - With the procedure (1106.006) referring to approximate EFW pump speeds (~910, ~900 rpm for ramp circuit to start effect), what is the uncertainty in the measured speed value. Could the pump actually be at 940 rpm and meet the ~910 rpm criteria in the procedure? Trying to determine if any of the distracters are partially correct.
One of the distracters changed from 940 to 960 rpm. Acceptable.
43 F
3 M
S B, CR, 41.7 - Originally, the proposed question was a Bank question. After discussion with the licensee the week of August 15, 2011, they proposed to change this question to a Modified question. Question reviewed, and it is acceptable.
44 H
3 X
B E
S B, CR, 41.5 - Similar to previous comment, distracter A appears plausible IF this minimum flow level is something the applicants have been trained on. If this old EFW minimum flow value is not part of the training program, then it isnt plausible.
Possible for licensed operators in requal, but not for initial classes.
Will look into the basis for the distracter.
The flow rate was from the 1990s. The distracter was changed to match a plausible SG EFW flowrate in RT-5. Resolved.
45 H
3 M
S C, CR, 41.5 - Originally, the proposed question was a Bank question. After discussion with the licensee the week of August 15, 2011, they proposed to change this question to a Modified question. Question reviewed, and it is acceptable.
46 H
3 B
E S
A, CR, 41.10 - The automatic actions in the A1 lockout appear to isolate all feed sources to it that may be suspect (aux transformer, start up transformers 1 and 2).
After the cause of the fault is known and corrected, it says to return power to the A1 bus. If breaker A-309 was closed prior to the transient (A1 supplying A3 bus), it is assumed that breaker A-309 opened on under voltage. With the EDG start supplying bus A3 after this occurs, would you want to verify that A-309 is open until the cause of the transient is identified and corrected? If A-309 doesnt operate as expected, it could complicate things. There is no step in the annunicator procedure to verify this.
The licensee will review procedure on this following exam. Accept question.
47 F
3 B
S B, CR, 41.7
ES-401 AN1-2011-08 6
Form ES-401-9 Page 6 of 13 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 48 H
4 B
S C, CR, 41.7 49 H
4 B
S D, CR, 41.7 50 H
3 B
S C, CR, 41.7 51 H
3 B
S A, CR, 41.7 - (Editorial) Correct the multiple misspellings of independent(ly).
Editorial changes have been made.
52 F
3 X
X N
U S
B, CR, 41.10 - K/A 2.1.32 tests the ability to explain AND apply system limits and precautions. The given question tests the ability to explain a Service Water system limit, but it doesnt test the ability to apply limits/precautions. It doesnt fully address the K/A statement.
Justify how the three distracters are credible pump failure mechanisms protected against by checking supply water level.
A new question was written. It is acceptable.
53 H
3 B
S C, CR, 41.5 54 H
3 X
B U
S B, CR, 41.7 - Verify that the cross-connection between Units 1 and 2 instrument air headers is normally open. If this cant be assumed, then it needs to be addressed in the question cue. If this assumption can be made, as written, an applicant could argue that none of the actions would be taken based on instrument air header pressure, since the correct answer isnt allowed by procedure until pressure is below 60 psig. In either case, there is no clear correct answer to the question. Resolve.
Cross connect is normally open. Per the procedure, no action is taken until air header pressure is below 60 psig.
Change made to include information in the stem allowing procedural action.
55 F
2 B
S C, CR, 41.5 - Repeat from 2008, Question 55 56 H
3 B
S A, CR, 41.5 (Editorial) In the answer explanation, it says that the D/P would be minimum with the draining of the reference leg. When the reference leg is drained, compared to the variable leg, the maximum D/P is seen on the D/P cell (I believe).
The text is correct. This is specific for LT-001 on their pressurizer.
57 F
2 B
S A, CR, 41.5 58 H
3 X
B S
B, CR, 41.7 59 H
3 X
B E
S B, CR, 41.7 - Distracter B is said to be incorrect because it sounds like this wouldnt be done until all ICS subassemblies are in Manual. Besides this being procedurally preferred, would selecting Tave to Loop B clear this condition? If so, it could be
ES-401 AN1-2011-08 7
Form ES-401-9 Page 7 of 13 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only argued that it is a correct answer as well.
Will revise the stem to tie it directly to procedural direction.
The question was revised and is acceptable.
60 F
2 B
E S
A, CR, 41.7 - Without knowing the correct power supplies, the applicant can eliminate two distracters (B and D) just by knowing that MCCs are designated with B-XX format vice B-Y format for Load Centers. NUREG-1021, Appendix B, C.2.m(2) -
Repeat from 2008, Question 60 Will take MCC out of the question stem.
Stem was revised. Question acceptable.
61 H
3 B
E S
C, CR, 41.5 - Recommend revising distracter B to state what you expect SFP level to do. It implies that it decreases.
Revision made, question acceptable.
62 H
2 X
X B
E S
C, CR, 41.7 - Recommend clarifying question by stating based on what happened..
Otherwise, someone could appeal and say that all the answers could result in a MFP trip.
Justify why high vibration is a credible MFP pump trip when the transient is a loss of power (distracter D).
Revised, question acceptable.
63 F
3 B
S C, CR, 41.7 64 H
2 X
B E
S B, CR, 41.2-.9 - The correct answer varies from the distracters in form and style.
Since all three distracters test whether specifically identified components do certain things, and the correct answer is generic, this can provide cueing to the applicant of the correct answer without testing their knowledge on the topic. See NUREG-1021, Appendix B, C.2.m(9). - The 21XX series procedure reference was not included in the reference disk. Need to look and see if more of these are referenced.
Revised, question acceptable.
65 F
2 B
S A, CR, 41.7 66 F
1 X
N E
S C, CR, 41.10 - Distracter A could be argued as partially correct because of COPD001, Attachment G, Step 2.6. This implies that there are situations where multiple communications at once may be acceptable.
Since there is no mention of clearing communications of unnecessary noise, it is difficult to see how distracter D is plausible.
Review procedure for necessary changes. No exception stated for Answer A.
EN-HU-102 was added as a reference to support distracter A as being incorrect.
ES-401 AN1-2011-08 8
Form ES-401-9 Page 8 of 13 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Question is acceptable.
67 F
2 X
X N
E S
B, CR, 41.10 - Writing questions with negatively stated stems is discouraged. See NUREG-1021, Appendix B, Section C.2.e. This needs to be revised to be consistent with the standard.
Justify why a credible distracter that implies there are times when doing actions per procedure is not advised (A). Distracter D implies that it is credible to consider not placing something back in AUTO that had no auto initiation signal is questionable too Discussed, with see if a revision can be made.
Revision made to make it a positively stated question stem. Question is acceptable.
68 F
2 X
X N
U S
C, CR, 41.10 - Answer C, if it is interpreted as a change in existing blood pressure medication, is not reportable. It is reportable if there is a change in medical condition.
This needs to be clarified.
Justify how a jury summons, which anyone can receive by law and has no negative connotations, can be a plausible distracter (D).
Will correct C, and research CFR for D.
Reviewed revised question. Question is acceptable.
69 F
2 B
S B, CR, 41.7,.10 70 H
3 B
S D, CR, 41.10 71 F
2 B
S B, CR, 41.10 72 F
1 X
X B
U S
B, CR, 41.11 - A question that ask what document Chemistry uses to calculate offsite doses with the Offsite Dose Calculation Manual is not discriminating. In addition, the distracter C refers to documents plural, when the stem refers to one document. That would be eliminated from consideration by the applicant just by the singular/plural text not matching, not because of plant knowledge.
At best, the question tests knowledge of plant procedures for radiation release measurement. It does not test the ability of the applicant to control radiation releases, as the K/A statement calls for. This question needs to be fixed.
Will review to propose changes.
This was a New question. A new question was selected from the Bank to replace this question. Question is acceptable.
73 F
2 M
E S
B, CR, 41.12 - The question focuses on the knowledge of the general occupational exposure limit. Recommend testing one of the less used limits, such as those stated as distracters.
Modified the Bank question originally submitted. Question is acceptable.
ES-401 AN1-2011-08 9
Form ES-401-9 Page 9 of 13 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 74 F
3 X
N E
S B, CR, 41.10 - Writing questions with negatively stated stems is discouraged. See NUREG-1021, Appendix B, Section C.2.e. This needs to be revised to be consistent with the standard.
The question was revised to resolve this. Question acceptable.
75 F
3 N
S C, CR, 41.10 - The stem of the question states that the performer of the Transient Checklist is directed to do so per procedure 1015.037. Per this procedure, Attachment G, Unit 1 Transient Checklist is NOT assigned to anyone to perform.
Check the roles/responsibilities, and search for procedure action to complete Attachment G. There is none. While it is agreed that some of the data would be picked up under Step 6.5, there is no clear assignment to complete Attachment G.
This could become an issue in an appeal on this question.
Site researched and procedure doesnt point to STA completing this. However, there are no cases where the other positions stated would be called on to complete this attachment. Leave question as is, and a corrective action on the procedure will be initiated later.
76 H
2 X
X N
U S
D, CR, 41.7 - The K/A statement deals with testing the SRO applicants ability to adhere to procedural limitations in the facilitys license and amendments. Examples of topics for SRO level of knowledge on this deal with reporting requirements for exceeding licensed thermal power output, administrative control usage in Tech Specs Sections 5 and 6, and others. The given question tests ability to recognize entry conditions for an EOP, which by itself is RO level of knowledge, let alone that it doesnt address the K/A statement.
A New question was written to replace the Bank question originally provided. The question is acceptable.
77 H
3 M
S A, CR, 43.5 78 H
2 B
S B, CR, 41.10 79 H
4 B
S D, CR, 43.5 80 H
3 X
X B
U S
A, OR, 41.10 - In this situation, if DC bus D01 is de-energized, then distracter C is correct too. It could be appealed that this is a correct answer as well because the operating crew takes concurrent action for both procedures. Please verify.
Will review and verify.
In order for this to be a SRO vice RO question, it needs to test more than knowledge of EOP/AOP entry conditions. That knowledge, coupled with knowledge of how to apply hierarchy selection of actions based on procedure type, is an example of an SRO only question.
Proposed another Bank question, Tech Spec related. Question is acceptable.
ES-401 AN1-2011-08 10 Form ES-401-9 Page 10 of 13 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 81 H
4 N
S A, CR, 43.2 82 H
3 N
U S
C, OR, 43.2 - When the rod drops and becomes inoperable, it is 1430. The clock for all LCO 3.1.4 Condition A actions start at that time. Action was taken to calculate SDM within COLR limits (Action A.1.1), but it was found that the SDM was NOT within the COLR limits. Therefore, Action A.1.1 is not complete. The alternate is to initiate boration to restore within SDM limits within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> of the inoperability. Either the SDM calculation showing the plant is within COLR limits OR the boration needs to start by 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, at 1530. The time stated in the provided correct answer is wrong.
Will revise the question to correct. It was incorrect. Agreed to make an Open Reference question.
Reviewed the revision to the question. It is acceptable.
83 H
3 X
B S
B, CR, 43.5 84 H
2 B
S D, CR, 43.2 85 H
3 X
B S
D, CR, 43.5 86 H
2 X
X B
U S
B, CR, 43.5 - An SRO-only question cannot be based solely on knowledge of AOP entry conditions. The question has to involve evaluation of a situation where the staff is already in an AOP/EOP, and the conditions when evaluated drive them to transition to another AOP/EOP. As written, it is a RO level question.
For the question given, the applicant could select answer A, which directs one to take actions stated in answer B. Both are correct answers.
Due to inability to address KA, rejected the original and re-selected a KA. Provided a new Bank question. Question acceptable.
87 H
2 X
M U
S B, CR, 43.5 - This is the same issue with this being a RO level of knowledge question testing AOP entry conditions, and not SRO level of knowledge.
Question modified to include testing level of knowledge of which procedure is required in this situation. It is acceptable.
88 H
3 X
X N
U S
D, CR, 43.2 - SRO-only knowledge isnt typically claimed for questions based solely on knowledge of 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> Tech Spec Action Statements. Verify whether ROs are required to know those as well per their learning objectives.
Per their learning objectives, the ROs are required to know actions of LESS THAN 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />. 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> or greater are SRO level of knowledge.
In addition, distracter B is also a correct answer. Tech Spec 3.3.1 Action Statement B (B.2.2) allows this.
Reviewed revised New question. It is acceptable.
ES-401 AN1-2011-08 11 Form ES-401-9 Page 11 of 13 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 89 H
4 M
S B, CR, 43.5 (Editorial) The notes for the question say it was modified for the 2010 exam. Since it wasnt found in the 2010 exam, it is assumed that the note was meant to say 2011.
90 H
2 B
S D, CR, 43.5 91 H
3 X
B U
S A, CR, 43.5 - An SRO-only question cannot be based solely on knowledge of AOP entry conditions. The question has to involve evaluation of a situation where the staff is already in an AOP/EOP, and the conditions when evaluated drive them to transition to another AOP/EOP. As written, it is a RO level question.
Revised the Bank question. It is acceptable.
92 H
2 B
S C, 43.2,.5 93 H
3 N
S A, CR, 43.5 94 H
3 M
E S
C, CR, 43.7 - This question, with altered words compared to its parent, is still the same question, with the same correct answer. This isnt a Modified question.
They changed some of the answer choices as well as the stem. Will count as Modified question.
95 H
3 B
S A, CR, 43.2 (Editorial) Change the references to plat to plant.
Question revised.
96 H
3 B
E S
B, CR, 43.2 - The CFT level stated in the cue is 12.5 feet. The Bases provide for Tech Spec 3.5.1 say that CFTs are operable as long as level is 11.95 feet and 14.00 feet. Justify why this is the reason the CFT is inoperable. The referenced procedure wasnt provided to verify the instrumentation inaccuracy allowances.
With the reference (to be provided), the operability band shows that the CFT level is inoperable based on these conditions. Procedure excerpt added to exam package.
97 H
3 X
X N
U S
C, OR, 43.2 - The K/A selected is 2.2.36. The statement of the K/A in the ES-401-3 is K/A 2.2.19, not 2.2.36. Then, the question provided to address this is written to K/A 2.2.17. Whether the K/A selected in the outline process is 2.2.36 or 2.2.19, the question does not match this K/A statement.
It was determined on outline selection that the question would be written to 2.2.36.
They attempted to write a question to this (new one), but had difficulties. Intended to place on rejection list, but did not. A second selection didnt work as well, so went to rejection, but didnt document. REPEAT ISSUE with exam outline quality.
For the given question, procedure COPD013, Section 4, 2nd bullet, allows for a sum greater than 4 for normal control room shifts.
Will add stipulation on Shift Manager control in stem to ensure that there is one correct answer.
ES-401 AN1-2011-08 12 Form ES-401-9 Page 12 of 13 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Revised question. It is acceptable.
98 H
3 X
X N
U S
C, CR, 43.4 - Per the table in the referenced procedure, the workers supervisor approval is NOT necessary for doses greater than 4 rem/yr but less than 4.5 rem/year. This would make none of the answers correct.
IF the workers supervisor approval is required per procedure somehow, then distracters A and B are a subset of answer C. This would be basis for making them correct answers on appeal. This question needs to be reviewed and revised.
Will revise question to tie to procedure question to make sure there is a correct answer.
Revised question to be in accordance with procedure. Question is acceptable.
99 F
3 B
S B, CR, 43.5 - Repeat of 2010 Question 99.
100 F
3 X
B E
S C, CR, 43.5 - Justify why distracter D is credible. It is saying that the TSC Director may take over the EDC role when the emergency has ended.
Will adjust wording for plausibility.
Revised the distracter. The question is acceptable.
RO TOTALS:
B= 51 F= 33 E= 22 Additional Notes: 4 Repeat questions from past 2 NRC RO exams.
Percent Unsatisfactory = 8/75 = 10.6% (resolved through comment resolution)
M= 9 H= 42 U= 8 N= 15 SRO TOTALS:
B= 14 F= 2 E= 3 Additional Notes: 1 Repeat question from past 2 NRC SRO exams identified by licensee.
Percent Unsatisfactory = 9/25 = 36% (resolved through comment resolution)
Overall Percent Unsatisfactory = 17/100 = 17% (resolved through comment resolution)
M= 4 H= 23 U= 9 N= 7
GENERAL COMMENT
S:
- 1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N
- 2.
Chief Examiner comments are indicated in blue.
- 3.
Average difficulty is __ 2-3 ___ on the RO exam and _3__ on the SRO exam.
- 4.
The 10CFR55.41/43 distribution is: RO / SRO 41.1 =
43.1 =
41.2 = 3/0 43.2 = 0/8
ES-401 AN1-2011-08 13 Form ES-401-9 Page 13 of 13 OBDI 202 - IOLE Process 41.3 = 1/0 43.3 =
41.4 = 1/0 43.4 = 0/1 41.5 = 12/0 43.5 = 4/12 41.6 =
43.6 =
41.7 = 35/1 43.7 = 0/1 41.8 = 4/0 41.9 =
41.10 = 20/2 41.11 = 1/0 41.12 = 1/0 41.13 =
41.14 =
- 5.
The answer distribution is: RO / SRO A = 19 (25%)
/
6 (24%)
B = 28 (37%)
/
7 (28%)
C = 20 (27%)
/
6 (24%)
D = 8 (11%)
/
6 (24%)
- 6.
There are 5 questions with attachments provided.
Written Exam Reviewers: Sean Hedger and Clyde Osterholtz