ML15265A316
| ML15265A316 | |
| Person / Time | |
|---|---|
| Site: | Arkansas Nuclear  |
| Issue date: | 09/09/2015 |
| From: | Vincent Gaddy Operations Branch IV |
| To: | Entergy Operations |
| References | |
| Download: ML15265A316 (11) | |
Text
Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Backw ard Q -
K/A SRO Only 1
F 2
N S
D, 41.7, CR 2
H 3
N S
A, 41.7, CR 3
F 3
M S
D, 41.10, CR, 1998 NRC Exam 4
H 3
E N
E S
C, 41.10, CR Need to add OP-2203.002 to the stem and make part 2 of the answers consistent (i.e. based on )
Will make the change.
The question was edited as recommended.
Question Sat 5
F 3
E N
E S
B, 41.10, CR The reference (Step 6) used in the answer justification for the 10-90% level is a contingency action for verifying natural circulation. Step 8 discussed the reasons for 10% and 90%.
Discussion on Answer C is confusing due to the limit of 90% is for water carryover in both shutdown and power operations. Recommend rewording the justification.
Will make the change.
The justification was revised.
Question Sat 6
H 3
B S
C, 41.5, CR, 2005 and 2006 NRC Exam 7
H 4
N S
A, 41.7, CR 8
F 2
N S
DC, 41.7, CR Changed order of the answers.
Question Sat Instructions (Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question a 1(easy) to 5 ( difficult); (questions with a difficulty between 2 and 4 are acceptable)
- 3.
Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
Cues: The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc.).
T/F: The answer choices are a collection of unrelated true/false statements.
Cred. Dist: The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
Partial: One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
Minutia: The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- / Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
Backward: The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only (K/A and license level mismatches are unacceptable)
- 6.
Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-401 Section D.2.f.
- 7.
Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any "U" Status ratings (e.g., how the Appendix B psychometric attributes are not being met).
Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Backw ard Q -
K/A SRO Only 9
H 3
N S
C, 41.2, CR Modified stem following validation Question Sat 10 F
3 E
N E
S A, 41.7, CR Cueing in Answer A. The stem says you have a tube rupture in 2 of the three given conditions and the correct answer actually says ruptured SG.
Maybe use minimize RCS break flow Will make the change.
The question was revised.
Question Sat 11 H
3 B
S D, 41.7, CR 12 F
2 E
E N
E S
B, 41.4, CR References discuss Main Feed Reg Valve bypass opening instead of MFWP Recirc Valve closing.
Confusing did you want to discuss the operation of the MFWP Recirc valve or the MFRV Bypass valve. Think the KA follows more with the MFRV Bypass valve operations.
Will up-date the references.
The references were revised.
Question Sat 13 H
3 N
S AC, 41.10, CR Modified Question due to change in EOP identified during validation Question Sat 14 F
2 U
N U
S A, 41.7, Answer B contradicts itself. The first part of the answer aligns SW to the suction of EFW and the second part states that it minimizes the possibility feeding the SGs with SW.
Will replace half of the two by two.
This question was revised.
Question Sat 15 H
3 B
S D, 41.8, CR, 2009 NRC Exam 16 F
2 N
S A, 41.7, CR 17 H
3 M
S B, 41.4, CR, 1998 NRC Exam 18 H
3 M
S B, 41.4, CR, 2012 NRC and 2013 Audit Exams 19 H
3 U
B U
S CA, 41.7 KA mismatch - does not test the applicants knowledge of operating/monitoring RPI following
Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Backw ard Q -
K/A SRO Only CRW.
This question is not a KA mismatch as written -
CEAC is used as part of the RPI.
Modified stem and answers to address comments during validation. Removed extra information from answers that was not required.
Correct answer now A Question is Sat 20 H
2 U
N U
S B, 41.5, KA mismatch - does not test the reason for the TS limits for response to a dropped rod.
Will reword part of the stem to correct.
This question was revised.
Question Sat 21 H
3 U
N U
S A, 41.5 KA mismatch - question does not test knowledge of the comparison of makeup and letdown flow.
This question is Sat as written.
22 H
3 N
S D, 41.7, CR 23 F
3 B
S A, 41.7, CR, 2000 and 2005 NRC Exam 24 H
2 N
S C, 41.10, CR 25 H
3 N
S C, 41.7, CR 26 H
3 N
S D, 41.5, CR Editorial corrections in response to comments made during validation.
Question Sat 27 F
3 N
S B, 41.7, CR 28 F
3 B
S B, 41.10, CR, 2000 and 2011 NRC Exam Editorial corrections in response to comments made during validation.
Question Sat 29 H
3 N
S B, 41.3, CR 30 F
3 N
S C, 41.3, CR 31 F
3 M
S D, 41.7, CR, 2012 NRC Exam 32 H
3 N
S A, 41.5, CR 33 F
3 E
B E
S D, 41.3, CR, 2012 NRC and 2013 Audit Exams Add the procedure reference (2103.002) in the stem Will add the procedure reference to the stem.
The procedure reference was added to the stem.
Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Backw ard Q -
K/A SRO Only Question Sat 34 H
3 E
M E
S D, 41.11, CR Delete may be from the last line of the stem.
Will change.
This question was edited.
Question Sat 35 F
3 N
S A, 41.4, Reference discussed in support of the correct answer is not contained in the documents.
OP-2104.028 Section 20.0 Will provide the reference.
Reference provided Question Sat 36 F
3 N
S B, 41.7, CR 37 F
2 N
S B, 41.6, CR 38 H
3 M
S D, 41.7, CR Answer justification needs to be corrected. List D as correct and incorrect.
Justification corrected Question Sat 39 H
3 E
N E
S A, 41.7, CR It could be argued distractors A and B are both correct (i.e. the overloads may be reset after the cause is determined. Recommend adding ONLY to distractors B and D.
Will make the recommended changes.
This question was revised as recommended.
Question Sat 40 F
2 N
S D, 41.4, CR 41 H
3 B
S C, 41.10, CR, 2012 NRC Exam Bank question since it is not significantly changed from original question.
Question Sat 42 F
3 N
S A, 41.7, CR 43 H
3 N
S A, 41.4, CR 44 H
3 N
S C, 41.5, CR Modified question in response to comments during validation.
Question Sat 45 F
3 N
S A, 41.7, CR Modified question in response to comments during validation.
Question Sat 46 F
3 N
S B, 41.4, CR
Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Backw ard Q -
K/A SRO Only 47 F
3 U
B U
S B, 41.4, CR, NRC 2011 KA mismatch because the question asks the effect of the DC system on the AC system rather than the effect of the AC system on the DC system.
This question is Sat as written.
48 H
2 E
B E
S C, 41.7, CR, 2006 NRC Exam Both C and D are correct answers since observing the voltage or maintenance testing could determine the severity of the ground.
The answer justifications are incorrect for the answers given. Need to provide correct justifications.
Will update justifications. Will revised the second distractor.
This question was revised.
Question Sat 49 F
3 E
B E
S C, 41.8, 1998 NRC Exam What is the reference for the loads off of 2Y13/24?
Load is spelled wrong in the stem.
Will correct spelling and add a reference.
Still need to change load to loads or loading in stem for it to read right.
The question was revised.
Question Sat 50 F
3 B
S B, 41.8, CR, 2009 and 2014 NRC Exam 51 F
3 M
S D, 41.7, CR, 2002 and 2008 NRC Exam 52 F
3 E
B E
S A, 41.12, CR It could be argued there is no correct answer since there is more than one required action to continue discharging. Recommend changing stem to read something like Which of the following is a REQUIRED action to continue the release Will revise the question.
This question was revised as recommended.
Question Sat 53 F
3 U
B U
S B, 41.3, CR KA mismatch - The question does not address the ability to predict or monitor changes associated with operating the SWS controls.
The cooldown rate is controlled through the use of SWS controls - this question is Sat as written.
54 F
2 E
N E
S B, 41.4, CR Answer A and B would both be correct from the given stem. Recommend change stem to read something like remain closed for a minimum of
_____ minutes.
Will revise the question.
Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Backw ard Q -
K/A SRO Only This question was revised as indicated.
Question Sat 55 F
3 B
S B, 41.9, CR, 2008 NRC Exam 56 H
3 N
S C, 41.6, CR 57 H
3 N
S CD, 41.3, CR Missing reference to support the flange level Will provide the reference.
Reference provided Rearranged answers and modified correct answer.
Question Sat 58 H
3 N
S B, 41.6, CR Modified answers to better align with AOP Question Sat 59 F
3 B
S A, 41.7, CR, 2006 NRC Exam 60 F
2 B
S D, 41.9, CR, 2011 NRC and 2012 Audit Exam 61 F
3 U
B U
S B, 41.13, CR 2005 NRC SRO only question due to requiring knowledge of the fuel handling equipment.
Answer A could also be considered correct if the hoist cable is considered part of the fuel grappling assembly.
Will replace the KA.
The KA and question were replaced.
Question Sat 62 H
3 E
E N
E S
D, 41.4, CR Stem stating that the vacuum is degraded due to air in-leakage cues that answer D is correct.
Need to rework the stem. Placing a lowering vacuum in the conditions and asking for what actions are required to restore condenser vacuum. That would have the applicant assess that the lowering vacuum is due to air in-leakage instead of being told as part of the stem.
Handswitch in the first sentence in the Notes section needs the n removed from the end.
Will revise question.
This question was revised.
Question Sat 63 H
3 N
S C, 41.13, CR Suggest changing answer B to same structural format as other three answers.
Distractor B was revised.
Question Sat 64 H
3 B
S A, 41.11, CR, 2012 NRC Exam
Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Backw ard Q -
K/A SRO Only 65 H
3 N
S D, 41.4, CR 66 F
2 M
S C, 41.10, CR, 2014-2 NRC Exam Need to correct the stem:
and the MAXIMUM.
requirement is within a This question was edited as recommended.
67 H
2 N
S DC, 41.10, CR Modified stem and answers to remove SRO aspects Question Sat 68 F
2 N
S B, 41.10, CR Justification notes need to annotate the correct answer as Answer B.
Corrected notes Question Sat 69 H
3 B
S A, 41.10, CR, 1998 NRC Exam 70 H
3 N
S D, 41.10, CR 71 F
3 B
S D, 41.13, CR, 2006 NRC Exam Bank question due to the question was not significantly modified Question Sat 72 H
3 B
S C, 41.13, CR, 2012 NRC Exam 73 F
2 B
S A, 41.8, CR, 2012 NRC Exam 74 F
2 E
B E
S C, 41.10, CR, 1998 and 2008 NRC Exams Edit, make the stem read MAXIMUM allowed interval Will revise the question.
This question was revised as recommended.
Question Sat 75 H
3 B
S D, 41.7, CR, 2014 NRC Exam Bank question due to the question was not significantly modified Question Sat 76(1)
H 3
U N
U S
BA, 41.5, CR This is an RO question because it can be answered using systems knowledge and alarm card actions. May want to consider selecting a new KA Will revise the question.
This question was revised.
Question Sat 77(2)
H 3
B S
B, 43.5, CR, 2012 NRC Exam Bank question due to the question was not significantly modified Question Sat
Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Backw ard Q -
K/A SRO Only 78(3)
H 3
E M
E S
BD, 43.2, CR, 2012 NRC Exam If answer B is correct then answer D would be correct in stating that the RCS is capable of accommodating pressure surges during all transients.
Will revise the question.
This question was revised.
Question Sat 79(4)
H 3
U N
U S
C, 41.7, CR This is an RO question since it tests on AOP entry conditions and associated immediate operator actions. It is a KA mismatch because it does not test on EOP interlocks, etc, associated with EOP entry conditions.
This question is Sat as written.
80(5)
H 3
M S
A, 43.2, CR 81(6)
H 3
U M
U S
D, 41.10, CR This is an RO question since it tests on AOP entry conditions and associated immediate operator actions.
This question will be revised.
Replaced question with modified bank question Question Sat 82(7)
H 3
U N
U S
DB, 41.7, CR Ro system knowledge only required to answer question This question is ok as written as it is from a contingency procedure.
Modified question due to overlap with audit exam JPM.
Question Sat 83(8)
H 3
B S
B, 43.7, CR, 2012 NRC Exam 84(9)
H 3
U N
U S
B, 41.10, CR This is an RO question since it tests on AOP entry conditions and associated immediate operator actions.
This question is Sat as written as it is from a contingency procedure.
85(10)
H 3
U N
U S
C, 41.10, CR This is an RO question since it tests on AOP entry conditions and associated immediate operator actions.
This question is Sat as written.
86(11)
H 3
N S
C, 41.10, CR Modified stem following validation.
Question Sat 87(12)
H 4
N S
B, 43.2, CR
Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Backw ard Q -
K/A SRO Only 88(13)
H 3
N S
C, 43.2, CR Justification notes have answer C as correct and incorrect. Need to change the incorrect C justification to D Justification corrected Question Sat 89(14)
H 3
U B
U S
A, 43.2, CR, 2014 NRC Exam This is an RO question since it tests on AOP entry conditions and associated immediate operator actions.
This question is Sat as written.
90(15)
H 3
U N
U S
D, 41.10, CR This is an RO question since it tests on AOP entry conditions and associated immediate operator actions.
This is an SRO question since the transition to the SPTAs is in the AOP making it a subsequent EOP.
Question Sat 91(16)
H 3
U N
U S
D, 41.10, CR This is an RO question since it tests on AOP entry conditions and associated immediate operator actions.
This question will be revised.
This question was revised. Change the cue to read the CRS would transition to. Instead of the CRS would use.
The question was revised.
Question Sat 92(17)
H 4
E N
E S
AB, 43.5, CR Based on the references provided, distractors A and B both appear to be correct depending on what procedure (the ACA or the OP) is referenced.
Will revise this question.
This question was revised.
Question Sat 93(18)
H 4
E N
E S
C, 43.2, CR Answer A and B should be changed to be placed in Level Cal.
We do not expect applicants to have the tables associated with TS memorized. Recommend providing TS 3.3.1 as a reference.
Will revise this question.
This question was revised.
Question Sat 94(19)
H 4
E M
E S
C, 43.5, CR In order to correctly answer this question, the applicant would have to have the procedure memorized, including the tables defining the different fire zones. This is not a realistic expectation. If the reference is provided, then it becomes a direct lookup.
Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Backw ard Q -
K/A SRO Only Similar to question 90 for fire actions.
Recommend changing the question Will revise this question.
Replaced with modified bank question.
Question Sat 95(20)
H 2
U B
M U
S A, 41.7, CR, 2011 NRC Exam This is an RO question since it is testing systems knowledge. It is a KA mismatch because this is a tier 3 question and it does not have a generic application. Bank question was a previously used RO question.
Will change the reference to ts bases.
New KA selected. Replaced question with modified bank question. Change cue to require prompt transition into. from required prompt entry into.
Question Sat 96(21)
F 3
N S
C, 43.2, CR 97(22)
F 4
U N
U S
DA, 43.2. CR, 2009 NRC Exam KA mismatch because this is a tier 3 question and it does not have a generic application. Previous use was a tier 2 question. In addition, we do not expect applicants to have 24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> action statement memorized.
Will revise the question.
Selected new KA and replaced with new question Question Sat 98(23)
F 4
B S
A, 43.7, CR 99(24)
F 3
E M
E S
D, 43.5, CR, 2009 NRC Exam Need to change the second bullet in the stem to Reactivity Control safety function instead of Reactivity safety function.
Will revise the question.
This edit was made as recommended.
Question Sat 100(2
- 5)
F 3
B S
A, 43.5, CR, 2012 NRC Exam
ES-401 Written Examination Review Worksheet Form ES-401-9 NUREG 1021, Revision 10 RO TOTALS:
B= 25 (33%)
F= 37 (49%)
E= 13 (17%)
Additional Notes:
M= 8 (11%)
H= 38 (51%)
U= 3 (4%)
N= 42 (56%)
SRO TOTALS:
B= 5 (20%)
F= 5 (20%)
E= 5 (20%)
Additional Notes:
M= 6 (25%)
H= 20 (80%)
U= 5 (20%)
N= 14 (56%)
GENERAL COMMENT
S:
- 1.
Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N
- 2.
Chief Examiner follow-up comments are indicated in blue.
- 3.
Average difficulty is _2.79 on the RO exam and 3.20 on the SRO exam. These values for this reviewer indicate the RO exam is average difficulty and the SRO exam is higher than average difficulty.
- 4.
The 10CFR55.41/43 distribution is: RO / SRO 41.1 = 0 43.1 = 0 41.2 = 1 43.2 = 7 41.3 = 5 43.3 = 0 41.4 = 10 43.4 = 0 41.5 = 6 43.5 = 5 41.6 = 3 43.6 = 0 41.7 = 24 43.7 = 2 41.8 = 4 41.9 = 2 41.10 = 13/1 41.11 = 2 41.12 = 3 41.13 = 2 41.14 = 0
- 5.
The answer distribution is:
A = 19 / 8 B = 20 / 5 C = 18 / 7 D = 18 / 5
- 6.
There are 2 / questions with attachments provided.