ML19239A206
| ML19239A206 | |
| Person / Time | |
|---|---|
| Site: | Perry  |
| Issue date: | 08/23/2019 |
| From: | Joseph Demarshall FirstEnergy Nuclear Operating Co |
| To: | NRC/RGN-III/DRS/OLB |
| Demarshall J | |
| Shared Package | |
| ML17215A919 | List: |
| References | |
| Download: ML19239A206 (20) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3. Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
Cues: The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length).
T/F: The answer choices are a collection of unrelated true/false statements.
Cred. Dist>: The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
Backward: The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SR-only. (K/A and license-level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).
Page 1 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
R-11 H
3 M
S Modified Perry NRC 2015 ILE Reference provided: PDB-A06, pp 3-7 2
R-12 F
2 X
N E
S Attachment provided: Control Room Instruments (various)
NRC:
- Distractors A and C can be readily eliminated given that both options include Instrument #3. Low plausibility when compared with the other instrument choices (#s 1,2,4,5,6), that Recirc Loop Flow Recorder C51-R614 (Instrument
- 3), would continue to provide accurate indication of Loop Flows following a Loss of All AC Power, since the Recic Pumps would also lose power during this event.
Replace Instrument #3 with a more credible instrument.
- Explanation for correct Answer D incorrectly references Containment multi-point recorder 1D23-R090A and excludes RPV Level recorder 1B21-R615.
- Explanations for Distractors A and B both reference RPV Level indicator B21-R723, which is not provided as one of the instrument options to choose from in the question.
- Explanation for correct Answer D does not address the fact that the associated instruments (i.e., 2, 4, 6) are Post-Accident Instrumentation.
ES-401 2
Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 2 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Response: Replaced #3 (Recirc flow recorder) with B21-R643, Rx Vessel Temperature recorder, renumbered the instruments on the attachment, revised the answers for C and D, and revised/corrected the pedigree explanations accordingly.
Added a description of Instrument #1 to the answer explanations for Distractors A &
B.
3 R-13 F
3 N
S 4
R-14 H
2 B
S NRC 2010 ILE 5
R-15 F
2 B
S NRC 2015 ILE (previous 2 NRC Exams) 6 R-16 F
3 N
S 7
R-17 F
3 N
S 8
R-18 H
3 B
S NRC 2013 ILE 9
R-19 H
3 B
S NRC 2007 ILE 10 R-20 F
2 B
S Attachment provided: Core Map NRC:
- LOK= F. Question requires recall of SRM locations in the core. Recoded the Question as F in the LOK Column.
- Question determined not to be Significantly Modified from Dresden 2009 NRC Exam Question #69. Recoded the Question as B versus M in the Source Column.
Response: Changed Pedigree sheet as requested.
11 R-21 H
2 X
N E
S NRC: Use of the words challenging containment integrity in the second sentence of the stem, can inadvertently cue the applicant to narrow their selections to the two answer options containing Primary Containment Limit (PCL). These words are unnecessary and their removal does not impact the question. Every competent applicant should know that an increase in drywell bypass leakage challenges containment integrity.
Response: Revised stem to delete words, challenging containment integrity and reworded.
ES-401 3
Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 3 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 12 R-22 H
2 X
N E
S NRC: Distractors A and C do not appear to be plausible. If A Channel C85 Pressure Regulator failed closed, resulting in a high Reactor Pressure condition (as evidenced by the receipt of two Reactor High Pressure annunciators), how could the CVs being open have led to an increase in Reactor Pressure?
Response: Deleted the annunciator information from the stem.
13 R-23 H
3 B
S NRC Early Review Question NRC 2015 ILE (previous 2 NRC Exams)
NRC: Question appears to be HCL. Question requires the recall and integration of two or more pieces of data. Accordingly, recoded the Question as H in the LOK column.
Response: Agree with NRC comment. Classified question as Higher Cog Level.
14 R-24 F
3 B
S 15 R-25 H
3 B
S 16 R-26 H
3 B
S 17 R-27 F
2 X
M E
S NRC: Stem Focus issue. As written, an applicant can readily eliminate Distractors A and B, using the logic that if SLC initiation prior to 120°F in the suppression pool will ensure the injection of Hot Shutdown Boron weight before suppression pool temperature exceeds the Part 2 answer, then SLC injection prior to 110°F, because it is a lower temperature, will ensure the same result.
Response: Reworded the question statement to address the stem focus issue.
18 R-28 H
3 N
S NRC Early Review Question NRC: What is 1G41-F145 and what is the basis for the 90 minute SBO Time Critical Operator Action (TCOA) to verify it closed? Is there an interrelation between 1G41-F145 and the Feedwater System? While the 90 minute TCOA in two of the distractors appears to be plausible from a time standpoint (60 minutes vs 90 minutes), the explanation of plausibility lacks sufficient detail regarding 1G41-F145.
Response: Perry has a procedure listing all the TCOAs. 1G41-F145 is an FPCC isolation valve and does not have a tie to FW. It was meant only as plausibility for the 90 minute time. Added additional information to Answer Explanation.
19 R-29 F
3 B
S NRC 2010 ILE
ES-401 4
Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 4 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 20 R-30 H
3 B
S NRC 2013 ILE NRC: Question is HCL as written (requires multiple mental processing steps, i.e.,
recall of information and integration of two or more pieces of data) and has been recoded as H in the LOK Column.
Response: Classified question as Higher Cog Level on Pedigree sheet.
21 R-31 H
3 X
N E
S NRC: Distractor C is not credible with RPV pressure 850 psig and rising.
Response: The applicant must know what the discharge pressure limit is for the RFBP. Changed first part of stem to, Power ascension following a refuel outage was in progress and changed RPV pressure to 500 psig from 850 psig.
22 R-32 F
1 2
X N
U S
NRC:
- LOD=1. Starting the standby containment cooling fans would be an obvious choice for Maximizing Containment Cooling for any applicant that has spent months training on the simulator in an ILT program.
- Distractors A and B are equivalent / synonymous options which rule out both options for an applicant who recognizes the equivalence. A CVCW Chiller needs its respective chill water pump running prior to start. Distractor A by itself would be plausible.
Response: Changed Distractor B to Start additional Nuclear Closed Cooling Pump since a malfunction of NCC could cause a high containment temp. Also deleted IAW SOI-M11 in two places. This increases the LOD.
23 R-33 H
F 2
X N
U S
NRC:
- Distractor B does not appear to be plausible. Why would an applicant believe that a Half Scram would be received without having also received a Rod Block on an IRM Upscale Trip?
- Distractor C does not appear to be plausible. Why would an applicant believe that a Full Scram would be received with only one IRM Upscale Trip?
- Balance of coverage concern with Question #37 (Exam Q #47) (i.e., IRM input to RPS resulting in the generation of a Half Scram signal).
- Question is HCL as written (requires multiple mental processing steps, i.e., recall of information and integration of two or more pieces of data) and has been recoded as H in the LOK Column.
Response: Wrote new question to address distractor implausibility and Balance of Coverage concerns. New question LOK classification is Fundamental (F).
24 R-34 H
3 B
E S
NRC 2001 ILE NRC: Stem should perhaps say what best describes RHR system operation since the correct answer does not address that RHR C pump should start immediately in LPCI injection mode.
ES-401 5
Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 5 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Response: Revised the question statement to read: Which of the following describes the response of the RHR system based on the conditions above?
25 R-35 H
3 M
S 26 (R36)
H 3
N S
NRC Early Review Question No Early Review comments.
Perry: Made minor enhancement to stem based on CE post walkthrough feedback.
27 R-37 H
3 N
S Reference provided: OAI-1703, Attachment 18. Reference modified to exclude Page 1 of 2, which removes the Drywell (DW) Hydrogen Deflagration Overpressure Limit (HDOL) information.
Perry: Deleted note on Page 2 to remove HDOL info. Page 1 removed for reason stated above.
28 R-38 H
2 X
N S
NRC: Distractor D does not appear to be plausible. Why would an applicant believe that operation of A RHR Upper Pool Cooling Isolation Valve under the conditions stated, could affect RPV Level, given that the valve is only used during refueling outages? Explanation on the pedigree sheet does not sufficiently address the plausibility of this Distractor.
Response: The applicant must recall the piping configuration to eliminate Distractor D. This line has a spectacle flange installed on the blank side.
29 (R39)
H 3
X B
N U
S NRC Early Review Question NRC:
(1) K/A mismatch. Question does not test the applicants ability to predict the impact of an EDG failure on RHR/LPCI Injection Mode. As written, there is no failure of the Division 1 EDG. The EDG responded appropriately to the receipt of a Crankcase Pressure High alarm following an automatic start signal on the LOOP/LOCA.
(2) Answer D (correct answer) is psychometrically flawed in that contains a double negative with readability confusion.
(3) Question source is listed as Modified. Question does not appear to have been significantly modified in accordance with the guidance in ES-401, Section D.2.f, last bulleted paragraph on Page 9 of 52. Accordingly, recoded the Question as B for Bank in Column 6.
Response: Replaced with a New question that includes conditions which result in the failure of an EDG.
30 R-40 H
2 B
S
ES-401 6
Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 6 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 31 R-41 H
2 M
E S
NRC: Adequate Core Cooling (ACC) is met by Submergence based on the conditions provided in the stem (i.e., RPV Water Level is +10 inches and rising).
Recommend changing the last bullet of the stem conditions to specify an RPV Level that is below TAF, but still satisfies ACC criteria via Steam Cooling, in order to enhance the discrimination validity of the question. Question is borderline LOD 2.
Response: Would not turn off a pump if < TAF. If RPV level >-25 then in a Do-Loop in EOP-01. If <-25 inches then SAMG entry is required. Currently ACC is met by submergence only. Changed last bullet in stem from rising to slowly rising to enhance discrimination validity.
32 R-42 H
2 B
S NRC 2013 ILE 33 R-43 H
3 X
N E
S NRC:
- Distractor B does not appear to be plausible. Why would an applicant believe that HPCS suction would automatically shift to the CST on High Suppression Pool Level if no such interlock exists?
- Explanation for the plausibility of Distractor A is unclear. Clarify the explanation with respect to Inadvertent ECCS initiation.
- Question is HCL as written (requires multiple mental processing steps, i.e., recall of information and integration of two or more pieces of data) and has been recoded as H in the LOK Column.
Response: 1) Replaced Distractor B with a more plausible distractor. Rearranged order of answers due to new distractor being longer than previous distractor. Correct Answer is now B. Updated Pedigree Sheet.
- 2) Changed explanation for Distractor A to read, Plausible since this is a manual Operator action for Inadvertent ECCS initiation. But, on a high Rx level the HPCS pump will run on Min Flow.
34 (R44)
H 2
B M
E S
NRC Early Review Question NRC: Overlap/Balance-of-Coverage concern with SRO Q20. Also, Distractors A and B can be easily eliminated knowing that RWCU failed to isolate. Every applicant should know that the expected action upon the failure of a system to automatically isolate, is to manually back up the isolation. Suggest keeping SRO Q20 and replacing RO Q44.
Response: Modified an existing Bank question (ID: 209410) to address the overlap concern with SRO Q20, and the issue pertaining to Distractors A and B described above.
35 R-45 F
3 B
S NRC 2007 ILE NRC: Question is Low Cog as written (memory recall) and has been recoded as F in the LOK Column.
ES-401 7
Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 7 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Response: Changed Pedigree sheet to low cog level.
36 R-46 H
2 B
S NRC: Question does not appear to have been Significantly Modified from the Hatch 2010 NRC ILE Question #10. Explain how the Hatch question has been Significantly Modified. Question coded as B versus M.
Response: Changed Pedigree sheet to Bank.
37 R-47 H
2 B
S NRC:
- Balance of coverage concern with Q #23 (Exam Q #33) (i.e., IRM input to RPS resulting in the generation of a Half Scram signal).
- Explanation for Distractor B states that the IRM reading is less than the value needed to generate a rod block or half scram. IRM B is reading 115/125, which is above the rod block value setpoint of 108/125. Revise the explanation for Distractor B to correct this error.
- Question is HCL as written (requires multiple mental processing steps, i.e., recall of information and integration of two or more pieces of data) and has been recoded as H in the LOK Column.
Response: Submitted new Q-23 which eliminated the Balance of Coverage concern.
Corrected the explanation for Distractor B and changed the Pedigree sheet to specify High Cog Level.
38 R-48 H
2 X
B E
S NRC 2005 ILE NRC: Distractor D does not appear to be credible. RO applicants would not be expected to make Operability determinations.
Response: Deleted Operability call from Distractor D and rearranged Distractors and Distractor Explanations.
39 R-49 H
3 B
E S
NRC: Editorial: Use of a double negative in Distractor D. Replace nor with or.
Response: Replaced nor with or. Also corrected Distractor lettering on Pedigree sheet.
40 (R50)
H 2
B E
S NRC Early Review Question NRC 2010 ILE NRC: Distractors A and C are fairly easy to eliminate based on information provided in the stem that APRM A is in Bypass. Suggest modifying the Question by replacing APRM A with APRM C in the first part of Distractors A and C. This will require the applicant to recall from memory the APRM/OPRM channel and associated cabinet assignments, given that A, C, and E APRMs are all A-side APRMs.
ES-401 8
Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 8 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Response: Replaced APRM A with APRM C in first part of Distractors A & C and updated the plausibility in the Answer Explanation.
41 R-51 F
2 N
S NRC: Pedigree sheet does not provide explanations for why Distractors A, B, and C are plausible.
Response: Added Distractor explanations to Pedigree sheet.
42 R-52 H
3 X
B U
S NRC:
- Answers A, B, and C would all be correct for the set of conditions specified in the stem (i.e., equivalent or synonymous options), given the phrasing of the question statement and the wording of Distractors A and B, and the fact that Answer C states Division 1 or 2.
- Last bulleted item in the stem conditions states Division 1 and 2 ADS LOGIC INHIBIT switches are in INHIBIT. Questioning when the ADS LOGIC INHIBIT switches were placed in the INHIBIT position if EOP-04-1, Alternate Level/Pressure Control, has not been entered. Stem states the plant is operating in EOP-1, RPV Control. Does not appear that EOP-1 provides direction to place the ADS LOGIC INHIBIT switches in INHIBIT.
Response: Question replaced with Perry Vision Question 204348, which was the bases for the originally submitted question.
43 R-53 H
3 M
S 44 R-54 H
3 X
B E
S NRC: Question statement is awkwardly written as it pertains to Distractor D, rendering the distractor implausible. The phrase In order to operate Safety Relief Valves to control RPV pressure, contradicts the two control switches will not work answer options in Parts (1) and (2) of Distractor D. SRVs could not be operated if the control switches on both panels were rendered inoperable/non-functional. Specific determiner that inadvertently provides a cue to correct Answer B.
Response: Reworded stem as follows, Based on these conditions, operation of the individual SRV 45 R-55 F
2 X
N E
S NRC: Questioning the credibility of Distractors B and C. Why would it be plausible for an applicant to believe that RFPT speed would either increase or decrease if both speed sensors were to fail to zero RPM? No explanation of plausibility has been provided for Distractors B and C.
Response: Changed Distractor B to: will go to the low speed stop and changed Distractor C to: will go to the high speed stop. Also changed Answer Explanation justifications.
46 R-56 H
2 B
E NRC: Stem states that the control switch for Annulus Exhaust Gas Treatment (AEGT) Fan B is taken to STOP, then placed in STANDBY? Reference
ES-401 9
Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 9 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only S
documentation SDM-M15, pp 13-14 indicated that the control switch is spring return to STANDBY from the STOP position. Clarification required.
Response: This was how AEGTS switches were operated prior to a recent procedure change. Changed the words, then placed in STANDBY to then released to STANDBY.
47 R-57 H
3 B
S NRC 2009 ILE 48 R-58 F
3 B
S NRC 2010 ILE 49 R-59 F
3 B
S NRC 2010 ILE 50 R-60 H
3 X
N S
Reference provided: SVI-R43-T1318 (modified)
Attachment provided: Division 2 DG TRA plot NRC:
- Pedigree sheet indicates that a modified version of reference SVI-R43-T1318 is to be provided to the applicants. Does not appear that the modified version of SVI-R43-T1318 has been provided with the submittal. Clarification needed with respect to how the reference material has been modified.
- TRA Plot is provided on separate pages. Is it possible to provide this information on one sheet of paper?
Response: Reference was provided. It was modified to eliminate dollar signs ($) to eliminate cue for question 70 (Note that Q-70 has been replaced. Reference remains modified as described.
This is how the TRA plot is provided in the plant.
Changed answers to Table format at the request of the CE.
51 R-61 F
3 B
S NRC 2010 ILE NRC: Revise Distractor C to specify Reactor Recirculation Pump Drive Motor instead of Reactor Recirculation Pump. Motor bearing oil reservoirs have oil coolers that are cooled by NCC.
Response: Recirc Pump is synonymous with Recirc Pump drive motor. Typically not referred to separately at Perry.
52 R-62 H
2 N
S 53 R-63 F
3 X
M E
S NRC 2003 ILE NRC: Questioning the credibility of Distractors C and D. Is it plausible that ESW Pump start would be a function of discharge valve opening time versus percent open
ES-401 10 Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 10 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only travel? Are there any pumps in the plant that start based on discharge valve opening time?
Response: Changed Distractors C & D to the same percentages in A & B. Also corrected Answer Explanations on Pedigree sheet.
54 R-64 H
4 B
S Reference provided: Drawing 302-872 (partial P&ID) 55 R-65 F
2 X
M E
S NRC: Distractor B does not appear to be credible. Why would an applicant believe that a low pressure condition downstream of 1G33-F033 would automatically close this valve when the discharge is to the condenser, which is under vacuum?
Response: Replaced Distractor B with a more plausible distractor. Updated Pedigree sheet for new Distractor.
56 R-66 F
3 N
S 57 R-67 F
3 X
N S
NRC: Question appears to be written at the SRO knowledge level. Would RO applicants be expected to recall EOP-3 Fuel Pool Temperature Control Leg and associated Bases information from memory?
Response: This is expected RO knowledge at Perry.
58 R-68 H
4 B
S NRC 2013 ILE 59 R-69 F
2 N
S NRC: Pedigree sheet has conflicting information. Question Source is coded as New, but Question History indicates Previous 2 NRC Exams; Perry 2017). Coded Column 6 as B versus N based on the Question History. Reconcile the Question Source and History information on the Pedigree Sheet.
Response: This is a new question. Deleted Perry 2017 from Question Source on Pedigree sheet.
60 R-70 F
2 M
E S
NRC: Suggest specifying a power level of 87% instead of 98% to raise the Discrimination Validity. Question is borderline LOD 2 as written.
Response: Changed stem to indicate plant power lowered to 80% to do a rod pattern adjustment. Changed Correct Answer to C. Updated the Answer Explanations on the Pedigree sheet.
61 R-71 H
2 N
E S
NRC: For Distractors B and D, suggest replacing the 308,000 gallons (in Part 1) with 282,000 gallons, to raise the discrimination validity of this question; 308,000 gallons is the upper end of the automatic CST level control band and 282,000 gallons is the lower end of the control band.
ES-401 11 Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 11 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Response: Change first part of Distractors B & D to 282,000 and updated Answer Explanations on Pedigree sheet.
62 R-72 H
2 X
N U
S NRC: K/A Mismatch. Question does not test the applicants Ability to predict and/or monitor changes in parameters associated with operating the REACTOR FEEDWATER SYSTEM controls including: Feedwater heater level. Understanding the effects of operating REACTOR FEEDWATER SYSTEM controls is not specifically addressed within the context of this question. Instead, question is asking about the effect on reactor power if no operator actions are taken in response to high level conditions in the 6A FWH resulting from the failure of Drain Valves that would function to maintain normal levels.
Response: Changed wording of stem to include operating FW System controls that affect FW Heater Level.
63 R-73 H
2 X
N E
S NRC: Distractors C and D do not appear to be credible. Why would it be plausible for an applicant to believe that condenser vacuum would not significantly degrade at 95% power following the loss of 1 of 3 Circulating Water Pumps?
Response: Changed Rx power to 82% in the stem and changed the third bullet to specify that outside ambient temperature is decreasing slowly instead of increasing slowly. This makes it more plausible that vacuum would remain the same.
64 R-74 F
3 B
S 65 R-75 H
2 B
S NRC 2009 ILE 66 R-01 H
F 3
X B
N U
S NRC 2013 ILE NRC:
- Question does not test the applicants ability to explain and apply a system limit or precaution as required by the K/A.
- Per ES-401, Paragraph D.2.a (last sentence), questions selected for Tier 3 are to maintain their focus on the plant-wide generic K/As, and are not to become an extension of Tier 2 Plant systems. As written, system-specific knowledge of Hot Surge Tank Level Control operation is required to answer this question correctly, making it an extension of Tier 2.
- Question is HCL as written (requires multiple mental processing steps, i.e., recall of information and integration of two or more pieces of data) and has been recoded as H in the LOK Column.
Response: Wrote new question regarding ECCS motor start limitations derived from multiple system P&Ls. New question satisfies criteria for Tier 3.
ES-401 12 Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 12 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Changed Pedigree Sheet to Low Cog Level, changed from Bank to New, and changed Learning Objective numbers.
67 R-02 F
3 B
S NRC 2009 ILE 68 R-03 H
2 B
E S
NRC: Suggest changing the time in Distractor D from 160 seconds to 150 seconds to raise the discrimination validity. IOI-01 states that a reasonable period is typically 60 to 150 seconds.
Response: Changed Distractor D to 150 seconds.
69 R-04 F
2 X
B E
S NRC 2015 ILE (previous 2 NRC Exams)
NRC: Distractor A does not appear to be credible. Why would it be plausible for an applicant to believe that a control room annunciator removed from service would be identified with a Danger Tag? DANGER Tags are used ONLY for equipment protection or personnel safety and not to inform operators that an alarm is OOS.
Response: Changed Danger Tag to Not In Service Sticker. Rearranged answer order.
70 R-05 F
1 2
B U
S NRC 2013 ILE NRC 2015 ILE (previous 2 NRC Exams)
NRC: LOD=1; Low Discrimination Validity. It should be obvious to all applicants that a Surveillance Test Acceptance Criteria step annotated with a dollar ($) sign, has Tech Spec applicability.
Response: Replaced with Perry 2015 # RO-04. (will put at limit for Previous 2 NRC Exam Qs).
71 R-06 F
4 N
S 72 R-07 F
3 M
S NRC Early Review Question NRC: Would classify this question as Modified vs Bank. Given the context of this question, can make the case that the change from 2000 to 10,000 dpm/100 cm2 would qualify as a pertinent change to the stem because 10,000 only differs from the High Contamination Area threshold of 100,000 dpm/cm2 by a single 0, slightly increasing the degree of discriminatory validity. This change, together with the changes to two of the distractors, would meet the intent of a significant modification as described in NUREG-1021.
Response: Agree with NRC comment. Classified question as Modified.
73 R-08 F
3 B
N E
S NRC: Per ES-401, Paragraph D.2.a (last sentence), questions selected for Tier 3 are to maintain their focus on the plant-wide generic K/As, and are not to become an extension of Tier 2 Plant systems. As written, system-specific knowledge of
ES-401 13 Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 13 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only individual radiation monitor signals that result in the automatic initiation of the D19 Post-Accident Radiation Monitor is required to answer this question correctly, making it an extension of Tier 2.
Response: Wrote new question to be more generic and meet criteria for Tier 3.
Question remains lower order but changed Pedigree sheet to show New vs. Bank.
74 R-09 F
2 X
B E
S NRC 2002 ILE NRC: Revise the stem to state that the actions require the concurrence of, similar to how the Perry Bank Source Question reads. Low plausibility to state that the RO is responsible to obtain the direction.
Response: Changed stem to state, the RO is required to obtain the concurrence of.
75 R-10 H
3 B
S NRC:
Response
76 (S8)
H 3
X X
N E
S NRC Early Review Question Reference provided: EAL Classification Matrix NRC: In order to lend plausibility to the loss of Divisional DC buses ED-1-A and ED-1-B (assuming EDGs are supplying onsite AC power to essential buses), suggest enhancing the stem by providing information that a seismic event ( OBE in magnitude) occurred at 0930, resulting in the loss of all Divisional DC buses.
Response: Per Perrys EALs, a seismic event in this situation could be classified as SA8.1, (ALERT) Hazardous event affecting a Safety System needed for the current operating mode. Per discussion with Chief Examiner, modified question to include seismic event and changed 1st part of Distractors A & B to Alert.
NRC (supplemental comment): Although previously coded as S under the Early Question Review evaluation, recoded the question as E due to the fact that the EAL Classification Matrix reference to be included can be used to directly evaluate the DC Bus Voltage values in the stem of Question 100 (SRO Q #7) and provide additional insight into answering the question.
Response to supplemental comment: Discussed with CE. Wrote new Question 100 (SRO Q #7) to address the concern identified above regarding the EAL Classification Matrix, and the fact that the original Question 100 (SRO Q #7) did not meet Tier 3 criteria (see NRC comments for Question 100 (SRO Q #7)). Early Review Question remains SAT on the basis of enhancements made to address the NRCs preliminary (Early) review comments/feedback.
77 S-9 H
2 B
S NRC 2010 ILE Reference provided: TS 3.3.1.1 (partial)
ES-401 14 Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 14 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 78 S-10 H
2 X
N E
S Attachment provided: Panel H13-P970 (partial)
NRC: Distractor A can be easily eliminated using RO plant / system knowledge (i.e.,
knowledge that is not unique to the SRO position) with the graphic of panel H13-P970 that is provided. Replace Distractor A with a more credible option.
Response: Replaced Distractor A as requested.
79 S-11 H
1 3
X X
N U
S Reference provided: EOP-02 (partial) Steam Tables NRC:
- Reference provided makes the question a direct lookup and provides cues to the applicant; LOD=1.
- Question is an exercise in transitioning to EOP-01, RPV Control, from EOP-02, Primary Containment Control, (while apparently still in EOP-01A, Level Power Control??). EOP-01 then requires the applicant (at the RC-2 IF / THEN Step) to exit EOP-1 and enter EOP-01A (which is already being implemented, i.e., 3 control rods failed to fully insert), without any apparent interpretation of the High Drywell Pressure / Drywell Temperature relationship; Question, as written, does not meet the intent of the K/A.
Response: Wrote new question. Updated Pedigree sheet to show that Steam Tables are provided as a reference.
80 S-12 H
3 X
M E
S NRC:
- The use of and/or in the first question statement introduces an element of ambiguity. Suggest the question statement read similar to the following:
With the plant at rated power, the number of SRVs required to be OPERABLE in the Safety Mode, together with the number of SRVs required to be OPERABLE in the Relief Mode, is ___(1)___?
- Separate the two question statements with a space, and drop the AND at the beginning of the second question statement.
- Clarify the plausibility explanations for Distractors A and B with respect to the EOC-RPT trip event. Explanations are confusing as written.
Response: Performed suggested changes in first two bullets. Clarified Answer Explanation for distractors A & B regarding the EOP-RPT trip event.
81 S-13 H
3 X
N E
S References provided: EOP-SPI Supplement Figures 4 & 4A NRC:
- Questioning whether Distractors B and D are credible with respect to Part (2) of the answer. An applicant would transition from EOP-01 to EOP-01A with the reactor not shutdown (3% power). Accordingly, why would it be credible for an applicant to believe that they would Anticipate ED per EOP-01?
ES-401 15 Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 15 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- Regarding correct Answer C, is it appropriate to raise Suppression Pool Level in this question, knowing that (a) HCL has not been exceeded, (b) Suppression Pool Level is 0.7 feet above the EOP-2 high level entry condition, and (c) raising level further will reduce the margin to SRVTPLL limits? Provide clarifying information in the explanation for Answer C.
Response: Changed Distractors B & D. Updated Answer Explanation on Pedigree sheet.
It is appropriate to initiate SPMU in this case. Reference EOP-2, Page 30 Discussion.
82 S-14 H
2 M
S Reference provided: TS 3.8.1 (partial - no Surveillance Requirements)
NRC: Explanation for Distractor B states that a DG is INOP if voltage is < 3800 V.
Note that SR 3.8.1.2 specifies a DG steady state voltage range of 3900 V and 4400 V. The 3800 V value referenced in the explanation for Distractor B appears to have been carried over from the Bank Question. Update the explanation with the correct voltage value.
Response: Corrected Distractor B explanation as requested.
83 S-15 H
2 3
M E
S Modified Perry NRC 2015 ILE NRC: First part of the question is RO level knowledge, and the selection of ONI-N62, Loss of Main Condenser Vacuum, is an obvious choice, given the fact that (1) the stem conditions specify degrading condenser vacuum, and (2) ONI-C71, Reactor Scram, contains post-scram actions. Suggest replacing ONI-C71, Reactor Scram, in the Distractors C and D, with OAI-1703, Attachment 21, Margins and Limits Hardcard, to raise the Discrimination Validity of this question. The SRO applicant would then have to evaluate whether a scram is required based on power/vacuum limits with the turbine on line (OAI-1703), or whether just tripping the turbine is the appropriate action (ONI-N62).
Response: Made changes to Distractors C & D as suggested. Also updated Distractor explanation.
84 S-16 F
H 3
X M
U S
Modified Perry NRC 2017 ILE NRC:
- K/A Mismatch. Table constructed in the stem does not meet the intent of the K/A with respect to the interpretation of Table Reference Material. The table lists parameter values for Steam Dome Pressure and Accumulator Pressure taken directly from the TS Bases Sections for Control Rod Scram Times and Control Rod Scram Accumulators. The Table is not Perry plant reference material.
- Question is Low Cog as written (memory recall) and has been recoded as F in the LOK Column.
ES-401 16 Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 16 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Response: Unable to develop an operationally valid and discriminating SRO level question to test the combination of selected K/A G2.1.25 (Ability to interpret reference materials, such as graphs, curves, tables, etc.) and APE #295022, Loss of Control Rod Drive Pumps. Based on discussion with CE, rejected K/As 295022 and 2.1.25, and re-selected the combination of EPE #295036, Secondary Containment High Sump/Area Water Level, and K/A G2.1.20, Ability to interpret and execute procedure steps, (IR=4.6) to maintain outline balance.
85 S-17 H
3 M
S 86 S-18 H
4 N
S Reference provided: Steam Tables 87 S-19 H
3 N
S 88 (S20)
H 3
B S
NRC Early Review Question NRC 2017 ILE (previous 2 NRC Exams)
NRC: Overlap/Balance-of-Coverage concern with RO Q44. Suggest keeping this SRO Question and replacing RO Q44. Question SAT as written.
Response: Modified an existing Perry Bank Question (ID: 209410) for RO Q44 to address the Overlap/Balance-of-Coverage concern with SRO Q20.
89 S-21 H
3 N
S 90 S-22 H
3 B
S NRC 2013 ILE Reference provided: TS 3.8.3 (partial) 91 (S23)
F 2
X N
E S
NRC Early Review Question NRC: Question: Explanation for Distractor A states that ONI-C11-1 refers the operator to SOI-C11 if no CRD pump is running. Accordingly, could Distractor A also be considered correct, given that ONI-C11-1 gets you to SOI-C11?
Response: Reworded stem as follows, The procedure that specifies the required actions to mitigate this condition is 92 S-24 H
3 N
S Reference provided: NOBP-OP-1015 93 S-25 H
3 B
S NRC 2007 ILE 94 S-1 H
3 X
N E
S NRC:
ES-401 17 Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 17 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- Distractor A does not appear to be credible. Why would it be plausible for an applicant to believe that entry into EOP-404, RPV Flooding, would be required, when RPV water level is provided in the stem?
- Explanation for Distractor C is confusing. Explanation implies that EOP-4-1, Alternate Level/Pressure Control, and the SAMGs would both be entered. This appears to contradict correct Answer D which states that the actions of all EOPs are discontinued when implementing the SAMGs. Also, if actions of the EOPs are discontinued when in the SAMGs, then why does EOP-01 circle back to Step RLC-1 downstream of SAMG entry step RCL-16? Additional clarification required with respect to the explanations for the Distractors B and C, and correct Answer D.
- Question is HCL as written (requires multiple mental processing steps, i.e., recall of information and integration of two or more pieces of data) and has been recoded as H in the LOK Column.
Response: Replaced Distractor A and rearranged distractors based on length of the new distractor. Updated Pedigree sheet accordingly. Also increased HPCS flow to avoid having 2 correct answers with new distractor.
For 2nd part of 2nd bulleted NRC comment above, see EOP-01 Bases Discussion on Page 68, Step RLC-16 (i.e., SAMG entry is required - not ready to enter yet - prep-steps in EOP-01 will be continued until entry into SAMGs is made). Updated Pedigree sheet plausibility statement for Distractor A to include this information.
Changed Pedigree sheet to High Cog Level.
95 S-2 F
3 B
S NRC: Question determined not to be Significantly Modified from Fermi 2008 NRC Exam Question #94, based on ES-401 guidance. Conditions in the stem were not significantly modified. Recoded the Question as B versus M in the Source Column.
Response: Changed Pedigree Sheet to Bank.
96 S-3 F
3 X
B S
NRC:
- Question tests knowledge of the Reactor Coolant System Pressure Safety Limit.
SR-only knowledge cannot be claimed for questions that can be answered solely based on knowledge of the safety limits, since Reactor Operators are typically required to know these items.
- Question determined not to be Significantly Modified from Columbia 2015 NRC Exam Question #97, based on ES-401 guidance. Conditions in the stem were not significantly modified. Recoded the Question as B versus M in the Source Column.
Response: First part of the question is RO knowledge. Changed Part 2 of Distractors A and C from RPV Feedwater Nozzles to Reactor Recirc Discharge piping.
ES-401 18 Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 18 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 97 S-4 F
3 N
S 98 S-5 H
2 X
B E
S NRC 2013 ILE NRC:
- The reference to D17-K836 in the last bullet of the stem provides a cue to the applicant that the answer is either choice B or D (Distractors A and C can be readily eliminated), because both options specify procedure ONI-D17, which includes the D17 designator. The correct answer is choice B.
- Per ES-401, Paragraph D.2.a (last sentence), questions selected for Tier 3 are to maintain their focus on the plant-wide generic K/As, and are not to become an extension of Tier 2 Plant systems. Question is borderline Tier 3 (i.e., aspects of plant-wide and system specific knowledge are tested).
Response: Question meets the intent of Tier 3 on the basis that ONI-N11 is entered for a Pipe Break Outside of Containment and ONI-D17 is entered for High Radiation Levels Within the Plant. Both of these ONI procedures are provided as answer selections. The entry conditions for ONI-N11 include alarming rad monitors on H13-P804, which could be indicative of a pipe break from one of several areas outside of containment, depending on the location of the break. Note that both ONI-D17 and ONI-N11 could be entered concurrently depending on plant location and severity of the event.
Also, deleted D17-K836 from last bullet in the stem to preclude any possibility of cueing the applicant.
99 S-6 F
2 B
S 100 S-7 H
F 2
X N
B E
S NRC:
- EAL Classification Matrix reference included in Question 76 (SRO Q #8) can be used to directly evaluate the DC Bus Voltage values in the stem and provide additional insight into answering this question.
- Distractor D is not plausible. Vital DC Voltage conditions specified in the stem place the Unit in a Site Area Emergency (SAE) IAW EAL SS2.1. Securing all the DGs would result in the loss of all Offsite and Onsite AC power capability, and intentional escalation to a General Emergency (GE) IAW EAL SG1.2.
- Per ES-401, Paragraph D.2.a (last sentence), questions selected for Tier 3 are to maintain their focus on the plant-wide generic K/As, and are not to become an extension of Tier 2 Plant systems. System-specific knowledge of Divisional DC Bus voltage and assessment of the need for emergency diesel generator power is required to answer this question correctly, making it an extension of Tier 2.
- Distractors C and D use blank lines instead of identifying the complete ONI-R42 procedure reference.
ES-401 19 Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 19 of 20 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Response: Replaced with a Bank question that tests applicant knowledge of normal /
transient alarm response protocol, in order to satisfy criteria for Tier 3. Updated Pedigree sheet to specify Bank for the question source and Fundamental for the question cognitive level.
ES-401 Written Examination Review Worksheet Form ES-401-9 Facility: PERRY NUCLEAR POWER STATION Exam Date: February 11 - 22, 2019 Page 20 of 20 RO TOTALS:
B= 37 F= 32 E= 22 Additional Notes: 8% of RO Questions assessed as unsatisfactory.
M= 10 H= 43 U= 6 N= 28 (14 HCL)
SRO TOTALS:
B= 9 F= 6 E= 8 Additional Notes: 8% of SRO Questions assessed as unsatisfactory.
M= 5 H= 19 U= 2 N= 11 (9 HCL)
GENERAL COMMENT
S:
1.
There are 7_ (RO) / 8_ (SRO) questions with attachments/references provided.
2.
Questions from the previous 2 NRC Exams: _4_ (RO) / _1_ (SRO) 3.
Average difficulty is __2.60 _ on the RO exam and 2.76 on the SRO exam.