ML15278A244

Waterford - WT-2015-09 Draft Written Exam Comments

ML15278A244
Person / Time
Site:	Waterford
Issue date:	09/14/2015
From:	Vincent Gaddy Operations Branch IV
To:	Entergy Operations
References
Download: ML15278A244 (14)

Text

ES-401 WT-2015-09 Written Examination Review Worksheet Form ES-401-9 Instructions [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable). 3. Check the appropriate box if a psychometric flaw is identified: $ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). $ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc). $ The answer choices are a collection of unrelated true/false statements. $ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable. $ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified: $ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). $ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). $ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). $ The question requires reverse logic or application compared to the job requirements. 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met). Attachment 8_Rev 1 Page 1 of 14 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 1 H 3 N S D, CR, 41.7 2 H 3 X N U S A, OR, 41.3 (Steam Tables) - With the initial conditions, the applicant cannot come up with a correct answer for the tail pipe temperature unless he/she has information on what the state of the PRT rupture disc is. If the rupture disc is intact, then tail pipe temperature is driven by saturation temperature for the PRT's pressure. If the disc has ruptured, then it is further along in the accident, and tail pipe temperature, influenced by superheated conditions in the RCS, will stabilize at ~285F (check plant-specific analysis data). Therefore, the initial conditions have to state whether the PRT pressure is increasing or decreasing. Right now, the question stem lacks to focus to elicit a correct answer. All answers can be incorrect. As discussed, added "with the Quench Tank rupture disc intact" to the list of initial conditions. 3 H 3 M S C, CR, 41.4 - Editorial: For the Reference, is TS 2.2.1 correct? These appear to be ES-401 WT-2015-09 2 Form ES-401-9 Attachment 8_Rev 1 Page 2 of 14 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only RPS trip set points, not ESFAS set points. Added Table 3.3-4 as a reference (ESFAS Instrumentation Trip Values). ESFAS set point same as the RPS set point. 4 H 3 X N E S A, CR, 41.7 - The references provided don't show why the 15% RWSP level is plausible, but wrong. Also, they don't provide any information about where RAS can be manually initiated. Training materials not provided. Add attachment 4.67 of OP-500-009 as a reference (RAS pre-trip). Other reference materials were provided. 5 H 3 X M U S C, CR, 41.10 - The Note at the beginning of Section E6 of OP-901-130 says that all of the stated temperature values used as thresholds for action are "for guidance only." Can it be argued that securing a RCP is not "required" at the specified temperature? The first part of the question has no correct answers if the applicant is specifying the procedurally "required" temperature. Changed the stem to state "provides the guidance to" instead of "the crew is required to". Alleviates the concern of the note at the beginning of the section that states the following temperatures are for guidance only. The way that Section E5 is written, securing all RCPs is not contingent on whether RCP reverse rotation is confirmed or not. Is that the intent? Added "only" to the two distracters. 6 H 3 N S C, CR, 41.7 7 F 3 N S B C, CR, 41.7 8 H 3 B S A, CR, 41.7 9 H 2 X B E S D, CR, 41.7 - The CPCs are part of the RPS measurement channel input in the PPS. Therefore, answers C and D are both correct. Added the word "first" to the stem. This is a CPC Aux trip generated because the pressurizer pressure value is outside the algorithm of the CPC DNBR calculation. 10 H 2 X N E S C, CR, 41.10 - Consider capitalizing IS and IS NOT in the answer choices. Recommended change has been made. 11 H 4 X N E S C, CR, 41.10 - Recommend stating in the cue that the CRS determines that the affected S/G is affecting the ability to cooldown the RCS. Use wording from the procedure. Added "and is limiting the RCS cooldown" to make it consistent with step 50 in OP-902-007. 12 H 4 B S D, CR, 41.10 13 F 3 X B E S A, CR, 41.8 - Provide the technical guide or procedure bases document that supports why the RCPs are secured. Include RCP operating curves justifying why ES-401 WT-2015-09 3 Form ES-401-9 Attachment 8_Rev 1 Page 3 of 14 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only the NPSH consideration is a plausible distracter. The RCP operating curves were provided to demonstrate the distracters' plausibility. 14 H F 3 X X N U S B, CR, 41.10 - The question does not test the ability to determine the operability/availability of safety-related equipment. It tests knowledge of a time requirement associated with equipment that the stem tells the applicant it is available, and the reason for the time requirement doesn't address the K/A statement either. This question has been changed. The same K/A has been kept. The change swapped the question's cognitive level from Comprehension/Analysis to Memory/Fundamental Knowledge. Aside from that, the subject matter of the question overlaps with JPM P3. One of the items has to be removed from the exam. This question has been changed. 15 H 3 X N E S B, OR, 41.10 (Steam Tables) - Per OI-038-000, Step 5.2.4, measuring subcooling margin is done with either Thot or the CETs, depending on the condition. Offering 15 and 20F subcooling as answers would better match this. Clarify that the Thot and Tcold temperatures are for one RCS loop. Changes made to enhance the answer options. 16 F 3 N S D, CR, 41.10 17 H 3 X N E S B, CR, 41.7 - The provided reference supports the correct answer, but doesn't support the distracters. It doesn't explain that at 102F, the Essential Chillers switch to "Wet Mode," etc. Additional reference provided. 18 H 3 X N E S C, CR, 41.4 - OP-901-314 doesn't specify the action to raise reactive load. Where is this specified as an approved action to address the situation? Added "out" to the 200 MVAR in the initial conditions. Increasing voltage in a lagging generator will raise MVAR. Added slides from generator theory to show this. Are RO applicants trained on the TRM bases? Yes. 19 H 3 X N E S A, CR, 41.2 - K/A is for actions to be taken if automatic safety functions have not taken place. What auto safety function did not take place for this question? Don't see how it meets K/A. Will provide additional reference to justify how it meets the K/A statement. The auto action that did not work is a CEA withdrawal prohibit (CWP). A CWP will occur on a single CEA deviation of 5.5 inches. The rod stepped out 8 inches. Provided a reference to support the explanation. 20 H 3 B S D, CR, 41.2 (2011 NRC Exam) 21 H 2 M S C, CR, 41.2 ES-401 WT-2015-09 4 Form ES-401-9 Attachment 8_Rev 1 Page 4 of 14 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 22 F 2 B S B, CR, 41.11 (2010 NRC Exam) 23 H 3 B S D, OR, 41.10 (Steam Tables, 2010 NRC Exam) 24 F 2 X B E S D, CR, 41.10 (2014 NRC Exam) - Justify why the Containment PIG rad monitors are plausible distracters. The containment PIG draws a sample from containment the same as the Containment High Range Monitors. The containment PIG is not located in containment and is not affected by rising containment temperature. The applicant must determine this. This statement has been added to the Explanation for distractors A and B. 25 H 3 N S A, CR, 41.4 26 F 3 B S B, CR, 41.9 (2009 NRC Exam) 27 H 3 X N E S D, CR, 41.7 - The question test the applicant's knowledge of procedural actions, but not the reason(s) for the actions. K/A mismatch. After review, it was determined that the question matched the K/A statement. Made changes to second part question. 28 H 3 B S D, CR, 41.3 (2011 NRC Exam) 29 H 2 X B E S B, CR, 41.5 (2011 NRC Exam) - Provide references that show how the RCP oil lift pumps operate in this situation. References provided. 30 H 2 X N S C, CR, 41.5 31 H 2 N S D, CR, 41.7 32 F 2 X B E S B, CR, 41.8 (2010 NRC Exam) - According to Attachment 11.3, valve SI-138A is LPSI Header to RC Loop 2B Flow Control Valve. Fix the valve number, or the valve description. Changed the valve identification to associate it with RC Loop 2B. 33 H 2 B S C, OR, 41.7 (2011 NRC Exam, OP-902-009, Attachments 2-E/2-F) 34 H 3 X B M U S A B, CR, 41.7 (2014 NRC Exam) - The K/A statement calls for testing knowledge of the effect on containment by a loss or malfunction of PRTS. Even if a rupture disc rupture is counted as a "loss" of the system, the question tells the applicant what happens to containment as a result. When this question was used in 2014, it was tied to another K/A statement. K/A mismatch. Modified the draft question to match the K/A statement. 35 F 2 X B E S A, CR, 41.10 (2007 NRC Exam) - With the stated CCW configuration, it is unclear if this occurs during power operations or shutdown. This could affect the actions, per ES-401 WT-2015-09 5 Form ES-401-9 Attachment 8_Rev 1 Page 5 of 14 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only Step 1, Section E2, OP-901-510. Has this been checked? Revised the question stem to state that the initial condition is with the plant at 100% power. The cited reference doesn't state that the basis for why the actions are taken (to prevent runout) is not stated within it. Reference provided. 36 H 2 N S B, CR, 41.7 - The question is satisfactory, but it overlaps with content of one of the operations test scenarios. Replace question. Replaced with another question addressing the same K/A statement. Satisfactory. 37 H 3 B S B, CR, 41.6 (2007 NRC Retake Exam) 38 H 3 X N E S C, CR, 41.4 - Need to provide technical reference showing prioritization of alarms. This is needed in case there's an appeal on the prioritization. Two annunicator response procedures are in use here: OP-500-006 (Cabinet F, M-16) and OP-500-009 (Cabinet K, F-12 and G-12). The focus of both is to restore S/G levels, so that isn't basis to prioritize one over the other. OP-500-009 does have the action to enter abnormal procedure OP-901-201. IF there is a reference that states that annunciator actions that drive the crew into abnormal procedures take precedence, then this makes the case. Otherwise, an applicant could argue both answers are correct, and both are in conflict with each other. The licensee provided references showing that an operator, with multiple annunciators, will prioritize the alarms, determining the most significant one. It is the licensee's position that an annunciator that indicates a pre-trip signal warning of a potential reactor trip is the most significant. Procedure issue: The Recommended Actions for S/G level LO Pretrip B/D alarms tell the operator to LOWER S/G level. This is totally incorrect. See OP-500-009 (Cabinet K, F-12 and F-14). Actions for LO Pretrip A/C are correct. Enter into corrective action program. 39 F 3 X N E S B, CR, 41.7 - SD-PPS, Figure 40, demonstrates that one train of Containment Spray is actuated when this action is taken. Are the figures the same for the other train? Need to understand to ensure second part of the question has the correct answer. References provided. 40 H 3 N S D, CR, 41.7 41 H 3 N S B, CR, 41.7 42 F 2 M B S A D, CR, 41.7 (2009 NRC Exam)- To count as Modified, at least one of the distracters has to be changed from the original Bank question. The question needs to be replaced. It overlaps with the content of simulator JPM S5. Randomly selected a new K/A (026 A4.01) and selected a new question (Bank) due to overlap with JPM S5. Question is satisfactory. 43 F 3 X B S A D, CR, 41.3 (2011 NRC Exam) - Question: If MS-319A fails all the way open, is ES-401 WT-2015-09 6 Form ES-401-9 Attachment 8_Rev 1 Page 6 of 14 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only there reference data/information that supports that the cooldown rate would be exceeded? In this case, the cooldown rate will be exceeded. Per the licensee, to do a rapid cooldown, the SBCV is only taken to 30% open, which exceeds the cooldown rate. No changes required. 44 F 2 X N E S D, CR, 41.7 - It isn't very plausible that a pump auto trip feature is active when the governor valves are closed. That means it is active even when it is not in use. A more plausible distracter will be to use one of the other active set points for the other trip features (pressure, pump speed, etc.). Discussed, and additional references provided to support plausibility of question. 45 H 3 N S C, CR, 41.7 46 H 3 X X N E S D, CR, 41.7 -Provide a reference that shows that the proposed condition will not result in a reactor trip. Reference provided. 47 H 4 X X N U S A, OR, 41.10 (Tech Spec 3.8.1.1; OP-903-066, Attachments 10.1 and 10.2) - The correct answer is contingent on the availability of the temporary emergency diesel generator. Nothing is stated, so an applicant could assume that it is available. If so, in reading the Tech Specs, that gives the plant 10 days vice 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> to restore the EDG. Therefore, there is no correct answer provided. Per the licensee, the applicants are taught that there is NOT a temporary emergency diesel generator available onsite. There is a mechanism to request one, and it would affect the Tech Spec action time once it is installed and operable. However, in this question, it is not onsite. Therefore, there is a correct answer provided. Adding text to Explanation to justify this. From review of Attachments 10.1 and 10.2, it is assumed that this confirms that either East or West Bus Feeder supplying meets operability with the OCB 7176/7172 and 7186/7182 conditions. Is this correct? Yes This tests LCO actions that are greater than 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />. That is SRO-only knowledge. The question has been revised to test RO knowledge regarding Tech Specs. 48 F 3 X B E S C, CR, 41.8 (2007 RO Retake Exam) - The reference says that with the 125 VDC loss, EDG control power is lost. It doesn't say that if the EDG is running, it is tripped.

Add supporting reference to ensure that that distracter A is not correct. Provided additional reference supporting distracter A as being incorrect. 49 H 3 X B E S C, CR, 41.7 - To clearly tie this to the (b) portion of the K/A statement, a cross-reference to the procedure needs to be made. The action needs to be supported by a specific action in the procedure. Review the EDG procedure and make sure this tie can be made. Added per OP-009-002, Emergency Diesel Generator, to the initial ES-401 WT-2015-09 7 Form ES-401-9 Attachment 8_Rev 1 Page 7 of 14 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only conditions. Verify what can cause EDG overcurrent trips. Provide reference that shows that this situation will not result in an overcurrent trip. This will determine if distracter B is correct or not. Added page 25 of OP-009-002 to the references. 50 F 3 N S D, CR, 41.7 51 F 2 X B E S B, CR, 41.7 (2007 NRC Retake Exam) - Reword question stem to say "can cause" vice "caused." Revision made. Add a reference that shows the Hi pH alarm for the Blowdown Proportional Sampler, and that it doesn't cause BD-303 to close. Additional reference provided. 52 F 3 N S B, CR, 41.11 53 F 3 X B E S A, CR, 41.7 (2010 NRC Exam) - The auto start signal for ACCW pump A is based on low pressure in the associated loop, not system pressure. See Item 9.8 in OP-002-001. If this is confirmed, then there is no correct answer. The stem is only applicable to ACCW Pump A. (one loop). Added "on the associated loop" to the stem. 54 F 2 X B M U S C, CR, 41.4 (2011 NRC Exam) - The proposed question doesn't test the applicant's ability to manually operate/monitor pressure gauges in the control room. It doesn't address the K/A statement. This question was used in the 2011 NRC exam (RO question 54), but it was associated with a different K/A statement, which it met. Changed part 2 such that the applicant must identify where the IA pressure gauge is located. Add OP-901-511 page 3 as a reference. This reference gives the location of the pressure gauge (IA-IPI-9700). For the given question, the descriptions of the valves should match their names in procedures. See Section C of OP-901-511. Changed the noun name of SA-125 to match OP-901-511. 55 F 3 X B E S A, CR, 41.7 (2009 NRC Exam) - The proposed answer is based on SBV-114A(B)'s opening/closing set points. Unless there a reference that shows the maximum amount that pressure will overshoot when the valve cycles, all the answers could be argued as correct. Changed "expected" to "designed" in the stem. Provide the references showing that the distracter pressure values (+2 and -10 INWG) are plausible. References provided. 56 F 2 B S C, CR, 41.7 (2009 NRC Exam) 57 H 3 N S C, CR, 41.7 ES-401 WT-2015-09 8 Form ES-401-9 Attachment 8_Rev 1 Page 8 of 14 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 58 F 3 N S C, CR, 41.2 59 F 3 X X B E S A C, CR, 41.10 (2008 NRC Exam) - The generic K/A statement is coupled (ability to do X and Y). For coupled K/A statements, NUREG-1021, ES-401, Section D.2.a., says that if a question cannot be written to address both conditions, then it is allowed to limit the scope of the question to that K/A statement aspect requiring the highest cognitive level. For this K/A statement, the ability to "interpret" the meaning of procedure steps requires more cognitive testing than the ability to "execute" procedure steps. The proposed question is written to test the lowest cognitive level. Therefore, the question does not meet the K/A statement. The interpret portion is knowing that the Hydrogen Analyzer containment isolation valves must be open for the hydrogen analyzer pump power to be turned to on. No changes are required. For the provided question, it needs a reference showing that the flammability limit is 4%. Also, the stem needs to say "..reaches a minimum flammable limit-" Added "minimum" to the stem. Added OP-902-002 page 14 and WLP-OPS-HRA page 29 as a reference. 60 H 3 X N E S C, CR, 41.7 - The wording of the question does not match the wording in the referenced procedure. For example, is the "Master Controller" in the procedure the same thing as the "Speed Controller M/A Station?" This introduces confusion for the applicant. Added "speed controller" to noun names to match OP-003-033. Changed "Main Feed Pump B" in the stem to state "SGFP B" to match OP-003-033. Added reference (SD-FWP page 23 revision 4) to indicate that speed control governor can no longer control speed when local controls are placed in manual. Provide a reference to support statement that once local controls are in AUTO, the speed governor control switch cannot be used to adjust SGFP speed. Added reference (SD-FWP page 23 revision 4) to indicate that speed control governor can no longer control speed when local controls are placed in manual. Editorial: Should "Main Feed Pump B" be "SGFP B?" Changed "Main Feed Pump B" in the stem to state "SGFP B" to match OP-003-033. 61 F 2 B S A C, CR, 41.7 (2010 NRC Exam) 62 F 3 X B E S D, CR, 41.11 (2010 NRC Exam) - Make the names of the radiation monitors consistent with their name in the referenced procedure. Revised to add unique identifiers for the different radiation monitors. 63 F 4 X X N U S D, CR, 41.5 - The first part of the question deals with knowledge of the Tech Spec bases. Unless it can be proven that RO applicants are trained on this information, this is SRO level of knowledge. In addition, the supporting reference for this question supports both answers for the first part of the question being right. On page B 3/4 1-5 of the Tech Spec bases, last ES-401 WT-2015-09 9 Form ES-401-9 Attachment 8_Rev 1 Page 9 of 14 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only paragraph, text used to write the question can be seen. However, the text before and after this paragraph talk about the insertion limits limiting radial peaking factors and the extent of radial xenon redistribution effects. An applicant could argue that both provided answers are addressed by the cited Tech Specs. Per the licensee: The radial xenon distribution effects are included in the basis for exceeding the long term and short term transient insertion limits, not the transient insertion limit. These are three different curves in TS. The question is asking for the reason for the transient insertion limit. The reason for the transient insertion limit and the short and long term transient limits are clearly defined in the TS basis. No need for any changes. The second part of the question deals with Tech Spec and/or procedure actions. The entry conditions of the referenced procedure, OP-901-103, do not cite issues with SDM associated with TS 3.1.3.5 or 3.1.3.6. The distracters have a 2 hour2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> TS action statement. Therefore, the only action supported by procedure or Tech Specs is the 2 hour2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> TS action statement. ROs are not required to know those. This appears to be a SRO question with two correct answers (distracters B and C). The question was revised to focus it on RO tasks in operating procedures. 64 F 2 X X B E S A, CR, 41.11 (2009 NRC Exam) - K/A mismatch. K/A is effect of a loss or malfunction of waste gas disposal on the ARM and PRM systems, but question is about the effect of a loss or malfunction of ARM/PRM on the waste gas system. Note that this is a bank question from a previous NRC exam, however, because the question on that exam matched that exam's K/A, this question will count towards exam submittal quality. It matches the K/A statement because GDTs are sampled prior to discharge to ensure that the radioactivity is at an acceptable level. If the discharge is underway, an indication of a malfunction in the system is radioactive monitors going off and isolating the system. Provided additional reference to support this claim. Question accepted. Procedure OP-901-413, which is not cited as a reference, calls the radiation monitor that closes GWM-309 "Gaseous Waste Discharge Radiation Monitor" and the "Gaseous Waste Management Monitor." Whatever the correct name for the monitor is, use it in the question. The name was corrected. 65 F 2 N S D, CR, 41.4 - Question: Does this portion of TCCW have an emergency/essential function? Since it isn't powered of the 3A or 3B busses, it doesn't appear so. Answer: The TCCW pumps are considered Essential because catastrophic damage to the Main Generator, Main Turbine or personnel can occur within two minutes of a loss of TCCW pumps. 66 F 3 B S C, CR, 41.10 (2012 NRC Exam) 67 F 3 X N E B, CR, 41.10 - Provide a reference that supports the water level being at 43 feet in ES-401 WT-2015-09 10 Form ES-401-9 Attachment 8_Rev 1 Page 10 of 14 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S the Refuel Cavity. Also, justify why this tests plant-wide knowledge in support of being a Tier 3 question. Provided a reference. Knowing the elevation of the reactor vessel flange is generic knowledge. This knowledge is important during refuel activities, draindowns, fill and vents, and responding to refuel cavity leaks. 68 H 3 X N E S A, CR, 41.10 - Is knowledge of EOP technical guides trained to the RO applicants? Provide a copy of this reference. RO applicants are trained on the tech guide's contents. A training reference was provided showing that RO and SRO applicants are trained on the basis for Immediate/Contingency Actions in the Standard Post Trip Actions. 69 F 3 X B E S B, CR, 41.10 (2014 NRC Exam) - Provide references that show a) what goes into a work package, and b) what goes in the Operability tab in PCRS. References provided. 70 F 3 X N U S D, CR, 41.3 - To answer this question requires knowledge of Tech Spec bases. Unless this is contained in procedures/training that the RO applicants are trained on, that is SRO applicant knowledge. Knowledge of where the containment isolation valve list is located may be RO knowledge. The question and its references were reviewed. The knowledge for the first part of the question was found in administrative procedures that the RO applicants are supposed to have knowledge of. Acceptable. 71 F 3 X N U S B, CR, 41.10 - While it is true that the SGTR procedure (OP-902-007) allows for manually operating the ADV on the affected steam generator to maintain its maximum pressure or to help reinitiate cooldown, Step 30.1 calls for doing the plant cooldown using the ADV on the least affected steam generator. Therefore, distracters C and D appear to be implausible. Modified the question stem to say that there is a SG tube leak in both steam generators. With the information in the cue, the applicant could assume that the tube leak is in one steam generator. It doesn't say. Someone could argue that there is no release from a steam generator without a leak, and say that there is no release. The second part of the question could cause the question to be thrown out. Recommend changing the cue to say "Any radioactive discharge through the Atmospheric Dump Valve(s)-" This would make it consistent with the conditional wording in OP-901-202, Step 20.5.3. Modified the question stem to say there is a SG tube leak in both steam generators. Therefore, there would be an unmonitored release. 72 F 3 N S C, CR, 41.10 73 F 3 X X N E S D, CR, 41.10 - Verify that the recent procedure change that provides for the distracters was trained to the applicants. If the change happened outside the scope of their training, this may not be a fair question to ask. The applicants have been trained on this revision.

ES-401 WT-2015-09 11 Form ES-401-9 Attachment 8_Rev 1 Page 11 of 14 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only Revise the question stem to say that this occurs upon entry into the EOPs. Added "and upon entry into the Emergency Operating Procedures" to the stem. 74 F 3 N S C, CR, 41.10 75 F 4 N S A, CR, 41.10 76 H 4 N S C, CR, 43.2 77 H 3 X N U S D C, CR, 43.5 - The K/A statement calls for testing the applicant's knowledge of low power/shutdown implications on large break LOCA mitigation strategies. If the LOCA occurred in low power/shutdown conditions, vice starting at power, what mitigation strategies change? The provided question doesn't address the K/A statement. The licensee randomly resampled for another K/A statement. K/A statement is a plausible tie to the drafted question. Question is satisfactory. 78 H 4 N S C, CR, 43.2 79 H 3 N S D, OR, 43.5 (OP-902-009, Attachment 2-A) Procedure comment: Part of the actions within Step 12a of OP-902-007 fit the condition when RCPs are operating, and part of them apply when RCPs are secured (natural circulation). This is supported by the discussion in the Technical Guide for this EOP (TGOP-OP-902-007, Rev. 306, page 29). However, the way Step 12a is written, it requires the operator to maintain RCS pressure at two conditions that contradict each other (if RCPs are in operation, maintain it above the NPSH curves, 1150 psia, WHILE maintaining pressure below 930 psia). This can lead an operator to choose one or the other and make a mistake. This procedure needs to be corrected. 80 H 3 X N E S C, CR, 43.5 - The question can be answered with power supply knowledge that a RO applicant has. There is only one abnormal procedure to enter, and only one section that needs to be completed within it, named after the power supply to PDP-90A. This is not a SRO only question. Based on discussion with the licensee, the second part of the question is SRO-only level of knowledge. The Explanation text has been revised to justify this. 81 H 4 N S B, CR, 43.5 82 H 3 N S A C, CR, 43.5 83 H 4 X N E S B, CR, 43.2 - Charging Pump B is not an option in Attachment 10.4, so distracter D isn't plausible. This question is a 4x2 format, so an applicant could know the answer to the first part of the question, and get the question right without having any knowledge of what the correct answer is for the question's second part.

ES-401 WT-2015-09 12 Form ES-401-9 Attachment 8_Rev 1 Page 12 of 14 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only Revised the question to address the identified issues. 84 H 4 B S A B, CR, 43.5 (2011 NRC Exam) 85 H 4 B S A, CR, 43.5 (2014 NRC Exam) 86 F 2 N S C, CR, 43.2 87 H 3 N S D, CR, 43.5 88 H 3 X N E S A, CR, 43.5 - In the question, state that Tech Spec Action statements are entered, not Tech Specs are entered. Proposed change has been made. 89 H 4 N S C, CR, 43.5 90 H 4 N S B, CR, 43.5 91 H 3 N S A, CR, 43.5 92 H 3 N S C, CR, 43.5 93 H 3 X N E S D, CR, 43.5 - The "sequencer" mentioned in the stem is assumed to be the EDG sequencer. Clarify. Also, spell out C/S. Proposed revisions were made. 94 F 3 N S C, CR, 43.1 95 F 3 X N U S D, CR, 43.2 - The question tests knowledge of primary chemistry surveillance requirements, but does not test knowledge of primary chemistry limits. K/A mismatch. The licensee randomly selected another K/A statement that was a plausible match for the draft question. Question is satisfactory. 96 F 3 N S A, CR, 43.5 97 F 3 B S B, CR, 43.4 (2011 NRC Exam) 98 H 4 B S B, OR, 43.5 (2009 NRC Exam; EP-002-052, Attachments 7.2 and 7.3) 99 F 3 B S B, CR, 43.5 (2009 NRC Exam) 100 F 3 X N E S D B, CR, 43.5 - Change the distracters for the first part of the question to better describe what an operator would do if they were manually operating a MOV. This will eliminate the ability to appeal saying that bleeding the air off of the AOV is "manually" closing it as well. Revision was made to address this concern. Provide a reference that supports why this action is taken, i.e. for the second part of the question. Reference provided.

ES-401 WT-2015-09 Written Examination Review Worksheet Form ES-401-9 Instructions [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable). 3. Check the appropriate box if a psychometric flaw is identified: $ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). $ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc). $ The answer choices are a collection of unrelated true/false statements. $ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable. $ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified: $ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). $ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). $ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). $ The question requires reverse logic or application compared to the job requirements. 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met). Attachment 8_Rev 1 Page 13 of 14 OBDI 202 - IOLE Process RO TOTALS: B = 29 (38.7%) F = 37 (49.3%) E = 31 (41.3%) Additional Notes: Peer review completed by T. Buchanan. Four questions are proposed that were used in the last 2 NRC written exams. M = 5 (6.7%) H = 38 (50.67%) U = 9 (12%) N = 41 (54.7%) S = 35 (46.7%) SRO TOTALS: B = 5 (20%) F = 7 (28%) E = 6 (24%) Additional Notes: Peer review completed by T. Buchanan. One question is proposed that were used in the last 2 NRC written exams. M = 0 H = 18 (72%) U = 2 (8%) N = 20 (80%) S = 17 (68%) GENERAL COMMENTS 1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N 2. Chief Examiner comments are indicated in blue.

ES-401 WT-2015-09 14 Form ES-401-9 Attachment 8_Rev 1 Page 14 of 14 OBDI 202 - IOLE Process 3. Average difficulty is __ 2.82__ on the RO exam and 3.32__ on the SRO exam. 4. The 10CFR55.41/43 distribution is: RO / SRO 41.1 = 0 43.1 = 1 41.2 = 4 43.2 = 5 41.3 = 4 43.3 = 0 41.4 = 6 43.4 = 1 41.5 = 3 43.5 = 18 41.6 = 1 43.6 = 0 41.7 = 27 43.7 = 0 41.8 = 4 41.9 = 1 41.10 = 21 41.11 = 4 41.12 = 0 41.13 = 0 41.14 = 0 5. The answer distribution is: RO / SRO A = 17 (22.7%) / 6 (24%) B = 17 (22.7%) / 6 (24%) C = 21 (28%) / 7 (28%) D = 20 (26.7%) / 6 (24%) 6. There are _5__ RO questions with handouts provided and __2__ SRO questions with handouts provided.