ML21298A149

2021 Point Beach Initial License Exam Form ES-401-9

ML21298A149
Person / Time
Site:	Point Beach
Issue date:	08/12/2021
From:	Chuck Zoia NRC/RGN-III/DRS/OLB
To:
Zoia C
Shared Package
ML20139A042	List: ML21298A146 ML21298A147 ML21298A149 ML21298A155 ML21298A212 ML21298A214 ML21298A220 ML21298A221
References
Download: ML21298A149 (18)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9

2021 Point Beach Initial License Examination

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/AOnly (B/M/N) (U/E/S)

2017 NRC exam; 2X2 B NRC: Explain in the pedigree how this question is Tier 1 F 2 N E 1. How does it test how to safety operate the plant ?

REPLY: Question replaced NRC: Question is now SAT NRC: Explain in the pedigree how this question tests Knowledge of the reasons for the following responses.

2 H 2 X B E REPLY: Updated K/A justification and modified the question statement to strengthen K/A match.

NRC: Question is now SAT NRC:

1. No credible distractors. A and D are only credible distractors if a cooldown is in progress. The distractor analysis says B is wrong because the MSIVs will be closed due to the cooldown. But the stem provides no RCS temperature trend.

Apparently, a cooldown is not in progress.

3 H 2 X N U 2. The and why portion of this question is not tested S because there are four unique actions. i.e. if the applicant knows to trip RCPs, C is the answer.

GENERIC COMMENT Any time half of a 2X2 question has 4 unique answers, the other part is not tested. This applies to several questions on this submittal.

REPLY: Question modified to removed second half, making it only 4 unique answer.

NRC: Question is now SAT 4 F 2 B S

EARLY SUBMITTAL 2012 NRC Exam NRC: GENERIC COMMENT Please redact (or make a separate document) any procedure portions in the pedigree that you do not w ant to be put into ADAMS.

REPLY: References moved to separate document.

5 H 2 B E NRC: Revise the pedigree to Cognitive Level = H Note: Procedure portions were only utilized for the early submittal, T-75 and later submittals have references moved to a separate document.

Change of cognitive level incorporated into pedigree.

NRC: Question is now SAT NRC: Need more detail in the pedigree regarding K/A match. Does this relate to a loss of reactor coolant 6 F 2 X B E makeup because if RCS pressure is too high, charging flow (makeup) will be too low?

REPLY: K/A match justification changed to address comment.

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/AOnly (B/M/N) (U/E/S)

NRC: Question is now SAT

7 H 2 B S

8 H 2 N S

2019 NRC Exam NRC:

1. Because the first part of the question has four unique answers, the second part does not actually test anything. The only knowledge necessary to answer the question is that the control switch must be initially turned to OFF to energize heaters in this 9 F 2 B E condition.

2. Add an explanation of why this is a Tier 1 question, i.e. what procedure directs this action?(See Operator licensing feedback 401.55).

REPLY: Question modified to removed second half of question making it four unique answers, and modified question stem to strengthen tier 1 tie.

NRC: Question is now SAT EARLY SUBMITTAL NRC:

1. FAQ 401.55 -- Tier 1 questions are required to test the operators knowledge of how to safely operate the plant during emergency and abnormal conditions. Satisfactory ways of meeting this objective include, but are not limited to, the following: (1) information contained in the sites procedures, including alarm response procedures, AOPs, EOPs, and their associated bases documents; (2) diagnosis that leads to selection of the procedures that should be used to respond to 10 F 2 X X N E the evolution, (3) the progression of an event, and (4) assessment of the integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions.

a. The question can be answered without the given conditions.

b. The knowledge being tested is fundamental system level knowledge unrelated to emergency or abnormal response procedures The only knowledge required to answer this question is that Rod Bottom lights have no power (only one answer does not rely on rod bottom lights), so it is not truly a two-part question.

REPLY: Question modified incorporating comments.

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/AOnly (B/M/N) (U/E/S)

NRC: Question is now SAT

EARLY SUBMITTAL NRC:

1. Revise the question statement to read: must take to mitigate the transient event. Using the phrase to stop the leak provide s an unintended cue.

2. Remove the specific loop/SG identifier (A,B)

a. Revise choice B to state Stop feeding the 11 H 3 X N E affected Steam.

b. Revise choice D to state: out of, the affected Steam.

There was no response to feedback 2 above.

Explain how this question tests knowledge of parameters and logic used to assess the status of safety functions. Also explain how D is credible (isolate the B SG when its conditions are all stable).

REPLY: Replaced question incorporating comments.

NRC: Question is now SAT NRC:

1. C is a subset of D making D not credible.

2. Since the first part of the question has four unique answers, the second part tests nothing i.e. the applicant can get the correct answer by knowing the requirement to establish 50 gpm to only one SG without any knowledge of the reason). As a result, the K/A is not tested (the reason).

3. Explain how this is a significant modification of the 12 H 2 X X M U original question. Only changing which AFW pump S is restored does not change the premise of the question.

4. Capitalize the word both the first time it is used in D for consistency.

REPLY: Question is being considered modified due to changing the CET trend from rising to lowering, thus changing the correct answer of the question. Also, further modified question incorporating changes and making a 2x2 question.

NRC: Question is now SAT NRC:

1. Explain how this is Tier 1 - how to safely operate the plant during emergency and abnormal conditions ?

13 F 2 X N E 2. Explain the K/A match. What are the operational implications of battery capacity i.e. does battery depletion change the mitigation strategy?

REPLY: Modified question stem, and K/A match justification to strengthen K/A match/Tier 1 tie.

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/AOnly (B/M/N) (U/E/S)

NRC: Question is now SAT

NRC:

Explain how this is Tier 1 - how to safely operate the 14 F 2 X N E plant during emergency and abnormal conditions?

REPLY: Question modified to enhance tier 1 tie.

NRC: Question is now SAT EARLY SUBMITTAL; 2017 NRC Exam NRC:

1. The listed annunciator is not needed since the question statement describes degradation of the air system. Consider removing from stem conditions.

2. RO applicants are generally not required to memorize subsequent actions of an EOP or AOP.

15 F 3 X B E As written, answering this question requires specific knowledge of an AOP Response Not Obtained for a subsequent action. If the question can be answered based solely on knowledge of the impact that the loss of air has on each of the choices, then consider removing per AOP-5B, Loss of Instrument Air.

3. Pedigree typo - Distractors C and D switched.

REPLY: Reference to AOP removed NRC: Question is now SAT NRC: Explain how this is Tier 1 - how to safely operate the plant during emergency and abnormal conditions?

16 F 2 X N E REPLY: Question and Justification for K/A match modified to strengthen tier 1 tie. Discussed during the on-site visit.

NRC: Question is now SAT 2003 NRC exam NRC: List the reference for establishing minimum injection flow in this lineup (charging pump drawing from VCT). L isted references do not contain this information.

17 H 2 B E Appendix B to ECA 1.1 says to adjust charging pump speed as necessary to maintain desired VCT level, not minimum injection flow.

REPLY: Distractor analysis modified to show minimum flow required information NRC: Question is now SAT 18 F 2 B S 2007 NRC Exam

19 H 2 M S Embedded reference

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/AOnly (B/M/N) (U/E/S)

20 H 2 B S

21 H 2 B S

NRC: The distractor analysis refers to parts 1 and 2 of distractors, but no distractor has a part 2. Correct 22 F 2 B E pedigree to address the current question.

REPLY: Distractor analysis corrected.

NRC: Question is now SAT NRC: Explain how this tests the operational implications for firefighting. There is nothing directly associated with fighting fires in the pedigree or the question.

23 F 2 X B E REPLY: The stem identified the crew was acting as required when fighting a fire which has caused the control room to be un inhabitable. Modified justification for K/A match.

NRC: Question is now SAT EARLY SUBMITTAL NRC:

1. Are ROs expected to monitor status trees? Usually this is a r esponsibility of the SRO or STA.

2. Selection of the appropriate Critical Safety Procedure (CSP) is based on evaluating (plotting) plant conditions against the conditions specified in the CSP-STs. Operators are not required to memorize the specific paths of a Critical Safety Function Status Trees (CSP-ST). While recognition that Containment Pressure being greater than 60 psig alone, constitutes a RED path CSP entry, entry into the CSPs identified in the other answer choices requires evaluation of 24 H 5 N E multiple conditions.

a. Entry into CSP-C.2 (Orange) requires evaluation of RCS Subcooling, RCP Status, Core Exit Thermocouples, and RVLIS. Given the plant conditions, entry into the procedure would be correct except that entry into a RED path procedure is a higher priority.

b. Entry into CSP-H.1 (Red), requires evaluation of Steam Generator Water Level and Total Auxiliary Feedwater Flow. Given the plant conditions, entry into this procedure would be incorrect because Steam Generator Level is high enough to preclude entry into the RED path procedure (the correct procedure would be CSP-H.2 (assumes SG pressure <1105 psig based on Cold Leg Temperature).

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/A Only (B/M/N) (U/E/S)

c. Entry into CSP-I.2 (Yellow), requires evaluation of Pressurizer Level (High and Low). Given the plant conditions, entry into this procedure would be correct except that entry into a ORANGE or RED path procedure are of higher priority.

3. NOTE - If both SG levels were less than 52% then entry into CSP-H.1 would be warranted and choice B would be the correct answer due to a higher priority RED path procedure and be more discriminating.

If an RO is expected to monitor CSFTs from memory, provide the learning objective(s) that spe cifies this requirement.

REPLY: Added additional learning objective to the pedigree to show the program tie to the knowledge of having the ability to recall/list the entry condition for each red path and orange path CSP. Added a copy of the lesson plan to the question references sent.

NRC: Question is now SAT 25 F 2 B S

26 H 2 B S

27 H 2 B S

28 H 2 B S

29 H 2 B S

30 F 2 B S

31 H 2 N S

32 F 2 B S

33 H 2 B S

34 F 2 B S 2015 NRC Exam

35 F 2 B S 2011 NRC Exam ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/A Only (B/M/N) (U/E/S)

36 H 2 N S

NRC: The only knowledge the applicant needs to know is that the deviation meter is nulled, since only one answer says this. The applicant does not need to know the output meter indication. The second part of the 37 H 2 B E question tests nothing - it could be removed.

Add a reference that explains the function and operation of the deviation meter.

REPLY: Question modified to incorporate comments (also, deviation meter answers removed).

NRC: Question is now SAT NRC: The distractor analysis proposes that the applicant may confuse OTT and OPT. But B, C and D are all true for OTT. That confusion would not lead 38 F 2 X B E the applicant to a single, incorrect answer. If it is the intent to make someone with this confusion pick t he wrong answer, the question must be revised.

REPLY: Modified distractor analysis based on comment.

NRC: Question is now SAT Embedded reference.

NRC: C is also a tripped condition. The distractor 39 H 2 N E analysis contradicts this.

REPLY: Modified distractor analysis based on comment.

NRC: Question is now SAT NRC: Initially provided a K/A that was not applicable to PB. PB does not have a core spray system. Replace d K/A with 013K4.19. R eplaced question.

40 F 2 B E Distractor analysis for C is incompatible with distractor.

REPLY: Modified distractor analysis for C based on comment.

NRC: Question is now SAT NRC: There is a better reference. The reason for starting BOTH RHR pumps is the caution prior to step 41 H 2 N E 13. The foldout page says to start pumps as necessary (not necessarily both). The caution says to start both.

REPLY: Modified distractor analysis based on comment.

NRC: Question is now SAT NRC: C is a subset of D (If D is correct, C is also correct). This makes D not credible ( if the applicant 42 H 2 X N E knows there cannot be two correct answers).

REPLY: Question modified corrected subset issue.

NRC: Question is now SAT NRC: Explain why the distractors with opposite unit 43 F 2 X B E power supplies are credible (i.e. are there any ESF loads powered from the opposite unit?)

REPLY: Modified distractor analysis based on comment.

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/AOnly (B/M/N) (U/E/S)

NRC: Question is now SAT

44 F 2 B S 2009 NRC Exam

45 H 2 B S

NRC: Revise the wording of the correct answer to match the FSAR reference. Specifically - isolate main 46 F 2 X B E feedwater flow to a faulted steam generator.

REPLY: Modified question as directed.

NRC: Question is now SAT 47 H 2 B S

2012 PB NRC exam NRC: B can be eliminated using fundamental knowledge of generators in parallel (i.e. it is impossible to change the speed). This does not require a 48 F 2 X B E knowledge of D/G load limi ts. It is not a credible distractor.

REPLY: Discussed with chief examiner and modified distractor analysis based on comment.

NRC: Question is now SAT 2019 NRC Exam; EARLY SUBMITTAL ; Embedded Reference NRC:

1. The first part of the question does not seem to match the first part of the K/A. Part (a) of the K/A is to predict the impacts on the DC electrical system;. The first part of the question instead asks for the expected plant response.

2. The second part of the K/A requires mitigative actions, not simply the selection of a procedure.

3. The question addresses an obvious loss of a major 49 H 1 X X X B E DC Bus. Choice D (the correct answer) is the only 2 choice that includes AOP -0.0.

4. Loss of a single DC train should not result in a Unit Trip, making choices A and B not very plausible.

Plausibility statements in answer justification refer to multiple failures not supported by the stem of the question.

UPDATE: Because the first part of the question has four unique answers, the second part tests nothing. (i.e. if the applicant knows that only Unit 1 trips, there is no need to even consider the second part) -this only tests the impact of the malfunction but does not the resulting procedure requirements. Also, the answers seem to ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/A Only (B/M/N) (U/E/S)

imply that replacing the fuses will solve the problem, but the stem says that the distribution system has multiple grounds. Maintenance needs to do more that determine the cause, maintenance need to correct the cause.

REPLY: Question was replaced to better match the K/A NRC: Question is now SAT NRC: Subset. If D is true, C is also true. This makes D 50 H 2 X N E not credible.

REPLY: Question modified correcting subset issue.

NRC: Question is now SAT 51 H 2 B S

52 H 2 B S 2005 NRC Exam

EARLY SUBMITTAL; 2019 NRC Exam

1. In the Source section of the p edigree, recommend adding a statement to indicate that the stem was reworded to correctly relate that the SW system operability is affected by the CW Pump Bay level 53 F 4 B E and not the components listed as the answers.

Also, that the K/A was changed to make the question applicable to the SW system.( therefore.,

the question was modified, but not significantly.)

REPLY: Comments incorporated NRC: Question is now SAT NRC: There are four unique answers to the second part (valve positions). If the applicant knows that the Service Air valves are OPEN and the IA dryer bypass valves are 54 H 2 X N U CLOSED, there is no need to evaluate compressor S status. As a result, the K/A is not tested ( power supply).

REPLY: Question modified incorporating comments.

NRC: Question is now SAT 2005 NRC Exam NRC: C is a subset of D (if D is true, C is also true).

55 H 2 X B E This makes D not a credible distractor.

REPLY: Question modified to correct subset issue.

NRC: Question is now SAT 56 H 2 B S

57 H 2 B S

2X2 58 H 2 B U NRC: Double jeopardy with #56. An applicant that does N S not understand the power mismatch portion of the automatic rod motion signal misses both questions.

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/AOnly (B/M/N) (U/E/S)

REPLY: Question replaced.

NRC: Question is now SAT NRC:

1. Explain in the pedigree why knowledge of the range of subcooling indication is operationally valid (i.e. why is this not trivia?).

2. An applicant would also need to know more details of the subcooling calculation to determine which channel is inaccurate (i.e. do both channels use the avg of all CETs or some other logic)? CETs never all read the same. Do both channels use 59 H 2 M E the same WR pressure monitors? WR pressure monitor indications typically vary somewhat. Is there a methodology in the procedure for manually calculating subcooling margin (i.e. use the average CET vice the highest 5)? Include this information in the references/pedigree.

REPLY: Pedigree modified to incorporate comment #1.

Question discussed during OSV; thus, made temperature difference between the channels given in the stem greater than 31°F.

NRC: Question (M) is now SAT 2011 NRC Exam NRC:

1. The FSAR and the justification section say hydrogen is generated as aluminum interacts with sodium borate solutions. The designated correct answer says that aluminum interacts with sodium 60 F 2 B hydroxide to produce hydrogen. The attached N E lesson plan also seems to contradict the FSAR source. Please reconcile these differences.

2. Also, explain why this question is operationally valid, i.e. does the operator have some role in minimizing aluminum in containment?

REPLY: Question replaced.

NRC: Question is now SAT NRC:

1. If the low-level condition has caused the loss of suction to the spent fuel cooling pumps, the pool may be boiling. (recent outage, high heat load).

This could make B a correct answer.

61 H 2 N E 2. Shine from the pool is credible only for monitors in the vicinity of the pool. Enhance the distractor analysis to take relative location into effect or explain why location is not important.

REPLY: Discussed during OSV. Question and pedigree modified incorporating comments and discussed item s.

NRC: Question is now SAT ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/AOnly (B/M/N) (U/E/S)

62 H 2 B S Embedded reference

2007 NRC Exam NRC:

All the applicant needs to know to get this question correct is that the turbine should trip when the reactor trips. This is knowledge that is known by all non -

licensed operators and most engineers.

Appendix B or NUREG 1021 says that items should be 1 revised if estimates of applicants selecting the correct 63 H 2 B E answer fall well below or above the 70- 90% range. See example C.1 in Appendix B. Because this information in the question should be known by all operators at all times, it has a low discriminatory value. I t should be revised because the estimate of applicant success is well above 90%.

REPLY: Question discussed during OSV - Distractor D changed based on CE comments..

NRC: Question is now SAT 64 F 2 B S 2015 NRC Exam

NRC: D should say cooling supply to the Steam Generator Blowdown system to match the structure of 65 F 2 X B E distractors A - C.

REPLY: Question modified incorporating comment.

NRC: Question is now SAT 2009 NRC Exam NRC:

1. B is not credible. If the license is active, there is no need to regain qualification.

66 H 2 X B E 2. Provide justification in the pedigree for requiring RO to answer this without references.

REPLY: Discussed during OSV, B was modified to remove the first issue, and distractor analysis modified to discuss RO knowledge requirement.

NRC: Question is now SAT 2012 NRC Exam NRC: The justification says that Sulfuric Acid and Sodium Hypochlorite are mentioned in the reference as examples of hazardous chemical systems, but the 67 F 2 B E reference given does not mention this. Provide a reference that identifies sodium hypochlorite as a hazardous chemical system.

REPLY: Modified references and distractor analysis incorporating comment.

NRC: Question is now SAT ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/A Only (B/M/N) (U/E/S)

68 F 2 B S

NRC: Provide reference that says to tag the control switch prior to the motor breaker. If not available, the question is still valid without mention of the control 69 F 2 B E switch.

REPLY: Modified question by removing control switch from the distractors.

NRC: Question is now SAT 70 F 2 B S

NRC: This question tests system knowledge, not the generic aspect of determining operability/availability.

B E ES-401 says Ensure that the questions selected for 71 F 2 X N Tier 3 maintain their focus on plantwide generic K/As and do not become an extension of Tier 2.

REPLY: Replaced question.

NRC: Question is now SAT 72 H 2 N S

NRC: This is a tier 1 question. It asks what a specific procedure requires. It is not tier 3.

REPLY: Replaced question 73 F 2 X B E NRC: Distractor analysis is incompatible with question.

A is marked as correct, but the reference seems to say that B is correct. Needs clarification.

REPLY: Distractor analysis corrected.

NRC: Question is now SAT NRC: Spell out the acronyms (AO, AOT, and PAB).

74 F 2 B E REPLY: Question modified incorporating comments.

NRC: Question is now SAT NRC: This question tests system and procedure 75 F 2 X N E knowledge and not plantwide generics.

REPLY: Replaced question NRC: Question is now SAT 76 H 2 B S 2012 NRC Exam

NRC: This question can be answered with RO knowledge. Since steam tables are included as a reference on Q #29, only RO knowledge is required to 77 H 2 X B U determine that subcooling at the core exit is over 140°F S and the subcooling margin alarm is inconsistent with the plant event. This makes the answer either A or B. Then the choice is whether to first enter EOP-2 or EOP-1, which is also RO knowledge (ES-401, Attachment 2).

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/AOnly (B/M/N) (U/E/S)

REPLY: Second half of question modified, incorporating comment.

NRC: Question is now SAT 2019 NRC Exam; Reference Provided NRC:

1) Stem focus. Consider pulling TSAC 3.8.1B, 3.8.1.E and 3.8.1 into the stem since it is common to all 78 H 2 X B E answers.

2) GENERIC COMMENT : For a REFERENCE PROVIDED, it should not be referenced in the question stem, but given in a packet of references.

REPLY: Question modified incorporating comment #2.

NRC: Question is now SAT 2012 NRC Exam NRC:

1. The first part of the question has no meaning. The term OPERABLE does not apply to a component that is not part of meeting any LCO. For TS purposes, a spare disconnected battery has the same status as a component in the warehouse. It is neither operable nor inoperable since it has no specified safety function. If the intent is to ask if 79 H 2 X X B E connecting this battery to the bus makes the LCO MET, much more information is necessary (have the associated surveillances been met within the specified frequency?).

2. Not SRO only because of procedure selection.

There is no procedure selection component to this question. The only restoration procedure listed is 0-SOP-DC--3.

3. Is a reference provided? NO REPLY: Question replaced NRC: Question is now SAT Reference provided GENERIC COMMENT: For a REFERENCE 80 H 2 B E PROVIDED, it should not be referenced in the question stem, but given in a packet of references.

REPLY: Question modified incorporating comment.

NRC: Question is now SAT 81 H 2 B S 2011 NRC Exam

1. Explain why the reason for each reactor trip is SRO-only knowledge in the pedigree. This is 82 H 2 X X B E commonly RO knowledge.A TS basis question is considered SRO only if it exceeds RO knowledge.

2. Explain how A(2) and C(2) are credible. How does a pressurizer level channel failure impact the ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/AOnly (B/M/N) (U/E/S)

ability of RPS to protect against DNB? The existing explanation is inadequate.

REPLY: Question discussed during on-site visit.

Pedigree modified incorporating comment #2.

NRC: Question is now SAT 83 H 2 X N S

EARLY SUBMITTAL NRC:

1. In answer choices A and B replace Complete with Continue in

2. In answer choice A, remove the comma after EOP-1.3 and replace when with until replace the period after established with a comma.

3. Answer choice C is structured differently from the 84 H 3 X N E other three choices. Recommend the following Transition to CSP -Z.2, locate and isolate the source of water entering the containment, then return to EOP-1.3.

4. In answer choice D delete word exit; and add return to EOP -1.3.

All the above changes are complete. However, Answer A is listed twice.

REPLY: Removed the second answer A distractor.

NRC: Question is now SAT 2017 NRC Exam NRC: Distractors are not credible. Only B says to trip the RCP wh en it has excessive and rising vibration. An 85 H 2 X X B E RO knows to trip an RCP in this case. No SRO knowledge is required to answer this question.

REPLY: Modified question to incorporate comment.

NRC: Question is now SAT 2015 NRC Exam; Reference Provided NRC: GENERIC COMMENT: For a REFERENCE 86 H 2 B E PROVIDED, it should not be referenced in the question stem, but given in a packet of references.

REPLY: Modified question to incorporate comment.

NRC: Question is now SAT NRC: The second part of the question is not worded properly. The LCO that is not met is 3.6.6 which requires four accident fan cooler units. Asking about 87 H 2 X X B E the fan is misleading and inaccurate. The fan itself is capable of its specified safety function (to move air).

But the cooler unit is not capable of removing heat.

REPLY: Modified question to incorporate comment.

NRC: Question is now SAT ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/AOnly (B/M/N) (U/E/S)

Reference Provided NRC:

1. C and D are not credible. If loss of a single steam supply always makes the affected pump inoperable, condition A has no purpose. i.e. if condition A always requires immediate condition B entry, condition A never imposes any actual limits, since condition B completion time is shorter than 88 H 2 X N U that of condition A.

S 2. It is more operationally valid to assign actual times and dates, instead of recently performed. The question proceeds to ask what is the latest date when none of the answers have dates.

3. GENERIC COMMENT: For a REFERENCE PROVIDED, it should not be referenced in the question stem, but given in a packet of references.

REPLY: Question modified as discussed during OSV.

NRC: Question is now SAT NRC: Contrary to the pedigree, LCO 3.8.4 basis does not specifically say that the battery remains OPERABLE following a loss of room ventilation. Explain the basis for this operability determination. While hydrogen buildup may only be a concern during an equalizing 89 H 2 B E charge, does the fan also serve a temperature control function? Does room temperature affect battery capacity?

REPLY: Pedigree and question modified to incorporate comments.

NRC: Question is now SAT EARLY SUBMITTAL 2015 NRC Exam NRC:

1. Last bulleted item in stem initial conditions has a typo: shift should be shifted

2. The K/A (G2.1.32) is the ability to [1] explain and

[2] apply system limitations and precautions. This question does not test the ability to explain any limitation or precaution. It actually tests knowledge of system interrelationships. Typically, this K/A 90 H 2 X B E relates to limitations and/or precautions identified in system operating procedures. Knowledge that a component will automatically trip due to a loss of a support system would be tested usi ng other system related K/As. The ability to apply a knowledge of a limitation or precaution c ould be to prevent the situation from occurring.

3. The original K/A applied to this question was the Ability to determine and interpret the following as they apply to the Loss of Instrument Air: Typical ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/AOnly (B/M/N) (U/E/S)

conditions which could cause a compressor trip (e.g., high temperature).

UPDATE: The word shift in the last bulleted item has not been revised. According to the record of rejected K/As, the proper K/A is now G2.1.7. The K/A is still incorrectly listed as G2.1.32 in the pedigree, but the words of the K/A statement are correct.

The explanation says that this is SRO because of procedure selection, but the only procedure that is selected is E-0, which is RO knowledge. The hierarchy is not tested because both answers that select E -0 are identical.

REPLY: Question modified incorporating comments.

NRC: Question is now SAT 91 F 2 M S

NRC: K/A mismatch. The K/A is to determine if CR indication correctly reflects the desired plant lineup. The stem says that the CR ADV indications are accurate (the ADV has failed open) and the applicant is told that 92 F 2 X M U the ADV is NOT in its desired plant lineup (all answers S are to stop the resultant steam flow ). This question does not ask if CR indications are valid or if the ADV SHOULD be open in this condition.

REPLY: Question modified as discussed during OSV NRC: Question is now SAT EARLY SUBMITTAL NRC:

1. Actuation of the Fire Protection System per the K/A is more than simply the inadvertent actuation of an alarm circuit (e.g. actuation of a sprinkler/deluge system, auto-start of a fire pump, actuation of a gas suppression system, etc.).

2. Proper classification of a non -event does not 93 F 2 X B E correct, control, or mitigate the consequences of the FPS actuation.

REPLY: Revised question provided. The event is an actuation (not just an alarm) and the response is disabling auto actuat ion and establishing a fire watch.

NRC: Provide the specific reference that shows that manual actuation is available w ith the palm switch in the depressed position.

REPLY: Reference provided.

NRC: Question is now SAT B E 2015 NRC Exam 94 F 2 X N U NRC:

M S 1. References provided were inadequate to evaluate this question. The only attached reference did not ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/AOnly (B/M/N) (U/E/S)

specify which RWP to use, the requirement to use a radio or the resulting prohibition against fuel movement through the transfer tube.

2. Explain further why this is SRO only. NUREG 1021 says SRO -only knowledge should not be claimed for questions that can be answered solely based on RO knowledge of radiological safety principles (e.g., radiation work permit) An RO is required to know which RWP applies for a given job, and only one answer lists the correct RWP.

REPLY: Replaced question NRC: The replacement (N) submitted for the proposed NRC bank question was NOT SRO-only, however, the (M) bank question submittal was SAT.

95 F 2 B S 2015 NRC Exam

NRC: The SRO justification is for 10CFR55.43(b)(5) but this question does not require the applicant to select a procedure. If this is SRO only due to a learning 96 F 2 X N E objective unique to the SRO position, please provide that specific objectives wording and applicability.

REPLY: Modified SRO-ONLY justification as discussed during on-site visit.

NRC: Question is now SAT NRC:

1. Explain how this is SRO only. It is not 43(b)(7) see below) and meets none of the examples in in Attachment 2 to ES-401 for 43(b)4). Also, a refueling alarm is probably dealt with by an RO, since there is no requirement for an SRO to be in the CR.

2. The fact that Fuel handling is in progress is window dressing (i.e. the answer is the same if fuel handling is NOT in progress). As a result, this 97 F 2 X N E question does not test fuel handling facilities and procedures.

3. This question does not require the applicant to analyze or interpret radiation and activity readings because the stem specifies that the alarm is valid.

This question only tests knowledge of an alarm response procedure.

REPLY: Question replaced NRC: This SRO question was also compared to Audit Exam RO question 71 due to K/A overlap and found to be different. Question is now SAT 98 H 2 B S 2012 NRC Exam ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q LOK LOD Stem Back Q - SRO Source Status 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units ward K/AOnly (B/M/N) (U/E/S)

2012 NRC Exam (Replacement Question)

NRC:

1. This question does not require the applicant to assess facility conditions and select a procedure.

No facility conditions are given to assess, and no procedure is selected. (not SRO-only)

U 2. This question does not ask for the basis for 99 H 2 X X B S prioritizing safety functions. (not a K/A match). It does not, for example, ask why HEAT SINK has a higher priority than INTEGRITY.

3. The first paragraph is window dressing, evidently added to make it appear to be a K/A match. It changes nothing about the question.

REPLY: Question replaced as discussed during OSV NRC: Question is now SAT 2015 NRC Exam NRC: Correct the SRO-only justification. This question cannot be SRO -only per 10CFR55.43(b)(5). It does not require the applicant to assess facility conditions and select a procedure. No facility conditions are given, and 100 F 2 X M E no procedure is selected. This does not test a decision point in E-plan implementation. However, i t could be SRO-only if it is tied to a facility learning objective that is unique to the SRO position. If so, p rovide this objective.

REPLY: Modified SRO-ONLY justification as discussed during on-site visit.

NRC: Question is now SAT Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question a 1(easy) to 5 ( difficult); (questions with a difficulty between 2 and 4 are acceptable)

3. Check the appropriate box if a psychometric flaw is identified:

• Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• Cues: The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).

• T/F: The answer choices are a collection of unrelated true/false statements.

• Cred. Dist: The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.

• Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

• Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• Minutia: The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

• # / Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• Backward: The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only (K/A and license level mismatches are unacceptable)

6. Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-401 Section D.2.f.

7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any "U" Status ratings (e.g., how the Appendix B psychometric attributes are not being met).