Text

2016-09-DRAFT Written Exam Comments
	ML16260A152
Person / Time
Site:	Arkansas Nuclear
Issue date:	09/01/2016
From:	Vincent Gaddy; Operations Branch IV
To:	; Entergy Operations
References
	Download: ML16260A152 (20)
	v • d • e

ES-401 AN1-2016-08 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 3 B S NRC 2005. OR.

NRC 1998. Stem would be more direct if simply phrased, What is the reason 1202.001, REACTOR TRIP, directs tripping RCPs within 2 minutes following a loss of subcooling margin? Use of the phrase what is the reason versus why is it 2 H 2 X B S desirable ties it directly to the Areva technical document; desirability could be a subjective term, and all of the distractors are desirable reasons. Lawyer thinking.

Revised as suggested.

NRC 2008. History is cut off. Which of the following actionS Corrected typo, printing is hit or miss sometimes. Question last used in 2008.

-12 FT BWST is pretty high for distractor D, unless there is something specific with 3 H 3 X B S that value. Make closer to 6 ft. Made it 8 ft.

-Change or add to stem that full ESAS flow has been occurring for 20 min, since there is a note that says 6 feet will be achieved in 25 min of full EAS flow. added Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

$ The answer choices are a collection of unrelated true/false statements.

$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

$ The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met). _Rev 2 Page 1 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only

-Make both LPI flows < 3050 but > 2800 to improve discriminatory value of distractor A. Changed A to 3000 and B to 2950

-Change CFR reference to a 55.41 reference, not .43. Done NRC 1998..

Re-phrase stem and answers as follows to improve focus of question and apply a common set of conditions to each of the answers/distractors:

The running makeup pump has tripped.

PZR level = [value < 55]. [50]

RCP seal bleedoff temp = [value > 180F]. [190F]

Restoration of normal makeup and seal injection is in progress.

Which of the following is a required action per 1203.026, Loss of Reactor Coolant Makeup, in order to restore seal injection?

A. Normal makeup must be restored BEFORE seal injection, to raise RCS inventory.

4 H 3 X B S B. Seal injection control valve CV-1207 must be quickly opened to restore previous flow rate and minimize time without seal injection, independent of normal makeup restoration.

C. Seal injection control valve CV-1207 must be slowly opened to minimize thermal shock to the RCP seals and prevent seal damage, independent of normal makeup restoration.

D. BWST outlet valve associated with the operating HPI pump must be verified closed prior to opening seal injection control valve CV-1207 to prevent borating the RCS.

Revised as suggested Change will allow maximum to will maximize.

U K/A MM not testing on a loss of RHR, testing on high flow alarm setpoint in a 5 H 3 X N S normal DHR configuration with 1 DH pump running. _Rev 1 Page 2 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 3 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Disagree. KA is the ability to operate and/or monitor LPI pumps as they apply to a Loss of RHR. The question conditions state that a DH pump has tripped and the opposite train pump has been placed in service (as a response to the DH pump trip.

The question conditions also state that the opposite trains pump was placed in service at minimum flow and that RCS temperatures are rising. The question stem then asks what maximum flow can be achieved with the B DH pump (to maximize RCS cooling) without causing the high flow alarm. This tests his ability to monitor the B DH pump flow following a loss of the A DH pump.

[DH Flow alarm setpoint is variable based on RCS level, therefore applicant does need to know what flow he can manually adjust to with one DH pump in response to a loss of the other in-service pump. Acceptable]

NRC 2004.

The reason the distractors are wrong is a matter of time and scale presumably even though the incorrect loads are small or have low D/P, given enough time they E would cause ICW surge tank level to rise and overflow, absent operator action, 6 H 3 X B correct? Add a qualifier to the stem clarifying that the surge tank overflows shortly S after radiation alarm receipt, or provide a specific time period in which the alarm comes in such that the largest leak on the other components would be incapable of overflowing the surge tank. Done [Provided additional explanation]

-CFR reference is not applicable to ROs. Done [41.6]

7 F 2 B S is the should be includes a in first line of stem. Done NRC 2007

-Dist C: given reactor vessel head voiding, step 40.E would have you open the reactor vessel head vents to eliminate void, correct? Not A and B Loop vents. You would in fact leave it open until void was eliminated. Concerned with phrasing.

3 N E Leave open for the remainder of the procedure?

8 F X 2 B S -Stem: Which of the following actions is procedurally-required for the above conditions, per 1202.006, Tube Rupture? Done Due to comment on SRO question #85 concerning overlap with this question, replaced K/A with EK1.02 and replaced question with bank question #0332. This question was last used on the 2007 exam. [Question replaced, SAT]

NRC 2008.

E Include in explanation of Dist C that this would be a correct action if both SGs are <

9 H 2 X B 600psig and dP is < 150psid per RT-6 step 6. Not incorporated, this would not be a S

correct action or the question would have 2 correct answers since both SGs ARE in that condition. [changed C to > 570 gpm] _Rev 1 Page 3 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 4 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Distractor D not credible with OTSG pressures as low as they are. Changed D to verify 340 gpm to each SG. [Minimum required flowrate per RT-6 when not in overcooling]

NRC 2004.

Not clear why it couldnt be an ICS malfunction (Dist B), as 2 of the given conditions E are entry conditions for the ICS Malfunction AOP. Per validator comment on same 10 H 3 X B S issue, added SASS mismatch alarm is clear to eliminate B as a possible correct answer. [No change necessary - SASS mismatch alarm eliminates ICS malfunction possibility]

-Clarify explanation for dist A/B re: AAC DG Availability. Deleted verbiage about AAC DG.

-Is AAC prohibited from being aligned to both A3 and A4 simultaneously? If it is, then a better distractor for C may be Energize both 4160VAC buses A3 AND A4 from the AAC DG. distractor C is plausible but a bit of an outlier. Changed per suggestion. [OK - also added to stem that no additional TS entries are to be made, to prevent invoking 50.54.(x) to restore power]

11 H 3 N S

-I was under impression that SU2 unit 2 loading was limited to 1 4160V and 1 6.9kV bus max - or is that just the allowed lineup for fast-transfer? Actually it is limited by procedure to 1 4160v bus (A1 and 2A1) per unit for auto transfer. If Unit 2 concurs, then we can align all of our buses for auto transfer when SU1 is not available (and no U2 bus aligned to SU2). Of course, the SU2 load shed circuits must be operable for this condition. [OK]

-CFR reference is not applicable to ROs. Done B and C - verbiage for 2nd parts differ - correct to due to the time delay relays which 12 H 3 N S sequence EDG loading. Revised both to say relays which prevent EDG over-loading. [OK]

The knowledge check which makes C the correct answer is sound - applicant has to understand how low power operations impact response to a loss of an NNI instrument bus. However, this is also the only choice powered from NNI Y, so the knowledge check is diminished. To improve this question, either:

1) Increase power some to a level where MFW pumps are not on dP control E and are therefore dont require HAND control, then make one of the 13 H 3 X N S distractors a component powered by NNI Y requiring manual control (new correct answer); OR

2) Keep stem as-written but make one of the distractors a component with a relation to NNI-Y but unaffected by loss of NNI-Y under these conditions.

Not incorporated since 1203.047 step 8 does not differentiate between actions for loss of NNI-Y AC and DC components, this step is performed if ANY NNI-Y power is _Rev 1 Page 4 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 5 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only lost. Thus any NNI-Y component would also be a correct answer. Additionally, C is correct ONLY if the MFW pumps are on DP control. MFW pumps are in AUTO at 30%. If the MFW pumps were on speed control, then there would not be a correct answer.

Also briefly explain in the justifications the power at which MFW pumps transition from dP control. Done

[All 4 choices have a relation to NNI X but applicant has to know that choice C is dependant on power level and method of MFW pump control at low power, which causes it to be powered by NNI Y. Acceptable as is - challenging question.]

NRC 2008.

14 H 2 B S

-CFR reference is not applicable to ROs. Done 15 F 3 N S Stem: Place Personnel are using Breathing Air for respiration, Pressure = 85 psig under the line about IA header pressure low alarm. Added Inst Air header press 70 16 F 3 N S psig and lowering. Alarm is at 75 psig. [OK]

Is there a specific threshold below which the plant s/d must occur? Yes Is it clear from the stem conditions that this point has not been reached yet? Yes [35 psig]

NRC 2005.

17 F 3 B S Modify stem to: Per 1202.004, Overheating, which of the following indications confirm adequate HPI core cooling? Done NRC 2014.

18 H 4 N S OE 19 H 3 B S NRC 1999.

NRC 2002.

Add units to 10E5 (cps), make a separate bullet for clarity Done Modify NI-4 to indicate 7E-11 Done E

20 F 3 X B Distractor D weak - Modify to Hold power and restore a SR channel to operable.

S Done What procedural guidance allows for controlled plant s/d (dist B)? If both SR and IR channels fail during a startup but there is still some indication of flux (NR-502), then 1203.021 allows a controlled shutdown vs. tripping reactor. [OK] _Rev 1 Page 5 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 6 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Distractor A - this band could, or would cause SCM to become inadequate?

Looking at fig 1 of EOP 1202.013, 970 psig at RCS temp 489F is either right on the line for +50 deg subcooled or to the right of the line which would make A correct as well; either way so close that it could not be substantiated against an appeal.

-Distractor D - not credible to maintain SG pressure above a safety setpoint during a tube leak/rupture.

-CFR reference is not applicable to ROs.

Disagree with comment on Distractor A. Right on the line or to the right of the line is NOT desirable (loss of SCM which requires tripping RCPs, highly undesirable in a SGTR) which is why A is not correct. It may be close but this is the only way to get U into this situation. Distractor is plausibleand also incorrect.

21 H 3 X X N S Disagree with comment on Distractor D: The pressure band is for RCS pressure, not SG pressure. The candidate has to recall the purpose of an emergency cooldown, the significance of RCS temperature being less than 500 °F, and of maintaining RCS pressure below the lowest SG safety, i.e., the RCS is now connected to the secondary, the steam lines could be filled with water, and RCS pressure must not cause a release through the safeties. None of this is given in the question.

However, a validator commented that the D range was not consistent with the other three. Changed temp to 490 so it is easier to find on graph. Changed distracter D.

Also revised explanations for answers.

D. 1060 to 1080

[Modify A to 950 to 970. Revisions SAT.]

Change if to when in the answer choices, to preclude the claim that D is also E correct (if vacuum is below 24.5 in, then its definitely below 26.5 in, and a trip is 22 F 2 X M S required). Using the term when vacuum drops implies it should be done at that time, and not before. Done Define CA-1.

Applicant only needs to know RCS vs Secondary components to answer question.

Disagree. The original question written for this KA was replaced following initial validation and only one person validated the replacement and that person missed the 1 U question (chose D). It is quite easy to confuse the distractor actions since they are all 23 F X N performed to reduce dose. FYI, A distracter is a primary function, not a secondary 2 S one.

CA-1 is the RCA exit point.

[Add stem info that A OTSG primary to secondary leakage <5 gpd (something less than 150 to meet TS and less than 5 to meet NRC commitments Unit 1) since B and D are in fact performed during a SGTR to reduce dose - wrong but more plausible] _Rev 1 Page 6 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 7 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 24 F 3 M S NRC 2005.

25 H 3 B S Add is in alarm to K01A1 line. Done Distractor C explanation - add a line describing corrective action taken/planned in 26 F 3 N S last year for this condition, e.g. new watertight door, access controls, etc. Credit this question as OE. Credited as OE.

Distractor A is partially correct - pressure will rise and eventually drop to 1650 due to cooldown. Not bounded.

Distractor B is not credible that increasing injection into solid RCS will be mitigated by E* heaters. Changed to PZR spray.

27 H 2 X M S Distractor C not credible that pressure will immediately drop. Also explanation doesnt match distractor. Expounded on explanation.

E* because used on NRC 2005 exam, otherwise would be U.

[Question replaced]

NRC 2014.

-This question is really about effects of a standard loss of seal injection (normal seal bleedoff) vs. effects of a loss of seal injection with excess seal bleedoff. The consequences in this case would be potential mechanical seal damage > 200F.

-Distractor C is not credible since the specified component does not exist.

-Change answer choices to be:

E A. Rise to potentially seal-damaging temperature > 200F, due to bleedoff in 28 F 3 X B excess of seal cooler capacity. (correct) Done S B. Rise to potentially seal-damaging temperature > 200F, due to loss of all seal cooling. This is too close to being correct. Added first part, left second part as is.

C. Rise to ~170F, due to bleedoff within seal cooler capacity. No seal damage expected. Done D. Remain the same due to seal recirc flow impeller circulation. Same as before.

[OK]

NRC 1999.

N E -What is reference for 1 psig minimum H2 gas volume to prevent corrosion? Revised 29 F 3 X B S explanation.

-A - Should be Exhibit A curve? Include curve in explanation. Done. _Rev 1 Page 7 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 8 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only

-B and C appear to be different way of saying the same thing inadequate H2 providing inadequate O2 scavenging, increasing corrosion risk. You are correct.

This was done to have another distracter rather than coming up with something implausible.

-Look at design basis for the VCT. Peer reviewer believes B&W design may allow VCT to be completely vented and still maintain NPSH on static head. Would result in no correct answer. Question is based on limit and precaution which appears in several procedures (1104.002 and 1102.012). Further investigation reveals this L&P is in error. ER980908 shows that adequate NPSH exists with a negative pressure in the tank.

Will replace question. Will submit PIFs following exam.

QID 258 chosen as replacement.

[Question Replaced - SAT]

30 H 3 B S NRC 2007.

Dist A: clarify what is meant by pressure surge path is isolated. Added explanation.

E Dist D: Does CV-1428 auto-open when P34A started? If not, explanation doesnt 31 H 3 X N really follow; a bypass too far open would reduce cooldown rate, and with DH Cooler S

outlet valve closed, bypass position has no impact on cooldown rate. Expounded on explanation. [Reviewed]

NRC 2013.

32 F 2 B S D: does LPI Flowpath mean LPI suction flowpath from BWST? Clarify. Yes, that is what is meant by LPI flowpath.

33 F 3 M S 34 F 2 B S NRC 2010.

35 H 2 B S NRC 2005.

Dist. D is an outlier - it is a parameter response, while the others are plant component responses. Also, can you confirm that NO increase in RCP seal temperature occurs with a loss of ICW to seal coolers, even with seal injection in E service? You are losing a cooling medium, however unnecessary, so if this causes 36 H 3 X X M S seal temps to increase by even a few degrees it could be considered a partially correct answer depending on the timing. Recommend a different component response for D. Be careful not to overlap with q35. Consider ICW Booster Pump 114A suction valve [###] closes. _Rev 1 Page 8 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 9 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Considered suggestion but rejected due to being implausible since this does not automatically occur except upon ESAS. Changed D to RCP seal temperatures rapidly rise.

[Replaced D with Nuclear ICW cooling loop inlet isolation valve isolates (component) which is true in ESAS but not in this case - more plausible]

37 F 2 N S 38 F 2 B S NRC 2002.

Make picture color, and highlight which indications are lit (may do via text box). Was 39 H 3 B S planning on making picture color for exam, scanner did not cooperate for submittal.

Is 1 out of 4 logic used in any protective system? No.

Not clear why there would necessarily be a release to the public, 1 train of RBS in service so containment integrity should not be challenged.

E Recommend modify C to Containment atmosphere Iodine fission product 40 H 3 X N S concentration would be higher Done.

Confirm that Containment atmosphere Iodine concentration is actually a design criteria may want to remove that term from stem. Done.

Explanation says that K/A requires candidate to know major purpose of logic buffers, but question is about bistables. Fixed explanation.

41 F 2 N S Include pages 28 and 29 of STM 1-65 in worksheet to show purpose of distractors.

Done.

42 F 2 B S NRC 2014.

Provide the normal spray termination criteria in the explanation, to demonstrate that the distractors do not meet these normal criteria. ANO-1 no longer has any normal 43 H 3 N S RB spray termination criteria, Spray is only secured if RB sump blockage is detected.

[add per 1202.10 att 1 to stem]

Modified NRC 2014.

Specify which EOP in question part of stem. Done.

44 H 3 M S Confirm that this specific TS basis information is expected RO knowledge at ANO-1.

This is expected RO knowledge. [OK]

NRC 2005.

45 H 2 B S Used on 4 previous NRC exams. [rearranged answers]

46 F 2 B S NRC 1999. _Rev 1 Page 9 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 10 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Is it necessary to specify the busses that were lost, or is degraded power sufficient irrespective of the affected busses? Degraded power is sufficient.

47 H 4 M S Confirm that this rule of thumb (2+ 0.024/psig above 800) is required knowledge.

This is required knowledge. [OK]

Modify stem to MUST be actuated to directly cause a fast transfer of to preclude D from being considered a correct answer. The use of must may be E sufficient in itself, but directly shores it up. Revised stem and slightly re-worded for 48 F 2 X N S clarity. Which of the following MUST be actuated to directly cause a "fast" transfer of the 4160v/6900v buses from the Unit Aux Transformer to the Startup #1 Transformer? [OK]

NRC 2005.

-Distractor D: Are lift oil pumps running at this power, or secured after RCP start?

Appears non-plausible. No, they are not normally running but they are a support E* system and would be used to shut down an RCP.

49 F 2 X B S -Distractor C: A loss of what would affect seal bleedoff flowpath isolation? The loss of power. [480V bus powered from A4, MOV fails as-is]

In general, use of the term most affected is not advised as it leaves too much room for argument and subjectivity. Deleted most.

Need a clearer, color picture or illustration. Color photo will be included in exam.

50 H 3 N S

[Made illustration]

51 H 3 B S NRC 2010.

Improve the focus on operational implication by rewording Question as:

What is the cause of these indications, and operational implication?

A. A crud burst from starting the 4th RCP released activated iron and nickel isotopes, letdown flow must be raised to increase clean up.

E B. The Iodine portion of the failed fuel detector is failing low, a mode change 52 H 3 X N S is not allowed.

C. The in-service letdown demineralizer is exhausted, and must be swapped.

D. RCS activity due to release of fission products is rising, a reactor startup may not commence (or plant heatup must be halted, if this is true).

Revised per suggestion. [OK]

53 H 4 B S E

54 F 2 X B NRC 2011. 4 previous NRC exams. [Rearranged answers]

S _Rev 1 Page 10 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 11 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Add a stem condition to the effect of Unit 2 has 1 degraded or inoperable IA compressor and both units at 100% power, to make answer B less directly obvious, and make applicant question if isolating IA cross-connect is correct. OK

-Simplify stem to, Which of the following is the procedurally-required response per 1203.024, Loss of Instrument Air? Added procedure but retained the restore or conserve to ensure only one answer is correct.

-Delete If ICW available from D. Done. [OK]

NRC 2004. 3 previous NRC exams.

-Distractor B not credible since this table doesnt exist. Most ILT trainees will not have seen it when it did exist. Replace. Done [replaced dist b]

E* -Distractor C is not grammatically aligned with stem question and other choices, and 55 F 2 X B does not actually direct the applicant to do or compare CIVs to anything, it is a S statement of fact. Replace. Done

-Add No components have been overridden to stem to handle the exception in RT-10 5.A. Done E because of prior exam use, but qualitatively a U.

NRC 2007. History cut off, more recent use? No

-Distractor A not credible, stem says only 1 rod dropped but A discusses relatching 2 rods. Corrected, question was revised due to validator comment, changed number of E rods dropped to 1 but neglected to change distracter.

56 H 3 X X B -Explanation discusses 2 dropped rods in several places, is something missing from S

question? Clarify, if Im missing something thats implied in the stem. Revised explanations, only one mentioning two dropped rods is explanation for distracter D since a Rx trip is only required for more than one dropped rod when greater than 2%

power. [oversight] [OK]

NRC 2005.

-Question has been used on 3 prior NRC exams, and is straightforward enough to be B E sight-recognized by an applicant who has reviewed publicly-available prior NRC 57 H 2 X M S exams. Change the power level to 85% and add a new correct answer, and credit as a Modified question.

Revised as suggested, now Modified.

58 F 2 N S OR.

E 59 H 2 X N -Since D is performed when entering Region 3, it is expected to have been performed S already, and if not, it should be performed correct? D appears partially correct.

Disagree, condition in question states All actions have been performed for the _Rev 1 Page 11 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 12 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only current Region. This makes D incorrect. [add parameters in first part of stem to clarify that region 3 is still applicable]

-There is a stray explanation at the bottom of the notes RE/ C. Deleted NRC 2011.

-Provide valve designator in stem. A line break has occurred downstream of in-service Spent Fuel Cooling Pump P-40A discharge valve #####. Added valve designator in last sentence of stem.

-What should happen with Done E -Modify B to The SFP will drain to the point at which fuel assemblies will become 60 F 2 X B uncovered. No other choices discuss compensatory actions. Revised with similar S

wording.

-Doesnt appear to be a technical justification for difference between A and B responses - consider changing A to reflect a value associated with cask loading area or fuel tilt pit discharge level, if different. Gate is normally installed. Technical justification is provided for B, A is ok as-is. [consider height of bottom of fuel pit gate as distractor for A - Done]

-For distractors A and B, is there a limit to which flow should be reduced to, which can be added to the distractor to make the question more operationally valid? A and B seem too vague to be found in a procedure. reduce flow to ##gpm due to exceeding Tube-to-Shell delta-T limit. Per RT-16 as long as SCM is adequate AFW E

61 H 3 X N flow may be adjusted as necessary to maintain adequate SCM, tube to shell DT S limits, and desired cooldown rate. There are flow limits only with EFW. [Added values to A and B associated with EFW limits, not AFW. Still wrong.]

-Improve stem to allow applicant to determine if tube-to-shell delta-T has been exceeded prior to snapshot given. Revised stem.

62 H 3 M S

-Stem is incomplete. A release from Waste Gas Decay Tank T-18A is in progress, when RE-4830 (name) alarms. Revised 2nd item in given conditions.

63 H 3 X M S

-Is there a name for the alarm? Theres a generic annunciator which sends one back to check out an entire cabinet of radiation monitors NRC 2008.

64 F 2 B S -Can you reference when distractor C changed? Im guessing 11/15/2000, the date for Rev. 1 of this question.

NRC 2007.

65 H 3 B S Add per 1203.001, ICS abnormal operation? to stem. Done _Rev 1 Page 12 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 13 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only

-Distractors A and B not credible that Electrical Maintenance Journeyman would evaluate continued need for a control room annunciator.

-Why say CBO instead of licensed operator?

This question is in a 2 x 2 format and any choice used has to be repeated, thus the danger with this format is creating two implausible distractors if one half of the U distractor is implausible. Electrical Maintenance journeyman was considered 66 F 2 X N plausible since they perform modification of the annunciator circuit boards. Again, S

using personnel actually involved in removal or modification of annunciators was deemed more credible than using other, un-involved positions. [Electrical Maint Supervisor replaced with I&C Superintendent - more plausible]

CBO was used since that is a common licensed operator position [revised stem to what positions are responsible from what two positions are since more than 2 positions for licensed operators.]

67 F 2 B S NRC 2011.

-Distractor A not credible; Waste Gas Decay Tank contains activity, not reactivity.

E Replace with something in the secondary that might be considered, like returning a 68 F 2 X N S condenser section to service. Replaced with Raising the seal injection flow rate to RCP P-32A. [OK]

NRC 2007. 3 previous NRC exams. [rearranged answers]

-Edit B to: Preparer and reviewer must include a licensed operator from each unit.

Done 69 F 2 B S

-Edit C to: if installation authorized by both Revised to Preparer and reviewer may be non-licensed if authorization is performed by a Unit Operations Supervisor from each unit.

-Are the applicants going to understand what an open configuration control loop is?

70 F 2 N S Yes, its a common term.

-Change C and D for improved discriminatory value.

-Change distractor C to: Approval of the on-watch Shift Manager (appears to be E required for VHRA only, verify) 71 F 3 X N S -Change distractor D to: Double PCs (Protective Clothing) (LHRA entry requirements appear independent of PC requirements)

Revised as suggested, this also required a change to the question stem.

72 F 2 B S NRC 2008.

73 F 2 X B E NRC 2010. _Rev 1 Page 13 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 14 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S -Change distractor D to Instruments designated for use during a loss of offsite power event. (or Station Blackout Event, whichever more plausible) Disagree, either one would make D a correct answer. Reg Guide 1.97 instruments are for accident conditions and those qualify.

[Go with Equipment designated as important to E-plan - from 1903.069]

NRC 1998.

-Edit C to stand-alone independent of B, e.g., All non-duty (or non-watchstanding)

E 74 F 2 X B Operations personnel S

-Is on-duty the same as on-watch? Make this phraseology consistent.

Made all choices on watch or non-watchstanding.

NRC 2011. This counts as a Modified question.

-Edit A to since 0400 to clarify that the surveillance started on night shift and 75 F 3 M S pump is still inop. Change all at to since.

Changed to Modified. Revised as suggested.

76 H 3 N S 77 H 3 M S

-Is going through the Main FW Block Valves a specific phrase? Feeding through the Main FW Block Valves? Yes, this is a specific B&W phrase.

E -[General Emergency is low credibility with the given conditions - can a lower value 78 H 2 X N of feedwater flow be added to stem that would make applicant consider higher S classifications more?] This is the only set of choices that are possible. None is not allowed and so there are no more choices to use. Revised FW flow to 1.4 e 6 lbm/hr.

[Significantly revised question. Choice is now between SAE and Alert, 2x2]

NRC 2010.

-Is it plausible in ANO EOP mitigating strategies that you would ever want to raise RCS pressure back to normal operating pressure following a major casualty, rather E* than implementing a controlled cooldown and depressurization?

79 H 3 X B S Plausible in the sense that RT-14 step 5 could allow one to do that, if one went along with a bad idea, which is what this question is all about: does the candidate recognize this as bad advice and does the candidate know the EOP bases? [OK could be allowed if RCS PT limits had not been violated]

80 H 3 M S OR. _Rev 1 Page 14 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 15 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only

-Modify first given condition to loss of both MFW pumps at time 1020 and delete 81 H 3 N S reference to time 1020 in 6th line. Done NRC 2011.

-Why is K/A statement 036 AA2.02 included in the description? Our description field contains the verbiage for A08 AA2.2. The APE title was at the beginning and has been deleted. [Will delete]

82 F 2 B S -Briefly explain how much time is available until a significant loss of shielding occurs, such that 1203.042 section 2 3.B does not apply. This is why 2/minute was chosen, the bridge is already over the core so there is enough time to set the assembly down in the core. A foot/min would lead to use of 2.3.B.

[Provided detailed math in the explanation showing that sufficient time exists to insert assembly into core before TS limit for water above TAF is exceeded.]

-Distractor B is not credible. TS do not require you to initiate a condition report.

Changed condition to special.

-Add per TS 3.7.4, Secondary Activity to B and C. Done

-Add per ODCM L2.3.1 to A and D. Done

-Use Suspend the release instead of Stop Done

-Clarify, if L2.3.1 Cond C were applicable, the release would not be required to be suspended immediately? See below E

83 F 2 X N -Stem says continuous release is in progress, but answer references L2.3.1B which S

is only applicable to batch releases - clarify basis. As the Notes state, only batch releases are allowed during a tube leak therefore the sampling requirements for a batch release would not be met and the release should be suspended. A continuous release is not allowed therefore L2.3.1.C is not applicable and would result in a continuous release of unknown activity while waiting on grab sample results.

[Added to explanation a logic path showing that when RI-3618 alarms, only a batch release is allowed at that point and sampling requirements have not been met.

1104.044 AND 1203.014]

OR.

84 H 3 N S -CFR 43.2 more applicable. Done U NRC 2008.

85 H 3 X B S -Substantial overlap with RO Question #8 - replace one or the other. Replaced #8.

E -What is the threshold for seal failure and RCP trip? Section 2 of 1203.031 entry 86 H 3 X M S condtions: _Rev 1 Page 15 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 16 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only

>10 gpm rise in RCS leak AND change in seal cavity pressure behavior RCP seal bleedoff or seal stage temp 200F AND no change in SI or ICW DP across a single stage = RCS press, with seal BO established.

-Distractor D: not credible to think you need to use to normal power reduction procedure to secure the reactor coolant pumps. Use the 2nd part of C instead. Done OR.

-Discriminatory question, but doesnt appear this question is testing the impact on ESAS - loss of DC causing letdown relief to lift doesnt have a relation to ESAS.

[peer review identified] Disagree, D01/02 are Vital DC buses and thus are Engineered Safety Features.

-Why is it necessary to give: attempts to transfer 125V DC panel D21 to its U emergency supply are unsuccessful in stem? Cues the malfunction, and applicant 87 H 4 X N S should be able to diagnose malfunction from first cue. If it is necessary to ensure entry into 1203.036, consider adding a qualifier to the first cue, actions to restore indication are unsuccessful. If this statement is not necessary to ensure entry to 1203.036, delete. The first action is to attempt this, if it is successful then power is restored and isolation of letdown is unnecessary.

-Modify stem question: which procedure should be implemented? Done

[replaced question]

OR.

-Distractor C: given an open-reference TS, not credible for an applicant to confuse CV-2613 with a TS 3.6.3 isolation valve but then apply a TS 3.7.5 action statement.

Only choice is to change the second half of C to match A which would make for two correct answers. C must be plausible since an SRO validator chose it. The TS do E not contain valve numbers. [2 validators chose C]

88 H 3 X N S -This question does test on an interesting operability nuance vis a vis CV-2613 vs.

CV-2663 for pump operability. Would be more discriminatory to make CV-2663 the affected valve, and make D the correct answer. This was considered and thought this might be too discriminating, best tested at requal after candidates have some experience. One of four SROs missed during validation. [Question OK as is. Very challenging]

-Distractor C: HPI cooling not credible given stem states an EFW pump is running, no reason to believe HPI cooling would be required. [peer review identified] Thats the E most plausible procedure possible without re-using Overheating.

89 H 3 X N S -Can you provide a more specific explanation of ANO procedure use hierarchy that specifies staying in Degraded Power EOP despite indications of heat transfer upset?

A1LP-RO-EOP07 lesson plan [add while continuing with 1202.007 to end of D. _Rev 1 Page 16 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 17 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 90 H 4 N S -gid should be grid in all answer choices. Corrected OR.

-Ensure inclusion of PAM equipment table in TS doesnt cue any other questions on RO exam.

-Question is direct lookup given the TS. Modify question to:

Per LCO 3.3.15 Action B.1, when is the special report to the NRC due?

A. 8 hours9.259259e-5 days 0.00222 hours 1.322751e-5 weeks 3.044e-6 months B. 24 hours2.777778e-4 days 0.00667 hours 3.968254e-5 weeks 9.132e-6 months C. 30 days (correct)

D. 60 days 1 M U 91 F X 3 N S -C requires applicant to know the tech spec bases. A is the 50.72 reporting time (not applicable), B is the 26.719 reporting time for licensed operator FFD violations, D is the reporting time for LERs under 50.73 (not applicable).

Disagree with this being a direct lookup. One has to know that Pzr level is in the PAM specification. The LCO refers to the table, the table has to be referred to and then one has to look back at the LCO. One has to comprehend what all of the specifications mean to determine which applies. I agree that this one is straight forward but disagree with direct lookup since there is no statement that states, If PZR LT-1001 or LT-1002 are inoperable, then.

[Suggested Change incorporated.]

-ATC reports to the control room communicator that Source Range court rate is 92 H 3 N S Done

-Add specific before consequences in stem. Very nuanced question and we need 93 H 4 M S to make sure distractors A, C, and D are not appealable. Done [facility endorses its use]

NRC 2014.

94 F 2 B S Its effectively the same question as Bank, just different start time (equivalent to changing a question to test on opposite train redundant SSC, which does not count as Modified per 1021). Credit as bank, used within last 2 NRC exams. Done

-Change stem question to: In order to change the fuel shuffle sequence the SRO in E charge of fuel handling and _________ must approve, per 1502.004, Control of Unit 95 F 2 X M 1 Refueling.

S A. Control Room Supervisor B. Operations Shift Manager _Rev 1 Page 17 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 18 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only C. Reactor Engineer D. Refueling Project Manager Revised as suggested.

-Delete statement in stem that the applicant is the Shift Manager, since they are not attaining that qualification. Done

-Compliance misspelled. Corrected 96 F 2 M S

-This and many other questions have strange alignment, where some lines cut off to a new line before others. Corrected.

-This is OE from Crystal River or Oconee. Yes 97 F 3 B S NRC 2014.

-Are radiation exposure limits to save a life part of rad worker training? If so it is RO level. ROs are not taught limits to save valuable eqpt, life, retrieve a body, etc?

-If this is in fact not RO knowledge, modify to remove C and D which dont actually have a technical basis, as follows:

Which of the following is the maximum time each team member can stay in this area?

(1 vs 3 minutes)

The [Shift Manager / Emergency Director / appropriate position] May / May not authorize a team member to extend this maximum time if they volunteer.

U 98 F 2 X M Still a fairly weak question for SRO exam.

S Disagree with this being a weak SRO question, a different version of this has been on two previous NRC exams as an SRO question. The original concept was obtained from an NRC developed exam.

Radiation exposure limits may be a part of rad worker training but exposure limits for protecting valuable equipment or to save a life is reserved for training to the Emergency Director and Shift Manager.

[Substantially modified question during validation week. SAT.]

E -Distractor C is not credible, as the EOFs purpose is elevated emergency response.

99 F 2 X N S Changed to TSC/OSC evacuation. [OK]

NRC 2011. 4 previous NRC exams.

E -Spell out ADH. Done 100 F 2 X B S -Distractor D, notifying the NRC within 4 hours4.62963e-5 days 0.00111 hours 6.613757e-6 weeks 1.522e-6 months simply isnt plausible. Change.

Changed to 30 minutes. Added 1903.011 to stem. _Rev 1 Page 18 of 20 OBDI 202 - IOLE Process

ES-401 AN1-2016-08 Written Examination Review Worksheet Form ES-401-9 B = 39 F = 37 E = 27 Additional Notes:

All final revisions SAT.

RO TOTALS: M = 11 H = 38 U= 4 N = 25 S = 44 B= 6 F = 10 E= 8 Additional Notes:

All final revisions SAT.

SRO TOTALS: M= 7 H = 15 U= 4 N = 12 S = 13

GENERAL COMMENT

S

1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N

2. Chief Examiner original comments are in black AND blue. Licensee provide follow-up comments in red.

Instructions