ML17081A540

4-COLUMBIA-2017-02 Draft Written Exam Comments

ML17081A540
Person / Time
Site:	Columbia
Issue date:	03/09/2017
From:	Vincent Gaddy Operations Branch IV
To:	Energy Northwest
References
Download: ML17081A540 (38)

Text

ES-401 CG-2017-02 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Are the abbreviations GE, LT, etc. standard usages at CGS that applicants are familiar with? If so - ok, if not - spell out.

Distractor B - what does restarting RRC-P1B have to do with placing Loop A controller in Manual? Implausible cueing.

Distractor D - should there be a target flow?

E CGS:

1 H 3 X N S GE, LT, etc. are standard abbreviations at CGS.

Distractor B - RRC-P-1A controller is placed in manual to allow manual control of the RUNNING RRC pump per step 4.3.3 of ABN-RRC-LOSS. It is not related to attempting to restart the tripped pump. The distractor wants to restart the pump that tripped. That pumps controller would be operated per the start procedure.

Distractor D - The distractor states that the target is operating the pump at 60 hz without regard to pump flow. 60 hz is the nominal 100% pump frequency.

Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

$ The answer choices are a collection of unrelated true/false statements.

$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

$ The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met). _Rev 2 Page 1 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Modified 10CFR part to 41.10

[NRC OK]

CGS:

2 H 3 N S Revised 10CFR section to 41.7

[NRC OK]

Modify stem to These alarms are consistent with a loss of voltage on Distractors C and D: In explanations, specify exactly which of the 4 alarms in the stem would alarm given a loss of the bus.

CGS:

Stem modified as directed.

3 H 3 N S Distractors C and D - An additional annunciator was added to the stem that is common to distractor C. Although distractor D does not share annunciators with the correct answer, a loss of DP-S1/7 will affect the same systems (RCIC, CRD, RFW) and will cause similar annunciators to alarm. Explanation was revised to reflect this.

10CFR changed to 41.10

[NRC OK]

Distractor D explanation - which will occur when the main turbine trips greater than 100% reactor power check for accuracy.

Explanations - describe difference between actual vs indicated RPV level in this transient and reason.

CGS:

Removed which will occur when the main turbine trips greater than 100% reactor power from distractor D explanation; not required for explanation.

4 F 2 N S Revised explanation to describe the differences between actual RPV level and indicated level, including the reasons for the difference.

[NRC - see comment on Q12. The concept is sound by the terminology needs to be precise. If an applicant were to pick A, he could point to question 12 and argue that the NRC considers RPV level to refer to downcomer level in another question, so it should apply in question 4 as well.]

CGS:

Modified stem to read in-shroud RPV level vice actual RPV level _Rev 1 Page 2 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 3 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only The question asks for the overall effect on in-shroud RPV level for the conditions given, which is a turbine trip from high power. This transient will cause in-shroud level to lower due to reduce core voiding. RRC pumps are tripped to minimize this effect.

When regarding only the RRC pumps tripped, RPV level, both in-shroud or indicated, will rise. However, when examined in the context of the conditions given in the stem, in-shroud level will lower.

See comments for Question 12.

[NRC OK]

Modify stem to Which of the following should have occurred since the DEH pressure is already below the trip setpoint.

The stem is cuing the Turbine Trip response because in the given picture of DEH-PI-21, the red band starts just above the trip setpoint of 1600#. Remove the colored bands from the picture of DEH-PI-21, or provide a numeric value instead.

DEH pressure is substantially below trip setpoint, move it up to 1500 # for better plausibility.

E 5 H 2 X X N Correct answer cued by SRO Q88 provided reference.

S CGS:

Stem modified as directed.

Color bands removed from DEF-PI-21.

DEH-PI-21 indication raised to 1500 psig.

SRO Q88 reference revised to remove cueing.

[NRC OK]

CGS:

6 F 2 B S Modified answer numbering for consistency throughout exam.

[NRC OK]

Im not great with this question because its just testing on whether the applicant knows which component is in the drywell and which are in the RX or Radwaste buildings not credible that applicant would think Fuel Pool Clg HX in drywell for 1 U example.

7 F X N 2 S Modify to some variation of: Degraded Reactor Component Cooling (RCC) flow, as sensed by (RCC Pump breaker position / RCC Pump discharge pressure / RCC Drywell Flow), causes an automatic isolation of RCC supply to (Radwaste building _Rev 1 Page 3 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 4 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only only / Radwaste and Reactor Buildings / etc.) in order to maximize available cooling to the Drywell. Still not a great K/A fit, but serviceable.

CGS:

Modified question as suggested.

Revised 10CFR part to 41.7

[NRC OK]

NRC 1996 Explanation states therefore the next action is for CAS-PCV-1 to close, but in stem Pressure = 92 psig, the statement immediately before this says CAS-PCV-1 OPENS at 75 psig, and the explanation for Distractor D state CAS-PCV-1 CLOSES at 75 psig. Errors confuse the intent.

E Explanation for distractor C is incomplete.

8 F 2 B S CGS:

Revised distractor explanations for clarification and to correct errors.

Completed distractor C explanation.

Revised 10CFR to section 41.4.

[NRC OK]

-I dont see anything that makes Distractor A, ABN-ELEC-SM1/7 plausible. No indications of a loss of electrical bus. Borderline LOD = 1, would be a better question if applicant had to assess that a normal SDC lineup not available and choose ABN-RHR-SDC-ALT as the correct answer, and something to make Dist D more plausible.

2 E 9 H X N 3 S CGS:

Modified question to determine alternate SDC method.

[NRC OK]

Need to attach parent question for modified bank questions.

What actions should be taken per PPM 6.3.2?

Distractors B and D are effectively the same.

E 10 H 2 X M CGS:

S Question was modified due to a recent procedure change.

Revised 10CFR55 part to 41.10 Dist C and D, make consistent: ... and obtain permission from ] _Rev 1 Page 4 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 5 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only CGS:

Distractor D revised to make consistent with distractor B.

[NRC OK]

Answer D: Can you provide actual design data or FSAR excerpt to support the statement they are designed to operate at higher primary containment temp?

CGS:

Modified correct answer (D) to read they are more reliable in post-LOCA conditions.

This represents the design information presented in the FSAR.

Added FSAR references to support correct answer.

Revised 10CFR55 section to 41.8

[NRC - The modified answer D is too generic to not be true. The distractors are all good and the original answer was good (presumably true) but needed to be bolstered with supporting design documentation to confirm true. Modify to some, not all, are qualified for long-term cooling post-LOCA. At least this makes the applicant question if all ADS valves are the same or not (theyre not). If you dont like this mod, pull the design data for the ADS SRVs to prove the original correct answer true, or put the SRV system engineer on the exam security agreement if your site procedure allows waiving the requirement to wear a badge.

Modify stem to When compared to other SRVs, one reason ADS SRVs are preferred E is that ]

11 F 3 N S CGS:

Modified stem as recommended Modified distractor D as recommended.

[NRC - Modify answer D to: some, not all, are qualified for long-term cooling post-LOCA, as described above. D as currently written is not fully correct. The FSAR excerpt provided on the 1-31 rev (not included with this rev) discussed that 3 ADS SRVs were qualified for long-term post-LOCA cooling, the other 4 were qualified for 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> only.]

CGS:

Updated the explanation to clarify the FSAR as follows:

All SRVs are environmentally qualified the same. The support systems for the ADS SRVs are designed to provide sufficient nitrogen to allow the ADS SRVs to remain operable through an entire post-LOCA cooldown, while non-ADS SRVs are only qualified to be operable up to 2 days.

There are two N2 banks that are credited in accident analysis. One bank supplies 3 ADS SRVs while the other supplies four ADS SRVs. The banks will automatically shift _Rev 1 Page 5 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 6 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only to supply the ADS SRVs if the N2 header pressure lowers to a specific value. The control power required to shift the banks is divisionally separated. Therefore, if one division of power is lost, at least 3 ADS SRVs will remain available.

Section 5.2.2.4.1 of the FSAR (page 5.2-8), reads Three ADS SRVs and their associated solenoid pilot valves (SPV) are qualified for the full post-LOCA time frame for long-term cooling. All other SRVs and their SPVs are qualified for 24 hr post-LOCA to provide overpressure protection capability. The intent of this statement is that during an event with a single fault that allows only one N2 bank to supply its ADS SRVs, at least 3 ADS SRVs will be available.

Therefore, answer D should read they are qualified for long-term cooling post-LOCA.

[NRC OK]

NRC 2009 Distractor D explanation cut off.

Distractor D doesnt make sense, run-on sentence overcomes power increase how?

Answer B: Switch to Increases voiding in the core which adds negative reactivity to match distractors A and C formats. Make Dist D same format once clarified.

CGS:

Revised distractor D for plausibility.

Revised distractor B as directed.

E Revised distractors to make them the same format.

12 F 2 X B S Changed 10CFR to 41.7

[NRC - Distractor D explanation directly contradicts the explanation for question 4 correct answer - its a matter of terminology regarding what is considered RPV level.

We need to ensure that the term RPV level is SPECIFIC to the level inside the core shroud above the core plate, and cannot be confused with the level in the downcomer region between the Pressure vessel and shroud. Find different terminology if the term RPV level is not explicit to one or the other.

CGS:

We consider RPV level to mean indicated or downcomer level.

Modified stem to be more specific about conditions; this question refers to the reason for automatically tripping RRC pumps during an ATWS with high RPV pressure.

Revised explanations for clarity. _Rev 1 Page 6 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 7 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Distractor D is incorrect. When examined separately, tripping RRC pumps will cause both in-shroud and indicated RPV level to rise in all cases.

In Question 4, the overall effect is that in-shroud RPV level initially lowers due to the large pressure transient caused by tripping the turbine. For this question, the pressure transient is not as pronounced and in-shroud level will rise until SRVs actuate.

[NRC - also Include the TSB 3.3.4.2 excerpt that you did with original question submittal in explanation.] OK Are the valve and pump positions / conditions given in the stem the expected conditions for LPCI mode? If so, dont need to provide in stem, applicant should know these conditions as-is, applicant just picking the relevant valves provided in the list.

Distractors A and D, RHR-V-48A is already full open, throttle open not credible terminology.

Revise question by making applicant self-identify that RHR is injecting in LPCI mode based on plant conditions (delete 4th bullet in stem re: injecting to recover level), make determination that RHR A needs to be placed in SPC mode due to wetwell temp, and test on method of doing so or some variation of the above.

CGS:

Modified the stem for plausible conditions where RHR pumps will be injecting in LPCI mode, and established that the crew is operating in the EOPs.

Recommend leaving the valve/pump conditions in the stem as written since they establish the current equipment lineup as RHR A is being shifted from LPCI to F E suppression pool cooling.

13 2 X N H S Modified answer and distractors to be more consistent.

Modified distractor explanations to be consistent with distractors.

Revised LOD to be consistent with Form ES-401-9 Revised LOK to F to be consistent with Form ES-401-9 Recommend changing LOK to H due to knowledge that the candidate needs to possess to correctly answer question:

• Needs to understand interlock between RHR-V-42A and RHR-V-24A

• Needs to know the difference in the suppression pool cooling lineup when operating in the EOPs - namely RHR-V-48A throttled or fully open.

• Needs to know the normal LPCI lineup - RHR-V-4A normally open.

[NRC - OK. There should be a comma in C. There is no D in explanation.]

CGS:

Revised distractor C as recommended. _Rev 1 Page 7 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 8 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Revised explanations to include explanation for distractor D.

[NRC - OK.]

NRC 2009 14 H 3 B S CGS:

No comments Dist B explanation, specify which WW level requires stop and prevent ECCS.

Dist D - what reference specifies operation of RHR pump A is prohibited with high room water level? Does the level at which alarm comes in physically prohibit operation of the pump?

CGS:

Revised distractor B explanation to include which pumps are a concern for vortexing.

15 H 3 N S Added PPM 5.2.1, table 18, Vortex and NPSH Limit as a reference.

Revised distractor D explanation to include the requirements contained in ABN-FLOODING concerning stopping RHR-P-2A when flooding in the RHR A pump room is detected.

Revised to Comprehension or analysis as listed on Form ES-401-9.

Changed 10CFR to 41.10

[NRC OK]

CGS:

Revised to Memory or Fundamental Knowledge as listed on Form ES-401-9 16 F 2 M S Changed 10CFR55 part to 41.9

[NRC OK]

Distractor B - not clear why this is plausible - why would this be an increased challenge to containment capacity with MSIVs open?

Distractor C - not plausible, its acceptable to cooldown because reactivity response for a partially borated core is unpredictable?

Distractor D - not plausible - the core is already at power.

U 17 F 2 X B S

CGS:

Distractor B- Revised explanation to discuss using SRVs to control RPV pressure during an ATWS. This could challenge containment, and is a credible distractor.

Distractor C - Revised explanation to discuss distractor plausibility. Hot Boron Shutdown Weight is discussed in PPM 5.0.10, Flowchart Training Manual. However, _Rev 1 Page 8 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 9 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only adding Cold Shutdown Boron Weight is required to ensure the reactor is shutdown in all cases. Revised distractor to be consistent with Answer A format.

Distractor D - Revised distractor for clarity. Revised explanation to clarify plausibility.

Revised attachment to include additional information on ATWS pressure leg.

Revised 10CFR part to 41.10

[NRC - Distractor C, is NOT permitted to start until Answer A - is not permitted to start until for consistency with C.

Distractor D - is permitted immediately CGS:

Revised distractors as recommended.

[NRC - OK, but improve distractor B by adding a qualifier is not permitted until [all rods inserted, all but one rod are inserted, all outer ring rods are inserted, etc. for better consistency with A and C, and plausibility.]

CGS:

Revised the stem to read: Plant conditions 15 minutes after the scram Revised distractor B to read is not permitted until all outer ring rods are inserted because additional heat load will be imposed on the primary containment that could lead to containment failure.

[NRC - In the question provided to me, the stem does not say plant conditions 15 minutes after the scram, as you state above. Add is not permitted to start to distractor B to be consistent with A and C.]

Statement above: Revised the stem to read: Plant conditions 15 minutes after the scram was incorrectly placed under Q17. It was supposed to go in the response notes for Q13.

Modified distractor B as recommended.

[NRC OK.]

CGS:

18 F 2 M S Revised 10CFR section to 41.8

[NRC OK]

Reference Provided Question is a direct lookup from the references provided. No resident knowledge 1 E 19 F N being tested on.

2 S CFR reference not provided. _Rev 1 Page 9 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 10 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only CGS:

Added ABN-FIRE bases discussing designation of starred steps as steps required for PFSS.

Revised distractors to present plausible choices for actions performed in the reactor building general area for a fire.

Revised distractor explanations to discuss criteria that steps must have a star to designate them as required PFSS actions.

Added 10CFR part 41.10.

[NRC OK]

Dist B explanation, describe the other sources.

CGS:

20 H 2 N S Distractor B explanation revised to include discussion that the Backup Transformer is still available and entry into ABN-ELEC-LOOP is not appropriate.

[NRC OK]

Can answer question without any knowledge of offgas system based on the 2nd part of question alone, since it is a 2 part question and all 4 of the second part choices are different, and relate to non-offgas plant response to rising P-condenser.

Distractor C cued as incorrect by SRO Q88 provided reference.

U CGS:

21 H 2 X X B Replaced question.

E Replacement question is a bank question.

[NRC - Part (1) should be close vs. remain open.]

Revised distractors as recommended

[NRC OK]

Explain when distracter 'D' would be applicable - i.e. when would an operator report cannot be determined.

CGS:

22 H 3 B S Modified distractor D explanation to discuss that a report of cannot be determined would be given if all RPV level instruments were outside usable range.

[NRC OK]

NRC 1999 E

23 F 3 X B Dist B - The given explanation, because step DT-1 is not clear, is not a good S

justification, and implies that perhaps the step itself needs to be revised for clarity. _Rev 1 Page 10 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 11 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Dist C - Is Ctmt Spray administratively prohibited from use to cool drywell absent a LOCA? If so then C is partially true.

Dist D - Drywell cooling equipment load shedding will result in a LOCA signal?

Distractor B - Revised explanation to clarify.

Distractor C - Revised explanation to discuss that containment spray is not prohibited without a LOCA signal. However, containment spray is initiated per PPM 5.2.1 and for the given conditions, containment spray cannot be initiated since DSIL not met.

Distractor D - Revised distractor to read This direction is given as an initial action since drywell cooling equipment will load shed if conditions degrade.

Modified distractor D explanation to explain distractor plausibility.

Changed 10CfR55 part to 41.10

[NRC -Distractor D explanation - If not restarted, the resultant effect of rising drywell temperature and pressure will not result in the fans load shedding. Is this what you intended to say?

Distractor D - since drywell cooling equipment will load shed if drywell conditions degrade.]

CGS:

Modified distractor D as recommended.

Revised distractor D explanation for clarity.

[NRC OK]

Explain why a full reactor scram occurs when the MG SET TRANSFER switch is taken to ALT B.

CGS:

Modified question to use graphics vice written description to improve operational validity and ease of understanding.

24 H 3 B S Modified explanation to provide an in-depth discussion of transfer switch indications and operation, including how a reactor scram may be inadvertently initiated during transfer switch operation.

Changed 10CFR55 part to 41.6

[NRC - OK]

Dist A - ... off-site radiation release rates release?

E Dist C and D - have you confirmed that there is not actually a concern with integrity of 25 F 2 X X B ECCS suction piping or pump components due to excessive static head? If you S havent, these could be potentially additional correct answers.

CGS: _Rev 1 Page 11 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 12 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Revised answer A to remove release at end of sentence Review of the FSAR shows static head of water/additional flow in ECCS systems not a concern. The limiting condition with containment level greater than 535 is the pressure capability of the girder joint of the primary containment.

Revised explanations to highlight that the pressure capability of the girder joint of the primary containment is the limiting condition with containment level greater than 535 Changed 10CFRpart55 to 41.9 Recommend revising LOK to H. Candidates need to know:

• Actual concern is containment pressure, not level.

• Change in limiting condition at 535 containment level

[NRC - while distractor A is not listed as the reason in 7.13.1, is it not a true statement regardless? Wont securing injection sources in fact restore the ability to determine off-site radiation release rates under these conditions?]

[^-- the above comment was provided on 2-3-17 but not incorporated in CGS 2-6-17 response for some reason. Comment still applies.]

CGS:

Replaced distractor A.

[NRC OK]

CGS:

26 F 3 N S No comments Spell out Equipment Drain in the first use of EDR in stem.

Subsequently, a steam leak occurs Is EDR-SUMP-R5 Pump the same thing as EDR Sump Pump, EDR-P-5A? If so, make consistent.

The EDR sump pump is running at the beginning of the question - does it automatically secure? If so, explain that in the explanation.

E CGS:

27 H 3 N S First use of EDR is spelled out.

Modified the stem so that EDR-P-5A is not initially running. After the Z signal, a high sump annunciator alarms. This should open the sump pump discharge valves and start the sump pump. However, with a Z signal locked in, the sump pump discharge valves will close. The sump pump will not start with the discharge valves closed.

Modified stem and answer configuration for clarity.

Terminology in current stem is correct. _Rev 1 Page 12 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 13 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Changed 10CFR55 part to 41.13 Revised LOK to be consistent with Form ES-401-9

[NRC -OK]

Dist A is true - LPCI injection valve does have an open signal for the first 10 minutes of ECCS injection, as well as after 10 minutes.

Dist C and D are not credible distractors - too obviously not related to ECCS initiation.

This is not a preferred style of questioning --- close to being a series of True/False statements which is discouraged by 1021. Reformat to test on RHR-V-48A behavior post-ECCS initiation in a more operational nature. Following an ECCS initiation U signal, RHR-V-48A will (go full shut to ensure maximum cooling / go full open to 28 F 2 X X B ensure ). After 10 minutes, the valve will S

CGS:

Modified stem to state Select the valve that has an automatic open signal only for the first ten minutes after an emergency core cooling system (ECCS) initiation signal.

This was done to ensure that distractors A and C are incorrect.

Revised explanations to clarify why answer B is correct and the other three are incorrect and to demonstrate the credibility of distractors C and D.

[NRC - OK.]

Drywell pressure peaked at 1.60 psig.

All other automatic actuations were successful.

Fix the - symbol to -135 inches in stem looks like a hyphen.

Why is it plausible that min flow bypass valve would be open with pump not running?

Why is it plausible that RHR-P-2B would not be running after attempting to manually actuate under these conditions? The stem tells the applicant that it failed to start automatically, meaning that there is a valid actuation signal. Would be better to have RPV level rise back above -129, and test if pump would start then.

E 29 H 3 X N 3-part answers are often problematic because you dont need to know all 3 parts to S

answer the question. In this case you can answer it just by knowing parts (1) and (2).

Sometimes its necessary, but Im not sure it is here.

CGS:

Modified stem as recommended.

Revised RPV pressure in the stem to 320 psig to enhance plausibility of distractor A.

Revised answer format to 2X2; removed 3rd part of answer requiring knowledge of pump status (running or not) as recommended. This removed the requirement for plausibility of the min. flow bypass valve open with the pump not running. _Rev 1 Page 13 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 14 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Revised explanations to be consistent with the revised answer format. Included discussion of difference in start signals between the automatic (F or A) signal and the manual initiation signal.

Changed 10CFR55 part to 41.8

[NRC OK - but modify the stem to include the highlighted line above]

CGS:

Modified stem to read All other automatic actuations were successful.

[NRC OK]

-Why is RPS A lost? Question doesnt state anything about a loss of DG-1, which should be powering SM-7 to power RPS A 3-part answers are often problematic because you dont need to know all 3 parts to answer the question. In this case you can answer it just by knowing parts (1) and (2).

Sometimes its necessary, but Im not sure it is here.

-Dist B - Explanation doesnt state why its plausible for both inboard and outboard SDC suction isolation valves to be open.

U 30 H 3 X X N CGS:

E Modified question to remove third part of answer (RHR pump status).

S Added abbreviation for shutdown cooling (SDC) in the stem.

Modified explanations to include a discussion of why both RPS busses deenergize on LOOP in Mode 4. Removed references to RHR pump status.

Modified distractor B explanation to show plausibility of distractor.

[NRC OK]

Stem says RPV pressure is 240 psig, explanation says it is 350 psig. Which is it supposed to be? The higher the better for plausibility. Explicitly state in explanation what LPCS-P-1 shutoff head is.

Explanation for D incomplete.

E CGS:

31 H 2 X N Modified explanation to read 290 psig. LPCS pump shutoff head is 320 psig S

Added references to LPCS shutoff head Completed distractor D explanation.

Changed 10CFR part to 41.7

[NRC OK]

Make the (1)s and (2)s consistent some questions its encircled, others its in 32 F 2 B S parentheses. _Rev 1 Page 14 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 15 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Simplify stem to, When the HPCS RPV INJ VALVE ILOCK OVERRIDE keylock sw (HPCS-) is place in the OVERRIDE position, HPCS INJ Valve HPCS-V-4 ... .

CGS:

Modified answer numbering to be consistent throughout exam.

Modified stem as recommended.

[NRC OK]

SLC 1-line flow diagram in SLC System training manual and operations study guide erroneously label SLC pumps as P-1A and P-2B - recommend correct.

Distractor D is a subset of Answer A, so if an applicant was confused about whether A or D were correct, he would select A knowing that it could be justified as a E subset of D if D was the intended correct answer. Correct subset issue by adding 33 H 3 X B and continue running to the ends of A, C, and D.

S CGS:

Error in system training manual recorded for correction following the exam.

Distractors A, C, and D corrected as recommended.

LOK changed to H to reflect Form ES-401-9.

[NRC OK]

Answer D - there should be an IRM MONITOR UPSCALE Annunciator in as well, correct?

Add IRM ACEG UPSCL TRIP OR INOP Annunciator to C Add status of 1/2 SCRAM to A and B.

E Why is C plausible that there would be a 1/2 SCRAM in, with NO Rod Block in? More 34 H 3 X X N S plausible to say IRM ACEG UPSCL TRIP OR INOP Annunciator, NO Rod Block, NO 1/2 SCRAM.

CGS:

Answer and distractors revised as recommended.

[NRC OK]

Explanation says Following APRM/IRM overlap at 5% power, all IRMs are inserted

. APRM power indicates close to 10% and IRM H is still withdrawn. - this appears E to be opposite actual conditions in stem.

35 H 2 X N S Distractor C - is applicant able to affirmatively determine that Q37 overlap requirements are met based on the given information in stem? Need to provide another IRM on range 10 below 40/125? _Rev 1 Page 15 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 16 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Distractor D explanation incomplete.

CGS:

Revised answer explanation. It incorrectly listed inserted instead of withdrawn in some cases.

Revised distractor plausibility statements and completed the D distractor description.

Added statement in the stem to clearly state that all IRMs were INSERTED prior to startup commencing.

Distractor C - Per the conditions in the stem and the fact that all other IRMs are withdrawn with their range switches on range 1, it would indicate that step Q37 is already complete and step Q42 has been performed incorrectly. Overlap is verified on each individual IRM and the conditions in the stem clearly show that overlap is satisfied for IRM H thereby making distractor C incorrect. The procedure would allow the startup to continue if overlap was verified on 3 other IRMs, but overlap is still verified on EACH IRM and it would be expected that a CR be generated if overlap was not met on an individual IRM channel.

[NRC OK]

Stem - Arent odd number ranges on a max scale of 40? Range 5 power should be 37/40 not 37/125?

PPM 4.603.A7 states IF CRS directs, THEN bypass the inoperable IRM channel.

This adds an element of subjectivity to the question modify stem to, Which of the following is the next action that should be taken, with CRS direction?

CGS:

E Revised Range 5 power indications to be XX/40.

36 H 3 X M S Revised stem as directed.

Revised 10CFR section to 41.6

[NRC OK - but stem should read Which of the following is THE next action ]

CGS:

Revised stem as recommended.

[NRC OK]

Can you provide digital pictures of the SRMs, as you have with the other Nuclear E

37 H 2 X X N Instrument questions?

S 3 part question, only need to know 2 of 3 to answer. _Rev 1 Page 16 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 17 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Why plausible that RPS B tripped but not RPS A under these conditions?

CGS:

SRM digital pictures provided for SRM indication.

Revised SRM counts to improve distractor plausibility.

Revised answer format to improve clarity and consistency with other similar questions in the exam.

Revised explanations to add enhanced plausibility statements.

[NRC - OK.]

CGS:

38 F 2 B S No comments.

Dist B - within 40 psig of WW pressure to ensure continued rate of energy addition is within capacity of primary containment vent path.

CGS:

39 H 3 N S Revised distractor B as recommended.

Revised distractors A and B, and answer D to maintain psychometric quality (all distractors are roughly the same length).

Changed 10CFR 55 part to 41.10 K/A mismatch. K/A requires knowledge of use of procedures to mitigate RCIC exhaust diaphragm failure. Question as written does not appear to be testing on procedural actions.

Cueing: Q41 Stem states that RCIC tripped on high exhaust pressure, which cues the first part of the correct answer in Question 40.

U 40 H 3 X X N CGS:

S K/A was replaced with 217000.A2.02. See Record of Rejected K/As for details.

Replaced Question.

Cueing issue removed from Q41.

[NRC OK.]

Cueing: Stem states that RCIC tripped on high exhaust pressure, which cues the first part of the correct answer in Question 40. Delete reason RCIC tripped.

E Distractor D - applicant cant know if ADS timers will be timed out or not, because he 41 H 3 X X B doesnt know the rate of level decrease - therefore D is not credible. Provide a rate S of decrease in stem that would allow him to identify that ADS timer is already timed out by time level 1 is reached.

CGS: _Rev 1 Page 17 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 18 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Removed reference to RCIC turbine tripping on high pressure.

Added RPV rate of decrease for distractor D plausibility.

[NRC - OK.]

State the actual Group 1 isolation trip setpoint in the explanation. -129 Why is it plausible that an inboard MSIV would be open but an outboard would be closed, and vice versa, given the design of the actuation circuitry (Dist B and C)?

Test on MSIVs Both closed or both open, then additionally test on MSL drains for the given conditions in stem since all channels are < Level 2 -50.

CGS:

Group 1 isolation setpoint included in the explanation.

Revised distractor explanations to discuss that Group 1 isolation logic is different than the other isolation groups. Group 1 is one out of two, taken twice. It has two divisions of trip circuitry with two inputs each. One trip signal in both divisions will F U cause all MSIVs to isolate. Other groups are two out of two, taken once. Two trip 42 2 X N signals in a division will operate the isolations in that division only. Therefore, it is H S plausible that a candidate will select an incorrect answer based on a misunderstanding of the isolation logic used. Examples of other group division dependent isolation valves is given.

Recommend revising LOK to H based on the information that the candidate needs to have to correctly answer this question:

• Understanding of the logic used in group 1

• Understanding of how a failed instrument is processed.

• Understand that group 1 valves do not operate by division as other isolation groups Changed 10CFR55 part to 41.9

[NRC OK]

43 F 3 B S Why does the explanation for A say See A?

Modify stem to add a condition that RFW-P-1B is being returned to service following a E pump trip, to make Distractor D more plausible.

44 H 2 X B S

CGS:

Removed See A from distractor A explanation. _Rev 1 Page 18 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 19 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Modified stem to say RFW-P-1B is being placed in service as the second Reactor Feed Pump per SOP-RFT-START, following a pump trip. Feedwater Level Control is aligned as follows:

[NRC OK]

Change first part of Dist C to Maintain SGT System A in standby, place SGT System B in standby, for consistency with other choices.

CGS:

Modified distractor C as recommended.

45 F 3 N S Recommend changing LOK to H based on the knowledge that the candidate must possess to correctly answer the questions. Specifically, the candidate must know that an SGT train with high charcoal temperatures should be placed in recirculation, vice standby, in an attempt to cool the charcoal bed.

[NRC - OK, still memory/fundamental knowledge to recall a procedure step given an alarm]

Subtle difference between lit and extinguished green light. Able to enhance?

Dist D explanation doesnt make sense with question - candidate must understand that SM-7 is powered by the startup transformer through CB-7/1. TR-S is tripped per stem. What about role of TR-N1?

46 H 2 N S CGS:

Enhanced green on lights.

Revised distractor D explanation.

[NRC - OK.]

Answer A says that MC-7F is a make-before-break transfer, but the explanation states that MC-7F goes through the static switch which is a break-before-make connection. Which is it?

CGS:

47 F 3 N S Modified answer A explanation to correctly classify this transfer as make-before-break.

Modified distractors B, C, D explanations to enhance reasons for plausibility.

[NRC - OK]

Not plausible that a battery charger or battery would not be required to be operable.

U DIST A/B.

48 F 2 X N E CGS:

Replaced question. _Rev 1 Page 19 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 20 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only The original K/A for this question (263000.2.2.44) was initially rejected and replaced with the current K/A (263000.2.2.22). After this question was determined to be unsat, it was impossible to find a new question to K/A 2.2.22.

A new question has been developed for the original K/A. CGS is requesting that the K/A for this question revert back to the original (263000.2.2.44).

[NRC - provide updated ES-401-1 outline.

Modify stem, what action, if any, should be taken?]

CGS:

Revised stem as recommended

[NRC - per above comment, stem should be, Based on these indications, what action, if any, should be taken? The to restore all indications to normal cues that the indications are not normal, and that D is incorrect.]

CGS:

Modified question to read Based on these indications, what action, if any, should be taken?

[NRC OK]

Question should ask "What actions should the crew take AND WHY?" This would more correctly setup the answer field.

Dist C - If increased battery electrolyte temperature negatively affects battery performance then this answer is correct also.

CGS:

E Modified stem as recommended.

49 H 2 X N S Revised distractor C to read install temporary ventilation in Battery Room No. 1 in accordance with ABN-HVAC, HVAC Trouble, to prevent battery electrolyte temperature from exceeding tech spec limits.

Revised distractor C explanation to match revised distractor.

Added TS bases as a reference.

[NRC OK]

Dist C - are there any DC-powered pumps on site?

CGS:

E Examples of DC pumps:

50 F 2 X B S DC powered pumps for DG-1 (DO-P-3A1,3A2, and DLO-P-2A1,2A2)

DC powered pumps for DG-2 (DO-P-3B1,3B2, and DLO-P-2B1,2B2)

TO-P-EOP (Main Turbine Emergency Oil Pump) _Rev 1 Page 20 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 21 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only SO-P-ASBU (Air Side Seal Oil Backup Pump)

RFT-P-EOP/1A (RFP 1A Turbine Emergency Oil Pump)

RFT-P-EOP/1B (RFP 1B Turbine Emergency Oil Pump)

[NRC OK]

CGS:

51 F 2 N S Revised LOD to match Form ES-401-9

[NRC OK]

CGS:

52 H 3 N S No comments 1st part answer is given in stem of Question 7.

2nd part is LOD = 1.

F 1 B U CGS:

53 X H 3 N S Replaced Question Revised LOK to H

[NRC OK]

CGS:

Revised LOD to match Form ES-401-9 54 H 3 B S Changed 10CFR55 part to 41.6

[NRC OK]

Rephrase stem to Which of the following conditions satisfies the Maximum Subcritical Banked Withdrawal Position, to ensure that there is sufficient SDM to keep the reactor shutdown under all conditions?

Delete the numbered list of conditions and just write out the combination in the answer choices - eliminates having to keep looking up and down over and over.

Modify the choices to:

55 F 3 B S A. two control rods at 08, all other control rods at 00.

B. one control rod at 08, one control rod at 04, all other control rods at 02.

C. one control rod at 48, one control rod at 04, all other control rods at 00.

D. one control rod at 48, two control rods at 02, all other control rods at 00. [correct]

CGS:

Modified question as recommended. _Rev 1 Page 21 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 22 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Revised LOD to match Form ES-401-9.

Changed 10CFR55 part to 41.6

[NRC OK]

Rephrase plant conditions 1 minute after the RFPT trip include:

CGS:

56 H 3 N S Modified stem as recommended.

[NRC OK]

Distractor B (189F) is a subset of A (Mode 4 = <= 200F). What is the basis for 189F?

CGS:

Modified distractor B to: After RHR is placed in shutdown cooling.

Modified distractor B explanation for new distractor.

E Changed 10CFR55 part to 41.10.

57 F 3 X B [NRC - Answer D, the provided references state that RWCU-V-31 shall not be S

open with reactor pressure GT 125 psig . Therefore, answer D should read When RPV pressure is LE 125 psig, not LT 125 psig.]

CGS:

Modified answer D as recommended.

[NRC OK.]

-Since it can be sometimes subtle to tell if a light is lit or not on a paper printout, It would be a good idea to provide applicants with a reference handout showing what various colored lights look like when they are lit or extinguished, such as blue, amber, 58 H 3 N S white, green, and red lights, etc.

CGS:

Developed a reference handout for light states in questions. [NRC - OK.]

Swap APRM 2 and 3 values so that APRM 2 is bypassed (B RBMs primary input for STP), so B RBM will automatically select its alternate, APRM 4. Answer should still be the same but it makes Distractors A and C more plausible if the applicant E doesnt know that if the primary APRM is in BYPASS, the RBM automatically selects 59 H 3 X N S the alternate.

CGS:

Modified question as recommended.

Provide a drywell temp in stem.

60 H 3 N S Specify which isolation valve is inboard and outboard. _Rev 1 Page 22 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 23 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only CGS:

Revised stem to include drywell temperature of 285°F.

Designated inboard and outboard isolation valves.

Changed 10CFR55 part 41.10.

K/A mismatch. Plant system is 239001 Main and Reheat Steam, but question is testing on system 239002 Safety Relief Valves which is already being tested on in Q43.

U Explanation for K/A match incomplete.

61 F 2 X N S CGS:

Replaced question.

[NRC - OK.]

Rephrase stem, If turbine thrust bearing wear rises to 15 mils, the turbine trip oil  :

K/A match statement is boilerplate.

Explanation for D does not make sense for the as-written distractor. Says that high wear on thrust bearing will not cause a turbine trip.

CGS:

Inserted K/A match statement.

Distractor D. Added the following to the explanation: There are two thrust bearings, the turbine thrust bearing and the pump thrust bearing. Both bearings will cause a Turbine A Vibration High alarm. However, only the turbine thrust bearing will cause F a turbine trip.

62 3 B S H Recommend revising LOK to H based on knowledge that the candidate has to possess to correctly answer this question:

• Which bearing will cause a turbine trip? There are two thrust bearings, the turbine thrust bearing and the pump thrust bearing. Both bearings will cause a Turbine A Vibration High alarm. However, only the turbine thrust bearing will cause a turbine trip

• The thrust bearing wear setpoint for the trip

• How the system responds to trip the feedwater turbine.

Changed 10CFR55 part to 41.7

[NRC OK - good distractor.]

CGS:

63 F 3 N S No comments _Rev 1 Page 23 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 24 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Dist A - what conditions cause control room ventilation fans to trip automatically? If none, not a plausible distractor. Revise to include an actuation for CR intake plenum high high rad alarm that is incorrect.

CGS:

There are no automatic trip signals for control room ventilation fans. Revised distractor A to read RWCU system isolates.

Changed 10CFR55 part to 41.9 Changed LOK to be consistent with Form ES-401-9

[NRC - original distractor A, control room ventilation fans trip, was more plausible in this question than a RWCU isolation due to high exhaust plenum radiation. Id rather replace with original distractor A, or change to control room ventilation realigns to E

64 H 2 X B recirculation mode, if this is in fact incorrect.]

S

[NRC - Update Ops Study Guide to remove discussion of CR intake plenum hi hi rad alarm, if this alarm has in fact been disabled as discussed during written exam review. The system training manual also has no discussion of control room recirculation mode as described in the ops study guide.]

CGS:

Replaced distractor A with original distractor as recommended.

Recommendations for changing the Ops Study Guide have been recorded. The Ops Study Guide is not a controlled document. It will not be updated prior to the 2017 NRC exam.

[NRC OK.]

CGS:

65 F 2 B S Changed 10CFRpart 55 to 41.4 This is just a series of true/false statements, which is explicitly disallowed by NUREG 1021. From Appendix B:

"Collections of true/false statements typically only test simple rote memory; F 1 U the examinee simply needs to recall a definition or condition. The questions 66 X X B elicit no comprehension or problem-solving; hence, they lack operational H 3 S validity. This type of question allows an examinee to answer the question without referring to the stem of the question and should be avoided on NRC examinations. _Rev 1 Page 24 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 25 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Also, 5 is deceptive, and would not hold up on appeal; at a minimum operators DO have to have an active RO license, and an SRO license allows the operator to perform all functions as an RO license.

Last NRC Exam field is blank.

CGS:

Replaced Question Revised 10CFR55 part to 41.10.

Revised LOK to H

[NRC - Modify stem, Per PPM 1.3.1, what are the ]

CGS:

Revised stem as recommended [NRC OK]

All 3 distractors are trying to stretch out the procedural direction for non-conditional steps to Obtain supervisor initials to show approval - its stretched too much.

Modify question to:

A. Document the reason next to the step, and obtain supervisor initials for approval.

B. Document the reason next to the step; supervisor initials are not required U for approval.

67 F 2 X N C. Obtain supervisor initials next to the step for approval; a documented S reason is not required.

D. No additional documentation or supervisor approval is required.

CGS:

Revised question answers as recommended.

Revised explanations to match new answer/distractors.

[NRC OK]

What is the plausibility of using no change in feed flow as a distractor? Use an increase in temperature instead for Distractors A and B.

CGS:

Modified initial conditions to Mode 2 for operational validity.

68 F 2 X N E Modified distractor A and B to use no change in reactor coolant temperature vice no change in feed flow.

Modified explanations to match new distractors.

[NRC OK] _Rev 1 Page 25 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 26 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only No correct answer. Reference procedure requires verbal PCN be authorized by CRS OR Shift Manager (CRS/Shift Manager is an or statement I believe), but answer D states authorization must come from CRS and Shift Manager.

Dist A - spell out EPN. Specify peer check is from CRO1.

E CGS:

69 F 2 X B S Revised distractor A as recommended. Additionally, removed initiate condition report and added no supervisory review is required, to make this distractor consistent with the other distractors.

Revised answer D to CRS or SM vice CRS and SM

[NRC OK]

MODE switch is in shutdown.

Place temperature at 200F exactly for increased plausibility. Distractors A and B are almost implausible / LOD=1 under given conditions.

E 70 F 2 X M CGS:

S Modified stem as recommended.

Changed 10CFR55 part to 41.10

[NRC OK]

71 F 2 N S 72 F 3 N S NRC 1999 Modify stem to At what classification are EN. NWs administrative exposure hold points first waived, and ..

Modify choices to:

A. Alert; 10 REM TEDE B. SAE; 10 REM TEDE E C. Alert; 25 REM TEDE 73 F 2 X B D. SAE; 25 REM TEDE S

CGS:

Modified stem as recommended.

Modified answer/distractors as recommended.

Modified explanations to match new answer/distractors.

Changed 10CFR55 part to 41.12.

[NRC OK] _Rev 1 Page 26 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 27 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Last NRC Exam field blank.

Provide the 10 CFR reference that it best matches.

CGS:

Revised Last NRC Exam field to N/A.

74 H 3 B S Changed 10CFR55 part to 41.10 Recommend changing LOK to H due to the knowledge that the candidate needs to possess to correctly answer the question.

[NRC OK. Was supposed to be H.]

CGS:

75 H 3 N S Revised LOK to H to match Form ES-401-9 Distractor D explanation - 2 howevers.

76 H 3 N S CGS:

Removed the second however. [NRC OK.]

Reference Provided Modify stem to which is the earliest required action that will preclude a shutdown?

Why is 3.8.7 a necessary handout?

Too much information being given out. Do not include SRs with 3.8.1.

E 77 H 2 X N CGS:

S Revised stem as requested.

Removed LCO 3.8.7 as a reference.

Removed LCO 3.8.1 SRs from the reference

[NRC OK.]

This question still only requires systems knowledge to answer both parts, which makes it RO knowledge. Applicant answers part 2) by knowing that RCIC physically trips on a loss of Div 1 DC power and is therefore not available. SRO knowledge would include the applicant choosing the event-specific sub-procedure or emx contingency procedure, or if he has to know how to prioritize among available U

78 H 3 X N alternate pressure control systems using station guidlines.

S Explanation for Distractor D doesnt make sense for the as-written distractor.

CGS:

Replaced question.

[NRC OK] _Rev 1 Page 27 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 28 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Reference Provided Q79 references attached to the worksheet include both pages 4 and 5 of 63 flowcharts, but the SRO references file includes only page 4 of 63 flowchart - what are you proposing to handout?

When stem says direct the equipment operators to does that mean the applicant is to choose an action that applies to the EOSS OR DGSS, or is it specific to the EOSS only?

Distractors A and D are direct lookup and therefore implausible because the provided reference directly states that they are to be performed by CROs, not EOs. Choices B U

79 F 3 X N and C are OK because they require the applicant to have knowledge of whether they S are to be performed in all cases or only in case of a fire, which is knowledge of the content of procedure which is not provided.

CGS:

Both pages 4 and 5 will be provided to the candidate.

Revised distractor A&D to remove direct references to SM-8 actions that could cue the candidate that these actions are performed by control room operators. Included discussion in explanation that distractor A&D actions are only performed if the control room is evacuated due to a fire, which is not the case per the conditions in the stem.

[NRC OK.]

Reference Provided N2 pressure 1600 psig down slow is too low a given pressure to be discriminatory.

E Give a steady pressure > 2000 psig.

80 H 2 X N S CGS:

Revised N2 pressure to 2050 psig and steady.

[NRC OK.]

Can unit be in mode 3 with RPV pressure < 30 psig for increased plausibility of Distractor D, or is that not a reasonable physical condition?

Procedure just says the previously-operating loop is the preferred restart loop, not the required one, which could lead to multiple correct answers on appeal. Modify stem to, what is the procedurally-preferred method to restore SDC?

E 81 H 3 X X N Lets evaluate if Dist B might be better as RHR Loop A using 5.1 RHR Loop A S

Shutdown Cooling Initiation, for increased plausibility since it wasnt in operation before.

No 10CFR reference provided. If its generic you pick the reference that best fits per the question you wrote - 43.5 in this case. Really same for all questions - you are not bound to put down what is listed in the KA catalog for reference. _Rev 1 Page 28 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 29 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only CGS:

Modified the stem initial conditions to Mode 3.

Revised stem question as requested.

Revised distractor B to Place RHR Loop A in SDC in accordance with SOP-RHR-SDC, section 5.61, RHR Loop A Shutdown Cooling Initiation.

Revised 10CFR55 part to 43.5

[NRC OK.]

Question as-written is not SRO-only knowledge. From NUREG-1021:

SRO-only knowledge should not be claimed for questions that can be answered solely using fundamental knowledge of:

• the basic purpose, the overall sequence of events that will occur, or the overall mitigative strategy of a procedure.

• any AOP entry condition.

• plant parameters that require direct entry to major EOPs; e.g., major Westinghouse EOPs are E0, E1, E2, E3, ECA-0.0, and Red/Orange Functional Restoration Procedures and major General Electric EOPs are Reactor Vessel Control, Primary Containment Control, Secondary Containment Control, and Radioactive Release Control.

• immediate operator actions of a procedure.

U 82 H 3 X N The Westinghouse E0 procedure is Reactor Trip, so similarly a GE SRO question S should not be able to be answered solely by knowledge of when to SCRAM the reactor. See examples on pages 30 and 31 of 50 in NUREG 1021 rev 10 ES-401.

Add something additional to the question that is SRO knowledge.

CGS:

As discussed in the written exam review during NRC validation week, this question is not asking for an immediate action or direct entry into an EOP. The candidate must identify that all actions have been taken per the ABN to reduce system fluctuations, including turning off the PSS, taking the voltage regulator to test, and ensuring that the system operator is not causing the disturbance. After these have been verified, and the generator fluctuations reach the limit, the candidate must decide that a reactor scram/turbine trip is required. This is SRO knowledge.

Revised the SRO Only section to add amplifying information.

Revised explanations for clarity.

[NRC OK.] _Rev 1 Page 29 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 30 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Modify stem to: This limit is established to specifically ensure that RCS pressure will not exceed [110% vs 125%] of reactor pressure vessel design pressure [in the steam dome vs. at the lowest elevation in the RCS].

83 F 4 X N S CGS:

Revised stem and answers as recommended.

[NRC OK.]

Is this question operationally valid? Specifically, PPM 5.0.10 page 274 of 320 makes it clear that primary containment venting is a last resort to be used only if primary containment pressure cannot be controlled with containment spray. Does this situation satisfy that requirement with the information given?

No 10 CFR reference provided. 43.4.

E CGS:

84 H 3 X N S Modified stem for drywell pressure at 1.75 psig, up slow, and drywell temperature at 200F, up slow. These parameters put the primary containment outside the Drywell Spray Initiation Limit, (DSIL). This enhances the plausibility of venting primary containment since, with these conditions, drywell spray is not allowed.

Revised 10CFR55 part to 43.4

[NRC OK.]

The first two sentences in the stem dont appear necessary. Or is it specifically to denote that only a single ARM is alarming, and not 2 or more?

The stem focus is not clear to me - is the question intending to test the conceptual basis for MSOV setpoint selection (the correct answer falls under this), or testing the ability to respond to exceeding MSOV given the specific operational conditions of the stem (a single RB ARM alarming, but not specifically above MSOV)? The explanations for the Distractors imply the latter. All 4 choices need to fall into the same category - make consistent.

U Distractors B and D appear partially correct if the question is not specifically testing on 85 F 3 X X N E the operational condition of a single ARM above MSOV (which the stem doesnt S clarify).

Distractor A explanation - how is it certain that the SAE EAL call will be made prior to emergency depressurization? Crew has 15 minutes to assess EAL after exceeding threshold - could they not ED before classifying as SAE?

CGS:

Revised the question stem to clarify the focus of the question, which is the reason for MSOV selection.

Revised distractor B clarify its intent. _Rev 1 Page 30 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 31 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Revised explanations to clarify reasons for plausibility and reasons why distractors are not correct.

[NRC - Would it be appropriate to include Table 24 in the question? MSOV values redacted if necessary?

Distractor B - high enough to confirm that an emergency depressurization is required to prevent core damage.

Distractor D - high enough to confirm primary system leakage into secondary containment, requiring a normal reactor shutdown.]

CGS:

PPM 5.3.1, Table 24 added to question (MSOV values redacted)

Distractors B and D revised as recommended. [NRC OK]

No CFR reference provided. 43.7 86 H 4 N S CGS:

Revised 10CFR 55 part to 43.7 [NRC OK.]

Provide RPV level trend.

How should adequate core cooling be achieved?

87 H 3 N S CGS:

Added RPV level trend: down slow.

Revised stem as recommended. [NRC OK.]

Reference Provided Cannot provide all of TS 3.3.1.1 - far too much information available in table 3.3.1.1-1 and the SRs. Id be inclined to allow the LCO Action Statements and the 1st 2 pages of table 3.3.1.1-1 that apply to IRMs and APRMs only, with the allowable value column redacted. Providing SR 3.3.1.1.6 makes the question direct lookup - what resident knowledge does the applicant need to have that cant be looked up in the given reference?

U 88 F 2 X X N Cueing: SR 3.3.1.1.12 and table 3.3.1.1-1 functions 8 and 9, Turbine Throttle Valve -

S Closure > 30% RTP cues the answer to RO question #5, and cues an incorrect distractor in RO Q21, at a minimum. There are probably other questions aided by the information in the table. This is something that needs to be verified for every reference that is provided on the exam.

Modify stem:

What is the earliest permitted action ? _Rev 1 Page 31 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 32 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Assume the action is taken just before the completion time. (this precludes the situation where someone argues within 14 hours1.62037e-4 days <br />0.00389 hours <br />2.314815e-5 weeks <br />5.327e-6 months <br /> could be any time between 0 and 14 hours1.62037e-4 days <br />0.00389 hours <br />2.314815e-5 weeks <br />5.327e-6 months <br /> and could therefore be the earliest)

No CFR reference provided.

CGS:

Modified stem to read Using the provided reference, what is the earliest permitted action that will satisfy all technical specifications requirements (Assume the action is taken at the time listed below)?

Modified all distractors to read at XX hours vice within XX hours.

Modified reference to provided TS 3.3.1.1 LCO action statements, and table 3.3.1.1-1 Intermediate Range information only. Removed Allowable Value column information from table 3.3.1.1-1. Removed all SR information.

Validated no cueing provided by revised reference.

Candidate must know that placing the reactor mode switch in RUN takes the reactor out of the mode of applicability for the conditions given in the stem, and that this action is allowed.

Revised 10CFR55 part to 43.2 Reference header and footer revised to include question number.

[NRC OK.]

Is RPV level +25 inches reasonable for this LOCA condition? Can it be made lower for better plausibility of B and D?

E Modify A to from low pressure injection sources or from LPCI and LPCS.

89 F 3 X M S CGS:

Revised RPV level to -127 inches, down slow.

Modified distractor A as recommended. [NRC OK.]

RO knowledge. Pump trips, standby doesnt start, start standby. Systems knowledge that RCC-V-6 closes on LT two RCC pump breakers closed.

No CFR reference provided. You need to pick the CFR reference that closest fits for U every question. 43.6 90 H 2 X X N E CGS:

S Replaced question Revised 10CFR55 part to 43.6

[NRC - Two correct answers, B and D. ABN-RCC 4.2.8 doesnt say to vent primary containment if Drywell Pressure rises outside of normal band, it just says to vent it if _Rev 1 Page 32 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 33 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Drywell pressure is rising due to inadequate RCC cooling, which it is. However, swapping CRD pumps is step 4.2.6 and venting Drywell is step 4.2.8; if the station procedure usage guide (I cant locate) directs that for an ABN the lower-numbered step is a higher priority and should be performed first, then the question is acceptable.

Explicitly state that in the explanation if so, or provide other procedure usage guidance.]

CGS:

ABN numbered steps are performed in order. The following references are attached to the question:

• OI-09, Operations Standards and Expectation, section 7.1

• PPM 1.3.1, Operating Policies, Programs and Practices, step 4.8.4.c

• SWP-PRO-01, Procedure and Work Instruction Use and Adherence, step 4.10.3 Revised drywell pressure to .26 psig; lower in band, but still up slow.

[NRC OK]

Dist C says maintain RHR Loop B in SPC, but stem says RHR Loop B is in SDC. Which is it supposed to be?

No CFR reference provided. You need to pick the CFR reference that closest fits for E every question.

91 H 3 X N CGS:

S Modified distractor C to read maintain RHR Loop B in SDC.

Revised 10CFR55 part to 43.5

[NRC OK.]

No CFR reference provided. You need to pick the CFR reference that closest fits for every question. It cannot be N/A.

92 H 2 M S CGS:

Revised 10CFR55 part to 43.7

[NRC OK.]

Reference Provided Explanation for A incomplete.

No CFR reference provided.

93 H 2 N S CGS:

Revised distractor A explanation to remove extraneous information ( and).

Revised 10CFR55 part to 43.5. _Rev 1 Page 33 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 34 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Revised reference header and footer. Added question number to header.

[NRC OK.]

10CFR Formats of answers are inconsistent with each other some are a condition only, some are approval authority only, some are both. Modify to:

When may control room operators perform actions that deviate from technical specifications or license conditions per 10CFR50.54.X?

If no adequate means of protecting public health and safety in an emergency is apparent consistent with tech specs A. , OR if failing to deviate from tech specs will result in unnecessary emergency equipment damage, with Ops Manager and NRC approval at a minimum.

B. , OR if failing to deviate from tech specs will result in unnecessary emergency equipment damage, with CRS and SM approval at a minimum.

C. ONLY, with Ops Manager and NRC approval at a minimum D. ONLY, with CRS and SM approval at a minimum.

I do like the license amendment distractor so that may work if you find a way to work it E in there. Could be license amendment vs. relief request vs. enforcement disgression 94 F 2 X B S vs. verbal approval. And then approval authority.

No CFR reference provided. Needs to be from 10CFR43. 43.1 Modified question stem and distractors as recommended. Unable to use license amendment in the distractors.

Modified distractor explanations to match new distractors.

Revised 10cFR55 part to 43.1.

[NRC - For exactness, stem should actually read 10CFR50.54.(x).

Does CRS/SM mean CRS AND SM or CRS OR SM?]

CGS: Modified the stem as recommended.

CRS/SM normally means either CRS or SM. Revised distractor D to read ONLY, with CRS or SM approval at a minimum.

[NRC - Change the CRS/SM in Dist. B to CRS or SM as well, since it was changed in D.]

CGS:

Revised distractor B as recommended. _Rev 1 Page 34 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 35 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only

[NRC OK]

Answer C, and after depressurization.

CGS:

95 H 3 N S Revised distractor C as recommended.

[NRC OK.]

Dist B - Would it be prohibited for the CRS to do this prior to installation? Could be partially correct, acceptable practice. Almost minutia.

Dist C - how do you verify independent verification is performed prior to installing jumper?

Distractors A & B are not explicitly the CRS responsibility per PPM 1.3.9 but if the CRS verified it, it wouldnt be wrong as part of his supervisory responsibilities.

U Multiple correct answers.

1 96 F X N E 2 Modify to test on whether simultaneous vs independent verification is required, plus S

another SRO responsibility.

CGS:

Modified question to test on simultaneous vs. independent verification and when plant drawings are updated (before vs. after TM installation).

Modified distractor explanations to match new distractors.

[NRC OK.]

D- why cant the VP-Engineering direct activation of OCC? Problem with a distractor that is higher than the approval authority is that it is usually within their roles and responsibilities to ensure such actions are directed by their subordinates. Can be got around by asking What is the lowest level of authority that can direct [an action]?

C- can the OCC be considered a subset of the ERO?

D- 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> or less.

97 F 3 X N E No CFR reference provided.

CGS:

Modified answer A to read when the fault causes entry into a LCO action statement of 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> or less.

Distractor C - Revised explanation to discuss that the OCC is NOT a subset of the ERO. When staffed at an EAL of Alert or higher, the ERO performs the actions of the OCC, but the OCC is not considered active and the OCC process of SWP-MAI-03 is not followed. _Rev 1 Page 35 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 36 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Distractor D - Revised explanation to clarify that the plant general manager has the responsibility to staff the OCC on a recommendation from the shift manager, work week manager, or duty manager.

[NRC OK.]

D- isnt this a correct secondary effect of ED, given injection of high capacity LP systems?

B- couldnt it be argued that increasing coolant injection to a damaged vessel limits the amount of fuel damage, which limits the amount of activity released?

U 98 F 2 X B Could modify to test on whether GE must be exceeded prior to ED.

S CGS:

Revised explanation to clarify the reason for emergency depressurization is required for the conditions given in the stem.

[NRC OK. Also revised stem to ask for specific reason per PPM 5.0.10]

2X2 not balanced, only need to know 2nd part since all 4 answers are different.

No CFR reference provided.

CGS:

E 99 H 2 X M Revised question to a balanced 2X2.

S Revised explanations to match new distractors.

Revised 10CFR55 part to 43.5

[NRC OK.]

No CFR reference provided.

CGS:

100 F 3 N S Revised 10CFR55 part to 43.3

[NRC OK.]

General Comments:

• Less than 50% of RO exam questions are Higher cognitive level by my assessment. In your edits, make at least 2 more RO questions HCL to put it at >50%.

• More formatting / incomplete answer /inaccurate explanation issues than I would have expected.

• I do like the use of control board indication pictures in the stem when available.

• Handouts that are intended for a specific question should be labeled as such. _Rev 1 Page 36 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 37 Form ES-401-9

• Every question needs to have a 10CFR55.41/.43 reference associated with it. The reference cited in the K/A catalog is just a suggestion - it is the question that you write which determines which 10CFR reference applies.

• Only half of the 14 available 10CFR55.41 references are cited in your submittal. Every 55.41 reference is not required to be represented on the exam, but it should be more than half. I would imagine that there are several existing questions which actually could fall into one of the unused 55.41 categories; look for them and re-cite.

• When a reference is provided, it has to be screened for possible cueing of other exam questions. _Rev 1 Page 37 of 38 OBDI 202 - IOLE Process

ES-401 CG-2017-02 38 Form ES-401-9 Results Table RO LOK -H 41 Avg RO LOD 2.48 Flaws 10 CFR Distribution RO LOK-F 34 AVG SRO LOD 2.64 Stem focus 17 41.1 22 43.1 1 SRO LOK - H 15 Overall LOD 2.52 Cues 5 41.2 2 43.2 3 SRO LOK - F 10 T/F 1 41.3 0 43.3 2

%  % Cred Dist 22 41.4 1 43.4 2 RO Bank 28 37.3 SRO Bank 2 8 Partial 11 41.5 11 43.5 14 RO Mod 5 6.67 SRO Mod 3 12 job link 0 41.6 4 43.6 1 RO New 42 56 SRO New 20 80 units 0 41.7 27 43.7 2

% minutia 0 41.8 2 Total Bank 30 30 backward 0 41.9 3 Total Mod 8 8 KA 3 41.10 22 Total New 62 62 SRO-only 2 41.11 0

%  % LOD = 1 2 41.12 2 RO Sat 38 50.7 SRO Sat 9 36 41.13 1 RO Unsat 9 12 SRO Unsat 5 20 41.14 0 RO Edit 28 37.3 SRO Edit 11 44 Answer Dist (%)

%  % RO-A 25 SRO-A 6 Total Sat 47 47 Total Unsat 14 14 RO-B 16 SRO-B 9 Total Edit 39 39 RO-C 15 SRO-C 5 RO-D 19 SRO-D 5 RO References Provided: 1 SRO References Provided: 5 _Rev 1 Page 38 of 38 OBDI 202 - IOLE Process