ML15009A304

2014-12 ES-401-9 Draft-Written Exam Comments (Resolved)

ML15009A304
Person / Time
Site:	Callaway
Issue date:	12/17/2014
From:	Vincent Gaddy Operations Branch IV
To:	Union Electric Co
laura hurley
References
50-483/14-012
Download: ML15009A304 (16)

Text

ES-401 CW-2014-12 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only D, CR, 41.7

- Recommend modifying stem to read What is the expected response of the Feed 1 F 3 N S Pump DELTA P setpoint after feed flow is shifted to the bypass valves? It eliminates the ability for an applicant to argue that he/she didnt know where they were in the process of shifting over. If it is in the process of shifting over, it could be argued that the 71 psid set point may still be in effect. - Made Recommended Changes. [SAT]

C, CR, 41.7

- The question should refer to Tech Spec Action Statement entries, not Tech Spec entries. By virtue of meeting the Applicability statement, the plant is applying 2 H 4 X M S the LCOs to their operation. Recommend similar revision on distracter A wording.

- Recommend saying RCS plant cooldown in question stem. - Made Recommended Changes. [SAT]

3 H 2 M S B, CR, 41.4 [SAT]

4 F 2 N S D, CR, 41.7 [SAT]

Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

$ The answer choices are a collection of unrelated true/false statements.

$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

$ The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met). Page 1 of 16 OBDI 202 - IOLE Process

ES-401 CW-2014-12 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only A, CR, 41.7

- Recommend adding a reference that describes how power is aligned in this shutdown condition, if one exists. The drawing shows all the connections, but doesnt spell out alignment during shutdown. - Unable to locate specific reference showing lineup during shutdown condition. Lineup can be derived from various procedures used during startup and shutdown (ie. OTG-ZZ-00002, OTG-ZZ-0005) It is 5 H 3 B S understood that when the Main Generator is NOT online then the Startup Transformer supplies the loads that would be supplied by the Unit Aux transformer which is supplied by the Main Generator when at power.

In the Explanation section, modify to say The variable heater group C is powered from PG24 via PA02.- Modified explanation as recommended. [SAT]

A, CR, 41.7

- In reviewing the Entry Conditions for procedures OTO-NK-00001 and OTO-NK-00002, having the NK01 Trouble alarm, at a minimum, satisfies the conditions to enter both procedures. Therefore, distracter B is also a correct answer. - Modified question by removing procedures as answer/distractors and replaced with component U

6 H 3 X X B which has failed.

S The Explanations imply that NN 11 INV TRBL/XFR alarm is in as well as NK01 Trouble. The stem conditions only show the NK01 Trouble alarm. Should both alarms be in? Any effects on the answers? - Revised Explanation Provide copies of the alarm response procedures for the alarms given in the conditions. - Procedures provided [SAT]

B, CR, 41.7

- The question stem refers to the starting air banks being cross connected, without specifying the inlet or outlet side of the banks. The inlets of the banks are cross-connected with check valves to protect them, the outlets are not NORMALLY cross-connected - but it appears there is a locked-closed cross-connect plug valve, V760A.

If the applicant recognizes that only one bank is necessary for EDG start, he/she E

7 H 3 X X B could argue that distracter A is also correct because the banks are cross-connected S on the inlet side with protective check valves. Both answers A and B would be accepted as correct answers.

Recommend specifying the discharge side of the banks in the question stem, as well as adding the word normally before cross-connected. - Modified Stem as Recommended. [SAT]

8 H 3 M S B, CR, 41.7 Page 2 of 16 OBDI 202 - IOLE Process

ES-401 CW-2014-12 3 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only D, CR, 41.7 9 H 4 N S - Ensure images are clearly readable on printed exam. If necessary enlarge and place on a separate sheet. Same for all questions with images. - Verified all images provided on this exam are clear. [SAT]

A, CR, 41.8

- Per OTO-NK-00001, Attachment B, Step B7, if NK02 voltage is not greater than 105 VDC, then the staff will take action to provide alternate supply. Does this need to E be accounted for in the stem, i.e., with a rider such as without additional operator 10 H 3 X M S action? - Made recommended changes to stem.

Provide the reference drawings listed. It would be expected that they have information to validate the amp hour ratings of the various batteries. If not, that needs to be provided as a reference as well. - Provided additional references. [SAT]

A, CR, 45.7 41.7

- Provide a reference that explains the interaction between the EDG load sequencer and MDAFAS signal logic. - Provided drawing 7250D64 Sheet 15 & E-22NF01 Since the starting sequence prevents frequency from dropping < 95% of 60 Hz, it also technically prevents it from dropping < 90% of 60 Hz; this makes B technically correct as well. Consider changing the second part of the answers to switch E between:

11 F 3 X N S So that at no time during the loading sequence does the voltage decrease to < 95 percent of 4.16 Kv (wrong) and So that at no time during the loading sequence does the frequency decrease to <

95% of 60 Hz (correct).

Modified question to ask for 75% or 95% of Voltage.

(The question needs to be associated with an element of 10 CFR 55.41. 10 CFR 55.45 is for operating test elements) Changed to CFR 41.7 [SAT]

C, CR, 43.5 41.7 Reference Attachments P and R of OTO-NN-00001 for this question.

12 H 3 N S

-RO questions need to be associated with elements of 10 CFR 55.41, not 55.43. -

Changed to CFR 41.7 A, CR, 41.7 M E - Justify why this is called a Modified question, or relabel to Bank. The question 13 F 3 X stem has not changed, and the wording of the answers is more verbose in the Bank B S question, but it is asking the same question. This appears to still be a Bank question Changed to Bank Question. [SAT] Page 3 of 16 OBDI 202 - IOLE Process

ES-401 CW-2014-12 4 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only A, CR, 43.5 41.4 14 H 4 N S -RO questions need to be associated with elements of 10 CFR 55.41, not 55.43.

Changed to CFR 41.4 [SAT]

15 F 3 B S A, CR, 41.8 B, CR, 41.7 16 F 3 B S - Provide the basis document for ECA-1.1 to justify the answers. - Provided additional reference. [SAT]

17 H 4 B S B, CR, 41.10 D, CR, 41.7

- Provide a reference that explains the operation of the manual load tap changers as a backup to the automatic load tap changers.

18 F 4 N S -As written C appears like it could be argued to be correct as well, since manual load tap changers do provide protection when in service. Are manual load tap changers taken out of service when automatic load tap changers are in service? Explain alignment. Only Auto OR Manual load tap changers are in service at one time. -

Discussed with Chief Examiner. Provided additional Reference. [SAT]

19 F 3 B S C, CR, 41.7 (2013 NRC Exam) 20 H 2 B S D, CR, 41.10 D, CR, 43.5 41.2

- Recommend adding a reference to describe how the pulse height discriminator works. - Provided Lesson Plan which discusses the basic operation of the pulse 21 H 3 B S height discriminator.

RO questions need to be associated with elements of 10 CFR 55.41, not 55.43.

Changed to CFR 41.2 [SAT]

B, CR, 41.7 22 H 2 N S - Add reference OTO-BB-00003 for the distracters. - Added Reference as recommended. [SAT]

23 F 3 N S A, CR, 41.10 24 F 3 N S A, CR, 41.10 25 H 2 N S C, CR, 41.7 26 H 2 X M E C, CR, 41.8 Page 4 of 16 OBDI 202 - IOLE Process

ES-401 CW-2014-12 5 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only B S - On distracter A, justify why it is plausible that if an applicant believed the Tavg/Tref mismatch decreased, that there would be a need to place Rod Control in Manual.

[procedurally required regardless of Tavg - Tref response.] -Discussed with chief Examiner the procedural requirement.

See NUREG-1021, ES-401, Section D.2.f, fourth bullet. If you compare this to the original bank question, it doesnt change any conditions in the question stem or alter the answers. The correct answer in the bank question is the same correct answer in the proposed modified question. This is still a Bank question. - Changed to Bank Question. [SAT]

(Procedure comment) OTO-SF-00001 Step 1 can be better worded as such: Check for indication of multiple dropped rods: Both of the following are met: .

B, CR, 41.7

- With PTS, actions need to be taken to reduce RCS pressure and stop a cooldown.

Distractor D also appears to be correct: FR-P.1, Step 2, RNO e.4).b) instructs the operator to ISOLATE all feedwater to faulted SG(s) that are NOT needed for RCS E

27 F 2 X B temperature control: Main Feedwater Reg Valve .

S Two correct answers. Modify distractor D.

Modified Stem of Question to indicate that FWIS has been generated, therefore D distractor is directed by the procedure, however will not have any impact on the PTS situation. [SAT - RX Trip coincident with Low Tavg = 564F causes FWIS]

A, CR, 41.5

- What is the vapor pressure minimum requirement for the CCPs? I expect it is lower than 15 psig, but it would be good to verify what it is, so that an applicant cant argue C is correct as well. If pressure was lowered down to the vapor pressure of the liquid it will cause gases to come out of solution. - Added 5 psig limit to C 28 H 3 N S explanation.

-Verify explanation for A: as VCT pressure is lowered, I believe the D/P between the

1. 1 seal leakoff and the #1 seal return line should increase, causing #1 seal return flow to rise (less backpressure from VCT). - Corrected explanation as recommended.

[SAT]

C, CR, 41.7

- Distractor A states (erroneously) that the flywheel prevents exceeding KW/ft limits, but the explanation refers to a protective function to prevent violating DNBR limit.

29 F 2 B S These are 2 different concerns; DNB limits prevent excessive fuel cladding temperature; limits on KW/ft are there to protect the fuel pellet temperatures within the cladding. Refine explanation. - Changed Explanation to correlate the KW/ft limit to the correct protective feature. [SAT] Page 5 of 16 OBDI 202 - IOLE Process

ES-401 CW-2014-12 6 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 30 F 3 N S C, CR, 41.5 B, CR, 41.5 31 H 3 N S Revise the Explanation for distracter A to say that reducing the flow through the RHR heat exchanger will raise the temperature out of the RHR system, not lower. - Made Changes as recommended. [SAT]

A, CR, 41.5

- In procedure OTN-EP-00001, Step 5.1.2, it says that RCS temperature has to be less than 350F for isolating the accumulators, in addition to being less than 1000 psig. If that is the case, then distracter D is the correct answer. Document BD-E-1, Page 30 of 47, refers to the temperature limit in terms of Thot. Verify if changes are U

32 H 3 X X B needed. - Made changes to stem to ensure RCS temperature and pressures are S below the limit to ensure that the SI accumulators have been isolated.

The isolation steps for the accumulators are provided in more detail in Addendum 6 to this procedure, not provided. Provide the reference. Provided additional reference.

The section describing if a reference will be given to the applicant is blank. Specify if open or closed reference. - Added No Reference to be Provided. [SAT]

B, CR, 41.5 (2005 NRC Exam)

- Procedure OTA-RK-0001, Addendum 34E says that the quench tank rupture disc is designed to relieve when pressure is in a nominal range of 86 to 100 psig. Modify stem to also say Which of the below values is within the actuation range of the PRT rupture disc? - Made changes to stem as recommended.

E Since quench tank temperature is used as part of the evaluation of the situation, it 33 F 3 X X B would be recommended to provide this to the applicant. - Added Temperature to S Stem It is not clear why the 50 psig setting in the proposed distracters is plausible. There doesnt appear to be a set point associated with the PRT that is 50 psig. Justify, or use a value that has some relationship to the quench tank or another tank. -

Changed distractor to 75 psig, VCT Maximum pressure, for greater plausibility.

[SAT]

A, CR, 41.7 34 H 3 N S - Provide the referenced drawings J-22EG01A and -22EG01B so that the answers can be validated. [1UE5ZA[1)) - References Provided [SAT]

35 F 2 B S D, CR, 41.8 36 F 2 X N S A, CR, 41.7 - Verify that the indication power is the same as the control power. -

Verified indication power is same as control power, Drawing E-23BB40 provided. Page 6 of 16 OBDI 202 - IOLE Process

ES-401 CW-2014-12 7 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only

[SAT]

37 F 2 N S B, CR, 41.5 C, CR, 41.7 38 H 3 B S - Provide Sheet 10 of drawing 7250 D64, the reference. Other sheets are provided, but not Sheet 10. - Provided Drawing 7250D64 Sheet 10. [SAT]

39 H 3 M S D, CR, 41.7 D, CR, 41.7 For the high vibration alarm, it would be assumed that there is an alarm response procedure, which may include additional actions, in addition to OTN-GN-00001.

Verify that there are no other actions that may affect the correct answer. [No ARP for 40 H 3 X B S Computer alarm] - Discussed with Chief Examiner that no ARP is available for Computer annunciators.

Recommend changing question to say, What action(s) is(are) required?- Made changes to stem as recommended. [SAT]

C, CR, 41.7

-Distractors B and D implausible for applicant to believe containment temperature would be maintained <120F on a design-basis LOCA.

U -Question does not test on the specific design feature of CSS which reduces 41 F 3 X X B S radiological hazard and corrosion, i.e., TSP Baskets. K/A mismatch.

Modified question to test on containment design pressure and which phase of ECCS actuation the TSP baskets are utilized. This corrected the K/A match issue and removed the implausible distractor of 120F. [SAT]

C, CR, 41.5

- Add a reference that shows the rated steam flows of the ASDs (i.e. 5% for 1 ASD).

42 H 3 N S

-Added reference FSAR Table 10.3-2 for ASDs providing approx. 4% flow each.

Reference provided. [SAT]

A, CR, 41.7 (2005 NRC Exam)

- Confirm that the information provided is sufficient determine there is not Degraded E Core Cooling. It will eliminate the argument that distractor C is correct. [Add CET 43 H 3 X M Temp] - Added CET Temperature to Stem.

S

- The cited reference doesnt directly support the basis for RCP trip criteria at 1425 psig. It provides criteria for establishing the trip point. There should be a site calculation that supports what the point is, and what its purpose is. Provide the Page 7 of 16 OBDI 202 - IOLE Process

ES-401 CW-2014-12 8 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only reference if available. Also, the action to trip the RCPs should be in operating procedures. Provide that reference as well. [E-0 Foldout page] - Added E-0 Fold Out Page as reference. Unable to locate site calculation reference for 1425 psig setpoint.

[SAT - ERG Background Information explains reason for RCP trip at 1425 psig]

B, CR, 41.10

- Explain why a reactor power increase on an intersystem LOCA is plausible. Affects distractors C and D. [Seal Water HX leak causes a dilution = power increase] -

44 H 2 M S Discussed with Chief examiner that Seal water HX leak = power increase.

-The annunciator response procedures mentioned in the question stem were not provided. Provide for review. - Provided additional references. [SAT]

D, CR, 41.8 45 H 3 B S - Explain that RHR pumps secured in E-1 prior to transition to ES-1.2. - Added explanation that RHR pumps secured per E-1, prior to transition to ES-1.2. [SAT]

B, CR, 43.5 41.3 46 H 3 B S RO questions need to be associated with elements of 10 CFR 55.41, not 55.43.

Changed to CFR 41.3 [SAT]

D, CR, 41.5 (2005 NRC Exam)

- The reference explains what is done to maintain RCS level in mid loop operations, 47 H 3 X B S but doesnt explain the expected response to a loss of RHR cooling in midloop.

Provide a reference that evaluates the response. [OOA-BB-00003] - Provided additional Reference OOA-BB-00003, to explain elevations of equipment during reduced inventory operations. [SAT]

48 H 3 B S B, CR, 41.7 (2005 NRC Exam)

C, CR, 45.7 41.7

- Provide the cited references (drawing 7250D64, Sheets 6, 17, and 18)

- Provided additional references as requested.

49 H 3 B S

-The question needs to be associated with an element of 10 CFR 55.41. 10 CFR 55.45 is for operating test elements Changed to CFR 41.7 [SAT]

D, CR, 41.10 (2011 NRC Exam)

- Verify that any transient effects with the steam generator safety valves would not 50 F 2 M S affect the distracters, making any of them plausible. If so, address in the question stem. [Explain that its 30 seconds or so until ASDs open and swell occurs - good enough for Initially] - Discussed with chief examiner and added discussion of Page 8 of 16 OBDI 202 - IOLE Process

ES-401 CW-2014-12 9 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only approximately 30 second delay to explanation. [SAT]

A, CR, 41.5

- See NUREG-1021, ES-401, Section D.2.f, fourth bullet. At least one distracter is M changed, but one pertinent condition in the question stem did not change. Unless 51 F 2 S B adding the fact that the plant is 100% power affects the answer of the question, this doesnt meet the definition of significant modification. - Changed to Bank Question

[SAT]

C, CR, 41.7

- The proposed reference (drawing J-22EF05, R2) explains what happens to EF-HV-51 when there is an SI signal. Provide cited references that support what the valves in the distracters do on a SI signal. [Only need ref for EF-HV-65] - Provided additional reference for EF-HV-65, as discussed with the Chief Examiner, and added M references to explanation.

52 H 3 X S B See NUREG-1021, ES-401, Section D.2.f, fourth bullet. The pertinent condition in the question stem did not change (80% power, SI signal). One distracter changed (the number of the containment cooler bypass valve), but if the applicant knows how the bypass valves function, then he/she knows the same thing for all bypass valves.

This doesnt appear to meet the definition of significant modification.

-Changed to Bank Question. [SAT]

C, CR, 41.7

- Clarify in explanation which circulating water pumps lose power in this scenario.

The configuration of the circulating water pump breakers have an effect on the correct answer. - Added explanation of normal power supplies for both Circwater and 53 F 3 X N S Service water pumps.

-The cited references werent provided (drawing 8600-X-88550 and procedure OTN-PB-00001, addendum 3). Please provide. - Provided additional references as requested. [SAT]

B, CR, 41.7 (2007 NRC Exam)

Recommend improving the Explanation section by explaining that the 105 psig set 54 F 2 B S point is a plausible distractor because it is the pressure that the inlet/outlet valves on the standby instrument air dryer train open with lowering header pressure. - Added this to the explanation for greater plausibility. [SAT]

B, CR, 41.7 (2005 NRC Exam) 55 H 3 B S - The given reference doesnt support the basis for distracter C. Provide a reference.

- Added Reference FSAR 9.4.6 to support Distractor C. [SAT]

56 H 3 N S C, CR, 41.7 Page 9 of 16 OBDI 202 - IOLE Process

ES-401 CW-2014-12 10 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only B, CR, 41.3

- The second part of the question (indication range) is not related to the K/A.

nd st Replace with a K/A-related 2 part, or modify 1 part to have 4 different plausible U

57 F 2 X N choices. - Modified question to ensure both parts of the 2 part question meet the K/A.

S

-Provide the reference for verification (ETP-BB-03138). - This reference is no longer applicable, provided reference Drawing M-22BB04 to support modified question.

[SAT]

58 F 2 N S B, CR, 41.5 59 F 2 B S A, CR, 41.5 (2007 NRC Exam)

D, CR, 41.7

- Justify plausibility of A, a skimmer pump having a low level shutoff setting.

60 F 3 B S [Explain that skimmer pump is limited by the length of discharge hose and will burn up if level lowers. No protective feature to prevent this] - Discussed with chief examiner the lack of protective feature for the skimmer pump if level lowers too low and added this explanation to the question pedigree. [SAT]

B, CR, 41.7 (2007 NRC Exam) 61 H 3 B S - The explanation for distracter D sounds like D is a correct answer as well. Should the explanation actually read if a Steam Pressure channel fails HIGH ? Verify. -

Changed distractor explanation to steam pressure channel fails HIGH. [SAT]

62 H 3 N S D, OR, 41.7 63 F 3 B S D, CR, 41.5 64 H 3 M S B, CR, 41.7 A, CR, 41.7 (2009 NRC Exam)

E - Since distracters B and C both test whether an alarm plus a trouble alarm in one 65 H 3 X B zone would cause a Halon actuation, it would be better if one of these distracters S

tested if 2 alarms in one zone would cause a Halon actuation. According to the Explanation, it would not. - Modified distractors as recommended. [SAT]

D C , CR, 41.1 (2007 NRC Exam)

- Minimum staffing, as implemented from Tech Specs Section 5, is typically a SRO U

66 F 3 X B only task. Justify whether RO are responsible for this at the site. Procedure ODP-S ZZ-00001 indicates that the Shift Manager is responsible for shift manning (Section 3.3.18), and the Shift Manager can delegate this and other responsibilities to the Control Room Supervisor (Section 3.5.10). The Unit Reactor Operators dont appear Page 10 of 16 OBDI 202 - IOLE Process

ES-401 CW-2014-12 11 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only to have a responsibility denoted for this. - Replaced Question with RO level question applicable to the same K/A. Provided reference APA-ZZ-00905. [SAT]

A, CR, 41.10

- Medical Emergency Response Team -- Explain that MERT is comprised of security 67 F 4 N S officers with EMT quals, and is not a subset of the fire brigade. Distractor B still incorrect. - Added this explanation to the question pedigree following discussion with the chief examiner. [SAT]

A, CR, 41.10 68 F 3 N S -Add Tech Spec LCO 3.4.16 as a cited reference for the question. It is needed to justify the plausibility of the distracters. - Added LCO 3.4.16 to the cited references for the question. [SAT]

D, OR, 41.10

- In the outline (Rev. 2), it says that this question is supposed to be written to K/A number 2.2.7. The proposed draft question is written to K/A number 2.2.39. The U provided question does not deal with special or infrequent tests, which is the subject 69 F 3 X B of K/A number 2.2.7. Form 401-4 does not list this K/A as being replaced. K/A S mismatch.

[Question resampled and replaced but outline and k/a reject list not updated] - No change to the question. Updated the Form ES401-4 as REV 1 and ES401-3 as REV

3. [SAT]

D, CR, 41.10 E - Revise stem to read Which of the following conditions alone would permit the 70 F 4 X B Independent Verification to be waived? This clears up any question about the S

exposure time, dose received from airborne, contamination, etc. - Modified Stem as recommended.

D, CR, 41.10

-This question is too system-specific for a Tier 3 generic K/A. It is testing on specific H N U aspects of a rod control malfunction. If a generic question cant be generated based 71 3 X on K/A, resample. - Replaced K/A and question. Provided new references required F B S for new question and updated Form ES401-4 as REV 1 and ES401-3 as REV 3.

-Question labeled as New but Last NRC Exam block is checked. - Question Replaced with Bank Question [SAT]

A, CR, 41.12 72 F 3 B S - Include references that show what the other positions have the power to do with regards to this. Their responsibilities are stated in the Explanation, but the supporting references arent provided. - Added reference ODP-ZZ-00001 to question pedigree Page 11 of 16 OBDI 202 - IOLE Process

ES-401 CW-2014-12 12 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only and provided this reference to NRC. [SAT]

73 F 3 B S C, CR, 41.11 (Vogtle 2012 Exam) 74 F 3 B S D, CR, 41.10 75 F 3 B S C, CR, 41.10 76 H 4 M S C, CR, 43.5 D, CR, 43.5 Answer D says that it is required that the plant transition to OTO-MA-00008. Step E E10 of procedure OTO-BB-00002, Attachment E, says that the plant can transition to 77 H 3 X B S Mode 3 in 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> using their choice of three procedures. Thus it is not necessarily required that the crew use OTO-MA-00008. Modify verbiage of stem to accommodate. - Modified stem to remove required action. [SAT]

B, CR, 43.2 - Provide reference procedure OSP-NE-00001. - Provided reference 78 H 3 N S OSP-NB-00001, the required reference for this question. [SAT]

D, CR, 43.5 K/A mismatch. The question doesnt test the ability of the applicant to determine and U interpret plant automatic actions that will occur. With a loss of instrument power 79 F 3 X N causing equipment to be operable/inoperable, there is no asking of what S

automatically will happen as a result of this, and what does this mean. - Modified question to test automatic action of the plant, and kept SRO portion to determine operability. [SAT]

A, CR, 43.5 - This question test systems level knowledge, and entry conditions for one abnormal procedure. There is no prioritization decision required on more than one applicable procedure requiring entries that need an SRO applicant to make a decision on how they should be coordinated. This is an RO level of knowledge question. - Modified question to meet SRO only criteria by having applicant choose U

80 H 3 X X B the applicable attachment of OTO-KA-00001 for the given conditions.

S In procedure OTO-KA-00001, the RNO for Step 3 says to conduct a load reduction per Attachment A. Answers A and B say that the load reduction is completed IAW OTO-MA-00008. Therefore, the proposed correct answer (A) is incorrect. There is no correct answer. - Modified answer to be in accordance with Attachment A of OTO-KA-00001 vice OTO-MA-00008. [SAT]

81 H 3 B S B, CR, 43.5 (2005 NRC Exam) 82 F 3 M S C, CR, 43.2 Page 12 of 16 OBDI 202 - IOLE Process

ES-401 CW-2014-12 13 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only D, OR, 43.4 and 43.5 - The applicant version of the test doesnt show the reference that would be provided. Assume this is the whole EAL matrix table set. Would this give information to the applicant on other questions that are not intended to be open 83 H 4 N S reference? Verify. - Reference is to be provided. This is the EAL classification matrix. It was provided to both RO and SRO candidates during all Validations to ensure no information available on the chart will assist in answering any other questions on the exam. [SAT]

D, CR, 43.4 84 F 3 N S -Stem should read what is the basis of this step? - Modified Stem as recommended. [SAT]

C, OR, 43.5 - The temperature provided in the stem is RCS temperature. Clarify RCS cold leg temperature is needed to use Figure 4a of the CSF status tree document. - Modified Stem to state RCS Cold Leg Temperature. Also, provide some information about the timeline since the reactor trip to this point. Otherwise, an applicant could argue that he/she doesnt know how long it took to get to the RCS temperature provided, so he/she cant answer the CSF status tree entry question.

E 85 H 3 X B With these two flaws, there is no way to determine a correct answer. -Added S statement to stem stating Rx trip occurred 15 minutes ago.

It says that transition to E-1 has occurred, but later in the question it asks what actions are taken upon leaving E-0. Deconflict. Modified stem to state crew is preparing to transition to E-1, and then ask what action is required upon transition to E-1. This ensures the correct answer remains correct by stating that transition back to E-1 is the correct path. [SAT]

B, CR, 43.2 - The intent of the second part of the K/A is to test how the situation is U addressed in procedures. The question as written tests on the basis for a SR. K/A 86 H 3 X N S mismatch. - Modified second half of the question to be SRO level and test the procedural action to be taken based on the current plant conditions. [SAT]

C, CR, 43.5 - If the OTN procedure allows the failed valve to be repositioned manually to the position it would be in the normal SI lineup, then this would be an action taken as a result of its inoperability as well. Verify if this is required; if so, Answers C and D are both correct. - Answer D provides direction to reposition the 87 H 3 N S opposite train valve, not the valve that has failed. Added this explanation to the distractor analysis.

Provide procedure OTN-EJ-00001, Addenda 5 and 6. - Provided additional references as requested. [SAT]

B, CR, 43.2 - Provide the references for this question. Procedure OTS-SA-00001 88 F 2 N S and the actions for ESFAS Annunciator 128B. - Provided additional references as requested. [SAT] Page 13 of 16 OBDI 202 - IOLE Process

ES-401 CW-2014-12 14 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only A, CR, 43.2 - Recommend modifying the second part of the question statement to say Which Technical Specification LCO(s) will require(s) Actions(s) to mitigate the E situation? More precise wording for Tech Spec usage. - Modified Stem as 89 H 4 X N recommended.

S For the References, add the T/S 3.8.9 Bases as well. - Added Reference to question pedigree. [SAT]

D, OR, 43.1 K/A mismatch.- The K/A statement calls for testing knowledge of EOP mitigation strategies with reference to Reactor Coolant Pumps. RCPs are not discussed in the proposed question, and it doesnt test knowledge of how to the EOPs are being used U to mitigate. It asks for knowledge of the EOP bases definition for a term, and it tests 90 H 4 X N ability to implement the EALs. - Modified question to test on RCPs, and Modified S

stem conditions to change EAL classification. Analyzed possible overlap with question 43 for RCP trip criteria. Determined that Q90 is testing on a different Major Event, and RCP status is different due to the unique situation of the plant during this specific situation. This is delineated in a caution of the E-3 procedure for SG Tube Rupture. [SAT]

A, CR, 43.1 - Does the applicant have the necessary information to determine if DRPI for the affected rod is within +10/-4 of the group demand position? Data is necessary to figure out if this is operable or not. - Added to explanation that due to Annunciator 80C Not being lit, the applicant can assume that rod H-8 isstill within 12 91 H 3 B S steps of the other rods within that same group, therefore DRPI can be determined to be operable.

For distracter D, explain how its plausible that the instrument has no accuracy. -

Added explanation to distractor D justifying plausibility. [SAT]

D, CR, 43.5 -

Per SRO-only guidance, SRO-only knowledge should not be claimed for questions that can be answered solely using fundamental knowledge of: the basic purpose, the overall sequence of events that will occur, or the overall mitigative strategy of a U procedure; or any AOP entry condition.

92 H 3 X N S

The first part of the question is systems knowledge. The second requires only fundamental knowledge of which procedure will mitigate the condition.

Not an SRO question. - Modified question to meet K/A by understanding the basis of the Boron Concentration required in T/S for the Spent Fuel Pool. Second part of the K/A met by asking T/S action required during this situation.

93 F 3 N S B, CR, 43.7 94 F 3 X B E A, CR, 43.7 - Is it possible that the Reactor Engineering representative would be a Page 14 of 16 OBDI 202 - IOLE Process

ES-401 CW-2014-12 15 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S licensed SRO? The way it is written, it is anyone from that department. If there is evidence of someone that fits that, the applicant could argue answer B is correct. - At Callaway, No members of the Reactor Engineering Group are Licensed SROs, maintaining that B is an Incorrect answer.

95 F 3 B S D, CR, 43.5 B, CR, 43.5 - In procedure MDP-ZZ-TR001, Attachment 2 (page 2 of 2), it says that the approval authorities for a Level 1 Troubleshooting Plan are the Shift Manager AND Duty Manager. In the Roles/Responsibilities section (Sections 3.2 and 3.3), it only denotes the Shift Manager as the approval authority for Level 1 and 2 E Troubleshooting Plans. I assumed that Section 3.2 says this because it appears that 96 F 3 X N the SM and DM can share the duties. Taking all of it together, it appears they both S

have to approve the plan.

Verify who the approval authorities are, then adjust the answers so that there is a correct answer. - Modified Stem to add Duty Manager and ____ must approve to address shared responsibilities of the DM and SM. [SAT]

97 F 2 B S C, CR, 43.3 D, CR, 43.4 - The question stem says a release is in progress, yet the first question asks about what would prevent a release. It needs to be clear on whether the release is in progress, or they are preparing for it. - Modified Stem to state the Crew E

98 F 3 X N is preparing to perform a release, and question asks what would automatically S prevent the release.

Provide a copy of reference HTP-ZZ-00006, Liquid Radwaste Release Permit. -

Provided Reference HTP-ZZ-02006. [SAT]

99 F 3 B S A, OR, 43.5 B, CR, 43.3 - See NUREG-1021, ES-401, Section D.2.f, fourth bullet. If you compare M this to the original bank question, it doesnt change any conditions in the question 100 F 3 S B stem, and alters one answer. This doesnt appear to meet the criteria to significantly modify the question to count it as Modified. - Changed to Bank Question. [SAT]

B= 41 (55%) F= 37 (49.3%) E= 10 (13%)

Additional Notes: Peer review by S. Hedger RO TOTALS: M= 9 (12%) H= 38 (50.7%) U= 7 (9%)

4 questions had issues on whether they were Bank or Modified - They are denoted in the Explanation section, Classified identified questions as Bank Questions.

N= 25 (33.3%)

B= 10 (40%) F= 12 (48%) E= 9 (36%)

SRO TOTALS:

M= 2 (8%) H= 13 (52%) U= 5 (20%) Page 15 of 16 OBDI 202 - IOLE Process

ES-401 CW-2014-12 16 Form ES-401-9 N= 13 (52%)

GENERAL COMMENT

S:

1. Bank / Mod / New questions are indicated in their respective column field above. Red Numbers indicate Modified totals for B/M/N

2. Chief Examiner comments are indicated in blue where multiple comments exist, otherwise they are in black.

3. Average difficulty is 2.85 on the RO exam and 3.08 on the SRO exam.

4. The 10CFR55.41/.43 distribution is: RO / SRO 41.1 = 1 43.1 = 2 41.2 = 1 43.2 = 5 41.3 = 2 43.3 = 2 41.4 = 2 43.4 = 3 41.5 = 12 43.5 = 11 41.6 = 0 43.6 = 0 41.7 = 37 43.7 = 2 41.8 = 5 41.9 = 0 41.10 = 13 41.11 = 1 41.12 = 1 41.13 = 0 41.14 = 0

5. The answer distribution is: RO / SRO A = 20 (27%) / 5 (20%)

B = 19 (25%) / 7 (28%)

C = 17 (23%) / 5 (20%)

D = 19 (25%) / 8 (32%)

6. There are 6 questions with attachments provided. Questions 62, 69, 83, 85, 90, and 99 require hand-outs Page 16 of 16 OBDI 202 - IOLE Process