Text

2018-09 Draft Written Exam Comments
	ML18285A122
Person / Time
Site:	Cooper
Issue date:	10/02/2018
From:	Vincent Gaddy; Operations Branch IV
To:	; Nebraska Public Power District (NPPD)
References
	Download: ML18285A122 (25)
	v • d • e

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3

1. LOK 2. LOD 6

7 Ans Letter CFR Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

/

units back-ward K/A SRO-only B/M/N U/E/S A/B/C/

D 55.41 55.43 1

F 2

X N

E C

5 RPS seems to always be right.

Maybe try the actual signals (flow-biased vs fixed). Add "According to Technical Specifications" to the part 1 question.

Based on discussion 8/14 this is an edit based on distractors being credible. Changing distractors to APRM scram signals.

2 H

3 X

N E

C 7

Add "NEITHER the AC or DC powered solenoids de-energize" to A.

3 F

3 B

E B

10 Do we need to add info on the plant has been shutdown at the beginning of the question?

Need to be more clear in the stem that IAW 5.3ALT-Strat, Att4, which valve does the procedure require using to control RCIC flow?

4 H

3 X

X N

E B

5 A and C don't seem plausible.

Could A be changed to high alarm and then to scram setpoint?

Could C be changed to scram setpoint and then return to a lower than normal value? Add level at normal operating value to the stem.

Distracters are not balanced. A for example, should be slightly rises then returns to previous value.

Based on discuusion 8/14 A is not credible and the only non credible so marked as an edit.

5 F

3 N

S C

6

3. Psychometric Flaws

4. Job Content Flaws

5. Other Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 to 5 (easy to difficult) rating scale (questions in the 2 to 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

a. The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

b. The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

c. The answer choices are a collection of unrelated true/false statements.

d. The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

e. One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

a. The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

b. The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

c. The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

d. The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

Place the answer letter here Place the 55.41(b)x or 55.43(b)y item here (example 1, 2, 3, 10, etc)

8. At a minimum, explain any Unsat ratings (e.g., how the Appendix B psychometric attributes are not being met).

Explanation 8

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 6

F 2

N E

D 7

I'd like to change at least C to a local indicator. One of the 903' indicators works. Also, consider using all local indicators since they are spread between the 903',

931', and the 976'.

7 F

3 B

E C

10 Is there another system in supplemental actions that can be used as a distractor (radiation monitoring, chemistry). The distractors need to be better balanced. Either balance the 2X2 or make it a 1X4.

12/2015 NRC Q7 8

F 2

N E

B 8

Capitalize MINIMUM in the stem.

9 F

2 X

X N

E B

10 Prefer to use MINIMUM instead of lowest. Where is the loss of shutdown cooling? Is there an AOP that warns against high flow when placing in service following a loss of SDC?

I don't think D is credible and C is marginal because these are at or near pump runout conditions. A better setup would be what value for flow do you enter the Off-Normal for loss of shutdown cooling and where is it read (what panel or inst)? This makes this Q not an F3 but an F2. Based on discussion on 8/14 this should have been marked an E based on K/A is matched and distractors are credible.

10 F

2 X

N U

D 10 The name of the valve gives away the first part of the answer 11 F

2 X

B S

A 7

Confirm this has never been used on an NRC exam.

12 F

3 X

X B

U B

10 This looks like a question that could be challenged. B and C could be correct based on how you draw the line. Are there differences in instrumentation when at high pressure that could be tested on? Replace the question.

This is not the KA. It is high DW Pressure versus SP level, so the stem should be asking how to determine/interpret SP level as s function of the other parameters.

13 F

3 N

E B

10 Move power level to the front of the question to make it more prominent.

14 F

3 X

N E

C 7

Should we consider adding no operator actions have been performed to the stem. What is the RHR qualification limit for components in the drywell?

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 15 H

3 X

X X

N U

A 10 B is not plausible. C and D seems to be also always correct.

The stem needs to be tied to the EOP's directly (by putting them in the stem, iaw) to establish a legal link to one and only one correct answer.

16 F

2 X

B E

C 10 Distracter A is unbalanced from the other three choices.

17 H

2 X

N E

D 6

Would this generate a recirc pump trip?

downscale in this context is for the purpose of answering the EOP question, is reactor power downscale?, not the alarm for downscale. This value is usually discussed well in the EOP bases and is 3% in the EOP flowcharts.

So a question on this aspect would meet the KA. The recorder aspect needs better setup to not be legally challenged since an operator has to manipulate the switch to get it there and it is not needed anyway. The number of neutron mon. systems needed to determined downscale for purpose of EOPs (or what if they are bouncing above and below 3%) would be good items for this KA. How could you be at a power level of 6% and be on the IRM?

18 F

3 X

N U

A 11 The question addresses signal status for any parameter for plant computer. The KA is wanting the applicant to determine a high offsite release with the plant computer. What screen is it on, how do you know if it is actually high (ie can you read a value and status on SPDS or ?). This is a great tier 3 question but not specific enough for this section.

19 F

2 N

E A

10 Capitalize primary in the stem

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 20 F

3 X

N E

C 10 Don't be vague when asking a question. Change to "following is the reason". You have 2 answers with large load protection as the goal are there 2 other large loads that can be used to replace A and B that have undervoltage protection?

Distracter A is not credible. The distracters are also imbalanced.

You ask for the reason then you give a short answer for A where you give answers with clarifying reasons for the other 3 choices.

For B, similar issue. You can also ask the value for the protection (ie is it 3600v or 3800v) and is it for undervoltage or overcurrent protection on those items that don't have that protection or for the attempted maintenance of the remaining bus voltages, etc.

Based on discussion on 8/14 this is an edit based on restructing balance between the question 21 H

3 N

S B

5 Did the stem validate okay as written? If so then I have no problems with it. I would have used words such as "As drywell pressure continues to drop, at what pressure, when reached, will the Torus pressure start to decrease?"

22 H

3 N

S B

4 23 F

2 N

S D

10 24 F

2 B

S B

6 3/2017 NRC Q23 25 H

3 X

M E

A 7

Are you determining or interpreting reactor pressure? If so how?

Mod from 2017 NRC Q42.

26 H

3 X

N E

B 5

Balance the 2X2. You have generator load and speed mixed together.

Distracter C is not credible.

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 27 F

2 X

N E

C 10 In accordance with 5.8.3 needs to be included in the question. This doesn't appear to have a lot of nuance to the question. Maybe set the meter at 20. A isn't really plausible.

No system knowledge is required to answer the stem as given. It would be appropriate to put the HCU numbers in the stem without the word "south" in the stem, then put the pictures of the north and south RM instruments in the stem and have the examinee determine which RM to use to determine if they can use std RP practices or use heightened practices due to dose. If an RO needs to know this, then the procedure aspect is also important so you could also toggle that in your choices.

28 F

2 M

S A

7 29 H

3 X

X N

S D

7 Per section 2.1.2.2, it seems to me that because RX water level is still low in the stem, that the LPCI initiation signal is still present and won't clear until the reset pushbutton is pushed to reset the condition WITH a cleared low water level condition, which would make C correct NOT D, if I am reading the reference materials correctly. Based on discussion 8/14 this should have been marked as SAT.

30 H

3 X

N E

B 4

Just ask which procedure is entered for the first part.

31 F

2 X

N U

A 7

First question is GFE knowledge.

Do you have guidance to shutdown individual trains if suppression pool level is low for NPSH?

Oscillations for cavitation is easy.

A better way to write the stem is to ask which indication would FIRST help in determining cavitation exists (eg. pump amps, suc press, disc press). If this pump has all three, then which indicator displays it first? If all three are affected equally and there are valid instruments for all three parameters then the way you have the question setup is fine.

Will look to replace.

32 H

3 N

S A

7

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 33 F

2 X

X N

S A

6 Question should have been scored as a 2 and all distractors are credible.

LOD=1, too easy as written with stem focus and distracter choices.

Better mixing always occurs with more water. You could ask the inner/outer tube aspect but the most important aspect of SLC injection in a BWR-4 is the power suppression is not homogeneous, so you get localized power suppression based on the location of the tube, until better mixing starts to occur. Putting the APRMs that are suppressed initially requires knowledge of where the sparger is connected relative to those APRMs. You could use one of the two items you are toggling but both are too easy when put together. You can get at this by directly asking them which APRMs are affected and then they would know if the tube is above or below the core plate, right? Check for overlap with op test also.

34 F

1 X

N E

C 7

Are you using the noun name for the squib valve ready light. Do the SLC pumps have min flow valves. Can they be part of the question instead of the ready lights? I don't think the lights are plausible. Should have been marked as a 2.

Do we have a SLC failure on a scenario? BORDERLINE LOD =

1. If not, it would be more plausible to have a SQUIB valve light and a SLC flow value that they have to determine is acceptable or not (ie if the valve opened correctly and fully) since these squib valves have some OE to fire the continuity circuit but not fully open the valve and the way that the operator knows that the valve opened FULLY is to check for proper flow.

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 35 H

3 X

N U

B 6

Does RPS have many single point vulnerabilities? Full scram from a single point isn't plausible. Backup scram valves not opening on full scram doesn't seem plausible.

Not a balanced 2X2 Single failure criterion for all nuclear plants prevent a single failure from causing a SCRAM, which makes A and B not credible. Also, this question may help answer another question or ask similar knowledge. Need to check for overlap with other questions on this power loss and RPS/Scram valve aspects.

36 F

3 X

X X

N U

C 10 As it is written the alarm card points to 2.1.5 so both could be argued as correct. This could be done as a 1X4 with voltages of 111, 119, 129, and 135. Or as a 2X2 with voltages and frequency (56.9, 61.2).

Single failure criterion for all nuclear plants prevent a single failure from causing a SCRAM (in this case entering/using the SCRAM procedure when not in a SCRAM), which makes B and D not credible. Also, this question may help answer another question or ask similar knowledge. Need to check for overlap with other questions on this power loss and RPS/Scram valve aspects.

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 37 H

3 B

E D

6 Should have been an edit with information to tell applicants the drive was succesful in starting to move.

FROM NRC AUDIT 3/2017 Is this off of an NRC Exam?

These audit exams are often previous NRC Exams. It is more appropriate to ask when DO you withdraw the IRM detectors in the startup. Anyone that has operated this system even once (and we do with JPM S-7 on this exam) knows that the buttons are continuous hold buttons, that you have to hold them down several minutes to get them fully withdrawn, and you also cue to that with the stem by stating that you then release the button. The distracter D does not need "only" in it since it contains both aspects.

You could setup the stem where you ask when is it done and if done early what are the op implications (ie once push the button you get the rod block almost immediately, right?). Rod block or SCRAM? This question will need to be changed due to overlap with JPM S-7.

38 H

3 M

E D

7 Why say the detector is failed?

Can say SRM C is reading Also, could use all four SRM in the picture with one above the high high, another high and the other 2 reading normal.

39 H

2 X

N E

B 6

Borderline LOD=1 question. C is not a credible distracter. Instead of just having inop, ask the examinee to determine the issue with the APRM by giving them more information and then toggle whether rods can be driven in or not (such as detector voltage issue, LPRM issues, etc). Not an H3 question either way.

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 40 F

3 X

X X

X B

E C

7 Don't need to tell them it's a Group 5 isolation. The 2X2 isn't balanced.

FROM NRC AUDIT 3/2017 Is this off of an NRC Exam?

These audit exams are often previous NRC Exams.

This question has a flaw for building on top of the KA. The KA is only initiation logic and by adding Isolation logic the question could possibly be challenged in an appeal because the KA does not include this aspect. Also, the NUREG discourages this technique. I would just ask Is it AA1, AA2, BB1, or BB2? A simple 1 X 4 structure and it hits the KA and doesn't add anything else and doesn't cue.

41 H

3 B

E D

7 I would spell out ADS and ASD each time to make sure it is clear the difference in the stem.

42 F

3 N

E B

13 I don't think that this is higher order because you know where RCIC drains to and you get to the answer, so this is a fundamental question of recall on where RCIC is located and what drains in that area.

43 F

3 B

S D

7 FROM NRC AUDIT 12/2015 Need to check that this was not on a previous NRC Exam also.

44 F

3 X

N E

C 7

AC power is not credible. Use DC power supplies for all answers.

Need to check for overlapping info on other questions. Also, DC power to SRVs is easy because the cart for emerg power to these valves during SBO is a battery.

Just ask, Is it AA1, AA2, AA3, or AA4 or AA1, AA2, BB1, BB2.

45 H

2 X

N E

D 7

Master controller in manual and the answer is master controller demand is a cueing issue.

Changing stem to match with C.

Cueing from stem straight to answer. Have to write this question another way.

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 46 H

3 X

B E

B 7

Could be higher order and difficulty if individual channels readings were added.

Distracter A is a "Nothing Happens" choice and is not allowed anymore on NRC Exams.

You could set it up with A. Does not start because only one division of isolation occurred.

B. Does not start because A is already running C. Starts and remains running because all logic is met D. Starts and then trips after the 30 second timer 47 F

2 X

X B

E D

4 I would prefer a system other than REC. What about circ water or turbine cooling? Per discussion on 8/14 B is not credible should have been marked as an edit.

Will replace system.

KA mismatch-This KA is intended to be used with the off-normal procedure for a malfunction of the AC electrical system, such as a loss of bus or part of the switchyard that takes out loads. A fast transfer event question only tests if they know all three buses are not lost, so no major loads are lost and this doesn't meet the KA.

The spirit of the KA could be met in a question where the fast transfer went slow or didn't occur, but does your procedure cover that? if you know the fast transfer gets all three buses with no losses then you aren't measuring anything else in the question so distracters B and C are not credible. Also, there are lots of questions on REC.

48 F

2 X

X N

U C

10 All information in the stem is repeated in the answer. Doesn't show an ability to interpret or execute the step.

KA mismatch. This question tests the basis for a step, which is an EK3 KA, not interpreting and executing a step.It also has cueing from the stem (normal and alt power) to the answer C (normal power..alt. power).

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 49 H

3 X

X B

E C

4 Per discussion on 8/14 should have been marked as an edit based on backwards logic.

KA mismatch and backwards or reverse logic question. First, How is the question monitoing auto operations of the DC electrical system if, in the ste, the operator caused the event? Secondly, for backwards logic, you give them the alarms (ie meters, dials, alarms) and then ask them how to reset the alarm. The KA wants the applicant performing a task (such as the transfer) and he gets several alarms and needs to determine if they are expected alarms or abnormal 50 H

2 X

X N

E C

7 B could also be correct because it is within 14 seconds. If at 11 seconds the bus was energized than B would be correct also. Add maximum to the question.

Distracter D is not credible because of standard design rules for all nuclear plants. If you use 13, 14, 24, and 27, that will work.

Based on discussion on 8/14 should have been called an edit based on clairifying information in the stem.

51 F

1 X

X X

N U

A 5

Basic engine knowledge can answer this question. How about using diesel non critical trip on lube oil as the basis for the question?

Distracters B and D are not credible. You wouldn't need a LO cooler if it stayed in band.

Everything heats up when started.

Also, part 2 of the question is a true/false setup. A good revision would be this:

What Lube Oil temperature band does the thermostat maintain while DG1 is in standby and when running at rated load?

A. 110 - 122 ; 145-160 B. 110-122 ; 160-172 C. 125-140 ; 145-160 D. 125-140 ; 160-172

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 52 H

3 X

X N

E A

4 Need to add all automatic actions have occurred in the stem. Don't think scram is credible without other problems happening in the plant. Could use open SA-PCV-609 as the second distractor.

Also since scram has been removed from the question the second part could ask what is the scram setpoint for this condition.

Per discussion on 8/15 this question should have been marked as an edit. Will change to include openeing PCV-609.

A better "predict the impact" part of the KA is when do you have to SCRAM. Dryer plugs, "predict" is the control air pressure starts to go down but you give them that in the stem with the alarm, then "use procedure" part of the KA would be 5.2AIR to combat the event.

Isn't there two gages nearly together in the CR, one for SA and the other IA, so you can directly see DP across them going up as this is happening (IA going down)? The more significant question for this KA is what value are you required to SCRAM per the procedure?

53 F

3 X

N E

C 4

Use another MCC for D. MCC-Q, RB, or Y are potentials Lots of REC stuff on the exam.

This is one where you have to ask it due to the KA. With "Non-critical" in the stem and distracter D being the 125 VDC Battery 1A, not sure this is credible for a safety power supply for a non-critical element (cueing). Just use 4 MCC choices.

54 H

3 X

N S

A 6

Should add the operator continues to hold continuous in to the stem. I think the first part should read rod insert and withdraw blocks are recived immediately OR are recived after full in and the operator releases the switch. Per discussion 8/15 should have been marked SAT originally.

Why are C1 and D1 credible?

Would it even be operationally valid to continue holding the button 'in' when the alarm comes in on RPIS pwr loss-Prudent operator Action would dicatate to stop what you are doing. This makes C and D not valid if so.

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 55 H

2 X

N E

D 5

The only link to the K/A is in the explanation. Per discussion on 8/15 should have been an edit to tie the K/A better.

KA mismatch because not directly measuring any knowledge of feedwater and RR. For example, at 50% power, RR A trips. What is rx and fw status:

A. 50% power and FW does x B. Scram and FW does x C. 50% power and FW does y D. Scram and FW does y.

You could also ask the bullets on the next page of the worksheet that clearly show a relationship between RR and FW 56 H

3 B

S A

4 57 H

3 X

X B

E B

7 Someone could argue that since there is no information that another pump is not available than D is also correct. Should have been marked as an EDIT based on needing to add info to the stem.

Have three open choices and one close choice (not balanced and A is not credible). Here is a good alternative:

A. Throttle closed MO-16A B.

Throttle closed MO-66A C.

Throttle open MO-12A D. Throttle open MO-27A 58 F

2 N

S A

4 59 F

2 X

X N

E B

10 Need to add both diesels are running loaded in the stem. Also need to establish a starting time for the event. Could argue that the condition lasted for more than 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> and A is also correct.

Based on discussion 8/15 should have been marked as an edit based on updates to the stem.

The examinee could read the stem as saying that RHR B is the only load on its respective DG.

Need to shore up the stem somehow for this. Instead of asking which DG, ask the kw loaded that the pump will add.

60 H

3 N

S B

6

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 61 H

4 X

N E

D 7

Based on 8/15 discussion should have been marked as an EDIT based on tier mismatch.

KA mismatch. Also, this is a system topic (you have an ATWS, etc) since in Tier 2 so should not be such a dire situation in the stem.Also answer topic already covered on another question (there is already a question on bypass valves not opening due to being below the setpoint due to SRV operation). A straighforward question is:

plant at full power, suddenly main turbine trips. Rx power stablizes at 15%. WHat is status of TBVs?

A. fully open for 3 seconds then 50% open B. fully open for 5 seconds then 50% open C. fully open for 3 seconds then 60% open D.

fully open for 5 seconds then 60%

open 62 H

3 M

E C

4 Can we add a 250V DC alarm that doesn't kill the supply to EBOP.

Mirror the question with the answers and put TGOP first and EBOP second in the answers or change the question order. Also D needs to be consistent with the others.

63 H

2 N

E D

7 Too much subset items going on.

Change B to Group 1 or Group 4 isolation 64 H

3 N

S A

10 Good as is Too much in the stem that is not needed for this question. Just ask:

Fire water system pressure slowly lowers due to pipe break. What is the setpoint for fire water system pressure for the auto start of diesel fire pump D and what methods.......

A. 1) 141 psig 2) no change B. 1) 141 psig 2) no change C. 1) 154 psig 2) no change D. 1) 154 psig 2) no change.

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 65 H

3 X

B E

D 7

Prefer to not use the 1 valve, 1 valve, 2 valves, 3 valves listing in the answers. A 2, 3, 2, 3 listing would be better.

Answer D is an "all of the above" choice and this is not preferred per the NUREG. You have several of these on the exam, also. Change HV-271 to open in the stem and then change the answer D to HV-270AV and HV-272AV ONLY. Also, the word "is" in the stem should be "are" after the words ventilation components.

66 H

2 B

S C

10 NOT Higher order 67 H

2 N

S B

10 68 F

2 N

S D

10 69 F

2 M

S D

10 Appears to be a modified bank to me.

3/2017 NRC Q69 70 F

3 B

S A

10 71 F

3 B

E D

10 2015 AUDIT Q70 Change distracter B to "imediately place mode switch in shutdown" and shuffle answers from top to bottom with least to most agrressive choices so TS 3.03 will be D and answer will now be A.

72 F

3 B

S C

12 2015 AUDIT Q71 73 F

2 X

X N

S B

10 Procedure selection is SRO knowledge because of needing to know the strategy to combat the event. Based on 8/15 discussion this just tests overall mitigation strategy so RO knowledge.

All the window alarms are not needed in this question. You have a high offsite rad release out of the offgas system. That is all you need before you get to the question. The reason that is tricky is the question has the appearance of being a tier 1 question but it really isn't, it is a tier 3 if you get rid of the system type stuff 9alarm windows).

74 F

2 N

S A

10 Are any of these other symbols actually used?

75 H

2 B

S D

10

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 76 H

3 N

S A

5 B appears to be a correct answer also. An applicant could determine that because LCO 3.0.6 applies only TS 3.8.4 needs to be entered for shutdown purposes. The other entries are for tracking purposes only. Also need to understand if this could be answered solely with systems knowledge and above the line information if 3.0.6 applies. Are you saying TS 5.5.11 has been applied to the procedure and the result is a loss of system safety function? More information is needed to determine a SAT question. For the question I'd prefer "When is the EARLIEST Mode 3 is REQUIRED..." I prefer stronger language in the question.

Changes were made question SAT.

Free Review Question 77 H

3 X

X N

E B

2 Why provide references in this question and not others REFERENCE NEEDED Stem clarity needs improvement.

"In addition to a CR, what action(s) is/are required now?"

Also you have too much Imbalance in the distracters. Need to modify them slightly. Make A. Restore cooldown rate to within limits within 30 minutes, ONLY B. Determine if RCS is accetable for continued operation, ONLY C. Restore cooldown rate......

D. Restore cooldown rate......

78 H

4 N

S A

2 REFERENCE NEEDED 79 F

3 N

S B

4 80 H

2 N

E C

5 Capitalize highest. Change the answer so SAE is the answer instead of a GE. Change power to less than 3%.

REFERENCE NEEDED

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 81 F

2 X

X N

S A

5 "yes" needs to be capitalized in C and D. It appears the first part of the question is answered with RO knowledge. The second part of the question appears to be a direct entry into an AOP which according to SRO guidance is an RO question. Since both halves of the question are answered with RO knowledge this is an RO question. Licensee provided justification why this is SRO knowledge. Free review and was not fixed as discussed during the free review.

Didn't capitalize all letters of YES as requested during free review to match NO. Two psychometric flaws with cueing.

82 H

4 N

S D

2 Couldn't find SR 3.8.1.1 requirements in our references. I would like to add transformer parameters that are out of spec that make the transformers inoperable if they are contained in SR 3.8.1.1. If the information isn't available than the first 2 sentences can be changed toThe plant is in MODE 1.

Question was replaced. With a SAT question. FREE REVIEW.

REFERENCE NEEDED The tap settings for SSST were not necessary-it is low no matter what setting you are on (it is way out of spec low at 4045v). Don't need that reference either but is okay to give to applicants.

83 H

3 M

S D

5 REFERENCE NEEDED, MOD FROM 3/2017 NRC Q80

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 84 H

1 X

N S

B 5

Take out the bullet about smoke.

From the distractor discussion there doesn't seem to be a diagnostic step in the EOP that asks for smoke in the area.

Maybe add EOP-5A as a distractor instead of A. I'd like to understand how 5.1INCIDENT is used in this scenario to rule out potentially multiple correct answers. Removed smoke. SAT.

Licensee replaced question based on validators comments.

Now UNSAT. K/A mismatch because no longer determining cause of high area temperature.

Will go back to the REV 1 question because they will go back to a SAT question.

Justify why new question written after free review completed is now SRO-only? The stem setup is too easy. If you had a 1 X 4 procedure selection that would be SRO-only, provided that the entry conditions for major EOPs is not the reason for the answer per the SRO only guidance.

85 F

3 N

S D

5 REFERENCE NEEDED 86 H

2 N

E B

2 Has pschometric flaws again with the Yes/NO. Is capital NO always part of the answer?

87 H

3 N

S D

5 I don't think the "Transition to is needed for the question. Will need to read more of the RCIC study material to ensure torus water temp isn't RO knowledge. If not then an SRO question. FREE REVIEW. Sat after review.

88 H

3 N

E C

2 I think the first question can be answered with RO knowledge based on the reference provided.

You should be using SRO knowledge to answer both halves of the question. Remember if there is an (a) and (b) part of a K/A it's acceptable to only write to the (b) part to get an SRO only question. WAS revised. SAT question.

have pschometric flaws again with the Yes/NO. Is capital NO always part of the answer?

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 89 H

3 X

B E

D 2

Can a 1X4 be made for this question. The second part is too obvious. Take the bold off the trains in the stem. Based on discussion 8/15 should have been marked EDIT based on high miss rate in requal and OP E at other facilites.

3/2017 AUDIT Q81 90 H

3 N

S B

5 The first part of the question seems to be RO knowledge.

Remember if there is an (a) and (b) part of a K/A it's acceptable to only write to the (b) part to get an SRO only question. Also, is the output breaker in AUTO during the test? I don't have full references to review. It seems like both are required and an applicant could argue on appeal that they would monitor voltage and frequency but would work to get the other diesel running making EMPWR a correct answer also. Revised to add bullett about critical trips. Replaced original question. FREE REVIEW.

91 H

3 N

S A

5 FREE REVIEW 92 F

3 N

E D

7 The "Excessive force" part of the answer needs to be more of a consequence. Take it to the next step and put the potential of damage to the grapple in the answer. Revised Q SAT Why not capitalize GRAPPLE after fuel in C and D also if it is a human factors issue?

93 F

3 X

X N

E C

5 I don't see the K/A match. I'm having trouble seeing the tie between starting with one more pump than necessary before an accident, and how that configuration will affect the accident mitigation strategy during a low power LOCA or loss of shutdown cooling. Revised Q SAT. Add more narrative. Go back to revised free review question.

After free review changes to question impacted the question.

Adding Sec containment established in stem reduces credibility of distracter B, so remove that. Also, the original distracter C that had ECCS and preventing 212F was more credible than the new distracter D.

Making the answer C is good because we will need less D answers on the SRO section when we get all the question details worked out.

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3 94 F

3 B

S D

3 3/2017 NRC Q96 95 F

2 X

X N

U C

5 This doesn't meet the SRO guidance. This is a tagging function not a question about multiple procedures and deciding which one has priority.

96 F

2 B

S D

3 4/2015 NRC Q96 97 F

2 N

S D

5 98 H

2 B

S C

4 3/2017 AUDIT Q97 99 F

1 X

B E

A 5

SAG 2 can never be correct because it doesn't appear to have direct entry requirements. Based on 8/15 discussion should have been marked EDIT based on being previously used on NRC initial exam. Will bold DIRECTLY.

3/2017 NRC Q99 easy question.

100 F

2 X

B U

B 5

A and C aren't credible. The emergency director is the only person who can approve. Could make a 2X2 and list positions staffed an they could determine if minimum staffing is met.

Not Higher order and not a 3 LOD.

Results Table RO LOK -H 34 Avg RO LOD 2.55 Flaws 10 CFR Distribution RO LOK-F 41 AVG SRO LOD 2.6 Stem focus 12 41(b)1 0

43(b)1 0 SRO LOK - H 14 Overall LOD 2.56 Cues 5

41(b)2 0

43(b)2 6 SRO LOK - F 11 T/F 0

41(b)3 0

43(b)3 2 Cred Dist 33 41(b)4 9

43(b)4 2 RO Bank 21 28 SRO Bank 6

24 Partial 7

41(b)5 6

43(b)5 14 RO Mod 5

6.667 SRO Mod 1

4 job link 3

41(b)6 9

43(b)6 0 RO New 49 65.33 SRO New 18 72 units 1

41(b)7 22 43(b)7 1 minutia 0

41(b)8 1

Total Bank 27 27 backward 0

41(b)9 0

Total Mod 6

6 KA 11 41(b)10 25 Total New 67 67 SRO-only 4

41(b)11 1

LOD = 1 4

41(b)12 1

RO Sat 25 33.33 SRO Sat 15 60 41(b)13 1

RO Unsat 9

12 SRO Unsat 2

8 Answer Dist ( in %)

41(b)14 0

RO Edit 41 54.67 SRO Edit 8

32 RO-A 17 SRO-A 5

RO-B 20 SRO-B 6

Total Sat 40 40 Total Unsat 11 11 RO-C 19 SRO-C 5

Total Edit 49 49 RO-D 19 SRO-D 9

Final is below

ES-401 CN-2018-09 Written Examination Review Worksheet Form ES-401-9 a Rev 3

1. LOK 2. LOD 6

7 Ans Letter CFR 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

/

units back-ward K/A SRO-only B/M/N U/E/S A/B/C/

D 55.41 55.43 Explanation 1

F 2

N S

D 5

All comments were resolved.

2 H

3 N

S C

7 All comments were resolved.

3 F

3 B

S B

10 All comments were resolved.

4 H

3 N

S A

5 All comments were resolved.

5 F

3 N

S C

6 6

H 3

N S

D 7

All comments were resolved.

7 F

3 B

S C

10 All comments were resolved.

8 F

2 N

S B

8 All comments were resolved.

9 F

2 N

S B

10 All comments were resolved.

10 H

3 N

S D

10 All comments were resolved.

11 F

2 B

S A

7 All comments were resolved.

12 H

3 N

S A

10 All comments were resolved.

13 F

3 N

S B

10 All comments were resolved.

14 F

3 N

S C

7 All comments were resolved.

15 H

4 N

S A

10 All comments were resolved.

16 F

2 B

S C

10 All comments were resolved.

17 H

2 N

S D

6 All comments were resolved.

18 H

3 N

S D

11 All comments were resolved.

19 F

2 N

S A

10 All comments were resolved.

20 F

3 N

S C

10 All comments were resolved.

21 H