ML15049A412

2015-02 Draft Written Exam Comments

ML15049A412
Person / Time
Site:	Cooper
Issue date:	01/27/2015
From:	Vincent Gaddy Operations Branch IV
To:	Nebraska Public Power District (NPPD)
References
50-298/15-002
Download: ML15049A412 (11)

Text

ES-401 CNS-2015-02 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only One could argue that C is a true statement, as the stem does not specify a procedure or basis to focus the answer. Distractors B and D are not credible because there is no mention of level or pressure in the stem, cuing that these are not the answer.

1 F 2 X X X X X N U Finally, if this reason comes from the tech spec bases, how is this an RO-level knowledge question?

Add per procedure 2.4RR Reactor Recirculation Abnormal in stem.

Distractor A is not set up like the others (which discuss whether HPCI must remain off or can be restarted). Reword distractor A to be more consistent: add but can be restarted if minimum battery charge is restored or similar. Need to edit C to be more consistent with procedure (I dont see where the from time injection flow reduced is 2 F 3 X X B U located - either revise C or provide that part of the reference). Also, from the procedure: If RCIC is unable to perform function, compensatory actions must be taken. Could argue that D is also correct because RCIC is not maintaining RPV level as described in 5.3SBO. Distractor C could be construed as correct because no later than 15 cold easily be thought of as approximately 10, and its second half is Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

$ The answer choices are a collection of unrelated true/false statements.

$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

$ The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met). Page 1 of 11 OBDI 202 - IOLE Process

ES-401 CNS-2015-02 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only correct. Add per procedure XXX in stem.

Minutia.

3 H(?) 4 X N U Dont see the higher cognitive level in this Q. Should probably be an F.

I dont think B is credible. C and D both reference procedures and A and B do not.

4 H 3 X X M E Add per procedure XXX or per plant procedures in stem.

Need to revise stem to make it clear the pressure spike was right after MSIV closure and is not indicative of pressure at the 10 minute mark of the question. Otherwise, none of the answers are correct based on TS lift settings.

5 H 2 X B E Distractor B does not appear credible. With 221 days of operation, what else could remove decay heat if no SRV was cycling with MSIVs shut?

Why is there a difference between the number of valves cycling for Questions 5 and 6?

B and C not credible. Why would the one NOT controlled stop cycling?

6 H 2 X M(?) U Essentially same as the original Q. I dont see any significant modification here.

Multiple correct answers per explanation. B is also correct because overfill can happen (even though its a secondary reason, it is still correct). D is also correct 7 F 3 X X N U because it is means of precluding stratification even if there are other ways to do so.

What provides for CRD motor cooling?

Add pump before motor in distractor A. Add per procedure XXX in stem.

Radwaste not inside drywell, why would someone think closing this would isolate the leak?

8 H 2 X X M U Add per procedure XXX in stem. Distractors C and D align an operable system with a faulted one (do not appear credible).

9 H 2 M S 10 F 2 N S 11 H 4 X B E Add per procedure XXX in stem.

Distractor D not credible. Why would flow increase if backpressure increases without 12 H 3 X B E turbine speed change?

Distractor B is not credible with the reference provided. I dont think the reference is 13 F 3 X X B E necessary. Add per procedure XXX in stem.

Its a direct lookup with the references as listed. Provide graph 9 but not the EOP3A 14 H 1 X M U sections. Add per procedure XXX in stem. Page 2 of 11 OBDI 202 - IOLE Process

ES-401 CNS-2015-02 3 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only K/A mismatch - answer is not on low level AND temperature. Change question to make NPSH more limiting. Also need more information to show calculation for 15 H 3 X X B U determining overpressure. I did not understand it based on the information provided.

NRC 2003. In stem directed to BOP should be directed the BOP. Add per procedure XXX in stem.

Give actual water level in stem, then add per procedure XXX, what group 16 F 2 X B E isolation(s) Put a water level in stem that changes the correct answer and make this a modified bank question.

Distractor A is a subset of Answer B, making it not credible.

17 H 3 X X X N E Need to add per procedure XXX, when MAY a normal depressurization commence.

Only answer that isolates offgas is the correct answer - makes correct answer 18 H 2 X N U obviously the only plausible distractor.

Distractor B is not credible. Distractor D doesnt make sense as written. How can 19 F 3 X X N U FW pumps keep operating if DC lube oil pumps only provide lubrication during coast down?

Stem of Q cues applicant that the problem is with the voltage regulator, rendering distractors A and B implausible (Why would you ask about voltage regulator operation unless the problem is with the voltage regulator?). D is not credible. If the voltage regulator is the one causing the problems, why would anyone want to switch 20 H 3 X X X N U it ON?

Also, stem should not ask what is causing the above indications, but which of the following should be causing the above indications. Also, what is the relevance of 70% reactor power in stem?

The first half of this Q asks for a simple location which is better tested on a scenario or JPM; not really appropriate for a written. Also, the second part appears to have no correct answer. The procedure (2.3 B-1) directs a scram if vacuum cannot be 21 H 3 X X X N U maintained greater than or EQUAL to 23. The stem indicates that condenser vacuum is equal to 23, at which time a scram would not be required (rapid power reduction more appropriate?). Also, what is the relevance of full power in winter months in the stem?

22 F 2 X N E Distractor A is not credible.

Distractor B not credible. How can you have shorter scram times without CRD 23 H 2 X B E pumps?

24 F 3 X X N U Distractor B basically says No ED, while Answer D says anticipate ED. Because these two are opposing, it indicates that one of the two is correct, rendering A and C Page 3 of 11 OBDI 202 - IOLE Process

ES-401 CNS-2015-02 4 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only implausible. Also, add per procedure XXX in stem.

Capitalize ONE, ONLY, and BOTH in answer choices. Reword C and D (delete ; and 25 F 2 X X X M E replace with AND). Wheres original bank question?

Also, add per procedure XXX in stem.

Revise second bullet due to bundle. Distractor B not credible. Why would 26 H 4 X X N E radiation rates go down following an accident?

Subset issue with A and C, B and D. Modify stem to add MINIMUM hydrogen concentration. Also change A and C to exceeding ODAM limits is NOT allowed 27 F 3 X X N U for clarity.

Need to modify stem to ask for the MINIMUM hydrogen concentration that requires action.

In stem, change does to should. Verify correct answer as it indicates the pump will 28 H 3 X N E provide flow at its shutoff head(?)

Add per procedure XXX in stem. There is nothing in the stem that would suggest any temperature indication is inaccurate. The student must assume normal 29 H 3 X X N U indications unless otherwise indicated in the stem. This renders distractors C and D implausible. Why would the student think theres an indication issue?

30 F 2 X N E Add technical specification required action to stem question.

31 F 2 N S Delete RFPT control goes to MDEM. This requires the operator to diagnose the 32 H 3 N E condition and improves distractor credibility and cognitive level of question.

33 F 2 X M E Distractor D contains cuing. It is the only non-MCC answer option. NRC Exam 2014.

Not sure about credibility of distractors A and D. How are 4 breakers causing 1/2 scram or 2 breakers causing full scram credible?

34 H 2 X?  ? M  ?

Are distractors A and D stand alone false statements? If so, they could be eliminated without reading the stem at all.

35 H 3 M S 36 H 4 B S APRM B is an option in all but one answer choice. Distractor B is the only one to not 37 F 3 X X B E contain APRM B, which is cuing. IRM H is not used as a possible answer choice.

38 H 3 X X N E Distractor D is not credible. Why would someone choose to make up to the ECSTs Page 4 of 11 OBDI 202 - IOLE Process

ES-401 CNS-2015-02 5 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only when action is based on torus level? Revise first part of answers for clarity.

Only correct answer credible. Three answers refer to indications while one refers to 39 H 3 X X M U an actual parameter, which cues the correct answer.

Someone could argue about when reactor pressure is < Gr.1 setpoint with switch still in run. Need to clarify in stem that reactor pressure falls after the switch is out of 40 H 3 X X N U RUN, or A could also be argued to be correct (as if reactor pressure goes down, so does steamline pressure).

Q doesnt appear to test knowledge of surveillance procedures, rather system 41 F 3 X X? N  ?

knowledge regarding SRVs. How does question test K/A? Also C is not credible.

42 F 2 X N E B is not credible.

43 H 2  ? N  ? How does question test K/A?

44 F 2 N S NRC 2012. Description says that distractor D is possible, so why is it wrong? Should 45 H 3 X X? B E probably specify with no operator action in stem.

Where in technical reference does it specifically list concern (due to fire)? If its located in the technical reference, need to add this to the procedure stem to ensure 46 H 2 X M  ?

only one correct answer. If not specifically listed somewhere, then the question is Unsat due to two correct answers.

Modify stem to clarify the question is looking for the FIRST item that would trip the diesel. In stem, delete (no oil flow) as it provides unnecessary cueing. Also, 47 F 3 X N E change what condition automatically trips to what condition should automatically trip.

Add statement that the system is in its normal cooling water lineup since REC is a backup source to B and C pumps. At what pressure do the cross-tie valves fail?

48 H 3 X N E In stem, change what condition automatically trips to what condition should automatically trip and change what automatically occurs to what should automatically occur.

B and D not credible. If non-critical loads automatically isolate, why would you need to isolate one? Also, who wouldnt try to start another REC pump when one trips?

49 F 3 X X N U Also, in stem (1) how does the system respond should be how should the system respond.

Need more information to determine question suitability. How was the percentage 50 H 3 X X  ? N  ? given in the explanation calculated (or did the value come from a specific reference)?

Based on the tank volume and % value, I determined the approximate gal per % Page 5 of 11 OBDI 202 - IOLE Process

ES-401 CNS-2015-02 6 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only level. Based on the pump gpm and 20 minute time frame, I calculated 23% level change, not 26%. How was 26% calculated? Either provide calculation or specific reference. Because 26% is the HSBW, then this could be the difference between a SAT and UNSAT question.

In stem, change what is the approximate to what should be the approximate. In distractors A and C, can we come up with a better term than NOT downscale?

How is it credible to have a rod block and/or scram while still on scale? Also, per 51 H 4 X X X N U lesson plan, Range 7 should be the (0-40) scale, not the (0-125).

52 F 2 X N E Distractor B is not credible.

53 H 3 X N E In stem, change is the effect to should be the effect.

54 F 3 X N E In stem, change is the flow to should be the flow.

Q is probably OK but poorly structured. When you have cascading distractors, the 55 H 3 X N S correct answer is invariably somewhere in between the two extremes (distractors A and D), which is the case here.

LOD = 1 due to the stem providing the answer.

56 F 1 X N U The stem asks about recirculation and the correct answer is the only answer that refers to recirculation.

57 F 2 N S 58 F 4 M S Distractor A refers to torus spray when the question is asking about drywell 59 H 3 X N E temperature and is not credible. Why would anyone pick A over B?

If there was no time delay, the question wouldnt be asking about it. This renders 60 F 2 X N U distractors A and C implausible.

Question 63 states in its stem that reactor pressure is 980 psig with the plant at 65%

power. This question states in its stem that reactor pressure is 940 psig at 25%

61 H 2 X X N U power. This cues the applicant that reactor pressure is greater than 980 psig at 100%

power. This renders distractors A and C (reactor pressure 958 psig at 100% power) implausible.

Distractors A and C not credible. An isolation valve going closed means isolation.

62 H 3 X N U So why would an operator think that isolating a feedwater heater would cause temperature to go up?

Note that the stem of this question answers half of question 61, rendering that 63 F 3 N S question UNSAT. Page 6 of 11 OBDI 202 - IOLE Process

ES-401 CNS-2015-02 7 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Need more information to understand system operation. When does the 15 minute 64 H 3 X? N  ? time delay occur and when does it not? The explanation section is not clear. How does this question match the K/A?

In stem, change does to should. None of the answers make any sense except the correct answer. In distractor A, why would dampers ever auto open on a fire?

65 F 2 X X N U Distractor C appears to be a normal ventilation lineup with no fire. In distractor D, why would supply fans continue to run with dampers isolated? Three implausible distractors.

Distractor A and B not plausible. Why would you remain at the valve unless you 66 F 2 X N U needed to be there to close it? Also, C and D arent grammatically correct (should be close the valve in event of an accident when instructed or similar wording).

67 F 3 B S How is Distractor D credible? Also, the three distractors are temperature readings 68 F 3 X X X N U while the answer is a pressure indication, cuing the correct answer.

Switch indication and location in stem.

69 F 3 N S 70 F 3 N S Need technical reference to fully evaluate question.

71 F 2 X N  ? Reword beginning of stem to read: Which of the following is the LOWEST accumulated dose ABOVE which. Also capitalize REQUIRED. Add procedure reference to stem.

Distractor A is the only one without a monitor scan in it - cuing. Also, a whole body 72 F 2 X X B U frisk is the same thing as a whole body scan, making B and C basically the same, leaving D as the only plausible answer.

Make question more discriminatory by making the dose rate at 1600 mrem/hr and asking if RPM (designee) approval is required.

73 F 2 N  ?

Is an SWP also considered an RWP? Because if so, then theres multiple correct answers.

74 F 3 X N E Add per procedure XXX in stem.

75 F 2 X N E Add per procedure XXX in stem.

There is nothing in the stem to indicate that there is anything that may be time critical 76 H 3 X X X N U (i.e. non-complicated automatic scram). Distractors B and D would therefore be implausible. Page 7 of 11 OBDI 202 - IOLE Process

ES-401 CNS-2015-02 8 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Distractor D and answer C are out of order. Also, there are directions on swapping heat exchangers using 5.2SW, so what would make D incorrect? Although entry criteria are not met, the overall administrative guidance specifically states that the 77 F 3 X  ? X M(?) ? entry criteria are not all inclusive and cant cover every situation, so why couldnt the SRO choose this procedure and the step to swap HXs to combat the transient?

Also, this looks like the same question from the 2011 exam (I dont see any substantive modification). Need to review modification guidance in NUREG-1021.

If the entry condition for IA pressure loss was required at 84 psig, then it would also be required at 75 psig. That means if 84 psig was the required entry condition for IA 78 H 3 X X X N U pressure loss, then there would be multiple correct answers. This eliminates 84 psig as a possibility, and therefore renders distractors A and C implausible.

Was the EAL sheet allowed as a reference on the 2011 exam? Because this 79 H ?3 X B  ?

question appears to be a direct lookup with the reference as provided.

Written exam questions typically do not ask about procedures that are not applicable.

80 H 3 X X N U Therefore, distractors B and D are not credible. Why would anyone pick no if the stem specifically asks about a procedure?

Written exam questions typically do not ask about procedures that are not applicable.

81 H 3 X X X N U Therefore, distractors C and D are not credible. Why would anyone pick no if the stem specifically asks about a procedure? Eliminate one from the stem.

Need to reword stem because pressure could have gone <68psig. We only know for sure that it went at LEAST that low (could have gone lower though). Need the TRM 82 H 3 X X X N  ? and bases to finish evaluating question.

TRM 3.11.2 should not be needed to answer this question (eliminate reference). In stem, capitalize MINIMUM.

Eliminate one in stem. Direct lookup with tech spec reference 3.4.10 provided and therefore is LOD = 1. Also, this question is fundamentally flawed. The stem has reactor pressure rising at 5 psig per minute. The stem also asks what would happen 83 H 1 X X X N U if these conditions persist. If reactor pressure continued to rise at 5 psig per minute, it would be 3675 psig by 2034 (safety limit is 1337 psig). Therefore, the reactor would have automatically tripped and would have been in Mode 3 long before any of the distractor answers.

84 H 3 X N S 85 H 4 X N S What is RHR pressure with the pump operating? Need to add this to the explanation 86 H 3 X X M E to show why the fire protection wont inject under these conditions. Page 8 of 11 OBDI 202 - IOLE Process

ES-401 CNS-2015-02 9 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only In stem in (1) change is to should be. In (2) change is to would be procedurally.

Probably OK - but open reference tech spec questions are normally evaluated on requalification written exams and during initial exam scenarios (or in some cases an 87 H 4 X N S admin JPM). These type of questions should be used minimally (or best case not at all) on initial written exams.

Subset issue - Per the question stem, it only asks for required TS actions. One doesnt just enter the most restrictive TS, but also all applicable TSs. This TS also allows separate entry for each condition. So, for an IRM inoperable, would always enter Condition A for that monitor. If required, condition B would also be entered. So if C was true, A would also be true, eliminating C as an answer (same is true for B 88 F 3 X X X X N U and D). Also, the explanation section is incorrect. If the IRM was in a different train, then each train would meet the minimum number of operable channels (3) and no TS entry would be required.

This Q essentially asks if IRM A and IRM G are in the same train. The tech spec portion is simply a direct lookup if TS 3.3.1.1 is provided to the applicant. Eliminate providing the tech spec reference.

No indication in the stem that actions were taken to stabilize the transient at the provided levels. In that case, without operator action, the trip setpoint could be 89 H 1 X X X N U exceeded prior to the TS allowed outage time expiration (no quantifiable rate increase provided). Also, by providing the TS, this becomes a direct look-up with an LOD = 1.

Reword (2) in stem to delete first required. No need to determine applicability as stated in explanation because only 3.8.1 provided. Per TS, one doesnt just enter the most restrictive, but all applicable action statements. Therefore, one would enter both Conditions A AND D, making B(2) and C(2) also correct, as it is a required TS 90 H 1 X X X X X N U action (just not the most limiting). Also, the applicant is told in the stem that the diesel is inoperable. Its not credible to think the applicant would not identify this as its given in the stem. Finally, with the TS provided as a reference, this would be a direct lookup and thus LOD = 1.

Reword stem for clarity. I had to read the explanation section and the procedure to 91 H 3 X X N E figure out just what the stem was even asking. At the least, replace performed with tested. Eliminate one in stem.

How is this SRO level knowledge? The first part of the question can be answered

?

with system knowledge and the second part is an immediate operator action, which 92 F 3 N  ? ROs are required to have memorized. Even though the question references the ARP X specifically, its the same as the AOP immediate operator action and thus the RO should also know the immediate scram RX level. Also need to reword answers to Page 9 of 11 OBDI 202 - IOLE Process

ES-401 CNS-2015-02 10 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only more closely match procedure step 4.3.2.

Need technical reference to evaluate question. Also add and to first bullet of stem.

93 H 3 X X X N  ?

In stem, (1) should read What should be the CRS response.

Need references to evaluate question. Also, clarify where the 4 shift requirement is coming from. The CFR does not require 4 shifts, it requires a minimum number of 94 F 2 X X X N  ? hours, spread out however. Provide the procedure reference in the stem.

Eliminate one in stem.

Need references to evaluate question.

Add per procedure XXX in stem. Put REQUIRES in caps in stem. Is the refueling 95 F 3 X X? X N  ? floor inside the refueling area? If so, distractor B is also a correct answer. Does the refuel floor supervisor have the authority to deny someone under vessel access if he/she deems it unsafe? If so, A is also correct.

Per 3.4.4 Step 4.1.3, a timely 10 CFR 50.59 should be performed. Should is not shall, so per procedure, it could go past the 90 days without violating the procedure (not a requirement). The only requirement has to do with if the TASM is 96 F 3 X X N U expected to last longer than 90 days, then the 50.59 shall be done before the TASM is implemented.

Add per procedure XXX in stem.

Even a less than minimally competent operator could recognize that SROs care about power reductions and not parts availability. B is also not credible, leaving A as 97 F 1 X X X N U the only credible value. This question then does not discriminate between a minimally competent and less than minimally competent operator and is UNSAT.

Eliminate one from stem.

This is supposed to be a Generic Tier 3 question, but this question uses system specific knowledge to answer the question, making it an extension of Tier 2. See 98 F 2 X X B U NUREG-1021 ES-401 D.2.a. This, therefore, does not match a Tier 3 K/A and is UNSAT.

Eliminate AND from stem. I dont see the relevance of the alert information in the 99 F 2 X X M E stem.

Need reference (O-EN-HU-106) to evaluate question. A and C are not credible, as by its name, an EOP would seem to have precedence. Also, the bank is Q43 on the 100 F 2 X X  ? M U 2011 exam, so how is this question SRO level question when it was considered RO level in the past? Also, eliminate one from stem.

RO TOTALS: B= 13 F= 38 E= 27 Additional Notes: 8 Questions need more information to determine whether SAT or Page 10 of 11 OBDI 202 - IOLE Process

ES-401 CNS-2015-02 11 Form ES-401-9 M= 12 H= 37 U= 27 UNSAT.

N= 50 B= 2 F= 10 E= 3 Additional Notes: 7 Questions need more information to determine whether SAT or SRO TOTALS: M= 4 H= 15 U= 12 UNSAT.

N= 19

GENERAL COMMENT

S:

1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N

2. Chief Examiner comments are indicated in blue.

3. Average difficulty is 2.64__ on the RO exam and 2.58_ on the SRO exam.

4. The 10CFR55.41/43 distribution is: RO / SRO 41.1 = 0 43.1 = 0 41.2 = 0 43.2 = 6 41.3 = 0 43.3 = 1 41.4 = 1 43.4 = 1 41.5 = 11 43.5 = 16 41.6 = 2 43.6 = 0 41.7 = 32 43.7 = 1 41.8 = 3 41.9 = 1 41.10 = 21 41.11 = 1 41.12 = 2 41.13 = 0 41.14 = 1

5. The answer distribution is: RO / SRO A = 21 (28%) / 6 (24%)

B = 18 (24%) / 6 (24%)

C = 19 (25%) / 6 (24%)

D = 17 (23%) / 7 (28%)

6. There are 12 questions with attachments provided. Page 11 of 11 OBDI 202 - IOLE Process