ML12297A443

10 Draft Written Exam Comments

ML12297A443
Person / Time
Site:	Cooper
Issue date:	10/30/2012
From:	Sean Hedger Operations Branch IV
To:	Nebraska Public Power District (NPPD)
laura hurley
References
Download: ML12297A443 (13)

Text

ES-401 CP-2012-10 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only A, CR, 41.8 - The explanation for distracters C and D dont explain why they are plausible. Could they think that a rise in RPV pressure causes a rise in RHR/LPCI E pressure, which causes increased flow? Is there some RHR/LPCI pump design 1 H 2 X X N rating that could be altered here to cause increased flow?

S Questionable as to whether this meets the K/A intent of loss or malfunction of LPCI.

The question was revised to address comments.

2 H 3 B S B, CR, 41.7 E C, CR, 41.7 - For distracter D, specify which control switch is being referred to.

3 H 3 X N S Revision made.

4 H 3 N S A, CR, 41.7 5 H 3 B S D, CR, 41.5 6 H 3 N S C, CR, 41.7 Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

$ The answer choices are a collection of unrelated true/false statements.

$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

$ The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met). Page 1 of 13 OBDI 202 - IOLE Process

ES-401 CN-2012-10 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 7 H 3 B S D, CR, 41.5 8 H 4 N S B, CR, 41.5 9 F 3 B S B, CR, 41.7 10 H 3 B S A, CR, 41.2 C, CR, 41.7 - It is difficult to see that distracter B is credible when the stated purpose E

11 F 2 X N of a source range monitor is to monitor the power range.

S Distracter B revised to enhance plausibility.

D, CR, 41.7 - (Editorial) Add the before distractors A and B. Recommend modify stem to read What positions must the associated meter function switches be placed E

12 H 3 N in to allow the control room operator to read the LPRM output for the above S illuminated indicating light?

Revised.

B, CR, 41.3- (Editorial) Recommend saying and into what feedwater line in the E

13 F 2 N question.

S Revised.

14 H 3 X N S C, CR, 41.5 15 F 2 B S D, CR, 41.7 16 H 3 N S D, CR, 41.7 17 H 3 X N S A, CR, 41.5 D, CR, 41.7 - Is there a way that the RFPT A speed probes can affect the operation of RFPT B? If not, then it isnt plausible with the given stem that the RFPT B speed control could change (distracter A).

E 18 H 3 X N Is it plausible that losing the speed sensor input for a pump would lead to a total loss S of control for the pump? (distracter B)

Question revised to add speed probe failures for RFPT B as well. Text about total loss of one of the RFPTs removed.

C, CR, 41.5 - The question doesnt address the (a) portion of the K/A statement (predicting the impacts of high containment pressure on the SGTS).

U Justify why distracter D is plausible. It says to place the preferred component in 19 H 2 X X N S standby, place the non-preferred component in run.

The question cue says that drywell pressure is going up, and within 5 minutes the drywell is being aligned to vent it out. Do site procedures direct venting a potential Page 2 of 13 OBDI 202 - IOLE Process

ES-401 CN-2012-10 3 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only LOCA through to atmosphere prior to understanding the cause and hopefully setting containment?

Revised to address K/A matching and issues discussed.

20 H 4 M S C, CR, 41.10 21 H 3 B S D, CR, 41.7 22 F 2 N S D, CR, 41.7 C, CR, 41.5 - The K/A statement focuses on knowledge of the operational implications (what effect does it have on operations?) of hydrogen generation during U battery charging evolutions. The question asks what damaging constituent can come 23 F 2 X N from battery charging, not what happens if hydrogen concentration isnt controlled, or S

asking what it is monitored at and why.

Question revised to address the K/A statement.

24 H 3 N S A, CR, 41.7 C, CR, 41.7 - Automatic can mean that a component moves by itself based on what it senses. Someone could argue that Distracter B is correct because as header E pressure goes down, the regulating valve automatically goes closed. Recommend 25 F 3 X B looking at air pressure control set points to see if these can be added into the S

question to make distracters absolutely incorrect.

Question revised and issues are addressed.

B, CR, 41.7 - This question can be answered correctly by knowing only the power E supply to 1A and 1B OR 1C and 1D. It doesnt test their knowledge of all four power 26 F 2 X N supplies.

S Revised and addressed.

B, CR, 41.6 - When rods are driven inward, drive flow is momentarily 8 gpm, then it settles to 4 gpm. This means, during the rod movement, indicated drive flow is 8 U

27 F 2 X N gpm, 6 gpm, and 4 gpm, depending on when you look at the gauge. The way the S question is constructed right now, there are 3 correct answers.

Revised to remove the multiple correct answers, providing one correct answer.

28 F 3 N S B, CR, 41.7 C, CR, 41.6 - From reading the explanation for distracter D, it sounds like it could be E partially correct. Is there anything that can be added to the stem to ensure that it is 29 F 3 X B completely incorrect?

S Revised and resolved. Page 3 of 13 OBDI 202 - IOLE Process

ES-401 CN-2012-10 4 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 30 H 3 N S C, OR, 41.10 31 F 2 M S A, CR, 41.5 D, CR, 41.7 - It appears an applicant could answer this question correctly with knowledge of what you can see after rod 34-27 movement alone. Knowing what E

32 L 4 X N RBM A meter reads after rod 26-37 is selected is not necessary to get the answer S right.

Revised to address comments.

33 H 3 B S A, CR, 41.7 B, CR, 41.7 (Editorial) Add the system name to the K/A statement - 226001 34 F 2 N S RHR/LPCI Containment Spray Mode Revised A, CR, 41.7 -In the question, change effect to effect(s). This cues the applicant E

35 F 3 X B that the correct answer only results in one change the way it is.

S Revised D, CR, 41.7 -Distracter C doesnt appear to be plausible - that the main turbine can E

36 F 3 X B be in a situation where it cant be tripped either in the control room or locally.

S Comment addressed.

C, CR, 41.4 - The K/A statement tests their ability to predict the impacts of the situation, as well as what procedure direction is correct. To predict the impacts of the TG hood exhaust temperature alarm on future operations, the operator needs to show how to determine if this is an alarm error, or if there is an actual high hood exhaust temperature, which demands actions to lower the temperature. The question U doesnt test the applicants ability with respect to the (a) portion of the K/A statement.

37 H 3 X X N S For distracter D, the explanation doesnt match with why the answer is wrong.

Lowering condenser backpressure/raising main condenser vacuum is an approved action, but opening a vacuum breaker valve does the opposite of this stated action, and it is not spelled out in the procedure.

Question revised to address the K/A statement and other comments.

38 H 2 B S D, CR, 41.7 39 H 3 B S C, CR, 41.2 - (REPEAT Question from 2011 Written Exam - Q 38) 40 H 3 N S C, CR, 41.5 41 H 3 N S A, CR, 41.10 Page 4 of 13 OBDI 202 - IOLE Process

ES-401 CN-2012-10 5 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 42 F 3 N S A, CR, 41.10 43 H 2 N S C, CR, 41.7 B, CR, 41.5 - The procedure cited tells what actions are taken. It is not a reference E for why actions are taken. The basis document for the procedure needs to be 44 F 3 X N checked to make sure the listed reason is correct.

S Revised explanation to address.

45 H 3 M S D, CR, 41.7 A, CR, 41.7 - Does the running of SGTS have any effect on HPCI operation? The explanation for distracter D says it is wrong because of what procedure is being E

46 F 3 N implemented, which technically it looks like two are being implemented. Be clear on S the basis for distracter D.

Revised distracter to address issue.

47 H 3 B S D, CR, 41.7 48 F 2 N S A, CR, 41.7 B, CR, 41.7 - Is there something that happens during this transient that changes in E one minute that could plausibly cause conditions to stabilize? Affects distracters A 49 F 2 X N and C.

S Revised to address issue C, CR, 41.10 - The K/A statement testing the knowledge of SRV tailpipe temperature/pressure relationships and their effects on operations. This question checks that the applicant knows what the expected conditions are, but it doesnt test knowledge of what effect given pressure/temperature indications have on what U actions are required moving forward.

50 H 4 X X N B is a subset of A. D is a subset of C. Someone could argue that D is correct S

because those indications are partially correct.

MAB/Sean: Add only in distracters b and d. Implication of operation in the future.

SDH Response: New question is satisfactory. Ensure that the applicants have copy of the steam tables available so that they can use the Mollier diagram if needed.

C, OR, 41.10 - Recommend changing the 156F distracter to 140F. This ensures that E additional knowledge is tested to come up with the answer, vice it being a direct look-51 H 3 X B up on the graph.

S Revised to address comment.

52 H 4 X B S C, CR, 41.5 Page 5 of 13 OBDI 202 - IOLE Process

ES-401 CN-2012-10 6 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 53 H 4 B S B, OR, 41.7 54 F 3 N S B, CR, 41.5 A, CR, 41.5 - Lowering the RPV level reduces power and reduces feed sub-cooling, mitigating thermal-hydraulic oscillations which could cause local overpower conditions, which could cause fuel damage. Justify why distracters B and C could not E be argued as being correct as well.

55 H 2 X B S Observation: Question ID 14478 used in 2011 exam, but question is different.

Revised explanations to support plausibility of distracters B and C. Revised the question to remove any ambiguity on the difference between answers A and B.

D, CR, 41.5 - The basis for distracter C is essentially it isnt correct because it isnt E

56 F 3 X B the correct answer. Verify why the answer is incorrect and document it.

S The Explanation for distracter C was modified to address the comment.

C, CR, 41.7 - Alter the stem wording to say potential hazard(s). They way it is E worded, it cues the applicant that there is more than one hazard involved, so he/she 57 F 2 X N can automatically rule out distracter A and B.

S Revised text to address comments.

B, CR, 41.7- The question tests knowledge of EOP entry conditions following a U generator/grid disturbance, but there is no test of knowledge of generator/grid set 58 H 3 X N points and interlocks.

S Question revised to address the K/A statement.

D, CR, 41.7 - According to procedure 2.4VAC, the first immediate action is to reduce power per procedure 2.1.10 to restore vacuum. There is no contingency on this step E that says if power is above/below a certain level, the step is N/A. Therefore, that 59 H 3 X B immediate action is required. Answers A and D are both correct.

S Distracter A revised to tie answer to a specific type of down power evolution that is not allowed by procedure in this case.

A, CR, 41.5 - Clarify whether the reactor is scrammed or shut down in the stem.

E The explanation given says that SRVs D and F are closing within their allowable 60 F 3 X B S value. The closing range s 855-895 psig, but the indicated low pressure is 825 psig.

Revised Explanation to support the basis for the correct answer.

C, CR, 41.3 - The provided references do not support the RPV level threshold of E concern being 48. Needs to be verified.

61 F 3 X N S (Editorial) Mark that this is a New question on the form.

Revised question relating carryover to high RPV level turbine trip and provided the Page 6 of 13 OBDI 202 - IOLE Process

ES-401 CN-2012-10 7 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only proper justification.

62 F 2 N S B, CR, 41.10 E B, CR, 41.5- Edit the question to say what is correct action per procedure(s).

63 H 3 X N S Revised to address comment.

64 H 4 B S C, CR, 41.7 D, CR, 41.9 - Clarify what above and below mean in the questions context.

65 F 3 N S Clarified as more negative and less negative as appropriate.

66 F 2 B S A, CR, 41.10 C, CR, 41.7 - To make this a test of generic abilities and not system specific for a given alarm card response, an alarm should be provided that has all of the options E provided (alarm name, Tech Spec, EOPs, automatic actions). The applicant, based 67 F 2 X N on what procedure 2.3.1 knowledge, answers as to what is required to be said.

S Reference NUREG-1021, ES-401, Section D.2.a, 1st paragraph.

Revised the question to better match the intent of a generic K/A statement.

68 F 3 N S D, CR, 41.10 A, CR, 41.5 - If the procedure says the operator is to take conservative action, why state in the correct answer to take conservative action and.. Unnecessary cueing of the correct answer.

E There is nothing in NPP 10.13 or procedure 2.1.1 that supports the reason that 69 H 3 X X B distracter C is incorrect. Need review of procedures to justify that it is incorrect.

S Revised to address cueing comment. Added more text to Explanation to show that distracter C because the rods needed to conduct this type of down power are not assigned by procedure until after a full reactor startup has taken place. Therefore, distracter C is plausible but incorrect.

C, CR, 41.10 - If the expectation is that the applicant applies LCO 3.0.6 knowledge to this to eliminate distracters, LCO 3.0.X series items are SRO knowledge.

The explanation implies that SE fan coil unit is a support system for the Core Spray ECCS sub-system (supported system). For LCO 3.0.6 to be applicable, both the U fan coil unit and the Core Spray sub-system have to have separate specific LCOs 70 H 4 X N S that this would help in arbitrating. The fan coil unit has no individual LCO.

Based on discussion, the question is based on the definition of OPERABILITY as stated in Technical Specifications. Part of that definition is that emergency cooling must be operable. The RO is required to know the definition of OPERABLE and apply it to LCOs. Changes made to remove reference to LCO 3.0.6. Page 7 of 13 OBDI 202 - IOLE Process

ES-401 CN-2012-10 8 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Revisions resulted in a question being provided that had two correct answers, which was unsatisfactory. Distracter D was revised to make plausible yet incorrect.

E D, CR, 41.12 - Revise to clarify reference to procedure 5.8.20 vice 5.8.22.

71 H 3 X N S Revised.

72 F N S B, CR, 41.12 B, CR, 41.10 - The questions focus is on plant system knowledge in the EOP bases, and not plant-wide generic knowledge. Reference NUREG-1021, ES-401, Section E

73 F 4 X B D.2.a, 1st paragraph.

S Replaced question with a generic question addressing multiple plant concerns vice being specific to one system A, CR, 41.10 - Distracter D says the reason it is completed is because low pressure E systems can rapidly be secured with individual control switches. So can the other 74 F 3 X N components. Not plausible as a discriminator.

S Revised to address distracter plausibility.

75 H 3 N S D, CR, 41.10 D, OR, 43.5 - The applicant can know the correct answer to the 2nd part of the E

76 H 3 X N question only and get the correct answer without knowing anything about the 1st part.

S Revised to 2x2 format to resolve comment.

B, CR, 43.5 - The applicant can know who the approval authority for action is and which procedure to use and answer this correctly. No knowledge of the appropriate E

77 H 3 X N protocol for pulling fuses is required. Recommend reducing each answer to two S parts.

Revised to 2x2 format to resolve comment.

B, CR, 43.2 - The K/A deals with knowledge of parameters and logic (use) of EOP U functions. The provided question is a system-specific Tech Spec call for ADS. This 78 H 3 X B doesnt match the K/A statement.

S Replaced the question to match the K/A statement.

79 H 3 N S C, CR, 43.5 D, CR, 43.5 - With instrument air pressure lowering outside the green band, the plant U has met entry conditions for procedure 5.2AIR. Entry into an alarm procedure for a 80 H 3 X N given alarm is common for all operators. This is RO level of knowledge.

S Systems knowledge of what valves IA-MO-80 and SA-MO-81 do and their normal positions (IA-MO-80 normally open) can be used to answer the second part of the Page 8 of 13 OBDI 202 - IOLE Process

ES-401 CN-2012-10 9 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only question correctly. This is RO level of knowledge.

Changed question to make it a SRO only level of knowledge.

C, CR, 43.5 - Unless the situation provided drives entry into both of the procedures listed, but a decision on which has to be entered as priority, this is an RO question. It U appears to be answerable with systems knowledge and AOP entry criteria only, 81 H 3 X X N which are expected for ROs.

S The wording on the cue has incomplete sentences up front as well.

Revised to provide a SRO level question.

82 H 3 N S B, OR, 43.2 A, CR, 43.5 - The K/A statement calls for testing the ability that alarms are consistent with plant conditions. In this question, alarms are received, plant parameters are E provided, and based on that it is given that conditions are consistent with the alarms 83 H 3 X N and they are signs of a displaced jet pump. It is questionable if this meets the K/A S

statement.

Question revised to address concern and to make a more challenging item.

84 H 3 N S D, CR, 43.5 A, CR, 43.5 - Knowing that throttling closed TEC-MO-139 would not improve the situation can be determined based on systems level knowledge [Lesson Number COR001-13-01 (1419),Section II.A.5]. It is also an immediate action in the alarm U procedure 2.3_B-2.

85 H 3 X N S From there, the procedure that is entered can be determined by knowledge of AOP entry conditions (rising H2 temperatures is entry condition to 2.4GEN-H2). This question can be answered correctly solely by RO level of knowledge.

Revised to make question SRO only in nature. Page 9 of 13 OBDI 202 - IOLE Process

ES-401 CN-2012-10 10 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only C, CR, 43.2 - Tech Spec LCO 3.4.7 requires two RHR subsystems to be OPERABLE with one in operation with ecirc pumps not in operation. RHR A is running, RHR C is inoperable, so what is the status of RHR B? This affects whether you are in an action statement already, or you are entering one with the RHR A failure.

The initial comment was intended to clarify for the applicant that RHR loop B was U operable. Since the initial conditions gave that RHR pump was in operation and RHR 86 H 3 X N pump C was de-energized for maintenance, RHR pump B needs to be operable to S meet the LCO 3.4.7 statement. With the revised question, RHR pump B is inoperable. Before anything happens to RHR pump A in the question, the plant already doesnt meet LCO 3.4.7. Since the initial time isnt given before RHR pump As loss, there are no correct answers that can be verified. Unsatisfactory as provided on revision.

After a second revision, the question is satisfactory.

87 H 3 B S B, OR, 43.5 A, CR, 43.2 - This question can be based solely on an applicants knowledge of Tech Spec LCO Action Statements that take 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> or less to complete. This is typically RO level of knowledge. There are greater than 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> actions in this LCO that can be combined with the 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> or less actions so that the K/A statement can be met, while providing a SRO only level question.

Also, since the question is testing knowledge of the correct action to be taken, add something to the question to the effect of what is the required operator action?

U Revised to address initial comments.

88 H 3 X N In the proposed question provided to address the above comments, the submittal S

was Unsatisfactory.: For there to be a correct answer to the changed question, there has to be a condition that the initial action taken was unsuccessful (in this case, taking the RMSS to Shutdown doesnt result in a scram, so heat continues to be input into the Torus causing a temperature rise of 1F/minute - if the scram was successful, the temperature rise rate would be something less than 1F/minute). With this, it could be assumed that the scram was successful (nothing tells you otherwise). Therefore, there was no correct answer.

In a subsequent revision, this was corrected.

B, CR, 43.5 - The K/A statement calls for testing an applicants ability to determine/interpret drywell/suppression chamber differential pressure. With the given question, the applicant can have knowledge of what EOP-3A says (BEFORE PC E

89 H 3 X N Water Level drops to 9.6 feet, scram and ED) and answer this correctly, strictly S based on knowledge of torus water level requirements. Testing of ability to determine/interpret the D/P isnt involved.

Question revised to address comments. Page 10 of 13 OBDI 202 - IOLE Process

ES-401 CN-2012-10 11 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only D, OR, 43.5 (Editorial) Denote on forms whether the question is New, Bank, or 90 H 4 N S Modified.

Revised to address comment.

91 H 3 N S C, CR, 43.5 B, CR, 43.5 - Knowledge of AOP entry conditions (RO knowledge) eliminates two of the answers (no entry into 2.4MC-RF). From there, knowledge of how feedwater heater level control valves are physically configured with the actual heater and how they operate can be used to get the correct answer (RO systems knowledge). This is U a RO question.

92 H 3 X N The knowledge of what the NLO does can be combined with application of 2.4EX-S STM Attachment 2 (graph and time requirements) to come to a SRO question.

The stem says that The RO has lowered power to rated. Does this mean rated power, at what level?

Revised question to test SRO level of knowledge.

C, CR, 43.5 - The question can be answered correctly by knowledge of AOP entry conditions based on power supply knowledge for the hydrogen monitors. Also, the applicant needs to know where hydrogen monitoring indications can be viewed in the control room, which is systems knowledge. This is an RO question.

The question was revised to test SRO level of knowledge. However, the SRO question provided on revision has the following issues:

U Step 1.1 (Purpose) of 5.9SAMG says that the SAMGs are to be entered only if 93 H 3 X X N directed by the EOPs. In the question, the applicant is in EOP 1A, 2A, 3A and 5A.

S Of those EOPs, only EOP 1A has an explicit entry condition stated for 5.9SAMG.

This is for the case when RPV level cannot be restored or maintained above -183 inches. Since this condition is not given as the case in the question, it is not clear how using the procedures drives the applicant into 5.9SAMG. The thresholds stated for DW radiation and primary containment hydrogen levels are not stated on the EOPs. With the given conditions, there is no correct answer (unsatisfactory).

Subsequent revision corrected this issue.

A, CR, 43.5 - The question should state whether the technician has been working consecutive days, what hours his work shift have been (count hours all in one day or E

94 H 3 X M half in two days, etc.). Also, what is defined as the end of the 9 day work period (time S on 6/30)?

Revised to address comment. Page 11 of 13 OBDI 202 - IOLE Process

ES-401 CN-2012-10 12 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only B, CR, 43.2 - The K/A deals with testing the ability to determine equipment operability and/or availability. The question provided tells the applicant all equipment is inoperable and the limiting TS Action Statement time needs to be determined. This doesnt match the K/A statement.

U Remember that this question needs to be focused on plant-wide generic knowledge 95 H 4 X N vice system-specific knowledge (NUREG-1021, ES-401, D.2.a 1st paragraph).

S The given question implies that once RHR C AND HPCI are inoperable, the plant is in Conditions A, C, and D. The condition they are in is fully described in Condition D, so that appears to be what controls the timeline. Answer C appears to be correct.

Question was replaced and determined to be satisfactory.

96 H 2 B S D, OR, 43.5 97 H 3 N S C, CR, 43.4 98 H 4 B S C, CR, 43.4 99 H 4 B S B, CR, 43.5 C, OR, 43.5 - The explanation says that a NOUE never applies, but in the E

100 H 3 X N justification for a couple of the distracters, it says it does. Resolve.

S Addressed in revision.

B= 25 F= 39 E= 28 Additional Notes: Percent Unsatisfactory 9%

RO TOTALS: M= 3 H= 36 U= 7 One repeat question from last two years exams.

N= 47 3 open reference questions proposed B= 10 F= 0 E= 6 Additional Notes: Percent Unsatisfactory 36%

SRO TOTALS: M= 1 H= 25 U= 9 6 open reference questions proposed.

N= 14

GENERAL COMMENT

S:

1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N Page 12 of 13 OBDI 202 - IOLE Process

ES-401 CN-2012-10 13 Form ES-401-9

2. Chief Examiner comments are indicated in blue.

3. Average difficulty is __ 2.72___ on the RO exam and _2.92__ on the SRO exam.

4. The 10CFR55.41/43 distribution is: RO / SRO 41.1 = 0 43.1 = 0 41.2 = 2 43.2 = 4 41.3 = 2 43.3 = 0 41.4 = 1 43.4 = 2 41.5 = 17 43.5 = 19 41.6 = 2 43.6 = 0 41.7 = 33 43.7 = 0 41.8 = 1 41.9 = 1 41.10 = 14 41.11 = 0 41.12 = 2 41.13 = 0 41.14 = 0

5. The answer distribution is: RO / SRO A = 17 (23%) / 4 (16%)

B = 16 (21%) / 8 (32%)

C = 23 (31%) / 8 (32%)

D = 19 (25%) / 5 (32%)

6. There are 9 questions with attachments provided.

Review conducted by S. Hedger and T. Farina Page 13 of 13 OBDI 202 - IOLE Process