ML12157A035

2012 Fermi Power Plant Initial License Examination Administrative Form Es 401-9

ML12157A035
Person / Time
Site:	Fermi
Issue date:	03/30/2012
From:	Bielby M Operations Branch III
To:	Detroit Edison, Co
Shared Package
ML11354A135	List: ML12157A035 ML12157A037 ML12172A189
References
Download: ML12157A035 (14)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 3 X Y N U NRC: Is the postulated scenario physically possible. 40% core flow would require both pumps to runback and the scenario only runs back the North Pump.

Licensee: Add other RR pump tripped to the stem. This puts plant in the scram region.

2 F 3 Y N E NRC:Shouldnt explanation of distracter D state loss of all offsite power rather than SBO?

Licensee: Changed explanation of distracter D to eliminate SBO.

3 H 3 Y B S 1/2x2 4 H 3 NO B U NRC: Knowledge or ability to use computer is not necessary to answer the question. Just simply stating that the information was Y E found on the computer does not test the ability to use the computer.

Licensee: This does match the KA. Information found on the computer, replaced bullets with a table of IPCS points.

5 H 3 Y B S 6 F 2 Y B S 7 F 2 N N U NRC: K/A is related to reason(s) for starting standby pump on loss of component cooling water. Question essentially asks pump start permissives (more related to system 400000 than to this K/A).

Licensee: Agree, does not match the KA. Wrote new question.

8 F 5 X Y N U 1/2 x 2; NRC: How do I know that the Center SAC is still running?

The fact that both CACs are running could imply that the Center E

SAC was lost for some other reason. Need a reason for the pressure drop.

Licensee: There was no indication of any problem with Center SAC, applicant cannot assume there was problem with Center SAC unless so stated. Further clarification was added.

9 H 3 Y B E NRC: Delete due to an electrical fault from fourth bullet. This could be interpreted as being an alarm only (i.e., no change in valve position).

Licensee: Incorporated comment.

1

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 10 H 3 X Y B E NRC: Distracter B is a subset of C. Recommend adding the word ONLY after Div 1 in distracter B.

Need to identify what monitor is associated with Channel 17 on recorder D21-R600.

Explanation of distracter B in the Comments should be more specific (i.e., similar to C and D).

Licensee: Added ONLY after DIV 1 SGTS in B. The monitor associated with channel 17 is identified. Explanation of distracter B in Comments made more specific like C & D.

11 H 3 X Y X B E NRC: Distracter D while credible for SROs, is not so for ROs. ROs are not required to know required actions for inoperability. Suggest replacing and Required Actions Time with but is now OPERABLE.

Licensee: Replaced D. Got rid of required action and completion time, replaced with but is now OPERABLE.

12 H 3 Y B S 13 H 3 Y N U 1/2 x 2; NRC: Not sure that part (2) of the correct answer is accurate.

S TSs require immediate shutdown at 110°F and subsequent cooldown. From the TS Bases: This limitation subsequently ensures that peak primary containment pressures and temperatures do not exceed maximum allowable values during a postulated DBA or any transient resulting in heatup of the suppression pool.

Licensee: 110 degrees is the BIIT. The purpose is so we do not exceed the BIIT.

14 F 3 X Y B E NRC: What difference does it may if DW pressure exceeds the scram setpoint if the Reactor is already shutdown (water level is low). Recommend changing question to Which of the following actions will mitigate the rising Drywell pressure trend?

Licensee: Incorporated comment.

15 H 4 X X Y N U NRC: It seems implausible that operators are required to memorize individual data points on a multipoint recorder.

S Licensee: Operations representative confirmed operators are expected to know these two EOP data recorder points.

2

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 16 H 2 X X Y B U NRC: Multiple correct answers. HPCI, RCIC, and SBFW can all be throttled from the Main Control Room to maintain the desired flow E

rated. Secondly the question should ask which pump combinations instead of systems. Also delete conversion factor from stem. Applicants are provided with steam tables which include conversion factors.

Licensee: Stem states systems operating at rated (ie, not throttled) flow which eliminates multiple correct answers. Deleted the conversion factor.

17 F 3 X X Y B U NRC: Why would an operator be required to memorize this number, since procedures specify required amounts and evolution would not S

be performed without a procedure?

Additionally C is a subset of D.

Licensee: Operators are expected to know the values of HSDB and CSDB weights. C is HSDB and D is CSDB weight; therefore, C not subset of D.

18 H 3 Y B S NRC: LOK=F vice H F

19 F 3 Y B S 1/2x2 20 H 3 X Y B E NRC: Distracter A is not credible. Adjusting voltage has no effect on frequency. Recommend changing to read, Reduce generator F

load to raise 4D132. Revise Comments section as appropriate.

Additionally, recommend removing procedure references in distracters since actions specified could be interpreted as Subsequent Actions which are not required to be memorized.

Tripping the Turbine is appropriate since an AUTO trip should have occurred (<58.2 Hz for 20 cycles).

LOK=F vice H since action is a memorized action.

Licensee: Revised A as follows: Reduce generator load to increase frequency". No auto trip at 58.2 Hz. Based on power and frequency makes LOK=H.

21 H 2 Y B S 3

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 22 H 3 X Y B U NRC: Distracter C is not credible. Initial conditions state that RCIC is isolated.

E I think of shrink and swell as being phenomena related to rapid changes in void fraction caused by changes in pressure or flow. The change in water level will be due to thermal expansion not swell.

Licensee: Removed RCIC condition from stem. Show where thermal expansion is defined as swell. Replaced swell with thermal expansion, and shrink with thermal contraction.

23 H 3 X Y B U NRC: Two correct answers - B is a subset of D F Explanation of Distracters B and C do not adequately explain why the answers are incorrect.

LOK=F vice H since ability to answer question is a function of memorizing Drywell Temperature Limitations.

Licensee: Added only to B to eliminate subset of D. Changed Drywell Coolers to Drywell Cooling Fans in stem and distracters.

24 H 2 Y M E NRC: Stem should state could have caused The change to the stem (power vs megawatts) is insignificant, but the change to the distracters justifies use of the question.

Licensee: Changed stem from caused the increase to could have caused the increase.

25 F 3 X Y B U NRC: ROs are required to memorize EOP entry conditions and not the sequence of EOP actions.

H S The second part of the question (i.e., reason for) is fair game.

LOK=H vice F, analyze set of conditions.

Licensee: Disagree-this is big picture Action for CRNSO and reason is not memorization of EOP step sequence.

26 F 2 Y N S 1/2x2 27 H 3 X Y N U NRC: ROs are required to memorize EOP entry conditions and not the sequence of EOP actions. Stated site learning objective is S

perform EOP/AOP actions as directed.

Licensee: Disagree-this requires a diagnosis of an event and actions that make sense for listed conditions. CRNSOs expected to take actions for fire protection.

28 F 3 Y B S 1/2 x 2; 4

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 29 F 2 Y M E 1/2 x 2; NRC: Stem should read Under what conditions .?

Need to explain how shutting the valve could lead to valve stem failure.

Licensee: Incorporated comment.

30 F 2 Y B S 31 F 2 X Y N U 1/2 x 2; NRC: Distracters A and C are not credible since no reason to believe that Torus level would change in the direction indicated with the suction aligned as indicated.

Licensee: Changed distracters A and C, and explained why they are plausible.

32 H 3 X Y N S 1/2 x 2; NRC: This question tests the ability of the applicant to recognize which answers are wrong to arrive at the correct answer.

To pick the correct answer would require memorization of the pump curve.

Licensee: Operations representative expectation is that operators know the discharge pressures.

33 F 3 X Y N U NRC: Choice C does not explain why SLC is not needed in Mode 3.

S Choice D (identified as the correct answer) does not explain why SLC is not required in Modes 3 and 4.

Licensee: Stem only asking for correct reason for the specified mode, not all modes. D is correct answer for question asked.

34 F 3 X Y B U NRC: The air released with deenergization of the B solenoids (RPS B 1/2 scram) is simply the air trapped between the A and B solenoid S

and would be a very short puff release. Is this really information important to a licensed operator or is it simply trivia.

Licensee: This is basic system knowledge that RO is expected to know. Taught and tested from Student Text.

35 F 1 Y B U NRC: Simple recall of power supply.

2 S Licensee: This is an exact match for the KA with an importance >

2.5, it is fundamental level of knowledge with valid distracters.

36 H 2 Y B S 1/2x2 37 H 2 Y M S Answer Explanation needs to be revised to state that W=53%

5

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 38 H 3 Y B E NRC: Stem should be revised to include some data associated with the suction source temperature (either a temperature value or statement indicating that suction is aligned to the CST.

Comments Section needs to be revised. Comments identify C as incorrect (should probably be D), but C is the correct answer.

Licensee: Corrected Comments Section answer.

39 F 3 X Y B U NRC: Multiple correct answers. C and D are both correct. Flow element is upstream of the split to the RPV Injection line and Test S

Return line. Flow element will measure flow to either line or combination of both.

Licensee: Correct as written. Added ..for this condition to the stem. Test line must be manually aligned, injection flow path is automatic. Change question from Modified to Bank.

40 F 3 NO M U NRC: K/A is knowledge of ADS design features provided to prevent inadvertent actuation. Question asks when Automatic actuation can Y B S be prevented when operating per EOPs.

Not a significant modification. Answer is essentially the same only worded differently.

Licensee: This does match the KA. Knowledge of Inhibiting ADS using switches is the design feature.

41 F 3 NO N U NRC: K/A asks for impact of a failure or malfunction of primary containment instrumentation on ADS, whereas question relates to malfunction of RPV level instrumentation.

If question is retained (after modification), then C should be reworded to state that ADS SRVs may be operated at least 5 times without operator intervention.

Licensee: Changed instrumentation to DW pressure instruments from RPV level instruments in the stem and added without operator intervention to C.

42 F 2 Y B S 43 H 3 X Y N U NRC: Answer is not supported by the reference and is not operationally valid. The answer is based on a note which does not provide procedural direction. As long as suppression pool cooling is able to maintain pool temperature below operating limits the plant would not be shutdown to reset or repair the leaking valve.

Licensee: Revised correct response D to make question operationally valid.

6

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 44 H 3 Y B S 1/2 x 2; 45 F 3 NO B U NRC: K/A is knowledge of design features that provide for decay heat removal. Question is related to adsorbtion efficiency of charcoal.

Licensee: Replaced question.

46 H 3 Y M S 47 H 3 Y B S 48 H 3 Y B S NRC: Typo in answer choice C - th(r)ough Licensee: Incorporated comment.

49 F 2 Y B S 50 H 4 X B U NRC: Dont believe there is a correct answer. Answer Explanation doesnt match up with discussion in ST0065 Appendix B.

S Discussion seems to indicate that not all bus lockouts result in prevention of diesel start especially if lockout occurs after EDG receives start signal.

Licensee: Reference 20.307.01, p.2, Note 1. Question is operationally valid and there is only one correct answer.

51 H X X Y M U NRC: Stem does not provide any information (e.g., a leak rate) as to when tank level dropped below the minimum required tank level.

Therefore answer B could be right.

ROs are only responsible for actions that must be completed within one hour. Question asks for Tech Specifications that must be completed. Choices A and B are the same required action, the only difference being the time frame. Since the time frame in choice B is greater than an hour, it is not credible.

Choice D is not credible in that ROs are not responsible for greater than 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> actions.

Licensee: Corrected the stem to indicate when EDG fuel oil leak started and when level was identified as low (0800). Change distracter D to a 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> action. B is 1.25 times the 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> limit (ie, 25% TS grace period) 7

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 52 F 3 X Y N U 1/2 x 2; NRC: No correct answer. East Compressor would eventually trip on high oil temperature.

E Licensee: As written, B is still a correct answer, it doesnt say it will run indefinitely, just doesnt go far enough to include tripping on high oil temperature. Added run until it trips on high oil temperature to B & C choices.

53 H 2 Y B E NRC: Change the stem to ask Which of the following describes the expected response to return the pump differential pressure to the normal band?

Recommend changing F405 will SHUT or OPEN to THROTTLE SHUT or THROTTLE OPEN Recommend changing distracter C to One of the operating TBCCW pumps should be shutdown Recommend changing can to should in distracter D.

Licensee: Incorporated following into last line of stem: which of the following describes the expected response to return the pump differential pressure to the normal band.

A&B. Change SHUT and OPEN to throttling OPEN/SHUT.

Change C to One of the TBCCW pumps should be S/D.

Change D can to should.

54 H Y B E NRC: Distracter B is not credible since stem focuses on control rod drifting out.

Choice D is only partially correct. Rod must be fully inserted first by procedure. Nothing in the question implies that the rod must be fully inserted before D can be performed.

Licensee: B focuses on an incorrect method of determining a stuck collet piston, similar to A and C.

Changed C, added Fully insert the control rod with normal in, and then. Changed D, added Fully insert the control rod with emergency in, and then.

55 H 3 Y B S 56 F 2 Y N E NRC: Recommend changing distracter B to place pumps after NRHX instead of switching order of NRHX and RHX.

Recommend replacing distracter A with a flow path similar to C but with pumps located after NRHX similar to above change.

Licensee: Changed B flow path by placing pumps after NRHX.

Made A flow path similar to C, but with pumps after NRHX.

8

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 57 H 3 Y B S 58 F 2 Y B S 59 F 3 Y N S 60 H 3 Y B S 61 H 2 X Y M E NRC: Distracter D needs to be changed to include a value that is at least on the curve.

Licensee: Changed D to 540 MWe.

NRC: Change last sentence in stem to What is the short term 62 H 3 Y N E impact on the condensate/feedwater delivery systems?

Delete from all answer choices: A decrease in Condensate will cause Delete low suction pressure in A.

Changed distracter C to cause a trip of the Heater Drain Pumps and Change distracter D to reduction in feedwater temperature Licensee: Changed last stem sentence to What is the short term impact on the condensate/feedwater delivery systems?

Deleted from all choices: A decrease in condensate will cause.

Deleted low suction pressure in A.

Replaced a loss of with Trip of.. in C.

Capitalize Reduction in D.

63 H 3 Y B S 64 H 2 Y N S NRC: Need to correct the Answer Explanation. Answer explanation describes the conditions associated with a loss of power to both the Jockey Pump and the Electric Fire Pump.

Licensee: Inserted correct explanation.

65 H 3 X Y M E NRC: Explanation of distracter A states that alarms indicate a dropped rod. This is an inaccurate statement. Alarms indicate an S

uncoupled control rod.

Need to indicate magnitude of power rise.

Licensee: Explanation of A correct as stated. Do not need to indicate magnitude of power rise.

66 F 2 Y B S 9

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 67 H 2 Y N E NRC: Clarify what about the activity is MODERATE; e.g., the work rate for this activity was identified as MODERATE.

S This is an OPEN-REFERENCE question.

Licensee: Activity clarification not needed. A reference will be provided for the applicant.

68 F 2 Y B S 69 F 2 Y B S 70 F 2 Y N E NRC: Answer choices should all be plural or singular and stem revised accordingly (Plant Manager is singular while all other choices are plural.)

Licensee: Incorporated comment.

71 H 2 Y B S 72 F 3 X Y B U NRC: The stem is too vague and the statement must perform is too strong. Visual inspection at the boundary is the normal method, but MOP04 does not exclude entry into the area, in which case all the remaining distracters are true statements.

Licensee: Changed stem from must perform to performs. Made this evolution a green trip ticket item, capitalized and underlined term Normal Day to Day to emphasize specific type of inspection defined by MOP04 to exclude the distracters.

73 F 2 Y B S 74 F 2 NO B U NRC: Question is not related to knowledge of emergency communication systems or techniques for making notifications.

Licensee: Question replaced.

75 H 4 X Y M U NRC: The knowledge required to answer this question is beyond that required to be reproduced from memory on a closed reference test. First the applicant must have memorized the TS release rate limit specified in TS 3.7.5 (minutia or trivia).

Secondly the applicant must have memorized an action level specified in the subsequent actions of an Abnormal Operating Procedure (AOP).

Licensee: Rewrote Question to only calculate the current Offgas Release rate, with three possible calculation errors and a correct answer.

10

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only LOK (F/H) 37/38 = 49%/50% 34/41=45%/55%

B/M/N 46/10/19 47/8/20=63%/11%/27%

UNSAT for RO Only 28/75 = 37% 11/75=15%

1 H 3 Y Y N S 2 H 3 X Y Y B U NRC: Conditions in stem require entry into 29.100.01 Sheets 1 and 2 as well as 20.000.18. EOP actions usually have priority over AOP S

actions. Question does not appear to be focused enough to justify B as the correct answer.

Licensee: Fire in control room require operators to take actions per AOP, B is the first action.

3 H 2 Y NO B U NRC: First part is knowledge of system actuation setpoints and ROs are required to know AOP entry conditions.

Y E Licensee: Revised question to change entry into AOP into a directed action in distracters A & B. Last line of stem, replaced taken with directed.

4 H 3 Y Y N S 1/2x2 5 H 3 Y Y N S 6 F 2 Y NO B U NRC: Not SRO only. ROs are responsible for knowing EOP entry conditions.

Licensee: Modified the question to require SRO recognition and prioritization of EOP entry Conditions/Actions.

7 H 3 X Y Y N U NRC: Not sure you can say that flooding is apparent if the sprinklers are isolated and water level is a foot below the Max Safe S

Operating Level.

Licensee: 8 inches of water in the HPCI room brings in the HPCI Room Flood Level alarm. With 2 feet of water in the HPCI room flooding is apparent.

8 H 3 Y S B S 9 H 2 Y Y B S 11

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 10 H 3 Y Y N S 11 H 3 Y Y N E NRC: Recommend changing LCO completion time to 7 days in distracter A.

Licensee: Made change to distractor A as requested.

12 H 3 Y Y B S 13 F 3 Y Y N S NRC: Will Tech Specs be provided as a reference? If so this may be too easy.

Licensee: TSs will not be provided.

14 H 3 Y Y B E NRC: Is an alarm a protective function? I dont believe so.

Recommend eliminating the mention of the RBM alarm from D Licensee: Deleted RBM alarm from D.

15 H 3 N Y N U NRC: Prioritization implies that more than one item is possible. As stated in the distracters, only one of the choices is possible and the E

second parts of the answer choices are not bases for decision making.

Licensee: A and C actions occur in the procedure. Reworded stem and distracters to be consistent. Changed distractor C to Closed a later action in the AOP. Prioritization action with reason make it H vice F.

16 F 3 Y Y N E NRC: Answer explanation should state that rod insertion and disarming are required (not simply should be done).

Licensee: Revised answer explanation as stated. Stem already stated operating at rated flow. Deleted the conversion factor.

17 H 3 X Y Y M U NRC: Imbedded table should be Table 21 from EOP 3A (ATWS Flooding)

Answer choice C is partially correct. Injection should have been restored when pressure dropped below 392 psig and increased until pressure returned above 392 psig. Simply opening another SRV will not ensure adequate core cooling unless injection is restored. If more than three SRVs could have been opened then additional SRVs (up to 5) should have been open.

Licensee: Change Table 8 to Table 21. C is incorrect because steam cooling vice injection is the cooling medium being implemented. The remainder of the question is SAT.

12

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 18 H 3 Y NO N U NRC: Question can be answer using system knowledge (interlocks). Could be fixed by including statements related to RPV level control actions (e.g., adding statements similar to restore level to control band, then restore torus cooling, or expand level band, then restore torus cooling).

Stem should state The CRNSO subsequently reports Licensee: Added The CRNSO subsequently reports to the stem. Added statement that CRS expands the level band to the stem.

19 F 2 X NO NO N U NRC: K/A is related to temporary management directives. Question is related to operations/management policy for monitoring reactivity changes.

I would think that ROs would also be responsible for knowing when a RMSRO is required.

Per ODE-1, a RMSRO would also be required for B and C. The explanation misinterprets the requirements of MOP19. The referenced statement restricts when the STA/IA can be used to fulfill the RMSRO.

Licensee: Replaced question.

20 F 3 Y Y N S 21 F 3 X NO NO B U NRC: K/A relates to ability to apply Technical Specifications, whereas question is related to knowledge of which instruments input into specific logic trip systems.

Knowledge of specific instrument channels inputs to a logic system is related to system level knowledge which is not SRO specific.

Memorization of specific instrument inputs is beyond that required for a closed reference examination.

Licensee: Wrote new question to identify the tech spec and what action shall be taken.

22 F 2 Y Y B S 23 H 3 Y Y B S 24 F 2 Y Y N S 13

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 25 F 2 Y Y B E NRC: Either move ENTER 29.100.01 Sheet 1, RPV CONTROL to the stem or add additional information in the stem that requires evaluation of entry into another EOP (e.g., Torus Level and Temperature below entry condition with a distracter entering Primary Containment Control.

Having entry into 29.100.01 Sheet 1 for all choices makes the question a low LOK (Fundamental) since there is little need to evaluate the initial conditions.

Licensee: Moved ENTER 29.100.01 to stem.

LOK (F/H) 10/15 = 40%/60% 9/16=36%/64%

B/M/N 11/1/13 11/1/13=44%/4%/52%

UNSAT for SRO Only 9/25 = 36% 5/25=20%

UNSAT Overall 37/100 = 37% 16/100=16%

Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

• The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

• The answer choices are a collection of unrelated true/false statements.

• The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

• One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

• The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

• The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

14