ML17103A263

2017 Clinton Power Station Initial Licensed Operator Examination Forms ES-401-9

ML17103A263
Person / Time
Site:	Clinton
Issue date:	02/09/2017
From:	Chuck Zoia Operations Branch III
To:	Exelon Generation Co, Exelon Nuclear
Shared Package
ML16308A462	List: ML17103A252 ML17103A263 ML17103A279 ML17103A286 ML17103A306 ML17103A318 ML17103A329 ML17103A339
References
50-461/17-301
Download: ML17103A263 (10)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 F 2 Y N S 2X2 2+ H 2 Y M S 2X2 3 H 2 Y B S 4 F 2 Y B S 2X2 (Effectively) 5+ H 1 Y N U 2X2 2 S NRC: LOD=1. Its clear that the reducing loads on 1B is required. Due to the 2X2 format, this results in eliminating 2 distractors. Licensee comments (below) were agreed to at OV.

LIC: The facility contends that the part 2 of the question is not LOD 1 for the following reasons:

• Question validated at 75% (3 of 4 validators answered correctly).

• Validators rated the question as LOD 3.

• There are multiple MCCs fed from 480V Unit Sub 1B (AB MCC 1B1, 1B2, 1B3, and 1B4), any one of which could logically provide power to DC MCC 1B Battery Charger.

Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

• The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

• The answer choices are a collection of unrelated true/false statements.

• The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

• One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

• The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

• The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

NOTE: + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 6 F 3 Y N S 2X2 7 H 3 X Y N U NRC: The stem says that automatic actions are the reason air is maintained to the selected components, but these were simply unaffected by the automatic S closure of the Ring Header isolation valves. The eliminated system was negatively affected by these valves closing. Licensee comments were agreed to at OV.

LIC: Question meets the KA (knowledge of the reasons for the following responses as they apply to partial or complete loss of IA: Service Air isolations) as written. SA header isolations are designed to automatically isolate to preserve air to critical components in the plant.

8+ F 2 Y N S 9 H 3 Y N S Reference provided: CPS 3105.05, Appendix A 10 H 3 Y B S 2X2 11 H 2 Y N S 2X2 12+ F 2 Y N S 13+ H 2 Y N S 14 F 2 Y N S 15 H 2 Y B S 16 F 3 Y B E NRC: Specify why the value given in CPS 4006.01, step 4.6.5, is required to be known from memory. Explanation provided at OV.

S LIC: The natural circulation level value is contained in the precautions sections of CPS 3312.01 Residual Heat Removal and CPS 3312.03 RHR-Shutdown Cooling (SDC) and Fuel Pool Cooling and Assist (FPC and A) and is tied to objective 1.10 of lesson plans N-CL-OPS-290002 Reactor Vessel and Internals and N-CL-OPS-203000 Residual Heat Removal.

17 F 2 X Y N E NRC: The NEITHER distractor (D) is implausible. Explanation provided at OV.

S LIC: The facility contends that distracter D is plausible because both loops of RHR share a common SDC suction line. If the examinee correctly recalls that LPCI is impacted by the SDC alignment, it is plausible to believe that neither loop will realign into LPCI mode.

18+ H 3 Y B S 19 F 2 Y N S 20 H 3 Y N S

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 21+ H 3 Y N S 2X2 22 H 2 Y B S 23 H 2 Y N S 2X2 24+ F 2 Y N S 2X2 25 F 3 Y N S 2X2, Reference provided: CPS 3005.01, Figure 1 26+ F 2 Y B S 2X2 27 H 2 Y B S CPS NRC Exam ILT 10-1 28 H 3 X Y N E NRC: Using distractor C at 1103 psig would be more discriminating, because 1165 psig is not associated with the three SRVs.

LIC: Changed distracter C to 1103. Re-ordered answers/distracters from low pressure to high pressure. Changed answer explanation.

29 H 3 Y N S 2X2 30 H 3 Y B S CPS NRC Exam ILT 10-1 31+ H 3 Y N S 32 H 2 Y B S 33 H 3 Y B S 2X2, CPS NRC Exam ILT 10-1 34+ H 3 Y B S 35 H 2 Y N S 2X2 36 H 2 Y B E 2X2 (Effectively)

NRC: SLIM acronym needs to be defined.

LIC: Added full noun name for the SLIM controller (Small Loop Interface Module) to the answer and distracter.

37 H 3 Y N S 38+ H 3 Y M S 2X2 (Effectively) 39+ F 3 Y B S CPS NRC Exam ILT 9-1

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 40 F 2 Y B S CPS NRC Exam ILT 10-1 41 H 2 Y N S 42 H 2 Y N E NRC: It appears that the procedural aspect of the KA could be addressed; what actions would be required in this situation?

S Question accepted as-is after discussion at OV.

LIC: Asking what procedural actions are required to mitigate the loss of DC would invalidate part 1 of the question. Left question as-is.

43+ F 3 Y N S 44 F 2 Y N S 45 H 3 Y N E NRC: Does Fail Point appear on actual plant indication? Yes S LIC: The meter is identical to the plant.

46+ F 1 Y N U 2X2 2 S NRC: LOD=1. It is obvious lights are dark at 12 oclock. Due to the 2X2 format, this results in eliminating 2 distractors. Discussed and agreed to at OV as indicated below.

LIC: The facility contends that the part 1 of the question does not contain implausible distracters. Novice candidates with limited experience paralleling a DG may select that the DG output breaker is closed when the synchronizing lights are bright (lit). In addition, 4 exam validators rated the question as LOD 2.8.

47 F 2 Y M S 2X2 48 F 3 Y N S 2X2.

49 H 3 Y B S 2X2 50+ H 3 Y N S 2X2 51 F 2 Y N S 52 F 2 Y N S 2X2 53+ H 3 Y B S Reference provided: CPS 4003.01F003 54 H 2 Y N S 55 H 3 Y N S

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 56 H 2 Y M S 2X2 57+ F 1 Y B U NRC: LOD=1. It is obvious that the velocity limiter is the choice among those given. Accepted replacement.

LIC: Replaced the question with a different T2G2 RO question deemed suitable by the CE.

58 H 2 Y N S 59 H 3 Y B S 60+ F 3 Y N S 61 H 3 Y N S 62 F 1 Y B U 2X2 2 S NRC: LOD=1. Its very clear that rod motion is inhibited. Due to the 2X2 format, this results in eliminating 2 distractors. Discussed at OV and considered acceptable.

LIC: The facility contends that the question is not LOD 1. Each control rod contains 2 position channels, only one of which is required to be operable. The examinee may plausibly believe that a single channel failure would not result in a rod block, making A1 and B1 plausible. Validators rated the question at 3.0 (average among 4 validators).

63+ F 2 Y B S 64 H 2 Y B S CPS NRC Exam ILT 10-1 65 H 3 Y B S 66 F X Y B E 2X2, 2010 Ginna ILT NRC Exam H NRC: This would be an H question, because the applicant would have to calculate that > 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> in a 7 day period would be exceeded. Also, the question would be better if asked what would be permitted, versus whats NOT allowed. Agreed to cog change.

LIC: Changed the question to high cog.

The facility also contends that the question meets the KA as written and would not be enhanced by asking what is permitted. Covered employees are required to ensure that any overtime worked does not violate 10CFR26 Work Hour Limits when accepting overtime to cover shift vacancies.

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 67+ F 1 X Y N U NRC: Distractor A is implausible and the entire question is LOD=1 for an ILT student who has performed this task many times.

2 S Discussed at OV and considered acceptable.

LIC: The facility contends that the question is not LOD1 and that distracter A is plausible. The question validated at 50% with an average LOD of 2.5. Additionally, the facility contends that a less than competent examinee may incorrectly believe that the SM can authorize entry into the MCR with no further authorizations required.

68 H 3 Y N S 2X2 69 F 2 Y B S 70+ F 2 Y B S 71 F 3 Y B S CPS NRC Exam ILT 10-1 NRC: Please explain why this is an RO-level question. Discussed at OV and considered acceptable.

LIC: Test item is linked to 10 CFR 55.41(b)(12) Radiological safety principles and procedures. CFR data is already documented in the General Data Table.

72 H 2 X Y B E NRC: Distractor B would be more plausible if the work could be done in 20 minutes. Change incorporated.

LIC: Changed distracter B to 20 minutes as requested.

73 F 3 Y B E CPS NRC Exam ILT 10-1 NRC: Please provide additional information to support LOD and plausibility of the distractors.

LIC: Included an elevation map of the containment to the answer explanation.

74+ H 2 Y N S 75 H 2 Y N S NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection 76 H 3 Y Y N S versus simply actions. SRO-only task referenced.

1 LIC: N/A

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: Distractor A is implausible. Also, 10CFR55.43 (b) (5) for SRO-only 77 F 3 X Y Y N E requires procedure selection versus simply actions. Explanation for 2 distractor A LIC: Modified distracter A to enhance the plausibility. Modified the SRO only justification to explain how the question constitutes a selection of procedures.

NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection 78+ H 2 Y Y N S versus simply actions. SRO-only task referenced.

3 LIC: N/A 79 H 2 Y Y B S 4

80 H 3 Y Y B S 2X2, CPS ILT 14-1 NRC Exam 5

81 H 3 Y Y N S 2X2 6 NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection versus simply actions. SRO-only task referenced for FH component.

LIC: N/A 82+ F 2 Y Y N S 2X2 7

NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection 83 F 2 Y Y B E versus simply actions.

8 LIC: Modified the SRO only justification to explain how the question constitutes a selection of procedures.

NRC: Higher Cog question because it requires figuring out where you are 84 F 3 Y Y B E and where you would be after a LOCA, etc. before applying the TS Bases 9 H information.

LIC: Incorporated change to cognitive level as suggested.

85+ F 2 Y Y N E 2X2 10 NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection versus simply actions. Also, the KA requires the ability to perform procedures versus recalling the actions performed by them.

LIC:

86 H 2 Y Y M S 11

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 87 H 1 Y Y N U 2X2 12 2 S NRC: The SRO-only part of the question is LOD=1, since the choice is limited to whether redundant required features are operable or not.

This, in turn, eliminates 2 distractors.

LIC: The facility contends that the question is not LOD1, because the examinee has to know that the ITS bases considers Div 3 DG redundant to Div 1 and 2 for the purposes of applying ITS 3.8.1 B.2.

This is not normally the case. The validators rated this question as LOD 3.

88 H 2 Y Y N E 2X2 13 U NRC: Replacing one of the OPEN direction ONLY with a new choice, CLOSED direction ONLY, would enhance the discriminatory value of the question.

LIC: Resolved by changing A.1 and B.1 to closed direction only.

Revised the answer explanation accordingly.

Additional Issue - Replaced bypassed with inserted in the stem (there were NO correct answers before this correction was made and communicated during administration of the written exam to ALL SRO applicants).

89+ H 3 Y Y N E 2X2 14 NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection versus simply actions.

LIC: Modified the SRO only justification to explain how the question constitutes a selection of procedures.

90 F 2 Y Y B S 2X2, CPS ILT 12-1 NRC Exam 15 91 H 3 Y Y M E NRC: If distractor D was changed to Scenarios 1 & 2, the choices would be more balanced - the correct scenario would only appear in 16 2 of the choices.

LIC: Changed Distracter D to scenarios 1 & 2 as requested.

Adjusted answer explanation accordingly.

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 92 H 1 Y Y N U 2X2 17 2 S NRC: LOD=1. While its clear that the Refueling Equipment interlocks are INOP, it is more obvious that the One-Rod-Out interlock would not be affected by any of the stem conditions.

LIC: This question validated at 75% (3 of 4) with an average LOD of 2.8. The validator that responded incorrectly chose D (refueling interlocks and refuel position one-rod-out interlock) were both inoperable. The validators, FR and FA believe this is a fair question for a novice candidate with no refueling experience.

93+ H 2 Y Y N S 2X2 18 94 H 3 Y Y B S 19 95 H 3 Y Y B S 2X2 20 96+ F 2 Y Y B S Reference provided: ITS 3.0 21+

97 F 2 Y Y N S 22 98 F 2 Y Y N S 2X2, SRO-only task referenced.

23 99 F 3 Y Y N S Reference provided: EP-AA-112-F01 24 References are contained in the development notebook for questions 24 and 25. References were reviewed by the CE during prep week activities.

100+ H 2 Y Y B S Reference provided: EP-AA-1003 Barrier Matrix 25 References are contained in the development notebook for questions 24 and 25. References were reviewed by the CE during prep week activities.

SRO-only task referenced.

Questions from Previous NRC Exam: 10 New Qs: 55 RO LOK: F - 30; H - 45

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017) 2x2 questions: 39 Modified Qs: 6 SRO LOK: F - 9; H - 16 Unsat questions: 1 (1 RO / 0 SRO) Bank Qs: 39