ML17103A263: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
 
(Created page by program invented by StriderTol)
 
Line 17: Line 17:


=Text=
=Text=
{{#Wiki_filter:ES-401 Written Examination Review Worksheet Form ES-401-9   Clinton Initial License Exam (2017)
{{#Wiki_filter:ES-401                                                           Written Examination Review Worksheet                                                                           Form ES-401-9 Clinton Initial License Exam (2017)
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 1.       2.         3. Psychometric Flaws               4. Job Content Flaws         5. Other   6.     7.                                           8.
: 4. Job Content Flaws
Q#  LOK      LOD (F/H)    (1-5)  Stem Cues         T/F   Cred. Partial Job- Minutia #/ Back-             Q= SRO B/M/N U/E/S                                          Explanation Focus                    Dist.        Link            units ward      K/A Only 1     F         2                                                                             Y           N     S     2X2 2+     H         2                                                                             Y           M     S     2X2 3     H         2                                                                             Y           B     S 4     F         2                                                                             Y           B     S     2X2 (Effectively) 5+     H         1                                                                             Y           N     U     2X2 2                                                                                                  S     NRC: LOD=1. Its clear that the reducing loads on 1B is required. Due to the 2X2 format, this results in eliminating 2 distractors. Licensee comments (below) were agreed to at OV.
: 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 1 F 2         Y N S 2X2 2+ H 2         Y M S 2X2 3 H 2         Y B S 4 F 2         Y B S 2X2 (Effective ly) 5+ H 1 2          Y N U S 2X2 NRC: LOD=1. It's clear that the reducing loads on 1B is required
. Due to the 2X2 format, this results in eliminating 2 distractors.
Licensee comments (below) were agreed to at OV.
LIC: The facility contends that the part 2 of the question is not LOD 1 for the following reasons:
LIC: The facility contends that the part 2 of the question is not LOD 1 for the following reasons:
ators answered correctly).
* Question validated at 75% (3 of 4 validators answered correctly).
1B3, and 1B4), any one of which could logically provide power to DC MCC 1B Battery Charger.
* Validators rated the question as LOD 3.
* There are multiple MCCs fed from 480V Unit Sub 1B (AB MCC 1B1, 1B2, 1B3, and 1B4), any one of which could logically provide power to DC MCC 1B Battery Charger.
Instructions
Instructions
[Refer to Section D of ES- Appendix B for additional information regarding each of the following concepts.]
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
: 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
: 1.         Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
: 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
: 2.         Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
: 3. Check the appropriate box if a psychometric flaw is identified:
: 3.         Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
* The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
* The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements. The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
* The answer choices are a collection of unrelated true/false statements.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified:
* The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). The question requires reverse logic or application compared to the job requirements.
* One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
: 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO
: 4.         Check the appropriate box if a job content error is identified:
-only (K/A and license level mismatches are unacceptable).
* The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
: 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES
* The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
* The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
: 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).
* The question requires reverse logic or application compared to the job requirements.
NOTE: .
: 5.         Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
ES-401 Written Examination Review Worksheet Form ES-401-9   Clinton Initial License Exam (2017)
: 6.         Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 7.         Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
: 4. Job Content Flaws
: 8.         At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
: 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 6 F 3         Y N S 2X2 7 H 3 X         Y N U S NRC: The stem says that automatic actions are the reason air is maintained to the selected components, but the se were simply unaffected by the automatic closure of the Ring Header isolation valves. The eliminated system was negatively affected by these valves closing
NOTE: + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.
. Licensee comments were agreed to at OV. LIC: Question meets the KA (knowledge of the reasons for the following responses as they apply to partial or complete loss of IA: Service Air isolations) as written. SA header isolations are designed to automatically isolate to preserve air to critical components in the plant.
 
8+ F 2         Y N S 9 H 3         Y N S Reference provided:   H 3         Y B S 2X2 11 H 2         Y N S 2X2 12+ F 2         Y N S 13+ H 2           Y N S 14 F 2         Y N S 15 H 2         Y B S 16 F 3         Y B E S NRC: Specify why known from memory.
ES-401                                               Written Examination Review Worksheet                                                           Form ES-401-9 Clinton Initial License Exam (2017)
Explanation provided at OV.
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other 6. 7.                                         8.
LIC: The natural circulation level value is contained in the precautions sections -Shutdown Cooling (SDC) and Fuel Pool Cooling and Assist (FPC and A) and is tied to  
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                                    Explanation Focus              Dist.        Link          units ward K/A Only 6   F     3                                                             Y       N   S   2X2 7   H     3     X                                                       Y       N   U   NRC: The stem says that automatic actions are the reason air is maintained to the selected components, but these were simply unaffected by the automatic S  closure of the Ring Header isolation valves. The eliminated system was negatively affected by these valves closing. Licensee comments were agreed to at OV.
-CL-OPS-and N-CL-OPS- 17 F 2   X     Y N E S NRC: The "NEITHER" distractor
LIC: Question meets the KA (knowledge of the reasons for the following responses as they apply to partial or complete loss of IA: Service Air isolations) as written. SA header isolations are designed to automatically isolate to preserve air to critical components in the plant.
("D") is implausible.
8+   F     2                                                             Y       N   S 9   H     3                                                             Y       N   S   Reference provided: CPS 3105.05, Appendix A 10    H     3                                                             Y       B   S   2X2 11   H     2                                                             Y       N   S   2X2 12+   F     2                                                             Y       N   S 13+   H     2                                                             Y       N   S 14   F     2                                                             Y       N   S 15   H     2                                                             Y       B   S 16   F     3                                                             Y       B   E   NRC: Specify why the value given in CPS 4006.01, step 4.6.5, is required to be known from memory. Explanation provided at OV.
Explanation provided at OV. LIC: The facility contends that distracter 'D' is plausible because both loops of RHR share a common SDC suction line. If the examinee correctly recalls that LPCI is impacted by the SDC alignment, it is plausible to believe that neither loop will realign into LPCI mode.
S LIC: The natural circulation level value is contained in the precautions sections of CPS 3312.01 Residual Heat Removal and CPS 3312.03 RHR-Shutdown Cooling (SDC) and Fuel Pool Cooling and Assist (FPC and A) and is tied to objective 1.10 of lesson plans N-CL-OPS-290002 Reactor Vessel and Internals and N-CL-OPS-203000 Residual Heat Removal.
18+ H 3         Y B S 19 F 2         Y N S   H 3         Y N S ES-401 Written Examination Review Worksheet Form ES-401-9   Clinton Initial License Exam (2017)
17   F     2                       X                                     Y       N   E   NRC: The NEITHER distractor (D) is implausible. Explanation provided at OV.
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
S LIC: The facility contends that distracter D is plausible because both loops of RHR share a common SDC suction line. If the examinee correctly recalls that LPCI is impacted by the SDC alignment, it is plausible to believe that neither loop will realign into LPCI mode.
: 4. Job Content Flaws
18+   H     3                                                             Y       B   S 19   F     2                                                             Y       N   S 20    H     3                                                             Y       N   S
: 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 21+ H 3         Y N S 2X2 22 H 2         Y B S 23 H 2         Y N S 2X2 24+ F 2         Y N S 2X2 25 F 3         Y N S 2X2 , Reference provided: CPS 31, Figure 1 26+ F 2         Y B S 2X2 27 H 2         Y B S CPS NRC Exam ILT
 
-1 28 H 3   X     Y N E NRC: Using distractor "C" at psig would be more discriminating , because 1165 psig is not associated with the three SRVs
ES-401                                               Written Examination Review Worksheet                                                 Form ES-401-9 Clinton Initial License Exam (2017)
. LIC: -ordered answers/distracters from low pressure to high pressure. Changed answer explanation.
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other 6. 7.                                   8.
29 H 3         Y N S 2X2 H 3         Y B S CPS NRC Exam ILT
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                              Explanation Focus              Dist.        Link          units ward K/A Only 21+   H     3                                                             Y       N   S   2X2 22   H     2                                                             Y       B   S 23   H     2                                                             Y       N   S   2X2 24+   F     2                                                             Y       N   S   2X2 25   F     3                                                             Y       N   S   2X2, Reference provided: CPS 3005.01, Figure 1 26+   F     2                                                             Y       B   S   2X2 27   H     2                                                             Y       B   S   CPS NRC Exam ILT 10-1 28   H     3                       X                                     Y       N   E   NRC: Using distractor C at 1103 psig would be more discriminating, because 1165 psig is not associated with the three SRVs.
-1 31+ H 3         Y N S 32 H 2         Y B S 33 H 3         Y B S 2X2, CPS NRC Exam ILT
LIC: Changed distracter C to 1103. Re-ordered answers/distracters from low pressure to high pressure. Changed answer explanation.
-1 34+ H 3         Y B S 35 H 2         Y N S 2X2 36 H 2         Y B E 2X2 (Effectively)
29   H     3                                                             Y       N   S   2X2 30    H     3                                                             Y       B   S   CPS NRC Exam ILT 10-1 31+   H     3                                                             Y       N   S 32   H     2                                                             Y       B   S 33   H     3                                                             Y       B   S   2X2, CPS NRC Exam ILT 10-1 34+   H     3                                                             Y       B   S 35   H     2                                                             Y       N   S   2X2 36   H     2                                                             Y       B   E   2X2 (Effectively)
NRC: "SLIM" acronym needs to be defined.
NRC: SLIM acronym needs to be defined.
LIC: Added full noun name for the SLIM controller (Small Loop Interface Module) to the answer and distracter.
LIC: Added full noun name for the SLIM controller (Small Loop Interface Module) to the answer and distracter.
37 H 3         Y N S 38+ H 3         Y M S 2X2 (Effectively) 39+ F 3         Y B S CPS NRC Exam ILT 9-1 ES-401 Written Examination Review Worksheet Form ES-401-9   Clinton Initial License Exam (2017)
37   H     3                                                             Y       N   S 38+   H     3                                                             Y       M   S   2X2 (Effectively) 39+   F     3                                                             Y       B   S   CPS NRC Exam ILT 9-1
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
 
: 4. Job Content Flaws
ES-401                                               Written Examination Review Worksheet                                                     Form ES-401-9 Clinton Initial License Exam (2017)
: 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only F 2         Y B S CPS NRC Exam ILT
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other 6. 7.                                     8.
-1 41 H 2         Y N S 42 H 2         Y N E S NRC: It appears that the procedural aspect of the KA could be addressed; what actions would be required in this situation? Question accepted as
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                                Explanation Focus              Dist.        Link          units ward K/A Only 40    F     2                                                             Y       B   S   CPS NRC Exam ILT 10-1 41   H     2                                                             Y       N   S 42   H     2                                                             Y       N   E   NRC: It appears that the procedural aspect of the KA could be addressed; what actions would be required in this situation?
-is after discussion at OV
S Question accepted as-is after discussion at OV.
. LIC: Asking what procedural actions are required to mitigate the loss of DC would invalidate part 1 of the question. Left question as
LIC: Asking what procedural actions are required to mitigate the loss of DC would invalidate part 1 of the question. Left question as-is.
-is. 43+ F 3         Y N S 44 F 2         Y N S 45 H 3         Y N E S NRC: Does "Fail Point" appear on actual plant indication
43+   F     3                                                             Y       N   S 44   F     2                                                             Y       N   S 45   H     3                                                             Y       N   E   NRC: Does Fail Point appear on actual plant indication? Yes S  LIC: The meter is identical to the plant.
? Yes LIC: The meter is identical to the plant.
46+   F     1                                                             Y       N   U   2X2 2                                                                            S   NRC: LOD=1. It is obvious lights are dark at 12 oclock. Due to the 2X2 format, this results in eliminating 2 distractors. Discussed and agreed to at OV as indicated below.
46+ F 1 2          Y N U S 2X2 NRC: LOD=1. It is obvious lights are dark at 12 o'clock. Due to the 2X2 format, this results in eliminating 2 distractors.
LIC: The facility contends that the part 1 of the question does not contain implausible distracters. Novice candidates with limited experience paralleling a DG may select that the DG output breaker is closed when the synchronizing lights are bright (lit). In addition, 4 exam validators rated the question as LOD 2.8.
Discussed and agreed to at OV as indicated below
47   F     2                                                             Y       M   S   2X2 48   F     3                                                             Y       N   S   2X2.
. LIC: The facility contends that the part 1 of the question does not contain implausible distracters. Novice candidates with limited experience paralleling a DG may select that the DG output breaker is closed when the synchronizing lights are bright (lit). In addition, 4 exam validators rated the question as LOD 2.8.
49   H     3                                                             Y       B   S   2X2 50+    H     3                                                             Y       N   S   2X2 51   F     2                                                             Y       N   S 52   F     2                                                             Y       N   S   2X2 53+   H     3                                                             Y       B   S   Reference provided: CPS 4003.01F003 54   H     2                                                             Y       N   S 55   H     3                                                             Y       N   S
47 F 2         Y M S 2X2 48 F 3         Y N S 2X2. 49 H 3         Y B S 2X2 H 3         Y N S 2X2 51 F 2         Y N S 52 F 2         Y N S 2X2 53+ H 3         Y B S Reference provided: CPS 54 H 2         Y N S 55 H 3         Y N S ES-401 Written Examination Review Worksheet Form ES-401-9   Clinton Initial License Exam (2017)
 
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
ES-401                                               Written Examination Review Worksheet                                                     Form ES-401-9 Clinton Initial License Exam (2017)
: 4. Job Content Flaws
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other 6. 7.                                     8.
: 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 56 H 2         Y M S 2X2 57+ F 1         Y B U NRC: LOD=1. It is obvious that the velocity limiter is the choice among those given
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                                Explanation Focus              Dist.        Link          units ward K/A Only 56   H     2                                                             Y       M   S   2X2 57+   F     1                                                             Y       B   U   NRC: LOD=1. It is obvious that the velocity limiter is the choice among those given. Accepted replacement.
. Accepted replacement
LIC: Replaced the question with a different T2G2 RO question deemed suitable by the CE.
. LIC: Replaced the question with a different T2G2 RO question deemed suitable by the CE.
58   H     2                                                             Y       N   S 59   H     3                                                             Y       B   S 60+    F     3                                                             Y       N   S 61   H     3                                                             Y       N   S 62   F     1                                                             Y       B   U   2X2 2                                                                            S   NRC: LOD=1. Its very clear that rod motion is inhibited. Due to the 2X2 format, this results in eliminating 2 distractors. Discussed at OV and considered acceptable.
58 H 2         Y N S 59 H 3         Y B S   F 3         Y N S 61 H 3         Y N S 62 F 1 2          Y B U S 2X2 NRC: LOD=1. It's very clear that rod motion is inhibited.
LIC: The facility contends that the question is not LOD 1. Each control rod contains 2 position channels, only one of which is required to be operable. The examinee may plausibly believe that a single channel failure would not result in a rod block, making A1 and B1 plausible. Validators rated the question at 3.0 (average among 4 validators).
Due to the 2X2 format, this results in eliminating 2 distractors.
63+   F     2                                                             Y       B   S 64   H     2                                                             Y       B   S   CPS NRC Exam ILT 10-1 65   H     3                                                             Y       B   S 66   F                                                                 X Y       B   E   2X2, 2010 Ginna ILT NRC Exam H                                                                                      NRC: This would be an H question, because the applicant would have to calculate that > 72 hours in a 7 day period would be exceeded. Also, the question would be better if asked what would be permitted, versus whats NOT allowed. Agreed to cog change.
Discussed at OV and considered acceptable.
LIC: The facility contends that the question is not LOD 1. Each control rod contains 2 position channels, only one of which is required to be operable. The examinee may plausibly believe that a single channel failure would not result in a rod block, making A1 and validators).
63+ F 2         Y B S 64 H 2         Y B S -1 65 H 3         Y B S 66 F H          X Y B E 2X2 NRC: This would be an "H" question, because the applicant would have to calculate that > 72 hours in a 7 day period would be exceeded. Also, the question would be better if asked what would be permitted, versus what's NOT allowed.
Agreed to cog change.
LIC: Changed the question to high cog.
LIC: Changed the question to high cog.
The facility also contends that the question meets the KA as written and would not be enhanced by asking what is permitted. Covered employees are required to ensure that any overtime worked does not our Limits when accepting overtime to cover shift vacancies.
The facility also contends that the question meets the KA as written and would not be enhanced by asking what is permitted. Covered employees are required to ensure that any overtime worked does not violate 10CFR26 Work Hour Limits when accepting overtime to cover shift vacancies.


ES-401 Written Examination Review Worksheet Form ES-401-9   Clinton Initial License Exam (2017)
ES-401                                               Written Examination Review Worksheet                                                   Form ES-401-9 Clinton Initial License Exam (2017)
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other 6. 7.                                       8.
: 4. Job Content Flaws
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                                  Explanation Focus              Dist.        Link          units ward K/A Only 67+   F     1                       X                                     Y       N   U   NRC: Distractor A is implausible and the entire question is LOD=1 for an ILT student who has performed this task many times.
: 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 67+ F 1 2    X     Y N U S NRC: Distractor "A" is implausible and the entire question is LOD=1 for an ILT student who has performed this task many times
2                                                                            S Discussed at OV and considered acceptable.
. Discussed at OV and considered acceptable.
LIC: The facility contends that the question is not LOD1 and that distracter A is plausible. The question validated at 50% with an average LOD of 2.5. Additionally, the facility contends that a less than competent examinee may incorrectly believe that the SM can authorize entry into the MCR with no further authorizations required.
LIC: The facility contends that the question is not LOD1 and that average LOD of 2.5. Additionally, the facility contends that a less than competent examinee may incorrectly believe that the SM can authorize entry into the MCR with no further authorizations required.
68   H     3                                                             Y       N   S   2X2 69   F     2                                                             Y       B   S 70+    F     2                                                             Y       B   S 71   F     3                                                             Y       B   S   CPS NRC Exam ILT 10-1 NRC: Please explain why this is an RO-level question. Discussed at OV and considered acceptable.
68 H 3         Y N S 2X2 69 F 2         Y B S   F 2         Y B S 71 F 3         Y B S -1 NRC: Please explain why this is an RO
LIC: Test item is linked to 10 CFR 55.41(b)(12) Radiological safety principles and procedures. CFR data is already documented in the General Data Table.
-level question
72   H     2                       X                                     Y       B   E   NRC: Distractor B would be more plausible if the work could be done in 20 minutes. Change incorporated.
. Discussed at OV and considered acceptable.
LIC: Changed distracter B to 20 minutes as requested.
LIC: principles and procedures. CFR data is already documented in the General Data Table.
73   F     3                                                             Y       B   E   CPS NRC Exam ILT 10-1 NRC: Please provide additional information to support LOD and plausibility of the distractors.
72 H 2   X     Y B E NRC: Distractor "B" would be more plausible if the work could be . Change incorporated.
LIC: Included an elevation map of the containment to the answer explanation.
LIC: 73 F 3         Y B E -1 NRC: Please provide additional information to support LOD and plausibility of the distractors
74+   H     2                                                             Y       N   S 75   H     2                                                             Y       N   S NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection 76   H     3                                                             Y     Y N   S   versus simply actions. SRO-only task referenced.
. LIC: Included an elevation map of the containment to the answer explanation.
1                                                                                          LIC: N/A
74+ H 2         Y N S 75 H 2         Y N S 76 1 H 3         Y Y N S NRC: -only requires procedure selection versus simply actions. SRO-only task referenced.
 
LIC: N/A ES-401 Written Examination Review Worksheet Form ES-401-9   Clinton Initial License Exam (2017)
ES-401                                               Written Examination Review Worksheet                                                       Form ES-401-9 Clinton Initial License Exam (2017)
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other 6. 7.                                         8.
: 4. Job Content Flaws
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                                  Explanation Focus              Dist.        Link          units ward K/A Only NRC: Distractor A is implausible. Also, 10CFR55.43 (b) (5) for SRO-only 77   F     3                       X                                     Y     Y N   E   requires procedure selection versus simply actions. Explanation for 2                                                                                          distractor A LIC: Modified distracter A to enhance the plausibility. Modified the SRO only justification to explain how the question constitutes a selection of procedures.
: 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 77 2 F 3   X     Y Y N E NRC: -only requires procedure selection versus simply actions.
NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection 78+   H     2                                                             Y     Y N   S   versus simply actions. SRO-only task referenced.
Explanation for distractor 'A' LIC: Modified distracter A to enhance the plausibility. Modified the "SRO only justification" to explain how the question constitutes a "selection of procedures".
3                                                                                          LIC: N/A 79   H     2                                                             Y     Y B   S 4
78+ 3 H 2         Y Y N S NRC: -only requires procedure selection versus simply actions. SRO
80   H     3                                                             Y     Y B   S   2X2, CPS ILT 14-1 NRC Exam 5
-only task referenced.
81   H     3                                                             Y     Y N   S   2X2 6                                                                                          NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection versus simply actions. SRO-only task referenced for FH component.
LIC: N/A 79 4 H 2         Y Y B S 80 5 H 3         Y Y B S 2X2, CPS ILT 14
LIC: N/A 82+   F     2                                                             Y     Y N   S   2X2 7
-1 NRC Exam 81 6 H 3         Y Y N  S 2X2 NRC: (5) for SRO
NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection 83   F     2                                                             Y     Y B   E   versus simply actions.
-only requires procedure selection versus simply actions. SRO
8                                                                                          LIC: Modified the SRO only justification to explain how the question constitutes a selection of procedures.
-only task referenced for FH component.
NRC: Higher Cog question because it requires figuring out where you are 84    F    3                                                              Y    Y  B    E  and where you would be after a LOCA, etc. before applying the TS Bases 9    H                                                                                      information.
LIC: N/A 82+ 7 F 2         Y Y N S 2X2 83 8 F 2         Y Y B E NRC: -only requires procedure selection versus simply actions.
LIC: Modified the "SRO only justification" to explain how the question constitutes a "selection of procedures".
84 9 F H 3          Y Y B E NRC: Higher Cog question because it requires figuring out where you are and where you would be after a LOCA, etc. before applying the TS Bases information.
LIC: Incorporated change to cognitive level as suggested.
LIC: Incorporated change to cognitive level as suggested.
85+ F 2         Y Y N E 2X2 NRC: -only requires procedure selection versus simply actions.
85+   F     2                                                             Y     Y N   E   2X2 10                                                                                          NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection versus simply actions. Also, the KA requires the ability to perform procedures versus recalling the actions performed by them.
Also, the KA requires the ability to perform procedures versus recalling the actions performed by them.
LIC:
LIC: 86 11 H 2         Y Y M S ES-401 Written Examination Review Worksheet Form ES-401-9   Clinton Initial License Exam (2017)
86   H     2                                                             Y     Y M   S 11
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
 
: 4. Job Content Flaws
ES-401                                               Written Examination Review Worksheet                                                   Form ES-401-9 Clinton Initial License Exam (2017)
: 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 87 12 H 1 2          Y Y N U S 2X2 NRC: The SRO-only part of the question is LOD=1, since the choice is limited to whether redundant required features are operable or not. This, in turn, eliminates 2 distractors.
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other 6. 7.                                       8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                                Explanation Focus              Dist.        Link          units ward K/A Only 87   H     1                                                             Y     Y N   U   2X2 12          2                                                                            S   NRC: The SRO-only part of the question is LOD=1, since the choice is limited to whether redundant required features are operable or not.
This, in turn, eliminates 2 distractors.
LIC: The facility contends that the question is not LOD1, because the examinee has to know that the ITS bases considers Div 3 DG redundant to Div 1 and 2 for the purposes of applying ITS 3.8.1 B.2.
LIC: The facility contends that the question is not LOD1, because the examinee has to know that the ITS bases considers Div 3 DG redundant to Div 1 and 2 for the purposes of applying ITS 3.8.1 B.2.
This is not normally the case. The validators rated this question as LOD 3. 88 13 H 2         Y Y N E U 2X2 NRC: Replacing one of the "OPEN direction ONLY" with a new choice, "CLOSED direction ONLY," would enhance the discriminatory value of the question.
This is not normally the case. The validators rated this question as LOD 3.
LIC: Resolved by changing A.1 and B.1 to "closed direction only". Revised the answer explanation accordingly.
88   H     2                                                             Y     Y N   E   2X2 13                                                                                      U   NRC: Replacing one of the OPEN direction ONLY with a new choice, CLOSED direction ONLY, would enhance the discriminatory value of the question.
Additional Issue
LIC: Resolved by changing A.1 and B.1 to closed direction only.
- Replaced "bypassed" with "inserted" in the stem (there were NO correct answers before this correction was made and communicated during administration of the written exam to ALL SRO applicants
Revised the answer explanation accordingly.
). 89+ 14 H 3         Y Y N E 2X2 NRC: for SRO-only requires procedure selection versus simply actions.
Additional Issue - Replaced bypassed with inserted in the stem (there were NO correct answers before this correction was made and communicated during administration of the written exam to ALL SRO applicants).
LIC: Modified the "SRO only justification" to explain how the question constitutes a "selection of procedures".
89+   H     3                                                             Y     Y N   E   2X2 14                                                                                          NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection versus simply actions.
90 15 F 2         Y Y B S 2X2, CPS ILT 1 2-1 NRC Exam 91 16 H 3           Y Y M E NRC: If distractor "D" was changed to "Scenarios 1 & 2," the choices would be more balanced  
LIC: Modified the SRO only justification to explain how the question constitutes a selection of procedures.
- the correct scenario would only appear in 2 of the choices.
90   F     2                                                             Y     Y B   S   2X2, CPS ILT 12-1 NRC Exam 15 91   H     3                                                             Y     Y M   E   NRC: If distractor D was changed to Scenarios 1 & 2, the choices would be more balanced - the correct scenario would only appear in 16 2 of the choices.
LIC: Changed Distracter D to scenarios 1 & 2 as requested. Adjusted answer explanation accordingly.
LIC: Changed Distracter D to scenarios 1 & 2 as requested.
ES-401 Written Examination Review Worksheet Form ES-401-9   Clinton Initial License Exam (2017)
Adjusted answer explanation accordingly.
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
 
: 4. Job Content Flaws
ES-401                                                 Written Examination Review Worksheet                                                   Form ES-401-9 Clinton Initial License Exam (2017)
: 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 92 17 H 1 2          Y Y N U S 2X2 NRC: LOD=1. interlocks are INOP, it is more obvious that the One
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other 6. 7.                                     8.
-Rod-Out interlock would not be affected by any of the stem conditions.
Q#  LOK    LOD (F/H)  (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                              Explanation Focus              Dist.        Link          units ward K/A Only 92   H     1                                                             Y     Y N   U   2X2 17          2                                                                            S   NRC: LOD=1. While its clear that the Refueling Equipment interlocks are INOP, it is more obvious that the One-Rod-Out interlock would not be affected by any of the stem conditions.
LIC: This question validated at 75% (3 of 4) with an average LOD of 2.8. The validator that responded incorrectly chose D (refueling interlocks and refuel position one-rod-out interlock) were both inoperable. The validators, FR and FA believe this is a fair question for a novice candidate with no refueling experience.
93+  H      2                                                              Y    Y  N    S  2X2 18 94  H      3                                                              Y    Y  B    S 19 95  H      3                                                              Y    Y  B    S  2X2 20 96+  F      2                                                              Y    Y  B    S  Reference provided: ITS 3.0 21+
97  F      2                                                              Y    Y  N    S 22 98  F      2                                                              Y    Y  N    S  2X2, SRO-only task referenced.
23 99  F      3                                                              Y    Y  N    S  Reference provided: EP-AA-112-F01 24                                                                                          References are contained in the development notebook for questions 24 and 25. References were reviewed by the CE during prep week activities.
100+  H      2                                                              Y    Y  B    S    Reference provided: EP-AA-1003 Barrier Matrix 25                                                                                          References are contained in the development notebook for questions 24 and 25. References were reviewed by the CE during prep week activities.
SRO-only task referenced.
Questions from Previous NRC Exam: 10                  New Qs: 55                            RO LOK: F - 30; H - 45


LIC: This question validated 2.8. The validator that responded incorrectly chose D (refueling interlocks and refuel position one
ES-401                               Written Examination Review Worksheet                 Form ES-401-9 Clinton Initial License Exam (2017) 2x2 questions: 39                   Modified Qs: 6                 SRO LOK: F - 9; H - 16 Unsat questions: 1 (1 RO / 0 SRO)   Bank Qs: 39}}
-rod-out interlock) were both inoperable. The validators, FR and FA believe this is a fair question for a novice candidate with no refueling experience.
93+ 18 H 2          Y Y N S 2X2 94 19 H 3          Y Y B S  95 H 3          Y Y B S 2X2 96+ 21+ F 2          Y Y B S  97 22 F 2          Y Y N S  98 23 F 2          Y Y N S 2X2, SRO-only task referenced.
99 24 F 3          Y Y N S Reference provided:
EP-AA-112-F01  References are contained in the development notebook for questions 24 and 25.
References were reviewed by the CE during prep week activities.
100+ 25 H 2          Y Y B S  Reference provided:
EP-AA-1003 Barrier Matrix References are contained in the development notebook for questions 24 and 25. References were reviewed by the CE during prep week activities.
SRO-only task referenced
. Questions from Previous NRC Exam:  New Q's: 55          RO LOK:  F - ; H - 45 ES-401 Written Examination Review Worksheet Form ES-401-9   Clinton Initial License Exam (2017) 2x2 questions:
39   Modified Q's:
6     SRO LOK: F - 9; H - 16 Unsat questions:
1 (1 RO / SRO) Bank Q's:
39}}

Latest revision as of 05:39, 30 October 2019

2017 Clinton Power Station Initial Licensed Operator Examination Forms ES-401-9
ML17103A263
Person / Time
Site: Clinton Constellation icon.png
Issue date: 02/09/2017
From: Chuck Zoia
Operations Branch III
To:
Exelon Generation Co, Exelon Nuclear
Shared Package
ML16308A462 List:
References
50-461/17-301
Download: ML17103A263 (10)
v • d • e


Text

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 F 2 Y N S 2X2 2+ H 2 Y M S 2X2 3 H 2 Y B S 4 F 2 Y B S 2X2 (Effectively) 5+ H 1 Y N U 2X2 2 S NRC: LOD=1. Its clear that the reducing loads on 1B is required. Due to the 2X2 format, this results in eliminating 2 distractors. Licensee comments (below) were agreed to at OV.

LIC: The facility contends that the part 2 of the question is not LOD 1 for the following reasons:

  • Question validated at 75% (3 of 4 validators answered correctly).
  • Validators rated the question as LOD 3.
  • There are multiple MCCs fed from 480V Unit Sub 1B (AB MCC 1B1, 1B2, 1B3, and 1B4), any one of which could logically provide power to DC MCC 1B Battery Charger.

Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
3. Check the appropriate box if a psychometric flaw is identified:
  • The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
  • The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
  • The answer choices are a collection of unrelated true/false statements.
  • The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
  • One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
4. Check the appropriate box if a job content error is identified:
  • The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
  • The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
  • The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
  • The question requires reverse logic or application compared to the job requirements.
5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

NOTE: + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 6 F 3 Y N S 2X2 7 H 3 X Y N U NRC: The stem says that automatic actions are the reason air is maintained to the selected components, but these were simply unaffected by the automatic S closure of the Ring Header isolation valves. The eliminated system was negatively affected by these valves closing. Licensee comments were agreed to at OV.

LIC: Question meets the KA (knowledge of the reasons for the following responses as they apply to partial or complete loss of IA: Service Air isolations) as written. SA header isolations are designed to automatically isolate to preserve air to critical components in the plant.

8+ F 2 Y N S 9 H 3 Y N S Reference provided: CPS 3105.05, Appendix A 10 H 3 Y B S 2X2 11 H 2 Y N S 2X2 12+ F 2 Y N S 13+ H 2 Y N S 14 F 2 Y N S 15 H 2 Y B S 16 F 3 Y B E NRC: Specify why the value given in CPS 4006.01, step 4.6.5, is required to be known from memory. Explanation provided at OV.

S LIC: The natural circulation level value is contained in the precautions sections of CPS 3312.01 Residual Heat Removal and CPS 3312.03 RHR-Shutdown Cooling (SDC) and Fuel Pool Cooling and Assist (FPC and A) and is tied to objective 1.10 of lesson plans N-CL-OPS-290002 Reactor Vessel and Internals and N-CL-OPS-203000 Residual Heat Removal.

17 F 2 X Y N E NRC: The NEITHER distractor (D) is implausible. Explanation provided at OV.

S LIC: The facility contends that distracter D is plausible because both loops of RHR share a common SDC suction line. If the examinee correctly recalls that LPCI is impacted by the SDC alignment, it is plausible to believe that neither loop will realign into LPCI mode.

18+ H 3 Y B S 19 F 2 Y N S 20 H 3 Y N S

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 21+ H 3 Y N S 2X2 22 H 2 Y B S 23 H 2 Y N S 2X2 24+ F 2 Y N S 2X2 25 F 3 Y N S 2X2, Reference provided: CPS 3005.01, Figure 1 26+ F 2 Y B S 2X2 27 H 2 Y B S CPS NRC Exam ILT 10-1 28 H 3 X Y N E NRC: Using distractor C at 1103 psig would be more discriminating, because 1165 psig is not associated with the three SRVs.

LIC: Changed distracter C to 1103. Re-ordered answers/distracters from low pressure to high pressure. Changed answer explanation.

29 H 3 Y N S 2X2 30 H 3 Y B S CPS NRC Exam ILT 10-1 31+ H 3 Y N S 32 H 2 Y B S 33 H 3 Y B S 2X2, CPS NRC Exam ILT 10-1 34+ H 3 Y B S 35 H 2 Y N S 2X2 36 H 2 Y B E 2X2 (Effectively)

NRC: SLIM acronym needs to be defined.

LIC: Added full noun name for the SLIM controller (Small Loop Interface Module) to the answer and distracter.

37 H 3 Y N S 38+ H 3 Y M S 2X2 (Effectively) 39+ F 3 Y B S CPS NRC Exam ILT 9-1

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 40 F 2 Y B S CPS NRC Exam ILT 10-1 41 H 2 Y N S 42 H 2 Y N E NRC: It appears that the procedural aspect of the KA could be addressed; what actions would be required in this situation?

S Question accepted as-is after discussion at OV.

LIC: Asking what procedural actions are required to mitigate the loss of DC would invalidate part 1 of the question. Left question as-is.

43+ F 3 Y N S 44 F 2 Y N S 45 H 3 Y N E NRC: Does Fail Point appear on actual plant indication? Yes S LIC: The meter is identical to the plant.

46+ F 1 Y N U 2X2 2 S NRC: LOD=1. It is obvious lights are dark at 12 oclock. Due to the 2X2 format, this results in eliminating 2 distractors. Discussed and agreed to at OV as indicated below.

LIC: The facility contends that the part 1 of the question does not contain implausible distracters. Novice candidates with limited experience paralleling a DG may select that the DG output breaker is closed when the synchronizing lights are bright (lit). In addition, 4 exam validators rated the question as LOD 2.8.

47 F 2 Y M S 2X2 48 F 3 Y N S 2X2.

49 H 3 Y B S 2X2 50+ H 3 Y N S 2X2 51 F 2 Y N S 52 F 2 Y N S 2X2 53+ H 3 Y B S Reference provided: CPS 4003.01F003 54 H 2 Y N S 55 H 3 Y N S

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 56 H 2 Y M S 2X2 57+ F 1 Y B U NRC: LOD=1. It is obvious that the velocity limiter is the choice among those given. Accepted replacement.

LIC: Replaced the question with a different T2G2 RO question deemed suitable by the CE.

58 H 2 Y N S 59 H 3 Y B S 60+ F 3 Y N S 61 H 3 Y N S 62 F 1 Y B U 2X2 2 S NRC: LOD=1. Its very clear that rod motion is inhibited. Due to the 2X2 format, this results in eliminating 2 distractors. Discussed at OV and considered acceptable.

LIC: The facility contends that the question is not LOD 1. Each control rod contains 2 position channels, only one of which is required to be operable. The examinee may plausibly believe that a single channel failure would not result in a rod block, making A1 and B1 plausible. Validators rated the question at 3.0 (average among 4 validators).

63+ F 2 Y B S 64 H 2 Y B S CPS NRC Exam ILT 10-1 65 H 3 Y B S 66 F X Y B E 2X2, 2010 Ginna ILT NRC Exam H NRC: This would be an H question, because the applicant would have to calculate that > 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> in a 7 day period would be exceeded. Also, the question would be better if asked what would be permitted, versus whats NOT allowed. Agreed to cog change.

LIC: Changed the question to high cog.

The facility also contends that the question meets the KA as written and would not be enhanced by asking what is permitted. Covered employees are required to ensure that any overtime worked does not violate 10CFR26 Work Hour Limits when accepting overtime to cover shift vacancies.

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 67+ F 1 X Y N U NRC: Distractor A is implausible and the entire question is LOD=1 for an ILT student who has performed this task many times.

2 S Discussed at OV and considered acceptable.

LIC: The facility contends that the question is not LOD1 and that distracter A is plausible. The question validated at 50% with an average LOD of 2.5. Additionally, the facility contends that a less than competent examinee may incorrectly believe that the SM can authorize entry into the MCR with no further authorizations required.

68 H 3 Y N S 2X2 69 F 2 Y B S 70+ F 2 Y B S 71 F 3 Y B S CPS NRC Exam ILT 10-1 NRC: Please explain why this is an RO-level question. Discussed at OV and considered acceptable.

LIC: Test item is linked to 10 CFR 55.41(b)(12) Radiological safety principles and procedures. CFR data is already documented in the General Data Table.

72 H 2 X Y B E NRC: Distractor B would be more plausible if the work could be done in 20 minutes. Change incorporated.

LIC: Changed distracter B to 20 minutes as requested.

73 F 3 Y B E CPS NRC Exam ILT 10-1 NRC: Please provide additional information to support LOD and plausibility of the distractors.

LIC: Included an elevation map of the containment to the answer explanation.

74+ H 2 Y N S 75 H 2 Y N S NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection 76 H 3 Y Y N S versus simply actions. SRO-only task referenced.

1 LIC: N/A

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: Distractor A is implausible. Also, 10CFR55.43 (b) (5) for SRO-only 77 F 3 X Y Y N E requires procedure selection versus simply actions. Explanation for 2 distractor A LIC: Modified distracter A to enhance the plausibility. Modified the SRO only justification to explain how the question constitutes a selection of procedures.

NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection 78+ H 2 Y Y N S versus simply actions. SRO-only task referenced.

3 LIC: N/A 79 H 2 Y Y B S 4

80 H 3 Y Y B S 2X2, CPS ILT 14-1 NRC Exam 5

81 H 3 Y Y N S 2X2 6 NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection versus simply actions. SRO-only task referenced for FH component.

LIC: N/A 82+ F 2 Y Y N S 2X2 7

NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection 83 F 2 Y Y B E versus simply actions.

8 LIC: Modified the SRO only justification to explain how the question constitutes a selection of procedures.

NRC: Higher Cog question because it requires figuring out where you are 84 F 3 Y Y B E and where you would be after a LOCA, etc. before applying the TS Bases 9 H information.

LIC: Incorporated change to cognitive level as suggested.

85+ F 2 Y Y N E 2X2 10 NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection versus simply actions. Also, the KA requires the ability to perform procedures versus recalling the actions performed by them.

LIC:

86 H 2 Y Y M S 11

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 87 H 1 Y Y N U 2X2 12 2 S NRC: The SRO-only part of the question is LOD=1, since the choice is limited to whether redundant required features are operable or not.

This, in turn, eliminates 2 distractors.

LIC: The facility contends that the question is not LOD1, because the examinee has to know that the ITS bases considers Div 3 DG redundant to Div 1 and 2 for the purposes of applying ITS 3.8.1 B.2.

This is not normally the case. The validators rated this question as LOD 3.

88 H 2 Y Y N E 2X2 13 U NRC: Replacing one of the OPEN direction ONLY with a new choice, CLOSED direction ONLY, would enhance the discriminatory value of the question.

LIC: Resolved by changing A.1 and B.1 to closed direction only.

Revised the answer explanation accordingly.

Additional Issue - Replaced bypassed with inserted in the stem (there were NO correct answers before this correction was made and communicated during administration of the written exam to ALL SRO applicants).

89+ H 3 Y Y N E 2X2 14 NRC: 10CFR55.43 (b) (5) for SRO-only requires procedure selection versus simply actions.

LIC: Modified the SRO only justification to explain how the question constitutes a selection of procedures.

90 F 2 Y Y B S 2X2, CPS ILT 12-1 NRC Exam 15 91 H 3 Y Y M E NRC: If distractor D was changed to Scenarios 1 & 2, the choices would be more balanced - the correct scenario would only appear in 16 2 of the choices.

LIC: Changed Distracter D to scenarios 1 & 2 as requested.

Adjusted answer explanation accordingly.

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 92 H 1 Y Y N U 2X2 17 2 S NRC: LOD=1. While its clear that the Refueling Equipment interlocks are INOP, it is more obvious that the One-Rod-Out interlock would not be affected by any of the stem conditions.

LIC: This question validated at 75% (3 of 4) with an average LOD of 2.8. The validator that responded incorrectly chose D (refueling interlocks and refuel position one-rod-out interlock) were both inoperable. The validators, FR and FA believe this is a fair question for a novice candidate with no refueling experience.

93+ H 2 Y Y N S 2X2 18 94 H 3 Y Y B S 19 95 H 3 Y Y B S 2X2 20 96+ F 2 Y Y B S Reference provided: ITS 3.0 21+

97 F 2 Y Y N S 22 98 F 2 Y Y N S 2X2, SRO-only task referenced.

23 99 F 3 Y Y N S Reference provided: EP-AA-112-F01 24 References are contained in the development notebook for questions 24 and 25. References were reviewed by the CE during prep week activities.

100+ H 2 Y Y B S Reference provided: EP-AA-1003 Barrier Matrix 25 References are contained in the development notebook for questions 24 and 25. References were reviewed by the CE during prep week activities.

SRO-only task referenced.

Questions from Previous NRC Exam: 10 New Qs: 55 RO LOK: F - 30; H - 45

ES-401 Written Examination Review Worksheet Form ES-401-9 Clinton Initial License Exam (2017) 2x2 questions: 39 Modified Qs: 6 SRO LOK: F - 9; H - 16 Unsat questions: 1 (1 RO / 0 SRO) Bank Qs: 39

Retrieved from "https://kanterella.com/w/index.php?title=ML17103A263&oldid=2009141"