ML14281A079: Difference between revisions
StriderTol (talk | contribs) (Created page by program invented by StriderTol) |
StriderTol (talk | contribs) (Created page by program invented by StriderTol) |
||
| Line 18: | Line 18: | ||
=Text= | =Text= | ||
{{#Wiki_filter:ES-401 Written Examination Review Worksheet Form ES-401-9 | {{#Wiki_filter:ES-401 Written Examination Review Worksheet Form ES-401-9 | ||
# questions reviewed in depth. | |||
: 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. | |||
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 | |||
3 F 3.0 | Question asks for a step buried in the subsequent actions of a 2 H 5.0 N U procedure. Replaced question. | ||
5 F 2.0 | Question asks for trivial information from the stations accident 3 F 3.0 x N U analysis. Replaced question 4 F 2.0 x N U Question requires sequence memorization. Replaced question. | ||
5 F 2.0 M S 6 H 5.0 N U Question needs references to answer. Replaced question. | |||
-only heaters. Replaced K/A, Question. | F 2.0 M U Station identified they do not have emergency-only heaters. | ||
7 Replaced K/A, Question. | |||
Instructions | Instructions | ||
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] | [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] | ||
: 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. | : 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. | ||
: 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable). | : 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable). | ||
: 3. Check the appropriate box if a psychometric flaw is identified: | : 3. Check the appropriate box if a psychometric flaw is identified: | ||
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). | * The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). | ||
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc). | * The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc). | ||
The answer choices are a collection of unrelated true/false statements. | * The answer choices are a collection of unrelated true/false statements. | ||
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable. | * The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable. | ||
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). | * One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). | ||
: 4. Check the appropriate box if a job content error is identified: | : 4. Check the appropriate box if a job content error is identified: | ||
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). The question requires reverse logic or application compared to the job requirements. | * The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). | ||
: 5. Check questions that are sampled for conformance with the approved K/A , and those that are designated SRO | * The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). | ||
-only (K/A and license level mismatches are unacceptable). | * The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). | ||
: 6. Enter question source: | * The question requires reverse logic or application compared to the job requirements. | ||
-401 Section D.2.f. | : 5. Check questions that are sampled for conformance with the approved K/A, and those that are designated SRO-only (K/A and license level mismatches are unacceptable). | ||
: 7. Based on the | : 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. | ||
: 8. At a minimum, explain any | : 7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? | ||
: 8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met). | |||
ES-401 2 Form ES-401-9 | ES-401 2 Form ES-401-9 | ||
: 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. | |||
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Question does not ask how to interpret valve position indications. | |||
8 2.0 N x x N U Replaced question. | |||
9 2.0 H M S 10 2.0 H N S 11 2.0 H B E Identified answer is incorrect. Updated correct answer. | |||
13 2.0 F | Update material to current station nomenclature. Updated 12 2.5 F B E procedure names. | ||
15 2.0 F | 13 2.0 F B S 14# 2.5 H B U Same as Question #42. Selected new K/A and question. | ||
19# 2.0 H | 15 2.0 F N S 16# 2.0 F N S 17 2.0 H B S 18 2.0 F B E Update correct answer. Updated correct answer. | ||
19# 2.0 H M S 20 2.0 H M S 21 2.0 H B S 22# 2.0 F M S 23 2.0 F B S 24 2.0 F N S Requires memorization of details that should be reviewed prior to a 25# 2.5 F x M U reactivity change. Replaced question. | |||
Update correct answer to current procedures. Updated correct 26 2.0 H B E answer. | |||
27 2.0 H B S 28# 2.0 F B S 29 2.0 H N E Correct the answer key. Should be (c.). Corrected answer key. | |||
: 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. | |||
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 30 2.0 H B S Delete the time-critical aspect of the question - not necessary, not 31# 2.0 H x S E part of the K/A. Removed the time critical portion of the question and answer. | |||
-critical aspect of the question | 32 2.0 F N S 33 2.5 H B S 34# 2.0 H B S 35 2.0 H B S 36 2.0 F B S 37# 2.0 H N S 38 2.0 F M S 39 2.5 H N S 40# 2.0 H B S 41 2.0 H B S 42 2.0 H B S Original question #14. | ||
- not necessary, not part of the K/A. Removed the time critical portion of the question and answer. | 43# 2.0 H B S 44 2.0 F N E Correct answer. Corrected answer 45 1.0 F x N U Question asks trivia. Replace Question Replaced question 46# 2.0 H B S 47 2.0 F x B U Q K/A. Replace Question. Replaced question 48 2.0 H B S 49# 2.5 H B S 50 2.0 F N S 51 2.0 F N S Q K/A. Doesnt address second part of the K/A. Replace Question 52# 2.5 H x B U Replaced question 53 2.0 F B S | ||
32 2.0 F | : 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. | ||
43# 2.0 H | Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Ice Condenser Containment - slight differences. Re-work question. | ||
Replace Question Replaced question 53 2.0 F | 54 2.0 F N U Reworked question 55# 2.0 F x M U Q K/A. Replace Question Replaced question 56 2.0 F N S 57 2.0 F M S 58# 2.0 H M S 59 2.5 H B S 60 2.0 H B S 61# 2.5 H B S Doesnt distinguish between competent/less-than competent 62 2.0 F x M U operator. Replace question Replaced question 63 2.0 H B S 64# 2.0 H B S 65 2.0 H N E Correct answer is (b.) - should be (a.) Corrected answer key 66 2.0 F N S 67 2.0 H B S 68 2.0 H B E Correct answer is (a.) - should be (b.) Corrected answer key 69 2.5 F N S 70 2.0 H N S 71 2.0 F N S 72 2.5 F B S 73 2.0 F M S Question requires SRO knowledge to make a transition to another 74 2.0 H B U procedure. Replace question. Replaced question. | ||
75 2.0 F N S 76# 2.5 H x B S 77 2.0 H x N S | |||
: 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. | |||
- slight differences. Re | Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 78 2.0 F x N S 79# 2.0 H x M E Remove unnecessary material from stem. Removed fluff. | ||
-work question. Reworked question 55# 2.0 F | 80 2.5 H x M S 81 2.5 H x B S 82# 2.5 H x M S 83 2.0 H x N S Station stated this tech spec call would be determined with the 84 5.0 H x N E assistance of engineering and is beyond the normal control room technical specification calls. Replaced question. | ||
-than competent operator. Replace question | Hydrostatic test referred to is an infrequent activity. Site engineering 85# 5.0 F x N E would participate in the application of the tech spec. Replaced question. | ||
- should be (a.) | 86 2.0 H x N S Question is RO level. This is a system question. Replaced 87 2.5 F x B U question. | ||
- should be (b.) | Rework question to conform with Palisades language. Revised 88# 3.0 F x N E question. | ||
75 2.0 F | 89 2.5 H x N E Some minor tweaks to the wording of the stem. Made revisions. | ||
90 2.0 H x B S 91# 2.5 H x N S Moved question to #99. Selected new K/A & Question for #91. | |||
92 2.0 H x N S 93 2.5 H x N S 94# 2.0 F x B S 95 2.0 F x N S Fix the answer and replace bad distractor. Added requirements to 96 2.5 H x B E the answer, replaced one distractor. | |||
80 2.5 H | 97 2.0 H x B S 98 2.0 F x B S Question was replaced by Q#91 - better question for 99 2.5 H x B E determining operator ability. | ||
: 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. | |||
90 2.0 H | Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Not required from memory - direct lookup if reference is provided. | ||
92 2.0 H | 100 2.0 F x N U Replaced question}} | ||
97 2.0 H | |||
- better question for determining operator ability. | |||
- direct lookup if reference is provided. Replaced question}} | |||
Revision as of 20:25, 31 October 2019
| ML14281A079 | |
| Person / Time | |
|---|---|
| Site: | Cook |
| Issue date: | 08/01/2014 |
| From: | Mcneil D NRC/RGN-III/DRS/OLB |
| To: | American Electric Power, Indiana Michigan Power Co |
| Shared Package | |
| ML13093A427 | List: |
| References | |
| 50-315/14-301, 50-316/14-301 50-315/OL-14, 50-316/OL-14 | |
| Download: ML14281A079 (6) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- questions reviewed in depth.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1
Question asks for a step buried in the subsequent actions of a 2 H 5.0 N U procedure. Replaced question.
Question asks for trivial information from the stations accident 3 F 3.0 x N U analysis. Replaced question 4 F 2.0 x N U Question requires sequence memorization. Replaced question.
5 F 2.0 M S 6 H 5.0 N U Question needs references to answer. Replaced question.
F 2.0 M U Station identified they do not have emergency-only heaters.
7 Replaced K/A, Question.
Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
- The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
- The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
- The answer choices are a collection of unrelated true/false statements.
- The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
- One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
- The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
- The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
- The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A, and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Question does not ask how to interpret valve position indications.
8 2.0 N x x N U Replaced question.
9 2.0 H M S 10 2.0 H N S 11 2.0 H B E Identified answer is incorrect. Updated correct answer.
Update material to current station nomenclature. Updated 12 2.5 F B E procedure names.
13 2.0 F B S 14# 2.5 H B U Same as Question #42. Selected new K/A and question.
15 2.0 F N S 16# 2.0 F N S 17 2.0 H B S 18 2.0 F B E Update correct answer. Updated correct answer.
19# 2.0 H M S 20 2.0 H M S 21 2.0 H B S 22# 2.0 F M S 23 2.0 F B S 24 2.0 F N S Requires memorization of details that should be reviewed prior to a 25# 2.5 F x M U reactivity change. Replaced question.
Update correct answer to current procedures. Updated correct 26 2.0 H B E answer.
27 2.0 H B S 28# 2.0 F B S 29 2.0 H N E Correct the answer key. Should be (c.). Corrected answer key.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 30 2.0 H B S Delete the time-critical aspect of the question - not necessary, not 31# 2.0 H x S E part of the K/A. Removed the time critical portion of the question and answer.
32 2.0 F N S 33 2.5 H B S 34# 2.0 H B S 35 2.0 H B S 36 2.0 F B S 37# 2.0 H N S 38 2.0 F M S 39 2.5 H N S 40# 2.0 H B S 41 2.0 H B S 42 2.0 H B S Original question #14.
43# 2.0 H B S 44 2.0 F N E Correct answer. Corrected answer 45 1.0 F x N U Question asks trivia. Replace Question Replaced question 46# 2.0 H B S 47 2.0 F x B U Q K/A. Replace Question. Replaced question 48 2.0 H B S 49# 2.5 H B S 50 2.0 F N S 51 2.0 F N S Q K/A. Doesnt address second part of the K/A. Replace Question 52# 2.5 H x B U Replaced question 53 2.0 F B S
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Ice Condenser Containment - slight differences. Re-work question.
54 2.0 F N U Reworked question 55# 2.0 F x M U Q K/A. Replace Question Replaced question 56 2.0 F N S 57 2.0 F M S 58# 2.0 H M S 59 2.5 H B S 60 2.0 H B S 61# 2.5 H B S Doesnt distinguish between competent/less-than competent 62 2.0 F x M U operator. Replace question Replaced question 63 2.0 H B S 64# 2.0 H B S 65 2.0 H N E Correct answer is (b.) - should be (a.) Corrected answer key 66 2.0 F N S 67 2.0 H B S 68 2.0 H B E Correct answer is (a.) - should be (b.) Corrected answer key 69 2.5 F N S 70 2.0 H N S 71 2.0 F N S 72 2.5 F B S 73 2.0 F M S Question requires SRO knowledge to make a transition to another 74 2.0 H B U procedure. Replace question. Replaced question.
75 2.0 F N S 76# 2.5 H x B S 77 2.0 H x N S
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 78 2.0 F x N S 79# 2.0 H x M E Remove unnecessary material from stem. Removed fluff.
80 2.5 H x M S 81 2.5 H x B S 82# 2.5 H x M S 83 2.0 H x N S Station stated this tech spec call would be determined with the 84 5.0 H x N E assistance of engineering and is beyond the normal control room technical specification calls. Replaced question.
Hydrostatic test referred to is an infrequent activity. Site engineering 85# 5.0 F x N E would participate in the application of the tech spec. Replaced question.
86 2.0 H x N S Question is RO level. This is a system question. Replaced 87 2.5 F x B U question.
Rework question to conform with Palisades language. Revised 88# 3.0 F x N E question.
89 2.5 H x N E Some minor tweaks to the wording of the stem. Made revisions.
90 2.0 H x B S 91# 2.5 H x N S Moved question to #99. Selected new K/A & Question for #91.
92 2.0 H x N S 93 2.5 H x N S 94# 2.0 F x B S 95 2.0 F x N S Fix the answer and replace bad distractor. Added requirements to 96 2.5 H x B E the answer, replaced one distractor.
97 2.0 H x B S 98 2.0 F x B S Question was replaced by Q#91 - better question for 99 2.5 H x B E determining operator ability.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Not required from memory - direct lookup if reference is provided.
100 2.0 F x N U Replaced question