ML13227A107: Difference between revisions
StriderTol (talk | contribs) (Created page by program invented by StriderTol) |
StriderTol (talk | contribs) (Created page by program invented by StriderTol) |
||
(6 intermediate revisions by the same user not shown) | |||
Line 2: | Line 2: | ||
| number = ML13227A107 | | number = ML13227A107 | ||
| issue date = 08/12/2013 | | issue date = 08/12/2013 | ||
| title = | | title = Facility Comments on Draft NRC-Developed Exam (Written) (Folder 2) | ||
| author name = Fish T | | author name = Fish T | ||
| author affiliation = NRC/RGN-I/DRS/OB | | author affiliation = NRC/RGN-I/DRS/OB | ||
| addressee name = | | addressee name = | ||
Line 9: | Line 9: | ||
| docket = 05000354 | | docket = 05000354 | ||
| license number = NPF-057 | | license number = NPF-057 | ||
| contact person = Jackson D | | contact person = Jackson D | ||
| case reference number = TAC U01872 | | case reference number = TAC U01872 | ||
| package number = ML13077A118 | | package number = ML13077A118 | ||
| document type = License-Operator, Part 55 Examination Related Material | | document type = License-Operator, Part 55 Examination Related Material | ||
| page count = 6 | | page count = 6 | ||
| project = TAC:U01872 | |||
| stage = Draft Other | |||
}} | }} | ||
=Text= | =Text= | ||
{{#Wiki_filter:ES-401 Written Examination Review Worksheet Form ES-401-9 | {{#Wiki_filter:ES-401 Written Examination Review Worksheet Form ES-401-9 | ||
: 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-Minutia #I Back-Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 1 H 2 M 2 H 3 B 3 F 3 B 4 H 4 B 5 F 3 B 6 H 3 N 7 H 4 B 8 H 2 N Instructions | : 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. | ||
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] | Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 1 H 2 M 2 H 3 B 3 F 3 B 4 H 4 B 5 F 3 B 6 H 3 N 7 H 4 B 8 H 2 N Instructions | ||
: 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question using a 1 -5 (easy-difficult) rating scale (questions in the 2-4 range are acceptable). | [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] | ||
: 3. Check the appropriate box if a psychometric flaw is identified: | : 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. | ||
. The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information) | : 2. Enter the level of difficulty (LOD) of each question using a 1 -5 (easy- difficult) rating scale (questions in the 2-4 range are acceptable). | ||
: 3. Check the appropriate box if a psychometric flaw is identified: | |||
.. The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information) . | |||
. | I The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc) . I The answer choices are a collection of unrelated true/false statements . | ||
. The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable . | |||
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem) . | |||
. | 4. | ||
Check the appropriate box if a job content error is identified: | |||
The question is not linked to the job requirements (i.e., the question has a valid KIA but. as written, is not operational in content) . ! | |||
: 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer's | .. The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory) . | ||
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons) . | |||
in need of (E)ditorial enhancement, or (S)atisfactory? | The question requires reverse logic or application compared to the job requirements . | ||
: 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met). | : 5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-on/y (KIA and license level mismatches are unacceptable). | ||
ES-401 2 Form ES-401-9 | : 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. | ||
: 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? | |||
21 H 3 B 22 H 2 M 23 F 3 B 24 F 3 M 25 H 3 X B u SRO level knowledge; Requires knowledge of EOP actions; Replaced question. | : 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met). | ||
26 H 4 N 27 F 2 N E Modified typographical error that made question invalid. | |||
28 H 3 B ES-401 2 Form ES-401-9 I 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-Minutia #I Back-Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 29 H 5 X B u Minutia-unusual and rarely used lineup; LOD 5; Replaced question. | ES-401 2 Form ES-401-9 | ||
30 H 4 N 31 F 3 B 32 H 4 B 33 H 3 B 34 H 3 N 35 H 3 B 36 H 4 B 37 H 2 N 38 H 3 N 39 H 3 B 40 F 3 N 41 H 3 B 42 H 3 B 43 F 2 B 44 H 2 N 45 F 3 N 46 H 3 B 47 F 2 N E Updated ARP tile nomenclature to match current plant. 48 H 3 B J;S-401 2 Form ES-401-9 | : 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. | ||
: 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-Minutia #I Back-Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 49 H 5 N B u LOD=5 w/out print; KIA MM; Replaced question. | Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 9 H 2 B 10 H 3 M 11 H 2 B 12 F 3 B 13 H 3 M 14 H 3 B 15 F 3 B 16 F 4 B 17 H 3 B 18 F 2 B 19 H 2 B 20 F 3 X B u Content queued answer to SRO 082; Replaced question. | ||
50 F 2 N 51 H 5 B 52 F 2 B 53 F 4 M 54 F 2 B 55 H 2 N 56 H 3 B 57 H 4 B 58 H 4 N 59 F 3 B 60 H 3 B 61 F 3 N 62 H 2 N 63 H 3 B 64 F 2 B 65 F 3 B 66 F 3 B 67 H 2 B 68 F 2 B ES-401 2 Form ES-401-9 | 21 H 3 B 22 H 2 M 23 F 3 B 24 F 3 M 25 H 3 X B u SRO level knowledge; Requires knowledge of EOP actions; Replaced question. | ||
: 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-Minutia #I Back-Q= SRO B/M/N UIE/S Explanation Focus Dist. Link units ward KIA Only 69 H 2 M 70 F 3 B 71 F 3 B 72 H 2 B E Provided print to prevent question from being LOD 5. 73 F 3 B 74 F 2 B 75 F 2 X B u Minutia; Bases for limit is not in any procedure or lesson plan. Replaced question. | 26 H 4 N 27 F 2 N E Modified typographical error that made question invalid. | ||
76 H 3 y B 77 H 4 y B 78 H 3 y M 79 H 3 y B 80 H 3 y N 81 H 3 y B 82 H 4 y N 83 H 5 y B E Provided sanitized EOP graphs to avoid LOD 5 question. | 28 H 3 B | ||
84 H 3 y N 85 H 3 y M 86 H 4 N N u Not SRO; Only need system knowledge to answer question; Rewrote question. | |||
87 H 4 y B 88 H 5 y B ES-401 2 Form ES-401-9 | ES-401 2 Form ES-401-9 I 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. | ||
: 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-Minutia #I Back-Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 89 F 3 y B 90 H 5 y N 91 H 5 y B 92 H 2 y B 93 H 1 X y B u Overlap with facility audit exam; Replaced question. | Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 29 H 5 X B u Minutia- unusual and rarely used lineup; LOD 5; Replaced question. | ||
94 F 2 y B 95 H 3 y M 96 F 2 y N 97 H 3 y N 98 H 3 y B 99 F 3 y B 100 H 2 y B}} | 30 H 4 N 31 F 3 B 32 H 4 B 33 H 3 B 34 H 3 N 35 H 3 B 36 H 4 B 37 H 2 N 38 H 3 N 39 H 3 B 40 F 3 N 41 H 3 B 42 H 3 B 43 F 2 B 44 H 2 N 45 F 3 N 46 H 3 B 47 F 2 N E Updated ARP tile nomenclature to match current plant. | ||
48 H 3 B | |||
J;S-401 2 Form ES-401-9 | |||
: 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. | |||
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 49 H 5 N B u LOD=5 w/out print; KIA MM; Replaced question. | |||
50 F 2 N 51 H 5 B 52 F 2 B 53 F 4 M 54 F 2 B 55 H 2 N 56 H 3 B 57 H 4 B 58 H 4 N 59 F 3 B 60 H 3 B 61 F 3 N 62 H 2 N 63 H 3 B 64 F 2 B 65 F 3 B 66 F 3 B 67 H 2 B 68 F 2 B | |||
ES-401 2 Form ES-401-9 | |||
: 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. | |||
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N UIE/S Explanation Focus Dist. Link units ward KIA Only 69 H 2 M 70 F 3 B 71 F 3 B 72 H 2 B E Provided print to prevent question from being LOD 5. | |||
73 F 3 B 74 F 2 B 75 F 2 X B u Minutia; Bases for limit is not in any procedure or lesson plan. | |||
Replaced question. | |||
76 H 3 y B 77 H 4 y B 78 H 3 y M 79 H 3 y B 80 H 3 y N 81 H 3 y B 82 H 4 y N 83 H 5 y B E Provided sanitized EOP graphs to avoid LOD 5 question. | |||
84 H 3 y N 85 H 3 y M 86 H 4 N N u Not SRO; Only need system knowledge to answer question; Rewrote question. | |||
87 H 4 y B 88 H 5 y B | |||
ES-401 2 Form ES-401-9 | |||
: 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. | |||
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 89 F 3 y B 90 H 5 y N 91 H 5 y B 92 H 2 y B 93 H 1 X y B u Overlap with facility audit exam; Replaced question. | |||
94 F 2 y B 95 H 3 y M 96 F 2 y N 97 H 3 y N 98 H 3 y B 99 F 3 y B 100 H 2 y B}} |
Latest revision as of 03:27, 6 February 2020
ML13227A107 | |
Person / Time | |
---|---|
Site: | Hope Creek ![]() |
Issue date: | 08/12/2013 |
From: | Todd Fish Operations Branch I |
To: | Public Service Enterprise Group |
Jackson D | |
Shared Package | |
ML13077A118 | List: |
References | |
TAC U01872 | |
Download: ML13227A107 (6) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 1 H 2 M 2 H 3 B 3 F 3 B 4 H 4 B 5 F 3 B 6 H 3 N 7 H 4 B 8 H 2 N Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 -5 (easy- difficult) rating scale (questions in the 2-4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
.. The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information) .
I The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc) . I The answer choices are a collection of unrelated true/false statements .
. The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable .
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem) .
4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid KIA but. as written, is not operational in content) . !
.. The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory) .
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons) .
The question requires reverse logic or application compared to the job requirements .
- 5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-on/y (KIA and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 9 H 2 B 10 H 3 M 11 H 2 B 12 F 3 B 13 H 3 M 14 H 3 B 15 F 3 B 16 F 4 B 17 H 3 B 18 F 2 B 19 H 2 B 20 F 3 X B u Content queued answer to SRO 082; Replaced question.
21 H 3 B 22 H 2 M 23 F 3 B 24 F 3 M 25 H 3 X B u SRO level knowledge; Requires knowledge of EOP actions; Replaced question.
26 H 4 N 27 F 2 N E Modified typographical error that made question invalid.
28 H 3 B
ES-401 2 Form ES-401-9 I 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 29 H 5 X B u Minutia- unusual and rarely used lineup; LOD 5; Replaced question.
30 H 4 N 31 F 3 B 32 H 4 B 33 H 3 B 34 H 3 N 35 H 3 B 36 H 4 B 37 H 2 N 38 H 3 N 39 H 3 B 40 F 3 N 41 H 3 B 42 H 3 B 43 F 2 B 44 H 2 N 45 F 3 N 46 H 3 B 47 F 2 N E Updated ARP tile nomenclature to match current plant.
48 H 3 B
J;S-401 2 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 49 H 5 N B u LOD=5 w/out print; KIA MM; Replaced question.
50 F 2 N 51 H 5 B 52 F 2 B 53 F 4 M 54 F 2 B 55 H 2 N 56 H 3 B 57 H 4 B 58 H 4 N 59 F 3 B 60 H 3 B 61 F 3 N 62 H 2 N 63 H 3 B 64 F 2 B 65 F 3 B 66 F 3 B 67 H 2 B 68 F 2 B
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N UIE/S Explanation Focus Dist. Link units ward KIA Only 69 H 2 M 70 F 3 B 71 F 3 B 72 H 2 B E Provided print to prevent question from being LOD 5.
73 F 3 B 74 F 2 B 75 F 2 X B u Minutia; Bases for limit is not in any procedure or lesson plan.
Replaced question.
76 H 3 y B 77 H 4 y B 78 H 3 y M 79 H 3 y B 80 H 3 y N 81 H 3 y B 82 H 4 y N 83 H 5 y B E Provided sanitized EOP graphs to avoid LOD 5 question.
84 H 3 y N 85 H 3 y M 86 H 4 N N u Not SRO; Only need system knowledge to answer question; Rewrote question.
87 H 4 y B 88 H 5 y B
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 89 F 3 y B 90 H 5 y N 91 H 5 y B 92 H 2 y B 93 H 1 X y B u Overlap with facility audit exam; Replaced question.
94 F 2 y B 95 H 3 y M 96 F 2 y N 97 H 3 y N 98 H 3 y B 99 F 3 y B 100 H 2 y B