ML13227A107: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
(Created page by program invented by StriderTol)
Line 19: Line 19:


=Text=
=Text=
{{#Wiki_filter:ES-401 Written Examination Review Worksheet Form ES-401-9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-Minutia #I Back-Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 1 H 2 M 2 H 3 B 3 F 3 B 4 H 4 B 5 F 3 B 6 H 3 N 7 H 4 B 8 H 2 N Instructions
{{#Wiki_filter:ES-401                                                           Written Examination Review Worksheet                                                                     Form ES-401-9
: 1.         2.         3. Psychometric Flaws               4. Job Content Flaws           5. Other   6. 7.                                       8.
Q#   LOK       LOD (F/H)     (1-5) Stem Cues         T/F   Cred. Partial Job- Minutia #I           Back-   Q= SRO B/M/N U/E/S                                     Explanation Focus                   Dist.         Link             units ward       KIA Only 1   H         2                                                                                         M 2   H         3                                                                                         B 3   F         3                                                                                         B 4     H         4                                                                                         B 5   F         3                                                                                         B 6     H         3                                                                                         N 7   H         4                                                                                         B 8   H         2                                                                                         N Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
: 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question using a 1 -5 (easy-difficult) rating scale (questions in the 2-4 range are acceptable).
: 1.         Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
: 3. Check the appropriate box if a psychometric flaw is identified: . The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information) . I . The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc) . I . The answer choices are a collection of unrelated true/false statements . . The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable . . One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem) . 4. Check the appropriate box if a job content error is identified: . The question is not linked to the job requirements (i.e., the question has a valid KIA but. as written, is not operational in content) . ! . The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory) . . The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons) . . The question requires reverse logic or application compared to the job requirements . 5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-on/y (KIA and license level mismatches are unacceptable).
: 2.         Enter the level of difficulty (LOD) of each question using a 1 -5 (easy- difficult) rating scale (questions in the 2-4 range are acceptable).
: 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
: 3.         Check the appropriate box if a psychometric flaw is identified:
: 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).
          ..          The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information) .
ES-401 2 Form ES-401-9 ------1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-Minutia #I Back-Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 9 H 2 B 10 H 3 M 11 H 2 B 12 F 3 B 13 H 3 M 14 H 3 B 15 F 3 B 16 F 4 B 17 H 3 B 18 F 2 B 19 H 2 B 20 F 3 X B u Content queued answer to SRO 082; Replaced question.
            ..
21 H 3 B 22 H 2 M 23 F 3 B 24 F 3 M 25 H 3 X B u SRO level knowledge; Requires knowledge of EOP actions; Replaced question.
I The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc) .                                                             I The answer choices are a collection of unrelated true/false statements .
26 H 4 N 27 F 2 N E Modified typographical error that made question invalid. 28 H 3 B ES-401 2 Form ES-401-9 I 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-Minutia #I Back-Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 29 H 5 X B u Minutia-unusual and rarely used lineup; LOD 5; Replaced question.
            .         The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable .
30 H 4 N 31 F 3 B 32 H 4 B 33 H 3 B 34 H 3 N 35 H 3 B 36 H 4 B 37 H 2 N 38 H 3 N 39 H 3 B 40 F 3 N 41 H 3 B 42 H 3 B 43 F 2 B 44 H 2 N 45 F 3 N 46 H 3 B 47 F 2 N E Updated ARP tile nomenclature to match current plant. 48 H 3 B J;S-401 2 Form ES-401-9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-Minutia #I Back-Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 49 H 5 N B u LOD=5 w/out print; KIA MM; Replaced question.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem) .
50 F 2 N 51 H 5 B 52 F 2 B 53 F 4 M 54 F 2 B 55 H 2 N 56 H 3 B 57 H 4 B 58 H 4 N 59 F 3 B 60 H 3 B 61 F 3 N 62 H 2 N 63 H 3 B 64 F 2 B 65 F 3 B 66 F 3 B 67 H 2 B 68 F 2 B ES-401 2 Form ES-401-9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-Minutia #I Back-Q= SRO B/M/N UIE/S Explanation Focus Dist. Link units ward KIA Only 69 H 2 M 70 F 3 B 71 F 3 B 72 H 2 B E Provided print to prevent question from being LOD 5. 73 F 3 B 74 F 2 B 75 F 2 X B u Minutia; Bases for limit is not in any procedure or lesson plan. Replaced question.
4.
76 H 3 y B 77 H 4 y B 78 H 3 y M 79 H 3 y B 80 H 3 y N 81 H 3 y B 82 H 4 y N 83 H 5 y B E Provided sanitized EOP graphs to avoid LOD 5 question.
            ..
84 H 3 y N 85 H 3 y M 86 H 4 N N u Not SRO; Only need system knowledge to answer question; Rewrote question.
Check the appropriate box if a job content error is identified:
87 H 4 y B 88 H 5 y B ES-401 2 Form ES-401-9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-Minutia #I Back-Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 89 F 3 y B 90 H 5 y N 91 H 5 y B 92 H 2 y B 93 H 1 X y B u Overlap with facility audit exam; Replaced question.
The question is not linked to the job requirements (i.e., the question has a valid KIA but. as written, is not operational in content) .                       !
94 F 2 y B 95 H 3 y M 96 F 2 y N 97 H 3 y N 98 H 3 y B 99 F 3 y B 100 H 2 y B}}
          ..          The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory) .
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons) .
The question requires reverse logic or application compared to the job requirements .
: 5.         Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-on/y (KIA and license level mismatches are unacceptable).
: 6.         Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
: 7.         Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
: 8.         At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).
 
ES-401                                                                         2                                                                 Form ES-401-9
                                                                              -       -----
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws     5. Other   6. 7.                                   8.
Q#   LOK LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #I     Back- Q=   SRO B/M/N U/E/S                             Explanation Focus             Dist.         Link         units ward   KIA Only 9   H     2                                                                           B 10   H     3                                                                           M 11   H     2                                                                           B 12   F     3                                                                           B 13   H     3                                                                           M 14   H     3                                                                           B 15   F     3                                                                           B 16   F     4                                                                           B 17   H     3                                                                           B 18   F     2                                                                           B 19   H     2                                                                           B 20   F     3           X                                                             B     u   Content queued answer to SRO 082; Replaced question.
21   H     3                                                                           B 22   H     2                                                                           M 23   F     3                                                                           B 24   F     3                                                                           M 25   H     3                                     X                                     B     u   SRO level knowledge; Requires knowledge of EOP actions; Replaced question.
26   H     4                                                                           N 27   F     2                                                                           N     E   Modified typographical error that made question invalid.
28   H     3                                                                           B
 
ES-401                                                                         2                                                             Form ES-401-9 I       1. 2.       3. Psychometric Flaws         4. Job Content Flaws     5. Other 6. 7.                                 8.
Q#   LOK LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #I     Back- Q= SRO B/M/N U/E/S                             Explanation Focus             Dist.         Link         units ward   KIA Only 29   H     5                                             X                           B   u   Minutia- unusual and rarely used lineup; LOD 5; Replaced question.
30   H     4                                                                         N 31   F     3                                                                         B 32   H     4                                                                         B 33   H     3                                                                         B 34   H     3                                                                         N 35   H     3                                                                         B 36   H     4                                                                         B 37   H     2                                                                         N 38   H     3                                                                         N 39   H     3                                                                         B 40   F     3                                                                         N 41   H     3                                                                         B 42   H     3                                                                         B 43   F     2                                                                         B 44   H     2                                                                         N 45   F     3                                                                         N 46   H     3                                                                         B 47   F     2                                                                         N   E   Updated ARP tile nomenclature to match current plant.
48   H     3                                                                         B
 
J;S-401                                                                         2                                                             Form ES-401-9
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws     5. Other 6. 7.                                 8.
Q#   LOK LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #I     Back- Q= SRO B/M/N U/E/S                           Explanation Focus             Dist.         Link         units ward   KIA Only 49   H     5                                                               N       B   u   LOD=5 w/out print; KIA MM; Replaced question.
50   F     2                                                                         N 51   H     5                                                                         B 52   F     2                                                                         B 53   F     4                                                                         M 54   F     2                                                                         B 55   H     2                                                                         N 56   H     3                                                                         B 57   H     4                                                                         B 58   H     4                                                                         N 59   F     3                                                                         B 60   H     3                                                                         B 61   F     3                                                                         N 62   H     2                                                                         N 63   H     3                                                                         B 64   F     2                                                                         B 65   F     3                                                                         B 66   F     3                                                                         B 67   H     2                                                                         B 68   F     2                                                                         B
 
ES-401                                                                         2                                                                 Form ES-401-9
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws     5. Other 6. 7.                                       8.
Q#   LOK LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #I     Back- Q= SRO B/M/N UIE/S                               Explanation Focus             Dist.         Link         units ward   KIA Only 69   H     2                                                                         M 70   F     3                                                                         B 71   F     3                                                                         B 72   H     2                                                                         B   E   Provided print to prevent question from being LOD 5.
73   F     3                                                                         B 74   F     2                                                                         B 75   F     2                                             X                           B   u   Minutia; Bases for limit is not in any procedure or lesson plan.
Replaced question.
76   H     3                                                                     y B 77   H     4                                                                     y B 78   H     3                                                                     y M 79   H     3                                                                     y B 80   H     3                                                                     y N 81   H     3                                                                     y B 82   H     4                                                                     y N 83   H     5                                                                     y B   E   Provided sanitized EOP graphs to avoid LOD 5 question.
84   H     3                                                                     y N 85   H     3                                                                     y M 86   H     4                                                                     N N   u   Not SRO; Only need system knowledge to answer question; Rewrote question.
87   H     4                                                                     y B 88   H     5                                                                     y B
 
ES-401                                                                         2                                                               Form ES-401-9
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws     5. Other 6. 7.                                     8.
Q# LOK   LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #I     Back- Q= SRO B/M/N U/E/S                               Explanation Focus             Dist.         Link         units ward   KIA Only 89   F     3                                                                     y B 90   H     5                                                                     y N 91   H     5                                                                     y B 92   H     2                                                                     y B 93   H     1           X                                                         y B   u   Overlap with facility audit exam; Replaced question.
94   F     2                                                                     y B 95   H     3                                                                     y M 96   F     2                                                                     y N 97   H     3                                                                     y N 98   H     3                                                                     y B 99   F     3                                                                     y B 100   H     2                                                                     y B}}

Revision as of 15:41, 4 November 2019

Facility Comments on Draft NRC-Developed Exam (Written) (Folder 2)
ML13227A107
Person / Time
Site: Hope Creek PSEG icon.png
Issue date: 08/12/2013
From: Todd Fish
Operations Branch I
To:
Public Service Enterprise Group
Jackson D
Shared Package
ML13077A118 List:
References
TAC U01872
Download: ML13227A107 (6)
v • d • e


Text

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 1 H 2 M 2 H 3 B 3 F 3 B 4 H 4 B 5 F 3 B 6 H 3 N 7 H 4 B 8 H 2 N Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question using a 1 -5 (easy- difficult) rating scale (questions in the 2-4 range are acceptable).
3. Check the appropriate box if a psychometric flaw is identified:

.. The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information) .

..

I The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc) . I The answer choices are a collection of unrelated true/false statements .

. The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable .

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem) .

4.

..

Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid KIA but. as written, is not operational in content) .  !

.. The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory) .

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons) .

The question requires reverse logic or application compared to the job requirements .

5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-on/y (KIA and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 2 Form ES-401-9

- -----

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 9 H 2 B 10 H 3 M 11 H 2 B 12 F 3 B 13 H 3 M 14 H 3 B 15 F 3 B 16 F 4 B 17 H 3 B 18 F 2 B 19 H 2 B 20 F 3 X B u Content queued answer to SRO 082; Replaced question.

21 H 3 B 22 H 2 M 23 F 3 B 24 F 3 M 25 H 3 X B u SRO level knowledge; Requires knowledge of EOP actions; Replaced question.

26 H 4 N 27 F 2 N E Modified typographical error that made question invalid.

28 H 3 B

ES-401 2 Form ES-401-9 I 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 29 H 5 X B u Minutia- unusual and rarely used lineup; LOD 5; Replaced question.

30 H 4 N 31 F 3 B 32 H 4 B 33 H 3 B 34 H 3 N 35 H 3 B 36 H 4 B 37 H 2 N 38 H 3 N 39 H 3 B 40 F 3 N 41 H 3 B 42 H 3 B 43 F 2 B 44 H 2 N 45 F 3 N 46 H 3 B 47 F 2 N E Updated ARP tile nomenclature to match current plant.

48 H 3 B

J;S-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 49 H 5 N B u LOD=5 w/out print; KIA MM; Replaced question.

50 F 2 N 51 H 5 B 52 F 2 B 53 F 4 M 54 F 2 B 55 H 2 N 56 H 3 B 57 H 4 B 58 H 4 N 59 F 3 B 60 H 3 B 61 F 3 N 62 H 2 N 63 H 3 B 64 F 2 B 65 F 3 B 66 F 3 B 67 H 2 B 68 F 2 B

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N UIE/S Explanation Focus Dist. Link units ward KIA Only 69 H 2 M 70 F 3 B 71 F 3 B 72 H 2 B E Provided print to prevent question from being LOD 5.

73 F 3 B 74 F 2 B 75 F 2 X B u Minutia; Bases for limit is not in any procedure or lesson plan.

Replaced question.

76 H 3 y B 77 H 4 y B 78 H 3 y M 79 H 3 y B 80 H 3 y N 81 H 3 y B 82 H 4 y N 83 H 5 y B E Provided sanitized EOP graphs to avoid LOD 5 question.

84 H 3 y N 85 H 3 y M 86 H 4 N N u Not SRO; Only need system knowledge to answer question; Rewrote question.

87 H 4 y B 88 H 5 y B

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 89 F 3 y B 90 H 5 y N 91 H 5 y B 92 H 2 y B 93 H 1 X y B u Overlap with facility audit exam; Replaced question.

94 F 2 y B 95 H 3 y M 96 F 2 y N 97 H 3 y N 98 H 3 y B 99 F 3 y B 100 H 2 y B

Retrieved from "https://kanterella.com/w/index.php?title=ML13227A107&oldid=2059681"