ML101270493: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
(Created page by program invented by StriderTol)
 
Line 16: Line 16:


=Text=
=Text=
{{#Wiki_filter:ES-401 Written Examination Review Worksheet Form ES-401-9 WT-2009-10 Instructions [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable). 3. Check the appropriate box if a psychometric flaw is identified:  
{{#Wiki_filter:ES-401               WT-2009-10                                            Written Examination Review Worksheet                                                               Form ES-401-9
$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much ne edless information).  
: 1.      2.          3. Psychometric Flaws              4. Job Content Flaws        5. Other      6.      7.                                        8.
$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).  
Q#    LOK    LOD (F/H)  (1-5)  Stem Cues T/F        Cred. Partial Job- Minutia #/ Back-             Q= SRO B/M/N U/E/S                                            Explanation Focus                Dist.            Link            units ward    K/A Only 1
$ The answer choices are a collection of unrelated true/false statements.  
to                                                                                                                      COMMENTS PROVIDED AND RESOLVED DURING VALIDATION 100 Instructions
$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.  
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified:  
: 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).  
: 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).
$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).  
: 3. Check the appropriate box if a psychometric flaw is identified:
$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with ques tion in gallons).  
      $   The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
$ The question requires reverse logic or application compared to the job requirements. 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer
      $   The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any A U@ ratings (e.g., how the Appendix B psychometric attributes are not being met).
      $   The answer choices are a collection of unrelated true/false statements.
Attachment 8 Page 1 of 1 OBDI 202 - IOLE Process
      $   The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
: 3. Psychometric Flaws 4. Job Content Flaws 5. Other  Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia#/ units Back-wardQ= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 1 to 100                COMMENTS PROVIDED AND RESOLVED DURING VALIDATION}}
      $   One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
: 4. Check the appropriate box if a job content error is identified:
      $ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
      $ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
      $ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
      $ The question requires reverse logic or application compared to the job requirements.
: 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
: 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
: 7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
: 8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met).                                                                                       Page 1 of 1                                                   OBDI 202 - IOLE Process}}

Latest revision as of 19:33, 13 November 2019

WT-2009-10-Draft Written Examination Review Worksheet
ML101270493
Person / Time
Site: Waterford Entergy icon.png
Issue date: 10/05/2009
From: Brian Larson
NRC Region 4
To:
Entergy Operations
References
50-382/09-301, WT-2009-10
Download: ML101270493 (1)
v • d • e


Text

ES-401 WT-2009-10 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1

to COMMENTS PROVIDED AND RESOLVED DURING VALIDATION 100 Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).
3. Check the appropriate box if a psychometric flaw is identified:

$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

$ The answer choices are a collection of unrelated true/false statements.

$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

$ The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met). Page 1 of 1 OBDI 202 - IOLE Process
Retrieved from "https://kanterella.com/w/index.php?title=ML101270493&oldid=2133369"