ML101270493

WT-2009-10-Draft Written Examination Review Worksheet

ML101270493
Person / Time
Site:	Waterford
Issue date:	10/05/2009
From:	Brian Larson NRC Region 4
To:	Entergy Operations
References
50-382/09-301, WT-2009-10
Download: ML101270493 (1)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9 WT-2009-10 Instructions [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable). 3. Check the appropriate box if a psychometric flaw is identified:

$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much ne edless information).

$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

$ The answer choices are a collection of unrelated true/false statements.

$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified:

$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with ques tion in gallons).

$ The question requires reverse logic or application compared to the job requirements. 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer

=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any A U@ ratings (e.g., how the Appendix B psychometric attributes are not being met).

Attachment 8 Page 1 of 1 OBDI 202 - IOLE Process

3. Psychometric Flaws 4. Job Content Flaws 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia#/ units Back-wardQ= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 1 to 100 COMMENTS PROVIDED AND RESOLVED DURING VALIDATION