ML102450367

From kanterella
Revision as of 22:20, 30 April 2019 by StriderTol (talk | contribs) (Created page by program invented by StriderTol)
Jump to navigation Jump to search
RB-2010-Draft Written Exam Review
ML102450367
Person / Time
Site: River Bend Entergy icon.png
Issue date: 04/12/2010
From: Garchow S M
Operations Branch IV
To:
Entergy Operations
References
50-458/10-04
Download: ML102450367 (7)
v • d • e


Text

ES-401 RB-2010-04 Written Examination Review Worksheet Form ES-401-9 Instructions [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable). 3. Check the appropriate box if a psychometric flaw is identified:

$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much ne edless information).

$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

$ The answer choices are a collection of unrelated true/false statements.

$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified:

$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with ques tion in gallons).

$ The question requires reverse logic or application compared to the job requirements. 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer

=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any A U@ ratings (e.g., how the Appendix B psychometric attributes are not being met).

Attachment 8 Page 1 of 7 OBDI 202 - IOLE Process

3. Psychometric Flaws 4. Job Content Flaws 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.

Dist. Partial Job-LinkMinutia#/

units Back-wardQ= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 1 H 2 N S C, CR, 41.10 2 F 2 N S A, CR, 41.7 3 F 3 N S A, CR, 41.7 4 H 2 M E S B, CR, 41.6 - Change "trips" to "trip" in distactors. This change will be made.

The text editing was made as recommended. 5 F 3 N S D, CR, 41.7 6 F 3 N S C, CR, 41.7 7 F 3 U B U S D, CR, 41.10 - This is a KA mismatch due to being focused on TBCW rather than CCW. This question will be rewritten.

ES-401 RB-2010-04 2 Form ES-401-9 Attachment 8 Page 2 of 7 OBDI 202 - IOLE Process

3. Psychometric Flaws 4. Job Content Flaws 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.

Dist. Partial Job-LinkMinutia#/

units Back-wardQ= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation This question was replaced with a new question. 8 H 3 N S B, CR, 41.4 9 F 2 N S A, CR, 41.8 10 F 3 N S C, CR, 41.4 11 H 3 N S A, CR, 41.3 12 H 4 N S B, CR, 41.5 13 F 3 M S A, CR, 41.7 14 F 3 N S D, CR, 41.9 15 H 3 N S C, CR, 41.10 16 H 3 N S B, CR, 41.2 17 H 3 N S D, CR, 41.10 18 F 3 N S B, CR, 41.13 19 F 3 N S C, CR, 41.10 20 H 4 N S A, CR, 41.5 21 H 3 N S B, CR, 41.5 22 H 3 N S A, CR, 41.9 23 H 3 N S B, CR, 41.7 24 H 3 U N U S A, CR, 41.6 - This question is U because distractor B is also correct. Distractor B will be revised. Distractor B was rewritten. 25 F 3 N S B, CR, 41.7 26 F 3 N S A, CR, 41.10 27 H 3 N S C, CR, 41.11 28 H 3 N S C, CR, 41.7 29 H 3 N S D, CR, 41.7 ES-401 RB-2010-04 3 Form ES-401-9 Attachment 8 Page 3 of 7 OBDI 202 - IOLE Process

3. Psychometric Flaws 4. Job Content Flaws 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.

Dist. Partial Job-LinkMinutia#/

units Back-wardQ= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 30 H 3 N S D, CR, 41.8 31 H 3 N S D, CR, 41.7 32 F 3 N S C, CR, 41.10 33 H 3 N S B, CR, 41.6 34 F 3 N S C, CR, 41.10 35 H 3 N S B, CR, 41.2 36 F 3 N S C, CR, 41.6 37 H 3 N S D, CR, 41.6 38 H 3 E N E S B, CR, 41.6 - Distractor A can also be argued as being correct since it is a subset of the correct answer. Distractor A will be revised. Distractor A was rewritten. 39 H 3 N S C, CR, 41.7 40 H 3 N S C, CR, 41.7 41 H 3 N S D, CR, 41.7 42 H 3 N S B, CR, 41.7 43 H 3 B S B, CR, 41.5 44 H 3 N S A, CR, 41.5 45 H 3 N E S D, CR, 41.7 - Need to discuss this question as it seems annulus pressure would normally be negative and would remain so under the described conditions in the stem. After discussing with the exam author, this question is correct as written. 46 H 3 N S B, CR, 41.7 47 H 3 M S A, CR, 41.7 48 F 3 N S A, CR, 41.7 49 H 3 N S D, CR, 41.7 50 H 3 B S C, CR, 41.7 ES-401 RB-2010-04 4 Form ES-401-9 Attachment 8 Page 4 of 7 OBDI 202 - IOLE Process

3. Psychometric Flaws 4. Job Content Flaws 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.

Dist. Partial Job-LinkMinutia#/

units Back-wardQ= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 51 H 3 N S D, CR, 41.7 52 H 3 N S D, CR, 41.7 53 H 2 B S B, CR, 41.10 54 F 2 E N E S D, CR, 41.3 - Discuss this question with exam author. This question is ok as written. 55 F 3 E N E S C, CR, 41.4 - Discuss this question with exam author. The stem will be revised to improve the readability.

Part of the question in the stem was highlighted to improve the focus of the question. 56 H 3 N S B, CR, 41.4 57 H 3 N S B, CR, 41.3 58 F 2 N S A, CR, 41.7 59 F 3 U N U S C, CR, 41.13 - This is a KA mismatch because it does not involve fuel handling. Suggest this KA be rejected and a new one selected. This question will be rewritten. This question was replaced with a new question. 60 F 3 N S B, CR, 41.10 61 H 3 E M E S A, CR, 41.5 - Revise distractor B so it reads 980 MW instead of 750 MW. This edit change will be made.

This change was made as recommended. 62 H 3 N S C, CR, 41.4 63 H 3 N S A, CR, 41.5 64 F 3 N S D, CR, 41.4 65 H 2 N S D, CR, 41.5 66 F 3 N S D, CR, 41.10 67 F 2 N S C, CR, 41.10 68 F 3 N S D, CR, 41.10 69 F 3 B S C, CR, 41.10 ES-401 RB-2010-04 5 Form ES-401-9 Attachment 8 Page 5 of 7 OBDI 202 - IOLE Process

3. Psychometric Flaws 4. Job Content Flaws 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.

Dist. Partial Job-LinkMinutia#/

units Back-wardQ= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 70 F 3 E N E S A, CR, 41.10 - Not sure this is a memory item; check with exam author. Based on feedback from the validating crew, this is a memory item. 71 F 3 N S B, CR, 41.10 72 F 3 N S A, CR, 41.9 73 H 4 N S A, CR, 41.11 74 F 3 U N U S C, CR, 43.5 - This is an SRO Only question in that it tests on the emergency plan notification requirements. This question will be revised. This question was replaced with a new question. 75 F 2 B S C, CR, 41.10 76 H 3 B S D, CR, 43.5 77 H 3 N S C, CR, 43.5 78 H 3 N S C, CR, 43.5 79 F 2 N S B, CR, 43.5 80 H 3 N S A, OR, 43.5 81 H 3 E N E S A, CR, 43.5 - Add another EOP related distractor other than the correct answer. This change will be made. Distractor "D" was rewritten. 82 F 3 N S D, OR, 43.2 83 F 2 N S A, CR, 43.5 84 H 3 M S B, CR, 43.5 85 F 3 N S D, CR, 43.5 86 H 3 N S B, CR, 43.5 87 H 3 N S C, CR, 43.5 88 H 3 N S A, CR, 43.5 89 F 3 N S A, CR, 43.2 ES-401 RB-2010-04 6 Form ES-401-9 Attachment 8 Page 6 of 7 OBDI 202 - IOLE Process

3. Psychometric Flaws 4. Job Content Flaws 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.

Dist. Partial Job-LinkMinutia#/

units Back-wardQ= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 90 H 3 N S D, CR, 43.5 91 H 3 N S B, CR, 43.2 92 H 3 N S C, CR, 43.5 93 F 3 E N E S A, CR, 43.2 - Candidates are not typically expected to have 12 hour1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br /> TS action statements memorized. The exam author will check with Ops on this. This question was replaced with a new question. 94 E N E S C, CR, 43.5 - This is a Tier 3 question and therefore needs to be generic and not symptom/system specific. This question was replaced. 95 H 3 E N E S D, OR, 43.5 - This is a Tier 3 question and therefore needs to be generic and not symptom specific. Recommend deleting the RHR status from the stem and change so it reads "Subsequent to the above conditions, a loss of all shutdown cooling occurs." This change will be made.

The question was revised as discussed above. 96 F 2 N S A, CR, 43.5 97 H 2 E N E S D, CR, 43.4 - Discuss this question with exam author. This question is ok as written. 98 F 3 N S B, CR, 43.4 99 H 2 E N E S B, CR, 43.5 - This is a Tier 3 question and therefore needs to be generic and not symptom specific. This question was replaced with a new question. 100 H 3 N S D, CR, 43.5 B= 2 F=32 (43%) E=7 M= 9 H=43 (57%) U=4 RO TOTALS: N= 64 Additional Notes: B= 1 F=8 (32%) E=6 SRO TOTALS: M= 1 H=17 (68%) U=0 Additional Notes:

ES-401 RB-2010-04 7 Form ES-401-9 Attachment 8 Page 7 of 7 OBDI 202 - IOLE Process N= 23

GENERAL COMMENT

S:

1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N
2. Chief Examiner follow-up comments are indicated in blue. 3. Average difficulty is _2.9 ___ on the RO exam and 2.8___ on the SRO exam. These values fall within the expected range for this reviewer
4. The 10CFR55.41/43 distribution is: RO / SRO 41.1 = 0 43.1 = 0 41.2 = 2 43.2 = 4 41.3 = 2 43.3 = 0 41.4 = 6 43.4 = 2 41.5 = 9 43.5 = 19 41.6 = 6 43.6 = 0 41.7 = 23 43.7 = 0 41.8 = 2 41.9 = 3 41.10 = 18 41.11 = 2 41.12 = 0 41.13 = 2 41.14 =
5. The answer distribution is: RO / SRO A = 18 (25%) / 7 (28%) B = 19 (25%) / 6 (24%) C = 19 (25%) / 5 (20%)

D = 19 (25%) / 7 (28%)

6. There are 0 / 3 questions with attachments provided.
7. This exam was reviewed by S. Garchow on March 9-10, 2010.
Retrieved from "https://kanterella.com/w/index.php?title=ML102450367&oldid=1565753"