ML18061A170

From kanterella
Jump to navigation Jump to search
2017-12 Proposed Written Exam Comments
ML18061A170
Person / Time
Site: Grand Gulf Entergy icon.png
Issue date: 12/11/2017
From: Vincent Gaddy
Operations Branch IV
To:
Entergy Operations
References
Download: ML18061A170 (10)
v • d • e


Text

ES-401 GG-2017-12 Written Examination Review Worksheet Form ES-401-9 Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question using a 1 to 5 (easy to difficult) rating scale (questions in the 2 to 4 range are acceptable).
3. Check the appropriate box if a psychometric flaw is identified:
a. The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
b. The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
c. The answer choices are a collection of unrelated true/false statements.
d. The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
e. One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
4. Check the appropriate box if a job content error is identified:
a. The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
b. The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
c. The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
d. The question requires reverse logic or application compared to the job requirements.
5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? Place the answer letter here Place the 55.41 or 55.43 item here
8. At a minimum, explain any Unsat ratings (e.g., how the Appendix B psychometric attributes are not being met).

1.

2. LOD 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6 7 Answer CFR 8 LOK stem cred #/ back- SRO- 55.41x Q# (F/H) (1-5) cue T/F partial job link minutia K/A B/M/N U/E/S A/B/C/D Explanation focus dist units ward only 55.43y Handout Provided Minor editorial edits Added revision numbers to handout material 1 H 3 B E C 10 Added space between bullets Question revised - Sat Sample Test Item Review 2 H 3 N S A 8 Minor editorial edits Added current plant conditions Re-ordered answers short to long 3 H 2.5 B E D 7 Corrected Technical references Question revised - Sat Unsat - K/A doesn't address impact of MT trip on pressure regulating system 4 H 3 X M U D 5 Submitted new question (Modified)

Question revised - Sat 2017 NRC Exam (1)

Editorial edits Removed information from each question and put in stem 5 F 2 B E B 7 Added plausibility statements Changed 10CFR to 7 Question revised - Sat a Rev 6

ES-401 GG-2017-12 Written Examination Review Worksheet Form ES-401-9 1.

2. LOD 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6 7 Answer CFR 8 LOK stem cred #/ back- SRO- 55.41x Q# (F/H) (1-5) cue T/F partial job link minutia K/A B/M/N U/E/S A/B/C/D Explanation focus dist units ward only 55.43y Handout Provided Minor editorial edits Changed inch symbol to inches, Changed wording in 6 H 3 B E B 10 distracter description to indicated level Question revised - Sat Editorial edits Changed to operating pump Changed stem verb to identifies and wording to without CCW flow 7 H 4 B E D 4 Swapped answers B and C Changed LOD to 4 Question revised - Sat Editorial edits Defined PAC in stem Editorial changes in answers 8 F 3 N E A 4 Added clarification in distractor explanation Question revised - Sat Minor editorial edits 9 H 3 B E A 8 Used bullets in stem Question revised - Sat Minor editorial edits 10 H 3 N E C 9 Gramatical errors Question revised - Sat Minor editorial edits 11 H 3 N E B 7 Used the word status instead of availability Question revised - Sat Editorial edits 12 H 3 N E B 14 Reworded stem and #2 question for clarity Question revised - Sat Handout Provided 13 H 2 B S C 5 Handout Provided - 2015 NRC Exam (2)

Minor editorial edits Added current data points Changed answer A from Fuel Zone Range only 14 H 3 B E B 10 Make only All Caps Changed 10 CFR # to 10 Question revised - Sat Minor editorial edits Moved "go" to the stem and deleted from each answer 15 F 2 N E D 5 Question revised - Sat Minor editorial edits Changed in answer D, FT to ft.

16 F 2 N E C 7 Question revised - Sat 17 F 2 N S C 7 a Rev 6

ES-401 GG-2017-12 Written Examination Review Worksheet Form ES-401-9 1.

2. LOD 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6 7 Answer CFR 8 LOK stem cred #/ back- SRO- 55.41x Q# (F/H) (1-5) cue T/F partial job link minutia K/A B/M/N U/E/S A/B/C/D Explanation focus dist units ward only 55.43y Minor editorial edits Changed answers from short to long 18 F 3 N E D 10 Added CAPS to components in stem Question revised - Sat Minor editorial edits Changed verb to identifies in stem 19 H 3 B E D 10 Editorial changes in stem and answers ALL CAPS on the word "only" Question revised - Sat 20 F 2 N S B 10 Sample Test Item Review Changed 10 CFR # to 3, MSIVs are components of the 21 H 3 N S A 3 primary system Minor editorial edits Editorial changes in stem and answers Added in inches to stem 22 H 3 N E C 4 Removed inch symbol in all answers Question revised - Sat Editorial edits Made question part of stem 2 parts 23 H 2.5 N E B 9 Made answers 2X2 Added space in bullets Question revised - Sat Changed LOD to 3 24 F 3 B S A 10 Sample Test Item Review Editorial edits 25 H 3 N E C 10 Created two line answers Question revised - Sat Editorial edits Editorial changes in stem and answers 26 H 3 N E A 10 Question revised - Sat Editorial edits Editorial changes in stem and answers 27 H 3 N E C 10 Question revised - Sat 28 F 2 N S B 8 Minor editorial edits Editorial changes in stem and answers 29 H 3 N E D 8 Question revised - Sat Minor editorial edits Editorial changes in stem and answers 30 H 3 N E A 7 Question revised - Sat Minor editorial edits Editorial changes in stem and answers 31 F 2 B E B 14 Re-ordered answers short to long Question revised - Sat a Rev 6

ES-401 GG-2017-12 Written Examination Review Worksheet Form ES-401-9 1.

2. LOD 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6 7 Answer CFR 8 LOK stem cred #/ back- SRO- 55.41x Q# (F/H) (1-5) cue T/F partial job link minutia K/A B/M/N U/E/S A/B/C/D Explanation focus dist units ward only 55.43y Editorial edits Editorial changes in stem and answers Removed third part (#3) of stem and third part of all 32 H 3 N E D 7 answers.

Question revised - Sat Editorial edits - rewrite question Editorial changes in stem and answers 33 H 3 N E A 7 Changed part one of answers per Chief's request Question revised - Sat 34 F 2 N S A 6 Editorial edits Changed stem to fill in the blank with 2 parts 35 H 2 N E B 6 Changed 10CFR to 6 Question revised - Sat Editorial edits Editorial changes in stem and answers 36 H 3 B E A 7 Removed reference to EPA breakers and third part of answers Question revised - Sat 2017 NRC Exam (3)

Minor editorial edits 37 F 3 B S C 7 Chnaged to Memory/Fundamental Minor editorial edits Changed answer B from anytime TO in all conditions Defined SRM/IRM overlap in Explanation 38 F 3 N E D 2 Changed LOD to 3 Question revised - Sat Minor editorial edits Minor changes to stem 39 H 3 N E C 2 Question revised - Sat Changed LOD to 3 40 F 3 N S C 7 41 F 2 N S A 3 Editorial edits Editorial changes in stem and answers 42 H 3 N E A 7 Added noun name of G33-F004 Changed answers short to long Question revised - Sat changed answer order for distribution 43 F 2 B S C 3 Editorial edits Editorial changes in stem and answers Added initially in stem 44 H 3 M E D 7 Replaced symbol with inches Question revised - Sat a Rev 6

ES-401 GG-2017-12 Written Examination Review Worksheet Form ES-401-9 1.

2. LOD 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6 7 Answer CFR 8 LOK stem cred #/ back- SRO- 55.41x Q# (F/H) (1-5) cue T/F partial job link minutia K/A B/M/N U/E/S A/B/C/D Explanation focus dist units ward only 55.43y Minor editorial edits Editorial changes in stem 45 H 3 B E C 11 Question revised - Sat Minor editorial edits Editorial changes in stem 46 H 3 B E B 7 Changed 3000 to 3.0 KV to match indicator Question revised - Sat Sample Test Item Review Minor editorial edits Editorial changes in stem and answers 47 H 3 N E C 8 Changed 10CFR to 8 Changed LOK to H and LOD to 3 Question revised - Sat Corrected spelling error on revised question.

48 H 3 B E B 7 Question revised - Sat 49 H 3 B S D 7 Editorial edits Editorial changes in answers 50 F 3 N E A 10 Question revised - Sat Editorial edits Editorial changes in stem 51 F 3 N E D 7 Changed to Memory Fundamental Question revised - Sat 52 F 2 B S A 4 53 F 2 N S D 4 Minor rev to make stem questions consistent 54 H 3 N E D 2 Question revised - Sat Used Recirc in stem Changed to Memory Fundamental 55 F 3 N S D 4 Changed to LOD 3 Changed 10CFR # to 12 56 H 3 N S C 12 57 H 3 N S D 8 Minor editorial edits Defined LOCA in stem Changed time for load rise to 18 min.

58 H 4 N E C 8 Changed LOD to 4 Question revised - Sat Editorial edits Editorial changes in stem 59 F 2 N E B 1 Changed 10CFR to 1 Question revised - Sat Editorial edits 60 H 3 B E B 4 Editorial changes in stem and answers Question revised - Sat a Rev 6

ES-401 GG-2017-12 Written Examination Review Worksheet Form ES-401-9 1.

2. LOD 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6 7 Answer CFR 8 LOK stem cred #/ back- SRO- 55.41x Q# (F/H) (1-5) cue T/F partial job link minutia K/A B/M/N U/E/S A/B/C/D Explanation focus dist units ward only 55.43y 61 H 3 B S A 4 62 F 2 N S D 5 Editorial edits 63 H 3 N E B 7 Editorial changes in stem and answers Question revised - Sat Editorial edits 64 H 3 N E D 4 Editorial changes in stem and answers Question revised - Sat Editorial edits 65 H 3 B E A 11 Editorial changes in stem and answers Question revised - Sat Changed to LOD 3 66 F 3 B S B 10 67 F 2 B S A 10 Editorial edits 68 F 2 N E B 10 Removed first sentence of the stem Question revised - Sat 69 F 2 B S A 10 Minor editorial edits Added noun name of P21 system 70 F 3 B E D 10 Changed LOD to 3 Question revised - Sat 71 F 2 B S B 10 Unsat - not a Tier 3 question Incorporated new question 72 F 2 X B U B 12 2017 NRC Exam (4)

Question revised - Sat Editorial edits Editorial changes in stem and answers Moved part of all answers to stem and made a fill in 73 F 2 B E C 12 blank Question revised - Sat Editorial edits - rewrite question 74 F 2 N E D 10 Changed answers to a 2X2 format Question revised - Sat Editorial edits Editorial changes in stem and answers 75 F 3 N E A 10 Question revised - Sat Handout Provided Minor editorial edit 76 H 3 N E C 2 Re-ordered answers hi to lo Added page numbers to Handout material Question revised - Sat Changed 10CFR # to 5 77 H 3 N S B 5 a Rev 6

ES-401 GG-2017-12 Written Examination Review Worksheet Form ES-401-9 1.

2. LOD 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6 7 Answer CFR 8 LOK stem cred #/ back- SRO- 55.41x Q# (F/H) (1-5) cue T/F partial job link minutia K/A B/M/N U/E/S A/B/C/D Explanation focus dist units ward only 55.43y Replace K/A with TS Bases to better comply with SRO-only. Same question with minor edits.

Editorial changes 78 F 2 B E A 2 Changed K/A to 2.2.25 Question revised - Sat Handout Provided Minor editorial edits Editorial changes, changed considered to available, moved second to third bullet.

Added sections of SOPP to given to the applicants 79 H 4 N E D 5 Changed LOD to 4 Changed 10CFR # to 5 Question revised - Sat Sample Test Item Review Editorial edits Editorial changes to stem Changed rad monitors name to EP-4 Table SC-1 80 H 3 N E C 4 description Changed 10CFR # to 4 Question revised - Sat Handout Provided Editorial edits Changed to Given the following 81 H 3 B E A 5 Editorial changes Changed answers to actions Question revised - Sat Handout Provided Minor editorial edits Added procedure number of handout material.

82 H 3 N E B 5 Added s to answers C and D to make plural.

Question revised - Sat Sample Test Item Review Minor editorial edits 83 H 4 N E A 5 Editorial changes to stem.

Changed to LOD 4 Question revised - Sat Changed 10CFR # to 6 84 H 3 N S C 6 Handout Provided Minor editorial edits Added and stable to stem 85 H 3 N E D 5 Added procedure numbers to Handout material Added another procedure to Reference material Question revised - Sat a Rev 6

ES-401 GG-2017-12 Written Examination Review Worksheet Form ES-401-9 1.

2. LOD 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6 7 Answer CFR 8 LOK stem cred #/ back- SRO- 55.41x Q# (F/H) (1-5) cue T/F partial job link minutia K/A B/M/N U/E/S A/B/C/D Explanation focus dist units ward only 55.43y Sample Test Item Review Editorial edits Editorial changes to stem 86 H 3 N E B 5 Moved water level from bullets Question revised - Sat Sample Test Item Review Not clear only 1 correct answer Changed to 2 questions, with impact first then 87 H 4 N E A 5 procedure used Changed answers to 2X2 format Question revised - Sat Sample Test Item Review Tech Specs in lieu of procedures ok within limits Revised question and answers 88 H 3 N E D 2 NO Handout provided Changed 10CFR # to 2 Question revised - Sat Editorial edits. Rewrite question to better address K/A Changed answers to match K/A NO Handout provided 89 H 3 N E D 5 Question revised - Sat Sample Test Item Review Editorial edits Editorial changes 90 H 3 N E B 2 Made answers two parts for A2 Changed 10CFR # to 2 Question revised - Sat Unsat - K/A mismatch. Doesn't include requirements of A2 K/A.

Added information to the stem, Made 2 questions to cover the A2 KA.

91 F 3 X N U A 2 Corrected the question info to match question. Change to a new question instead of bank Question revised - Sat Editorial edits - rewrite question Added refueling platform position to stem.

Added IAW Tech Spec SR.

92 F 3 N E C 7 Changed answers small to large.

Question revised - Sat Unsat - K/A mismatch. Does address low power/shutdown implications. Reselect K/A, same question with minor edits.

Changed K/A, randomly selected 2.4.8 93 H 3 X N U D 5 Editorial changes in stem Changed 10CFR # to 5 Question revised - Sat a Rev 6

ES-401 GG-2017-12 Written Examination Review Worksheet Form ES-401-9 1.

2. LOD 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6 7 Answer CFR 8 LOK stem cred #/ back- SRO- 55.41x Q# (F/H) (1-5) cue T/F partial job link minutia K/A B/M/N U/E/S A/B/C/D Explanation focus dist units ward only 55.43y 94 F 2 B S B 5 2015 NRC Exam (1)

Changed LOD to 3 95 F 3 B S B 6 Changed 10CFR # to 6 Changed LOD to 3 96 F 3 N S A 3 Changed 10CFR # to 3 Editorial edits Editorial changes in stem Changed answers 97 F 3 B E B 5 Changed to LOD 3 Question revised - Sat Unsat - overlap with Op Test JPM Changed emergency classification to UE Change explanations Changed to LOD 3 98 F 3 X N U C 4 Changed from Bank to NEW Changed 10CFR # to 4 Question revised - Sat Minor editorial edits Changed to LOD 3 and minor grammar (tense) change 99 F 3 B E C 5 Question revised - Sat 100 F 2 N S B 5 a Rev 6

ES-401 GG-2017-12 Written Examination Review Worksheet Form ES-401-9 1.

2. LOD 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6 7 Answer CFR 8 LOK stem cred #/ back- SRO- 55.41x Q# (F/H) (1-5) cue T/F partial job link minutia K/A B/M/N U/E/S A/B/C/D Explanation focus dist units ward only 55.43y Results Table RO LOK -H 42 56 Avg RO LOD 2.71 Flaws 10 CFR Distribution RO LOK-F 33 44 AVG SRO LOD 3.00 stem focus 0 41.1 1 43.1 0 SRO LOK - H 15 60 Overall LOD 2.78 cues 0 41.2 3 43.2 5 SRO LOK - F 10 40 T/F 0 41.3 3 43.3 1

%  % cred dist 0 41.4 9 43.4 2 RO Bank 29 38.67 SRO Bank 6 24 partial 0 41.5 4 43.5 14 RO Mod 2 2.667 SRO Mod 0 0 job link 0 41.6 2 43.6 2 RO New 44 58.67 SRO New 19 76 units 0 41.7 18 43.7 1

% minutia 0 41.8 7 25 Total Bank 35 35 backward 0 41.9 2 Total Mod 2 2 KA 4 41.10 19 Total New 63 63 SRO-only 1 41.11 2

%  % LOD = 1 0 41.12 3 RO Sat 24 32 SRO Sat 6 24 41.13 0 RO Unsat 2 2.667 SRO Unsat 3 12 Answer Dist ( in %) 41.14 2 RO Edit 49 65.33 SRO Edit 16 64 RO-A 19 SRO-A 6 75

%  % RO-B 19 SRO-B 8 Total Sat 30 30 Total Unsat 5 5 RO-C 17 SRO-C 6 Total Edit 65 65 RO-D 20 SRO-D 5 a Rev 6

Retrieved from "https://kanterella.com/w/index.php?title=ML18061A170&oldid=2413870"