ML061770026: Difference between revisions
StriderTol (talk | contribs) (Created page by program invented by StriderTol) |
StriderTol (talk | contribs) (Created page by program invented by StriderTol) |
||
Line 17: | Line 17: | ||
=Text= | =Text= | ||
{{#Wiki_filter:NRC COMMENTS TO THE PROPOSED WRITTEN EXAMINATION, ES-401-9 FOR THE BRAIDWOOD INITIAL EXAMINATION | {{#Wiki_filter:NRC COMMENTS TO THE PROPOSED WRITTEN EXAMINATION, ES-401-9 | ||
- | |||
FOR THE BRAIDWOOD INITIAL EXAMINATION MAY 2006 | |||
ES-401 Written Examination Form ES-401-9 Rp&wMxksbWnrknhpPt Instructions | |||
[Refer to Section 0 of ES-301 and Appendix B for additional information regarding each of the following concepts.] | [Refer to Section 0 of ES-301 and Appendix B for additional information regarding each of the following concepts.] | ||
: 1 | : 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. | ||
Check the appropriate box if a psychometric flaw is identified: The stem lacks Sumcient focus to elicit the correct answer (e.9.. unclear intent, more information is needed, or too much needless information). | : 2. - - - | ||
Enter the level of dimculty (LOD) of each question using a 1 5 (easy dimcult) rating scale (questions in the 2 4 range are acceptable) | |||
: 3. Check the appropriate box if a psychometric flaw is identified: | |||
The stem lacks Sumcient focus to elicit the correct answer (e.9.. unclear intent, more information is needed, or too much needless information). | |||
The Stem or distractors contain cues (i.e.. clues, specific determiners. phrasing, length, etc). | The Stem or distractors contain cues (i.e.. clues, specific determiners. phrasing, length, etc). | ||
The answer choices are a collection of unrelated truelfalse statements. | The answer choices are a collection of unrelated truelfalse statements. | ||
The distractors are not credible; single implausible distradors should be repaired, more than one is unacceptable. | The distractors are not credible; single implausible distradors should be repaired, more than one is unacceptable. | ||
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the aoorooriate box if a iob content error is identified: | One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). | ||
: 4. Check the aoorooriate box if a iob content error is identified: | |||
Thk qbeslion ,s not linked to the ,OD requirements ( e the q.esl on has a valid KJA 0.1 as wrinen. 5 not operat Onal n content, Tne q.estion reqL res the recall of know edge tnat s too Specifc for tne c osed reference test moae 11 e ,l 1s not requ red 10 be ?mown from memory) | Thk qbeslion ,s not linked to the ,OD requirements ( e the q.esl on has a valid KJA 0.1 as wrinen. 5 not operat Onal n content, Tne q.estion reqL res the recall of know edge tnat s too Specifc for tne c osed reference test moae 11 e ,l 1s not requ red 10 be ?mown from memory) | ||
Tne o.es1 on contains data wtn an .nrea s1 c eve of accdacv of mons srent units | . | ||
: 5 | Tne o.es1 on contains data wtn an .nrea s1 c eve of accdacv of mons srent units l e 0 Dane meter tn Dercent with OJesl on In aa onsl The 4uestion requires feverre logic or application comparedtdthe job requirements. | ||
Based on the reviewer's judgment. | : 5. Check questions that are sampled for conformance with the approved WA and those that are desiqnated SRO-only (WA and license level mismatches are unacceptable). | ||
is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditbrial enhancement, or (S)atisfactory? | : 6. Based on the reviewer's judgment. is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditbrial enhancement, or (S)atisfactory? | ||
At a minimum. explain any 'V ratings (e.g., how the Appendix B psychometric attributes are not being met). | : 7. At a minimum. explain any 'V ratings (e.g., how the Appendix B psychometric attributes are not being met). | ||
Written Examination Form ES-401-9 | |||
Stem and distractor | ES-401 Written Examination Form ES-401-9 R p V i p W t Explanation Bank. | ||
New. Stem and distractors do not match. Stem asks for action, distractors provide action and reason for action. d. does not provide any reason for action. Suggest removing causes or revising stem and addina cause to distractor d. Stem and distractor d. mcdified. | |||
S New. | |||
I S New. | |||
S S | |||
Question not modified. | New. | ||
S Bank ES-401 Written Examination Form ES-401-9 m | INew. | ||
III1 U | |||
S New Tnis Dewmes a matn problem witn tne reference providea Quesrron modified Strll a math problem but the operator must make choces basea on the malh results New. | |||
I I | |||
LOO = 2, there are different power supplies a1 different power levels, making lhis more lhan simple memory | S IBank. | ||
~~~ ~ | |||
II T | |||
New. Question's answer has to be known from memory - no method of determining the correct answer unless memorized. Not higher skill. | |||
tt S Distractor a. is not plausible. Question does have a thought process to am've at the correct answer. Since pressure can continue to rise under certain conditions. distractor a. is Dfausibfe. Question not modified. | |||
S Bank | |||
ES-401 Written Examination Form ES-401-9 m teR | |||
- | |||
- 3. Psvchometric Flaws 4. Job Content Flaws 7 I | |||
Q= SRO Explanation WA Only X S New. | |||
I S | |||
S New. | |||
Bank. I I I S | |||
S E | |||
New. | |||
New. | |||
]New. Chanae ODeratorS to ODerator's in the stem. Chanoe made. | |||
~~ | |||
I 11 S New. | |||
New. Distractor a. appears to not be gramatically wrrect ,or is missing E information. Reworked dislractor a. lo be accurate, put key words in another dislractor. | |||
II I tl S New. Question is not discriminatory. LOO = 2, there are different power supplies a1 different power levels, making lhis more lhan simple memory However, question modified lo lest lhe non-nomalmwer lineup. | |||
New. Question is not discriminatory Question is discriminatory, tt E however, dislraclor a. was not plausible and was changed. | |||
I S New. | |||
S S | |||
New. | |||
New. I S | |||
S S | |||
New. | |||
New. | |||
New. | |||
IU | |||
ES-401 Written Examination Form ES-401-9 PPt | |||
ES-401 Written Examination Form ES-401-9 rn R p V l p W t | |||
+ | |||
3 Psychometric Flaws Dist. | |||
I 4 Jobcontent Flaws I 5 Other I/E/S Explanation S INew. II S New. | |||
S Bank. | |||
S S | |||
S New. | |||
New. | |||
INew. | |||
I II | |||
~~ | |||
New. Question is not discriminatory. This material is required to be tf S known by licensed operators. Auxiliary operators are not required lo know this infomation. Question not modified. | |||
X S New. | |||
X S INew. II X II X S INew. II New. Question is not discriminatory for an SRO. All of the plant conditions are nonessential. Once the operator knows a Site Area Emergency has been declared, all the remaining items are not X | |||
necessary. The question is not a Higher skill. but memorized. Question replaced, new K/A selected - unable to write a question with this combination event and generic WA. | |||
- | |||
New. Attempt to replace distractor c. implausible. Correct the stem to include a reason that the action is being taken (since the distractors X E already provide the reason.) Distractor c. was accepted as is. stem was modified, | |||
ES-401 Written Examination Form ES-401-9 R p V i p W t | |||
.OK FIH) | |||
I | |||
~ | |||
H I LOD (1-5) 3 I | |||
Cues TIF Ro UIEIS nly x S Bank. | |||
Explanation New. 1) The question does not match the WA. The WA requires H 2 x U knowledge of which events are reported to outside agencies. The question deals with whenlwho gets the report. Queslion was moditied New. Doesn't match WA. Distradors a., b.. and c. are not plausible. | |||
H 2 x et S Reviewed distractors with the utility, distractors determined to be plausible as written. No changes made. | |||
H 3 New. Distractom c. and d. are implausible. Because power is c3G% | |||
H 2 x et S where auto trips occur, distractors c. and d. bemme implausible. No changes made to question. | |||
F 2 x S New. | |||
New. Used an 'all of the above" format. Change distrador format. | |||
F 2 x E Added more distraclor info, resulting in removal of 'all of the above" format. | |||
New. Material is not required to be known from memory. A station H 3 objective requires this material be known from memory. No change made to the ouestion. | |||
H H | |||
2 2.5 x | |||
x I S | |||
S Bank. | |||
INew. | |||
I II Bank. Doesn't match WA. The WA refers to causeleffed between fuel handling and shutdown monitor. The question refers to nuclear F 2 X et instrumentation. Changed WA, lhe question was retained. The exam author believed the Shutdown Monitor referred to nuclear instruments. | |||
New. No analysis is reauired. Knowinq that any Adion Level 3 requires H 2 x u a reanor sn-taown makes ~ n i sa s mp; memo, question Station agreed Cognrlwe level changed lo fundamenral U | |||
ES-401 Written Examination Form ES-401-9 | ES-401 Written Examination Form ES-401-9 m R p v i p W e t Explanation}} | ||
Revision as of 18:06, 23 November 2019
ML061770026 | |
Person / Time | |
---|---|
Site: | Braidwood ![]() |
Issue date: | 05/15/2006 |
From: | Mcneil D NRC/RGN-III/DRS/OLB |
To: | |
Shared Package | |
ml061510490 | List: |
References | |
ES-401-9 | |
Download: ML061770026 (9) | |
Text
NRC COMMENTS TO THE PROPOSED WRITTEN EXAMINATION, ES-401-9
-
FOR THE BRAIDWOOD INITIAL EXAMINATION MAY 2006
ES-401 Written Examination Form ES-401-9 Rp&wMxksbWnrknhpPt Instructions
[Refer to Section 0 of ES-301 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. - - -
Enter the level of dimculty (LOD) of each question using a 1 5 (easy dimcult) rating scale (questions in the 2 4 range are acceptable)
- 3. Check the appropriate box if a psychometric flaw is identified:
The stem lacks Sumcient focus to elicit the correct answer (e.9.. unclear intent, more information is needed, or too much needless information).
The Stem or distractors contain cues (i.e.. clues, specific determiners. phrasing, length, etc).
The answer choices are a collection of unrelated truelfalse statements.
The distractors are not credible; single implausible distradors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the aoorooriate box if a iob content error is identified:
Thk qbeslion ,s not linked to the ,OD requirements ( e the q.esl on has a valid KJA 0.1 as wrinen. 5 not operat Onal n content, Tne q.estion reqL res the recall of know edge tnat s too Specifc for tne c osed reference test moae 11 e ,l 1s not requ red 10 be ?mown from memory)
.
Tne o.es1 on contains data wtn an .nrea s1 c eve of accdacv of mons srent units l e 0 Dane meter tn Dercent with OJesl on In aa onsl The 4uestion requires feverre logic or application comparedtdthe job requirements.
- 5. Check questions that are sampled for conformance with the approved WA and those that are desiqnated SRO-only (WA and license level mismatches are unacceptable).
- 6. Based on the reviewer's judgment. is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditbrial enhancement, or (S)atisfactory?
- 7. At a minimum. explain any 'V ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 Written Examination Form ES-401-9 R p V i p W t Explanation Bank.
New. Stem and distractors do not match. Stem asks for action, distractors provide action and reason for action. d. does not provide any reason for action. Suggest removing causes or revising stem and addina cause to distractor d. Stem and distractor d. mcdified.
S New.
I S New.
S S
New.
INew.
III1 U
S New Tnis Dewmes a matn problem witn tne reference providea Quesrron modified Strll a math problem but the operator must make choces basea on the malh results New.
I I
S IBank.
~~~ ~
II T
New. Question's answer has to be known from memory - no method of determining the correct answer unless memorized. Not higher skill.
tt S Distractor a. is not plausible. Question does have a thought process to am've at the correct answer. Since pressure can continue to rise under certain conditions. distractor a. is Dfausibfe. Question not modified.
S Bank
ES-401 Written Examination Form ES-401-9 m teR
-
- 3. Psvchometric Flaws 4. Job Content Flaws 7 I
Q= SRO Explanation WA Only X S New.
I S
S New.
Bank. I I I S
S E
New.
New.
]New. Chanae ODeratorS to ODerator's in the stem. Chanoe made.
~~
I 11 S New.
New. Distractor a. appears to not be gramatically wrrect ,or is missing E information. Reworked dislractor a. lo be accurate, put key words in another dislractor.
II I tl S New. Question is not discriminatory. LOO = 2, there are different power supplies a1 different power levels, making lhis more lhan simple memory However, question modified lo lest lhe non-nomalmwer lineup.
New. Question is not discriminatory Question is discriminatory, tt E however, dislraclor a. was not plausible and was changed.
I S New.
S S
New.
New. I S
S S
New.
New.
New.
IU
ES-401 Written Examination Form ES-401-9 PPt
ES-401 Written Examination Form ES-401-9 rn R p V l p W t
+
3 Psychometric Flaws Dist.
I 4 Jobcontent Flaws I 5 Other I/E/S Explanation S INew. II S New.
S Bank.
S S
S New.
New.
INew.
I II
~~
New. Question is not discriminatory. This material is required to be tf S known by licensed operators. Auxiliary operators are not required lo know this infomation. Question not modified.
X S New.
X S INew. II X II X S INew. II New. Question is not discriminatory for an SRO. All of the plant conditions are nonessential. Once the operator knows a Site Area Emergency has been declared, all the remaining items are not X
necessary. The question is not a Higher skill. but memorized. Question replaced, new K/A selected - unable to write a question with this combination event and generic WA.
-
New. Attempt to replace distractor c. implausible. Correct the stem to include a reason that the action is being taken (since the distractors X E already provide the reason.) Distractor c. was accepted as is. stem was modified,
ES-401 Written Examination Form ES-401-9 R p V i p W t
.OK FIH)
I
~
H I LOD (1-5) 3 I
Cues TIF Ro UIEIS nly x S Bank.
Explanation New. 1) The question does not match the WA. The WA requires H 2 x U knowledge of which events are reported to outside agencies. The question deals with whenlwho gets the report. Queslion was moditied New. Doesn't match WA. Distradors a., b.. and c. are not plausible.
H 2 x et S Reviewed distractors with the utility, distractors determined to be plausible as written. No changes made.
H 3 New. Distractom c. and d. are implausible. Because power is c3G%
H 2 x et S where auto trips occur, distractors c. and d. bemme implausible. No changes made to question.
F 2 x S New.
New. Used an 'all of the above" format. Change distrador format.
F 2 x E Added more distraclor info, resulting in removal of 'all of the above" format.
New. Material is not required to be known from memory. A station H 3 objective requires this material be known from memory. No change made to the ouestion.
H H
2 2.5 x
x I S
S Bank.
INew.
I II Bank. Doesn't match WA. The WA refers to causeleffed between fuel handling and shutdown monitor. The question refers to nuclear F 2 X et instrumentation. Changed WA, lhe question was retained. The exam author believed the Shutdown Monitor referred to nuclear instruments.
New. No analysis is reauired. Knowinq that any Adion Level 3 requires H 2 x u a reanor sn-taown makes ~ n i sa s mp; memo, question Station agreed Cognrlwe level changed lo fundamenral U
ES-401 Written Examination Form ES-401-9 m R p v i p W e t Explanation