ML061770026

NRC Comments to the Proposed Writtten Examination, ES-401-9 for the Braidwood Initial Examination - May 2006

ML061770026
Person / Time
Site:	Braidwood
Issue date:	05/15/2006
From:	Mcneil D NRC/RGN-III/DRS/OLB
To:
Shared Package
ml061510490	List: ML061770016 ML061770023 ML061770025 ML061770026 ML061770028 ML061770037 ML061770044 ML061770049
References
ES-401-9
Download: ML061770026 (9)

Text

NRC COMMENTS TO THE PROPOSED WRITTEN EXAMINATION, ES-401-9 FOR THE BRAIDWOOD INITIAL EXAMINATION MAY 2006

ES-401 Written Examination Form ES-401-9 Rp&wMxksbWnrknhpPt Instructions

[Refer to Section 0 of ES-301 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. - - -

Enter the level of dimculty (LOD) of each question using a 1 5 (easy dimcult) rating scale (questions in the 2 4 range are acceptable)

3. Check the appropriate box if a psychometric flaw is identified:

The stem lacks Sumcient focus to elicit the correct answer (e.9.. unclear intent, more information is needed, or too much needless information).

The Stem or distractors contain cues (i.e.. clues, specific determiners. phrasing, length, etc).

The answer choices are a collection of unrelated truelfalse statements.

The distractors are not credible; single implausible distradors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the aoorooriate box if a iob content error is identified:

Thk qbeslion ,s not linked to the ,OD requirements ( e the q.esl on has a valid KJA 0.1 as wrinen. 5 not operat Onal n content, Tne q.estion reqL res the recall of know edge tnat s too Specifc for tne c osed reference test moae 11 e ,l 1s not requ red 10 be ?mown from memory)

Tne o.es1 on contains data wtn an .nrea s1 c eve of accdacv of mons srent units l e 0 Dane meter tn Dercent with OJesl on In aa onsl The 4uestion requires feverre logic or application comparedtdthe job requirements.

5. Check questions that are sampled for conformance with the approved WA and those that are desiqnated SRO-only (WA and license level mismatches are unacceptable).

6. Based on the reviewer's judgment. is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditbrial enhancement, or (S)atisfactory?

7. At a minimum. explain any 'V ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 Written Examination Form ES-401-9 R p V i p W t Explanation Bank.

New. Stem and distractors do not match. Stem asks for action, distractors provide action and reason for action. d. does not provide any reason for action. Suggest removing causes or revising stem and addina cause to distractor d. Stem and distractor d. mcdified.

S New.

I S New.

S S

New.

INew.

III1 U

S New Tnis Dewmes a matn problem witn tne reference providea Quesrron modified Strll a math problem but the operator must make choces basea on the malh results New.

I I

S IBank.

~~~ ~

II T

New. Question's answer has to be known from memory - no method of determining the correct answer unless memorized. Not higher skill.

tt S Distractor a. is not plausible. Question does have a thought process to am've at the correct answer. Since pressure can continue to rise under certain conditions. distractor a. is Dfausibfe. Question not modified.

S Bank

ES-401 Written Examination Form ES-401-9 m teR

- 3. Psvchometric Flaws 4. Job Content Flaws 7 I

Q= SRO Explanation WA Only X S New.

I S

S New.

Bank. I I I S

S E

New.

New.

]New. Chanae ODeratorS to ODerator's in the stem. Chanoe made.

~~

I 11 S New.

New. Distractor a. appears to not be gramatically wrrect ,or is missing E information. Reworked dislractor a. lo be accurate, put key words in another dislractor.

II I tl S New. Question is not discriminatory. LOO = 2, there are different power supplies a1 different power levels, making lhis more lhan simple memory However, question modified lo lest lhe non-nomalmwer lineup.

New. Question is not discriminatory Question is discriminatory, tt E however, dislraclor a. was not plausible and was changed.

I S New.

S S

New.

New. I S

S S

New.

New.

New.

IU

ES-401 Written Examination Form ES-401-9 PPt

ES-401 Written Examination Form ES-401-9 rn R p V l p W t

+

3 Psychometric Flaws Dist.

I 4 Jobcontent Flaws I 5 Other I/E/S Explanation S INew. II S New.

S Bank.

S S

S New.

New.

INew.

I II

~~

New. Question is not discriminatory. This material is required to be tf S known by licensed operators. Auxiliary operators are not required lo know this infomation. Question not modified.

X S New.

X S INew. II X II X S INew. II New. Question is not discriminatory for an SRO. All of the plant conditions are nonessential. Once the operator knows a Site Area Emergency has been declared, all the remaining items are not X

necessary. The question is not a Higher skill. but memorized. Question replaced, new K/A selected - unable to write a question with this combination event and generic WA.

New. Attempt to replace distractor c. implausible. Correct the stem to include a reason that the action is being taken (since the distractors X E already provide the reason.) Distractor c. was accepted as is. stem was modified,

ES-401 Written Examination Form ES-401-9 R p V i p W t

.OK FIH)

I

~

H I LOD (1-5) 3 I

Cues TIF Ro UIEIS nly x S Bank.

Explanation New. 1) The question does not match the WA. The WA requires H 2 x U knowledge of which events are reported to outside agencies. The question deals with whenlwho gets the report. Queslion was moditied New. Doesn't match WA. Distradors a., b.. and c. are not plausible.

H 2 x et S Reviewed distractors with the utility, distractors determined to be plausible as written. No changes made.

H 3 New. Distractom c. and d. are implausible. Because power is c3G%

H 2 x et S where auto trips occur, distractors c. and d. bemme implausible. No changes made to question.

F 2 x S New.

New. Used an 'all of the above" format. Change distrador format.

F 2 x E Added more distraclor info, resulting in removal of 'all of the above" format.

New. Material is not required to be known from memory. A station H 3 objective requires this material be known from memory. No change made to the ouestion.

H H

2 2.5 x

x I S

S Bank.

INew.

I II Bank. Doesn't match WA. The WA refers to causeleffed between fuel handling and shutdown monitor. The question refers to nuclear F 2 X et instrumentation. Changed WA, lhe question was retained. The exam author believed the Shutdown Monitor referred to nuclear instruments.

New. No analysis is reauired. Knowinq that any Adion Level 3 requires H 2 x u a reanor sn-taown makes ~ n i sa s mp; memo, question Station agreed Cognrlwe level changed lo fundamenral U

ES-401 Written Examination Form ES-401-9 m R p v i p W e t Explanation