ML061290227

04-2005-Written Exam Comments

ML061290227
Person / Time
Site:	San Onofre
Issue date:	04/01/2005
From:	Operations Branch IV
To:	Southern California Edison Co
References
50-361/05-301, 50-362/05-301, ES-401, ES-401-9
Download: ML061290227 (8)

Text

ES-401 SONGS April 2005 - Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 3 Y S minor edit dist C, to use vice using fixed 2 H 2 Y S 3 H 3 x Y change dist. C or D to replace Technical Specifications with the E(S) acceptance criteria in SO23-2.30,Determination of Adequate Core Cooling (fixed) 4 H 2 Y S 5 F 2 Y S 6 H 2 Y S 7 F 1 x Y U(S) GFES (Replaced with BANK question) 8 H 2 Y S 9 F 2 Y S 10 F 1 x Y U(S) GFES as written. Stem background is not needed to answer the question. Add an airborne component and options for wearing breathing protection, and consider adding local exposure x limits. (Replaced with NEW question) 11 H 2 x x Y E(S) Answer stands out. Consider rewording answer to Rupture of air supply to condensate pump A suction isolation valve. (Fixed) 12 H 2 Y S 13 H 2 x Y E(S) Need more stem/distractor linkage. A ;- are applicants required to know the switch by number? B, C and D; which manual transfer switch?

(Replaced with BANK question) 14 H 2 Y E(S) editorial. Add to stem the start of the answer: By procedure ....

A: replace can with should only B, C: replace can with could, add from the control room, before but C: add and stopped after started D: reword to the emergency diesel should be placed in maintenance lockout and not started (Fixed)

ES-401 SONGS April 2005 - Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 15 F 2 x Y E(S) A is not very credible... replace with another false normal function.

D: replace source with makeup, add a after of. (Fixed) 16 H 2 Y S minor edits. Reword stem: For the following valves, which of the following is the correct failure position due to a loss of air to the respective pneumatic valve operator D: replace fails so flow goes with diverts (Fixed) 17 F 3 Y S minor edits. Add F to end of stem (or remove F throughout exam).

(Fixed) 18 F 2 Y S minor edit. Be consistent with procedure nomenclature throughout examination... ie, EOI SO23-12-1, Standard Post Trip Actions, and D: ensure all appropriate units are included throughout examination...

%NR or %WR? (Fixed) 19 F 2 Y S 20 F 3 Y S 21 F 3 Y E(S) Need to verify this level of detail is expected from memory (Considered OK as-is) 22 H 3 Y S 23 F 3 Y S 24 F 2 Y S Edit explanation in reference exam... currently says A is correct, should be C (Fixed) 25 H 2 x Y U(S) As worded, A and C are not valid distractors. Reword stem to state which of the following is the minimum (least restrictive) required to meet Technical Specification containment closure requirements. (Fixed)

ES-401 SONGS April 2005 - Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 26 H 2 Y S Edits. Be consistent with format of lists in the stem throughout the exam... use either bullets, hyphens, checkmarks, etc, only. (Fixed)

Second bullet in stem: incomplete... says Containment High Pressure initiated.. Alarm? Setpoint? (Fixed)

Fourth bullet in stem: no need to say actual, and is psia correct? psig?

(Fixed)

Use same font in all questions. Stem also states the procedure directs the operators... should be specific and replace with EOI SO23-???,

???? directs the operators...(did not change)

Correct tab on B, C and several other questions throughout the exam (Fixed) 27 F 2 x Y S edit. Consider replacing A or B with a system related affect.

Correct the font on D. (Fixed font; did not change A or B) 28 H 2 Y S 29 H 3 Y S 30 H 3 Y S 31 H 2 Y S 32 F 3 x x Y E(S) make post accident choices higher to be more credible, such as, 380, and 395. (Fixed) 33 H 3 x Y E(S) Edit stem: add as directed by after SIT volume. Add approximately to end of question. Adjust indent for distractors/answer. Also, consider making the distractors more believeable by changing 71% to 76% in A and B, and change 570 to 594 in A and C. This would be a 6% level drop and 36psia pressure drop... consistent with the 1:6 ratio of the answer.

(Fixed) 34 H 2 Y S minor edit: In stem, replace while with the with following a normal change PCS to PZR. (Fixed) 35 H 3 Y S minor edit. Spell out NCL in stem (Fixed) 36 H 2 Y E(S) The intent of the stem is unclear. other? (Fixed)

ES-401 SONGS April 2005 - Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 37 F 2 Y S minor edit, correct indent on C (Fixed) 38 H 3 Y S minor edit, stem, third bullet, delete has also (Fixed) 39 H 3 Y S minor edit. C needs reworded to ensure intent: Reactor trip breakers 2 and 4 (only) open. (Fixed) 40 H 3 Y S 41 F 2 Y S 42 F 2 Y S 43 H 3 Y S minor edit. C: delete the, add during CSS operation to end of dist.

D: correct indent, Delete Inability of the Shutdown Cooling System and replace with Failure (Fixed) 44 H 2 Y S minor edit, adjust indent dist A and B (Fixed) 45 H 2 Y S 46 H 2 Y S 47 F 2 Y S minor edit. In stem, delete the, and make operability all CAPS. (Fixed) 48 H 3 Y E check reference. If the breaker will auto open on some trips, D could be argued as incorrect, thus no correct answer. (Check during validation) 49 H 2 Y S 50 F 2 Y E(S) edits for clarity. In stem, delete always. A: add at all times B: correct font, C: delete unless there is and replace with ,except when, and add is after signal.

D: add at all times after active, delete or if there is and replace with and when, and add is after signal (Fixed) 51 H 3 Y S check if train names are needed in stem. A: add Only a to front of dist.

B: add Only (Fixed)

ES-401 SONGS April 2005 - Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 52 H 2 x Y E(S) Consider changing last sentence in stem to read After approximately 15 minutes, without operator action, RCS temperature is now _______.

Which of the following describes the expected shutdown cooling system response? (The temperature given should be such that SDC is allowed)

(NO CHANGES MADE) 53 H 3 Y S minor edits. In stem, replace when the with when a valid, and add due to a small air leak after received. In each dist, add instrument before air compressor. In C and D, delete self-regulating pilot operated. (Fixed) 54 F 3 Y E(S) Clarify stem: Which of the following conditions would prevent an emergency containment entry, in accordance with EOI SO23-3-2.34,

???

for dist C, replace with a high temp condition that would not prevent entry, such as Containment temperature is 158 F Dist D: give a reading... Containment atmosphere oxygen level is 16%

(Fixed) 55 F 2 x x Y E(S) Clarify stem. Is spent fuel being moved? Remove bold font from stem.

(Fixed) 56 H 2 N E(S) Note, question does not address KA / QPTR. This is a prior NRC exam Q. (Replaced with another BANK question; QPTR is called azimuthal power tilt in CE plants) 57 F 3 x Y S minor edit. In stem, add in response to a LOCA after in service.

Delete rather than the expected 1150-1400 degrees since this is a cue to the answer. (Fixed) 58 F 3 Y S 59 H 3 N S minor edit. Correct font in stem. This is a questionable KA match. (NO CHANGE MADE) 60 F 2 x Y E(S) GFES as worded. Add plant specfic readings that require the applicant to determine level is high. Give a power level, Tav, and PZR lvl. Then ask the applicant to determine the potential consequences of continuing to operate with these conditions. (Fixed)

ES-401 SONGS April 2005 - Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 61 H 3 Y S 62 F 3 Y S 63 F 2 x Y U(S) This is the same question as #60. (Replaced with BANK question) 64 H 3 Y S 65 H 2 Y E(S) This is a similar question as #53, and may provide cues. Consider replacement or reword to focus on the affect on service air of instrument air. Also, do instrument air compressors come on first? (Replaced with new KA and BANK question) 66 F 2 Y S minor stem edit. Adjust font. Correct procedure nomenclature, add Fire before Barrier Impairment (Fixed) 67 F 2 Y S 68 H 2 Y E(S) Clarify stem. Delete high after failed. Reword third sentence to read What operator action is required to allow continued automatic response of the SBCS? The operator must place the: (Fixed) 69 F 2 x x Y S 70 F 2 Y S 71 H 3 Y S 72 F 2 Y S 73 H 3 x x Y E(S) Need more information in stem to answer the question. How long does it take for one operator to complete the task? How long for two operators?

(Fixed) 74 F 2 Y S minor stem edit: delete the second Protection. (Fixed) 75 F 4 x x Y E check with facility on knowledge of these indications from memory.

(Check during validation) 76 H 2 Y Y S

ES-401 SONGS April 2005 - Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 77 H 3 Y Y S 78 H 3 Y Y S 79 H 3 x Y Y S minor stem edit. Consider deletion of resulting in a complete loss of CCW This is diagnosable with the given information and a cue to the answer. (Fixed) 80 H 3 Y Y S reference given does not support answer as clearly the only answer.

(Clarified the explanation of why the reference supports answer) 81 F 3 Y Y S 82 F 2 Y Y S 83 H 2 Y Y S 84 H 3 Y Y S 85 H 3 x x Y Y E(S) The stem cues the procedure part of the answer and is not needed in the stem. Change to first sentence in the stem to say The reactor has been tripped from 100% power, and the control room has been evacuated due to toxic gas in the control room.

Change the procedure in either A or C to a different procedure. (Fixed) 86 H 2 x Y Y E(S) As written, both B and D could be argued correct. Change B to a different procedure, or add in stem WHY the procedure is entered.

(Fixed) 87 F 2 Y Y S 88 H 3 Y Y S 89 H 4 Y Y S 90 H 3 x Y Y E Change one dist to an OPERABLE condition. As written, only the answer is an OPERABLE condition. (No change made) 91 H 4 Y Y S 92 H 3 Y Y S

ES-401 SONGS April 2005 - Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 93 F 2 Y Y S 94 H 2 Y Y S 95 H 2 Y Y S 96 H 3 Y Y S 97 H 2 Y Y S 98 F 2 Y Y S minor edit, A. and B: delete generated by and replace with obtained from (Fixed) 99 F 2 Y Y S 100 F 3 Y Y S Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

• The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

• The answer choices are a collection of unrelated true/false statements.

• The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

• One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

• The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

• The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

7. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).