Text

2009-03 - Draft Written Exam Comments
	ML091490493
Person / Time
Site:	Fort Calhoun
Issue date:	03/27/2009
From:	Brian Larson; Operations Branch IV
To:	; Omaha Public Power District
References
	50-285/09-301, ES-401, ES-401-9 50-285/09-301
	Download: ML091490493 (9)
	v • d • e

ES-401 FC-2009-03 RO Written Examination Review Worksheet Rev 1 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 2 B E B/10 A/5 - EOP-00 (SPTA) step 11 has the operators close the block valve. This question assumes operation in EOP-03 (LOCA); however, the crew had to go through EOP-00 2 F 2 X N U first. There is no caution in EOP-00 about solid plant conditions and PORV operation. The applicant would correctly assume the PORV block is already closed, and there would be no correct answer. Revised question A/5 - Doesnt address KA - time available for action. Also, what is reference SHB:

3 H 2 X B U 07-15-12? Also, ref states thumb rule is 10%/deg F - is it 10% or 1%. Also, ref is for uncomplicated reactor trip but KA applies to SBLOCA. Replaced question 4 H 3 N E D/5 - Handout C/7 - Reword to operational event. Plant is in Mode 4. SDC is in operation per procedure XXX, with LPSI pump XX in service. A loss of off-site power occurs.

5 H 2 B E EDGs operate as designed. What action (if any) (or something similar). Also, STM-15, para 3.19 on page 71 is a better reference. Fixed D/10 - What is the reactor trip criteria? What is the failure? A leak (surge tank level 6 H 3 X B U 8 and lowering) and/or trashed pump (loud rumbling noise)? Revised question 7 H 3 X N U D/14 - Rewrite into bullet format and let applicant determine system response Done Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

$ The answer choices are a collection of unrelated true/false statements.

$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

$ The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met). Page 1 of 6 OBDI 202 - IOLE Process

ES-401 FC-2009-03 2 Rev 1 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 8 H 2 X M E D/10 B/11 - The references do not support the correct answer or explain why the others are incorrect. Nothing in the lesson plan 7-15-33 or SO-G-105 supports the correct 9 F 2 N U answer. The material states that RM-54 MAY alarm first for small leaks IF blowdown is in service. Revised 10 H 3 N E C/10 C/10 - Need ref for the why part of question. Also, need procedure reference in 11 H 3 N E stem. Also, revise stem to remove conclusions. Done 12 H 3 M E B/7 13 H 2 B S D/5 B/10 - Rewrite in bullet format as it would be more operationally valid. Done Also, 14 F 3 N E are FCS RO applicants expected to know AOP bases? Facility Rep accepts question as acceptable for RO applicants C/10 - If Panel 1 is lost, will the low voltage annunciator also alarm? Yes added to 15 H 3 N E stem Also, according the AOP reference provided, this is one pushbutton on 1A1-1A3 Aux Power Compartment. Revised 16 H 3 N S D/7 D/10 - The AOP states that MFPs are stopped to prevent the RCS cooldown 17 F 2 B E following the trip, not overfilling the generators (although overfilling would have the effect of overcooling.) Prefer to follow wording of the AOP. Done 18 H 3 N E B/10 19 F 3 B E A/7 20 H 2 B E A/5 - Handout.

B/6 - Doesnt address KA - startup termination on SR loss. Backwards logic - asks a 21 F 3 X X N U question about what is needed prior to s/u yet a s/u is already in progress. New KA 22 H 2 N E B/6 23 F 2 X N E D/13 B/13 - Doesnt address KA - same as above but for accidental gaseous radwaste 24 F 2 X X B U release. Distractors C and D are not credible. Revised 25 H 3 M E D/7 - Worksheet says Source: NEW - should this be MODIFIED BANK? Yes 26 F 2 X N U B/5 - Backward logic. Give parameter(s), then choose a procedure. Revised 27 H 3 N E A/3 Page 2 of 6 OBDI 202 - IOLE Process

ES-401 FC-2009-03 3 Rev 1 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 28 H 3 N E C/10 29 H 3 N E D/7 30 H 2 N S C/11 31 F 2 N S B/10 32 H 3 X X N E B/5 - Stem doesnt state reactor is critical. Fixed C/10 - Question has not been modified significantly. It is a BANK question. Updated 33 F 2 B E Ref for venting NCGs to the PQT or VCT? OI-CH-3 34 H 3 B E A/5 B/10 -The way this question is worded, the operator could assume that the prerequisite checklist is being performed, and the discharge valve for AC-3A is being 35 F 3 U U N E opened as required by OI-CC-1-CL-A (Page 29 of OI-CC-1). This would make C the correct answer. Add to stem - prerequisites for OI-CC-1 are complete. Fixed 36 H 3 N S B/7 - Ref not adequate to support answer provided 37 H 3 N E D/10 C/6 - Stem - provide CR indications of a loss of 125 VDC power and ask impact on 38 H 3 N E DSS. Otherwise cognitive level is LOW. Fixed 39 H 3 N E B/10 - Cognitive level is HIGH. updated D/7 - Doesnt address part b of KA - use procedures to correct, control, or mitigate.

40 H 3 X B U Also, refs do not detail impact of 10 sec drop in voltage. fixed B/4 - Reword stem to read From where do the CEDM Cooling Fans, VA-2A and VA-41 F 2 M E 2B, take suction? Then remove the phrase The CEDM cooling fans take (cool) air..

from each of the answers. done 42 H 3 N E C/10 43 F 3 N E C/5 A/10 - Does not address auto operation of MFW and level programming. This addresses AFW and safety function checks of the EOPs. Also, to meet the safety 44 H 2 X N U function, must be feeding the S/Gs - stem states BOP stopped feeding. Rewrote question C/10 - The words prior to: in the stem would make the correct answer incorrect.

45 F 3 N E The continuous action step (15) of the EOP states that once through core cooling is performed if there is a rise of >5F. Revise the stem. Fixed D/10 - Assuming Step 8 of AOP-31 is performed sequentially, bus 1A4 is 46 H 3 X X X N U deenergized first, and 1A4 loads are started manually after EDG start. Then bus 1A3 is deeneergized and operators ensure loads started after EDG start. Answers C & D Page 3 of 6 OBDI 202 - IOLE Process

ES-401 FC-2009-03 4 Rev 1 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only state that both buses are deenergized and then loads are started. Reword C & D.

Also, eliminate values of 3750 and 430 volts as they could be used to assist in answering Q87 (state less than minimum bus voltage). Fixed nd 47 F 2 X B U B/10 - Does not address 2 part of the KA. Fixed 48 F 2 N E D/7 49 H 2 N E A/5 A/11 - Backward logic (see 1021 Appendix B). Give the alarm and have applicant 50 H 3 X M U determine system response. New KA 51 H 3 B E C/11 -

52 F 3 X M U B/7 - Backward logic. Rewrote question 53 H 3 N E B/7 - Ref provided doesnt contain info on sequencer starting AC-10C. ref provided 54 H 3 N E D/7 55 F 2 N E B/9 56 H 3 M E C/10 57 F 2 B E A/9 58 H 3 N E C/10 D/5 - Does the reactor have to be > 60% for the valves to stroke full open? Yes If 59 H 3 M E so, add initial power to stem. Done B/7 - Ref provided doesnt contain info on 43/FW switch or trip signals to FW-8A.

60 F 2 N E Ref provided nd 61 H 3 X N U D/7 - Doesnt address 2 part of KA - use procedures to correct, control . Fixed D/13 - Doesnt address the KA - loss or malfunction of Rad Monitor on Liquid 62 F 2 X B U Radwaste System (this is an admin type question). Also, this is not a significant modification. SOURCE is BANK. New KA and question 63 H 3 N E B/13 64 F 2 N E D/6 A/10 - Distractors B and C are not credible. Also, this is not a significant modification.

65 F 3 X N U SOURCE is BANK. New KA 66 F 2 N E A/10 67 F 3 X N E C/10 68 F 2 B E D/10 Page 4 of 6 OBDI 202 - IOLE Process

ES-401 FC-2009-03 5 Rev 1 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 69 H 2 N E B/5 70 H 3 X N E B/5 - Add power statement to stem. Done 71 F 2 X N U D/12 - Distractor A is not credible. Fixed 72 F 3 B E A/11 - This is not a significant modification. SOURCE is BANK. Fixed 73 F 2 N E A/13 74 F 2 N E B/10 B/6 - The training manual states that the zero power mode bypass automatically removed when power is above 10-4%. The answers B and D state the Rx would trip 75 H 3 N E when power reaches 10-4%. This is contradictory. Change B and D to read exceeds 1 x 10-4%. Fixed B= 18 (24%) F= 31 (42%) E= 52 (69%) Additional Notes:

M= 10 (13%) H= 44 (58%) U= 18 (24%) 1. Any performance data available on Bank questions?

RO TOTALS: 2. DRAFT submittal should include explanation of why answers are correct and why N= 47 (63%) S= 5 (7%)

distractors are incorrect but credible.

3. Average difficulty is lower than normal (2.57 vs 2.8).

GENERAL COMMENT

S:

1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N.

2. Chief Examiner comments are indicated in blue.

3. Average difficulty is 2.57 on the RO exam.

4. The 10CFR55.41 distribution is:

41.1 = 41.11 = 5 41.2 = 41.12 = 1 41.3 = 1 41.13 = 5 41.4 = 1 41.14 = 1 41.5 = 13 41.6 = 5 41.7 = 14 41.8 =

41.9 = 2 41.10 = 27

5. The answer distribution is: RO A = 14 (19%)

B = 25 (33%) Page 5 of 6 OBDI 202 - IOLE Process

ES-401 FC-2009-03 6 Rev 1 Form ES-401-9 C = 15 (20%)

D = 21 (28%)

6. There are 2 questions with handouts (Q4, Q20). Page 6 of 6 OBDI 202 - IOLE Process

ES-401 FC-2009-03 SRO Written Examination Review Worksheet Rev 1 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 76 F 3 N S A/2 - Note: second question on PDIL (Q20)

C/5 - How credible are distractors A & B - are there any EOP or EOP Attachment 77 H 3 N E transitions to the TDBs? Changed to EOP-AOP Attachment 2 How credible is distractor D - what is PRC? Plant Review Committee - acceptable C/5 - Doesnt address loss of RC flow as identified in the KA - cause of RCP failure.

78 H 3 X N U Modified stem to ensure operationally valid for Plant Operator response 79 H 3 N E A/5 80 H 3 U N E D/5 C/5 - RO knowledge. Also, doesnt address KA for ability to determine and interpret.

81 F 2 X X N U No assessment of plant conditions to determine course of action. Revised question D/5 - Doesnt address KA - using incore/excore instrumentation, incore or loop 82 H 3 X N U temperature measurements Revised question Instructions