ML102300258

DC-2010-01-DRAFT Written Exam Comments

ML102300258
Person / Time
Site:	Diablo Canyon
Issue date:	08/18/2010
From:	Apger G NRC Region 4
To:
References
50-275/10-301, 50-323/10-301, DC-2010-01
Download: ML102300258 (9)

Text

Additional Review information is documented following Form ES-401-9. This exam was developed by the NRC with significant review and technical support from the facility.

ES-401 DC-2010-01 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 F 2 N S B41.3 2 F 3 N S C41.7 3 H 3 M S A41.7 4 F 3 N S B41.7 5 F 3 N S B41.10 6 H 2 B S A41.5 7 H 2 M S A41.7 Verify this. Good.

8 H 2 B S C41.14 9 F 3 N S C41.7 10 H 3 B S A41.8 Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

$ The answer choices are a collection of unrelated true/false statements.

$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

$ The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met). Page 1 of 6 OBDI 202 - IOLE Process

ES-401 DC-2010-01 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 11 H 4 N S B41.7 This does require detailed knowledge of P logic that causes the PK 12 H 2 N S D41.7 Added a period at end of first sentence.

13 F 3 N S A41.8 14 F 3 N S B41.8 Minor edit to answer.

B41.4 This is a main steam KA, and the answers require knowledge of the main 15 F 3 N S steam system valves up to the turbine.

16 F 2 N S D41.4 Swapped C and D to balance answers 17 H 3 B S A41.7 18 H 3 M S C41.8 B41.7 Changed distractor A from None. This would not be plausible. OK as was.

19 H 3 B S Returned to original.

20 H 3 N S B41.10 Modified stem so there is no way that A could be considered correct.

21 F 2 B S C41.5 22 H 2 B S C41.8 Fixed Spelling error in C.

23 F 2 B S A41.11 24 F 3 N S D41.4 Swapped C and D for balance.

25 F 2 B S A41.7 FIXED JUSTIFICATIONS D41.7 SELECTED ONLY OTHER KA TOPIC SINCE FACILITY DOES NOT HAVE 26 F 2 B S EMERGENCY AIR COMPRESSORS. (K2.01 IA) 27 H 3 N S C41.7 28 H 3 M S A41.7 29 F 3 N S A41.7 30 F 3 N S C41.7 31 F 3 N S C41.6 D41.7 Operators should know that there are no automatic selections in the NN 32 H 3 N S intruments.

33 F 3 N S D41.10 Page 2 of 6 OBDI 202 - IOLE Process

ES-401 DC-2010-01 3 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 34 F 2 N S B41.8 35 H 3 N S A41.7 36 H 2 N S B41.5 37 F 3 N S D41.11 SWAPPED B AND D FOR BALANCE 38 F 3 N S D41.10 39 H 3 B S D41.5 ACCEPTED REPLACEMENT 40 H 3 N S B41.8 41 F 2 N S A41.10 42 H 3 N S C41.10 43 H 3 N S A41.3 C41.10 This may seem to be SRO, but it does not discriminate between any EOP or 44 H 3 N S AOP. It is asking what is implemented (either one or both) and what is the hierarchy.

A41.8 Justifications reference alarms, but no alarms are present in the stem. Should 45 H 4 N E?

this be fixed? Justifications removed references to alarms.

46 F 2 N S D41.7 47 H 2 N S D41.10 48 F 3 N S D41.7 B41.8 This is OK although a similar KA was selected for another question. This is 49 F 2 N S written sufficiently different and requires different knowledge.

50 H 3 N S C41.5, 14 51 H 3 N S A41.7 Differs enough from SRO 91.

52 H 3 N S C41.5 53 H 3 N S D41.10 A41.4 Contact NRC about this question. Need to fix justifications and verify correct 54 F 2 N E?

answer. Good.

55 H 3 N S C41.10 56 H 3 N S B41.10 Page 3 of 6 OBDI 202 - IOLE Process

ES-401 DC-2010-01 4 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only D41.10 Verify this would be correct IAW AOP-5. Changed distractors to make it less 57 H 2 N S?

like question 32 and match the KA.

58 H 3 B S B41.11 Changed drawing.

59 H 3 B S B41.1 60 F 3 N S D41.10 61 H 2 N S D41.10 62 H 3 B S C41.5 63 F 2 B S C41.10 64 H 3 N S B41.10 65 H 3 B S D41.11 66 H 3 N S C41.7 67 F 3 N S D41.10 68 H 3 B S A41.10 69 H 3 B S D41.10 70 F 3 N S A41.10 71 H 3 B S D41.12 72 F 3 B S A41.11 73 F 3 N S B41.10 74 F 3 N S A41.7 75 F 2 N S D41.10 76 H 3 N S D43.5 77 H 3 B S B43.5 78 H 3 N S B43.2 TECH SPEC BASIS 79 H 3 N S C43.5 80 H 3 B S B43.5 Page 4 of 6 OBDI 202 - IOLE Process

ES-401 DC-2010-01 5 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 81 H 4 B S C43.2 ADDED TS BASIS INTO STEM 82 H 2 N S C43.5 83 H 3 N S C43.2 84 H 3 N S D43.5 85 H 3 M S B43.5 86 H 3 B S A43.5 DCPP-2009-02 87 H 2 M S A43.5 88 H 3 N S C43.5 89 H 2 B S C43.5 90 H 3 N S A43.5 91 F 3 N S D43.5 92 F 2 N S B43.5 93 H 3 N S A43.2 OPEN REFERENCE 94 F 3 N S B43.2 95 F 2 N S C43.1 96 F 2 N S A43.3 97 F 3 N S A43.2 98 H 3 B S C43.4 OPEN REFERENCE 99 F 2 N S B43.5 100 F 2 B S A43.5 B= 20 / 27% F= 34 / 45% E= 2 RO TOTALS: M= 4 / 5% H= 41 / 55% U= 0 Additional Notes:

N= 51 / 68%

SRO TOTALS: B= 7 / 28% F= 8 / 32% E= 0 Additional Notes:

M= 2 / 8% H= 17 / 68% U= 0 Page 5 of 6 OBDI 202 - IOLE Process

ES-401 DC-2010-01 6 Form ES-401-9 N= 16 / 64%

GENERAL COMMENT

S:

1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N.

2. Chief Examiner comments are indicated in blue.

3. Average difficulty is __ 2.73 ___ on the RO exam and _2.72__ on the SRO exam.

4. The 10CFR55.41/43 distribution is: RO / SRO 41.1 = 43.1 =

41.2 = 43.2 =

41.3 = 43.3 =

41.4 = 43.4 =

41.5 = 43.5 =

41.6 = 43.6 =

41.7 = 43.7 =

41.8 =

41.9 =

41.10 =

41.11 =

41.12 =

41.13 =

41.14 =

5. The answer distribution is: RO / SRO A = 20 / 7 B = 17 / 7 C = 16 / 8 D = 22 / 3

6. There are 2 questions with attachments provided. Page 6 of 6 OBDI 202 - IOLE Process

Changes made following first facility review.

Post Validation changes

1. 1 - changed RCP to 1-4. this will prevent someone from seeing 1-2 and somehow thinking #2 seal.

1. 2 - no change, should be operator knowledge. Added OIM reference.

1. 3 - changed setup and distractor D. added rate back, felt the cooldown could be continuing but the rate could be reduced.

1. 4 - more direct question, modified distractors. Underlined minimum (Policy on underline, bold, caps etc.??)

1. 5 - modified C, added parameters to setup. Modified A.

1. 6 - focused question on letdown effects from CCW flow

1. 7 - modified term used to describe PORV low pressure close interlock. Changed to two heater groups to reflect what is usually done.

1. 9 - added automatically to turbine trip to clear up any thought that the operator would trip the turbine.

Removed P-4 from question, focused on the turbine trip.

1. 11 - no action, however requires knowledge of PK status rather than the Ps

1. 13 - moved valve descriptor to stem, added clarification to first sentence of answers.

1. 15 - modified answers to reflect answer closer to reason for OPC. added OIM reference.

1. 17 - changed question due to potentially misleading information in the procedures. E-0.1 has the operator cycle the reactor trip breakers and then reset FWI. Cycling the trip breakers is only necessary for FWI due to P-14 and SI. The operator could answer based on what was in the EOP, which would be correct or what is actually required, also correct. Therefore, to make one answer clearly correct, changed condition to FWI due to SI (a bank question).

1. 18 - modified A. added reasons to C and D to more closely match the answer structure of A and B. modified format of question. Added name of the LCVs

1. 19 - P-14 setpoint 90% w/new steam generators and modified time delay. Thought time delay may be due to signal time from time of P-14, to trip MFPs and THEN start AFW pumps.

1. 20 - added that U2 is on Startup (not critical but it could be aligned to the 500 kV yard).

1. 21 - replaced with bank question - more operationally focused (what an operator could actually observe). Added that the battery is supplying the bus, to remove any confusion of response if the charger was available.

1. 22 - minor change to stem, comment that if the 1-1A is not completely depressurized, it may help the start

1. 25 - changed to U1 - otherwise potential double jeopardy with question 20

1. 28 - replacement due to better KA match.

1. 29 - modified C to make it clearly wrong

1. 30 - put PK back into question, it is needed to match the KA.

1. 31 - no changes, should be operator knowledge. There is enough info to determine the correct answer. Added

1. 33 - no action at this time, will see how it validates a second time. Removed middle bullets, unnecessary and could lead someone to answer the RCPs based on being told that the ECCS pumps were not running.

1. 35 - modified to make it less critical to know which P-4 does what action (B).

1. 36 - offered as a suggested change. It just not seem plausible that the reactor would trip on OPdT from such a low power.

1. 38 - enhancement to answer

1. 39 - replaced, original question not RO knowledge

1. 40 - modified A, it could be argued that core cooling is being maintained by the ECCS flow.

1. 41 - modified B and C. changed answer. Given answer was not a tripping requirement.

1. 44 - Submitted question appears to be SRO. Also, not sure there is a good tie available to KA. Suggest resample. Our stab at it, is possibly a stretch.

1. 45 - modified to rule out a non-vital header leak in containment.

1. 47 - Replaced with Ruptured, not faulted. Reformatted question, and A very close to correct, modified.

1. 48 - modified. Phase B is not needed to cause the MSIVs to close. Both happen at the same setpoint.

1. 49 - modified answer to make it clear. NOTE, this is the second time this KA is tested and it should have been resampled. (#21)

1. 52 - replaced - would not use AP-10 if the standby pump starts and if DP is high, the action per the AR PK would place the standby hx in service. This appears to be procedure knowledge not at the RO level.

1. 54 - changed question to make it a more "operate the controls" to meet the ka

1. 56 - modified D, (Operators (being operators) would start the standby pumps). Added reason for power reduction to meet ka, may not still be a good match.

1. 63 - modified D, pressurizer will rapidly increase once the RCS depressurization is stopped if SI is not promptly stopped.

1. 66 - changed to PK, this is typically what the operators check

1. 67 - there are 2 answers as written, for instance, AP-1, step 1 has the operator initiate SI (which causes a trip),

this is supported by OP1.DC10, step 4.6l2. modified A to be incorrect and changed answer to D

1. 68 - ambiguous as to what the correct answer is and could be SRO. Submitted bank question for recommended replacement.

1. 70 - better suited for SRO, replaced.

1. 72 - minor -added entered AP-14 to setup

1. 73 - replaced, not a good match of KA.

1. 74 - all are either on PAM1 or listed in TS as PAM. Changed to PAM1

1. 75 - added "number" after step. If the step is enclosed, its continuous action (A)

1. 76 - capitalize Hot Leg Recirc. Minor formatting changes to setup

1. 77 - changed modes .

1. 78 - formatting, removed "isolate the affected system", felt it was very close to the actual reason.

1. 80 - changed to address the concern with the loss of power, RHR.

1. 81 - modified D, last half unnecessary. Added that the 120VAC bus is energized, by its backup.

1. 82 - reformatted question. Removed reference to P250, replaced with PK. Modified answers. Added PRT pressure to remove possible thought that the PRT has ruptured

1. 83 - modified answers and setup.

1. 84 - does not match KA (no dropped rod in the question). replaced

1. 85 - lowered power to make the cooldown more plausible. Put temperature as the crew exits E-0 above the red/magenta path entry condition (initially temperature was a magenta path).

1. 86 - not SRO, replaced with #87 from February exam

1. 87 - with one train of cont. spray in service, the containment integrity is only a yellow path. Changed answer to A and modified. raised RHR flow, although technically correct, fairly unrealistic to only have 150 gpm if RCS is at 60 psig

1. 88 - changed. A lot of loss of power at low MODE questions.

1. 90 - if seal cooling is restored, than the evaluation has been done and RCP 2 would be started. Modified question. Removed C.1 reference, typically when we think of RCPs in C.1 its start them and darned the torpedoes.

1. 91 - replaced, question, as written has nothing to do with loss of NIS.

1. 93 - not related to KA, replaced

1. 94 - changed to place both units in a mode that did not require an STA (common to both units and required in modes 1 - 4)

1. 96 - original not really SRO, replaced with another stab at the TMOD process

1. 97 - replaced. This question is just too deep in the weeds, I know, I wrote it for the 2005 exam and it was

1. 99 - minor mod to C and D

1. 100 - minor mod

Changes made following final facility review.

Question 2 - changed 500 kV is lost to 25 kV to Auxiliary Transformer 1-2 is lost. Power to aux transformer 1-2 is actually 25 kV (stepped down 500 kV).

Question 3 - underlined reduce, common error is to misread the question. Hopefully underlining the word (as we did in question 4) will prevent this from happening.

Question 5 - modified C. the question really would require knowledge of what the PK has the operator do and its not a common task. Modifying C to containment structure sump makes it a more system focused question.

Question 9 - chronic problem involving the RPS part of the answers. Modified answers/distractors.

Question 21 - minor change to question.

Question 26 - minor point, but there really isnt a unit 1 or unit 2 instrument air system. Its shared. Modified question.

Question 35 - modified slightly, the answers to reflect that the system blocks not fails to actuates certain steam dump groups.

Question 52 - modified B. as written it was also correct.

Question 57 - changed D to letdown break Question 71 - underlined admin limit. Again, automatically assumes admin guideline (which is C).

Question 84 - modified 5th bullet. If tave and power are decreasing, one could argue that there is a problem that would require a reactor trip. For a dropped rod, both decrease then stabilize, with power eventually increasing to approximately its orginal value.

Question 85 - raised RCS temperature (first bullet). The value was very close to a Magenta path and w/o a reference is hard to discern.

Question 94 - problem is an STA is not addressed as mode specific position in the plant procedure (OP1.DC37). changed modes of the plants and answer to yes/yes.