ML022700156

Draft - NRC Comments

ML022700156
Person / Time
Site:	Susquehanna
Issue date:	07/23/2002
From:	Alan Blamey Division of Reactor Safety I
To:	Division of Reactor Safety I
Conte R
References
50-387/02301, 50-388/02301, ES-401-7 50-387/02301, 50-388/02301, ES-301-3
Download: ML022700156 (16)

Text

MEMORANDUM TO: Exam File PPL Susquehanna, LLC Susquehanna Steam Electric Station DATE: July 23, 2002 tS 37 /?3/?.

FROM: Alan Blarney Senior Operations Engineer

SUBJECT:

COMMENTS ON SUSQUEHANNA STEAM ELECTRIC STATION UNITS 1 & 2 EXAM - Pre and Post VALIDATION.

The purpose of this memorandum is to document comments on the Susquehanna exam received on Thursday June 20, 2002. (Note: the exam review was completed in 13 working days instead of 10 due to availability of reviewers - Pilgrim Prep. Week) The written exam was reviewed by A. Blamey, Senior Operations Engineer and C. Sisco, Operations Engineer. The operating exam was reviewed in its entirety by A. Blarney, with portions of the operating exam reviewed by C. Sisco. The comments were provided to R. Chin Examination Developer, Susquehanna during the prep week, July 14, 2002. The comments are as follows.

ES-401-7 Written Examination Quality Checklist do See ES 401-9, "Written Exam Review Worksheet," for comments.

_ _ The Reactor Recirculation System speed control logic is overweighted.

Question 41 for RO I SRO has been changed to focus only on the feedwater heaters.

dt Verify the method that was used to ensure no overlap between the audit and written exam.

ES-301-3 Operating Test Quality Checklist Cat. A Admin Section General Comments

>z Very that the JPMs have the required reviews as per the PPL program.

Direct "look up" question can be asked but they must be closed reference questions. ES-301, Attachment 1 provides guidance on what constitutes a look up and what is not a look up. However, ES-301-3, item 2.b, states that Category A are predominantly open refernce.

Senior Reactor Operator Administrative JPM Comments

_ SRO, A.1.a, This question is a direct look up. Additionally, would you expect your SRO to know that the individual should not

take the shift when he is on Codeine or would an MD make this determination? There is no procedural guidance on this issue.

This was changed to the individual failing his eye exam and not being able to obtain glasses prior to assuming licensing duties. The SRO applicant must now understand license restrictions and actions that must be taken. There is procedure guidance on this issue.

SRO, A.1.b, This question is a direct look up.

The stem was changed to a unit supervisor instead of a shift supervisor to allow notification of the shift supervisor.

The answer was changed to be more realistic and encompasses important actions in procedure ON-155-001, step 3.6 as well as the original answer in procedure OP-AD-002.

SRO, A.1.c, This question is a direct look up.

The question was changed to require the applicant to determine the minimum shift staffing requirements and then take appropriate actions.

SRO, A. I .d, Day shift hours should be listed for accuracy.

No change needed, accept as is.

SRO, A.2 The K/A for this JPM is 2.2.24, which is effect maintenance has on equipment LCOs. The question is on a failed surveillance. The outline was approved on surveillance procedures and the K/A for surveillance is more significant.

Change the K/A number to the correct one 2.2.12.

Why does this JPM have to be performed in the simulator?

This JPM will not be performed in the simulator.

SRO, A.3.a No comments SRO, A.3.b This question is a direct look up.

Question is changed to require a calculation to determine dose rate and then take appropriate actions in accordance with station procedures and TS.

a SRO, A.4 The JPM says that this can be performed at the completion of a scenario for the US. During this exam it can not be performed in this manor because it would be repeated on different days.

_ _ This JPM is time critical and should be less than 15 minutes.

_ _ Is this the responsibility of theeor RO?

PPL - this is the responsibility of the SRO. However, there were several items that were incorrect in the JPM.

-to The JPM must have the PICSY weather conditions.

-(ii) The JPM must be updated for current job requirements, i.e., delete the step 4 and the associated critical task.

4lii) Provide a completed ENR form.

Reactor Operator Administrative JPM Comments RO, A.1.a, No Comments RO, A.1.b, This is a direct look up.

The stem question (a) was changed from position discrepancy to condition. The operator must now identify the misspositioned control rod based on the information in the stem.

I; RO, A.1.c This is a direct look up. The wording temporary relief gives the answer away and makes direct look up.

The question was modified so that the person would report off site for a random drug test. He must understand that the testing is off site and since he is leaving the control structure he must perform a short term relief turn over.

_ RO, A.1.d This is a direct look up, the question can be reword to give answer, and K/A match not adequate.

This question has been change. The individual will receive an incomplete procedure and must determine what actions have to be taken to obtain a new 'good" procedure.

RO, A.2, No Comments

RO, A.3.a, No Comments RO, A.3.b This is a direct look up.

The stem of the question will now require the applicants to determine (calculate) dose rate and then determine what approvals I controls are required.

RO, A.4.a This is a direct look up question and this was asked on the written exam.

The question will be replaced with a simplex fire alarm.

The applicant must take the alarm, find the area, and then determine action based on a fire in that area.

___ RO, A.4.b The outline states ALERT the question states UNUSUAL EVENT. Change the question to an ALERT to match sample plan.

Cat. B Job Performance Measures (JPM)

B.1.a Step 19 should be a critical step.

___ B.1 .c Bypassing SRM channel "C" Rod Block Input to RMC.

This JPM was modified to remove the control rod withdrawal at the beginning of the JPM. This activity provided addition time and set up that did not effect the required action of the JPM.

___ B.1.d Is this JPM a direct bank JPM or modified and if modified then what is the date / rev. on the JPM. -

B.1.e Step 7 should have the instrument air cross-tie valves listed as 126172 and 126167.

B.2.a Fire Protection System Crosstie to RHRSW.

This JPM was changed during validation week from Fire Protection System Crosstie to RHRSW,,to Connecting the Standby Liquid Control Tank to the RCIC System in accordance with ES-150-002. The basis for this change was to increase the discrimination ability of the JPM portion of the exam and minimize dose.

ES-301-4 Simulator Scenario Quality Checklist General Comments Very that the scenarios have the required reviews as per the PPL program. The original scenarios that were sent with the approved exam was not approved by the supervising manager, as required.

Verify that the simulator modeling is not altered for the exam.

Review the simulator scenarios that were used for remediation during the prep. week and determine if there any adjustment must be made to the NRC exam scenarios.

Scenarios NRC0201 Page 21, The power decrease may be less than 20%.

___ NRC0202 Page 16, The power decrease may be less than 100 MWe.

Page 22, The scenario should provide the information that both CRD pumps are damaged beyond repair so that we will not have to wait 20 minutes to shutdown.

Review the level of difficulty at the simulator. This scenario has 4 failures after the EOP entry conditions which are required to get the plant to an emergency depressurization state.

The insertion of control rods should not be a critical task because the actions are taken locally. This will remain a CT because the actions are directed by the applicant and in accordance with Appendix D, D. 1. a, effectively directing can be considered a CT.

NRC0203 Page 7, Why is identifying that the RPV level indication is no longer valid a Critical task. If they do flooding don't they recognize this? - JOT C-T.

L NRC0204 Page 20, why isn't inhibiting ADS a critical task and why isn't there a step to run recircs back to minimum and trip one at a time?

ADS not critical in this scenario and Recirc pumps will be tripped due to EXC RPT.

Page 24, first step, PCOM should be a critical step, manual rod insertion. This is a continuation of a critical task from the previous page.

Page 25, instructor activity 1 & 2, should these activities be based on the actual time to perform the activity. For example, activity 1 will take about 15 minutes, not five.

NRC0205 None

( 401 Susquehann ken Examination Form ES-401-9 (R(

)

Review Worksheet

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD _ -

(F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only BI F 2 x Y U N The stem should state the minimum water level. If the stem does not state the minimum then (a) & (b) are also correct.

ANS The was corrected to state the minimum water level.

SROs will have EOP 113, therefore, this question is a direct look up and not acceptable for a memory level question.

ANS To determine the water level the applicant will have to work through the EOP or know from memory based on high power to find answer. Not direct look up.

B2 H 2 Y S N B3 H 3 Y S N B4 H 3 Y S N B5 H 3 Y S N B6 H 3 Y S B B7 H 3 Y S N Shouldn't level be increased to promote natural circulation?

The answer agrees with the procedure but I do not technically understand why you would not raise level to promote natural circulation?

ANS This answer is technical correct because if level was increased to enhance natural circ. the J-Tube and stub tube welds would see a rapid temperature change /

increase in thermal stress.

B8 F 2 Y S N

I I,

1. 2. 3. Psychometric Flaws 4. Job Content Flaws - Jther 6. 7. l Q# LOK LOD _ _ I (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO UIEIS Explanation Focus I Dist. Link l units ward K/A Only Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

• The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

More than one distractor is not credible.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-onl, (K/A and license level mismatches are unacceptable).

6. Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

7. At a minimum, explain any "U"ratings (e.g., how the Appendix B psychometric attributes are not being met).

I

( ( -I-.'.

(

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

LOK LOD (F/H) (1-5) Stem ICueR T/F Cred- IPartial IJob- I Minutia I / IRack- = ISRO 1I/IFI5 Pynilnntinn

\-~~~~~~~~~~"t 0ist.

Focus Link W-units ward K/A Only I ,\.V . . ..........................

B9 H 3 _ S B B10 F 2 Y S N B11 H 2 Y S N B12 H 2 Y S N B13 H 2 Y S B Question is sat., however, on the answer key the proposed answer should state that the point is above the curve not below.

B14 F 2 Y S N B15 F 2 Y S M B16 H 2 Y S B Ventilation recirculation system reduces the dose but if it failed I l the off site dose is expected to rise but not exceed 10 CFR 100.

B17 F 2 Y U M Isspecifically there a correct answer? The answers & distractors state "only," and do not address the ECCS room cooler (safety related loads). Based on your plant configuration both ESW loops will automatically supply the Room Coolers.

ANS The question was reworded to "only" look equipment in question and disregard the ECCS room coolers.

B18 F 2 _ S N B19 H 3 Y S M B20 F 2 Y S M B21 H 3 _ U M Is there a correct answer? The fans stop, with the breakers still closed, not tripped. They restart when power is restored. The wording should be changed to actually reflect the system /

component status.

ANS TRIP was removed from the distractors (b) & (d)

B22 H 3 _ S B Could answer (a) also be correct? Once isolated the N2 will expand as it heats up and if not vented could actuate the high DW pressure scram result. If the system is isolated at 1# will the expansion reach the 1.72 scram?

ANS PPL stated that the expansion will not result in pressurizing the drywell to 1.72#.

B23 F 2 Y S M B24 F 2 Y S B B25 H 2 Y S M

I I

( 1. 2. 3. Psychometric Flaws 4. Job Content Flaws ,cher 6. 7.

LOK LOD _

(F/H) (1-5) Stem ICues T/F l Cred. Partial Job- lMinutia #/ Back- Q= SRO U/E/S Explanation Focus I I I Dist. Link I I units I ward I K/A I Oniv B26 F 2 Y S N B27 F 3 Y S M B28 H 2 Y E B The applicant should be given a power level or other information to be more specific that not in mode 4.

__ ANS Stem now states in MODE 1 Q#

1.

LOK 2.

LOD

3. Psychometric Flaws 4. Job Content Flaws 5. Other J 6. 7.

(F/H) (1-5) Stem Cues T/F 1Cred. Partial Job- Minutia #/ Back- Q= SRO U/EIS Explanation Focus

- - Dist. _ Link _ +/-

lunits ward K/A l Only l B29 F 2 Y S B B30 H 3 Y S B B31 F 2 Y S B B32 H 3 Y S N B33 H 2 X Y E N Change the high pressure trip to RPT breaker to be consistent with the stem. In addition the high pressure trip may also lead the individual to ATWS and quickly reducing reactor power.

ANS distractors change to RPT breaker.

B34 H 2 Y S N B35 F 2 Y S M B36 F 2 Y S N The K/A address high off site release rate the question address high rads in secondary containment.

ANS The basis discuss limiting releases into the secondary containment to minimize off site release.

B37 F 2 X X l U M The last sentence of the stem should state ... PSL curve always...

to ensure that distractors (a) & (b) are incorrect.

ANS Stem changed to "as a minimum"

I,

( 1. 2. 3. Psychometric Flaws 4. Job Content Flaws I -T-r (her 6. 7.

I I

LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- lMinutia l #/ l Back- Q= lSRO U/E/S Explanation Focus I Dist. Link I I units I ward I K/A I Oniv B38 H 3 X X X Y U N The stem must be focus more on a control rod that has the RBM functional.

(C)- A CR on the core periphery will not be stopped by the RBM nor will it provide a scram, unless specific conditions exist.

Provide more specifics - rod location.

(d) - control rods do not drop from 24 to 00 they drop from 00 to 24.

ANS Stem focus was improved and distractors were corrected.

B39 F 2 Y S M Based on your JTA is this expected to be know from memory.

(Hot box - have all personnel read the hot box or have the license applicants procedures been froze before the hot box was issued?)

ANS Yes they have been trained and expect to know from memory B40 F 2 Y S B EOP 113 (sheet 1&2?) will be provided to the applicants. This changes the question to a simple lookup. This is not acceptable for a memory question.

ANS Sheet will NOT be provided B41 H 3 Y E N The question answer is correct but for the wrong reason. The Hi-Hi alarm will not result in an isolation. Level will continue to rise until 47 is reached and then a 30 second time delay to isolate. Not a 30 second time delay after the 35' Hi-Hi alarm. So the actual isolation will occur when the heater fills up to 47" and a 30 second time delay - hence after 30 seconds is correct.

B42 H 1 X X Y U N The correct answer is the only answer/distractor that contains the fuel pool cooling and cleanup system, same as the stem. In addition, (C) Is also correct because of water level decreases to this level the fuel pool cooling pumps / system will trip.

ANS (c) changed to the SDHRS and ques in (d) removed.

B43 H 2 Y S B B44 H 3 Y S N B45 H 3 Y S N B46 F 2 _ S N

I l

/

ti I,

1. 2. 3. Psychometric Flaws 4. Job Content Flaws - her 6. 7.

LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO UIEIS Explanation Focus I Dist. Link I I units ward I K/A I Only B47 H 3 Y S N Would the other feedwater pumps be able to make up the water loss from the min. flow line and if yes would you receive a low suction pressure alarm and trip the pumps?

ANS this information is not required to be know because the initial response is to open and divert flow.

B48 F 2 _ _ Y S B

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 15. Other 6. 7.

Q# LOK LOD l I T 1 1 (F/H)

I (1-5) Stem lCues TIF Cred. Partial Job- Minutia #/ Back- Q= SRO UIEIS Focus I I Dist. I1 ILink I units ward IK/A Only II Explanation B49 H 3 Y S N B50 F 3 Y S N lB51 F 2 X X Y U N Could the other distractors also be correct or do they have higher EQ temperature values. They may not have higher temperatures and therefore a, b may also be correct.

ANS Changed components to ones that would be greater than 340 F.

B52 F 3 Y S N B53 H 3 Y S N B54 F 2 V S N B55 H 2 X Y U N Do you expect your operators to know these procedural steps from memory? Are these immediate operator actions? What does you Knowledge objective require?

ANS No the operators are not expected to know this from memory. The procedural reference will be provided.

B56 H 3 Y S N B57 H 2 Y S N B58 H 2 Y S N B59 H 4 Y S N B60 H 3 Y S N B61 H 3 Y S M B62 F 2 Y S B B63 H 2 Y S M

/= /,,

- - r

1. 2. 3. Psychometric Flaws 4. Job Content Flaws I cher 6. 7. 'K LOK LOD (F/H) (1-5) Stem Icues T/F Credi Partial

- - onh- 6Minutia.....- I It/ I BRk- OIA=

I----RR I I//IC I M-rzl---

LAididLIVI

_ Focus I I Dis..

Iink I... units I ward K/A In yI B64 F 2 Y S M B65 H 3 Y S N B66 F 2 Y S N B67 H 3 Y E M A time period of greater than 20 minutes could be used to prevent any ambiguity on the source of power.

ANS 30 minutes were provided in the stem.

B68 H 3 Y S N B69 H 2 Y S N B70 F 2 = _ _ = - _ S IN

3. Psychometric Flaws 4. Job Content Flaws l 5. Other 6. 7.

Q# L K LO D - - - - - - - - - -I (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation H I ; I Focus IIIL Dist. I _ Link I ] units ward K/A Only R i r .

B71 H 2 Y E N The stem wording should be changed "determine identify" I I II I I I I I I l lANS Removed identify from stem.

B72 F 3 N U N K/A miss match IANS Question was replaced.

B73 F 2 Y S N B74 F 3 Y E N The vacuum breakers could be changed to the condenser vacuum breakers for clarity.

ANS Condenser added for clarity.

B75 H 3 Y S N SI H 2 Y Y S N The second part of 43.5 is not addressed, and immediate actions steps.

ANS 43.5 Is appropriate because they must diagnose plant conditions and understand that RPV control is entered I land then directs ATWS procedure.

S2 I H l 3 lY Y I S N 10 CFR 55.43 (4) may be more appropriate for the reference.

7

( 1.

LOK 2.

LOD 1

3. Psychometric Flaws

4. Job Content Flaws

. i.

I =

ucher 6. 7.

7 i

(F/H) (1-5) Stem Cues T/F Cred. IPartial Job- lMinutia l #/ Back- Q= ISRO UIEJS Explanation

__._____- Focus I I Dist. I _. Link I I lnits II. ward

_ a../I\ I4l &A

. WA Ank, Z [MA a .

S3 F 2 X Y YIN U N Unsat because this question does not meet the criteria for an SRO, is this condition in the facility license? The closest item would be 55.43.5 and if this is the case then selection of appropriate procedures is not met.

ANS Changed to reflect conditions and actions.

S4 F 2 Y Y/Y S B S5 H 2 N YIN U N K/A Knowledge of system status criteria that requires the notification of plant personnel. The question does not address this K/A it just ask how to make the announcement, it does not address the criteria for the announcement.

Question changed to classify event at a level that would require a site assemble.

S6 F 3 Y Y/Y S N Is this a knowledge objective for the SRO?

ANS Yes S7 F 3 Y Y/Y S N S8 F 2 Y Y/Y S N S9 H 2 Y Y/Y S N S10 H 3 Y Y/Y S N S11 F 2 Y Y S N The criteria for SRO is 10 CFR 55.43 (4) not (5)

S12 F 3 Y Y/Y S N S13 H 3 Y Y /Y E N Should the stem state that HPCI and RCIC is operable ( all ECCS is operable) or do you expect the Individual to assume that it is operable because of the testing?

ANS The should know this by PPL practice of testing. HPCI and RCIC Is not inop at the time of planned maintenance test.

This meets the criteria in 43 (2) not (1)

S14 H 3 Y Y/Y S N The criteria for SRO is 10 CFR 55.43 (2) not (1)

S15 H 3 X Y Y/ Y U N Would you expect the applicants to know this level of detail with the procedure?

I_-__ ANS No. Procedure provided.

II

( 1'

1. 2. 3. Psychometric Flaws 4. Job Content Flaws

=

,Aher 6. 7.

I, LOK LOD . . _ _ -

(F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO UIE/S Explanation Focus I Dist. Link I I u1nitsi wI rId I K/A I Onlv 816 H 2 Y Y/Y S N 817 F 2 Y~ 'YY S N S18 H 3 Y( YIY S IN 819 H 3 Y YIY S N this should be 43(4) not(5)

S20 F 2 Y Y/ Y U N Must add directly In stem to prevent more than 1 correct answer.

ANS Added directly in stem.

S21 H 2 Y YIY S N S22 H 3 Y Y/Y S N S23 H 2 Y Y/Y S N S24 F 2 Y Y S NThe criteria for SRO is 10 CFR 55.43(4) not (I)

S25 F 2 Y Y/Y S N RI F 2 Y S N R2 H 3 X Y E M The last sentence of the stem should state which one of the following best describes the appropriate TS action ( if any), and the shift supervisor should be removed from all the distractors and answers.

ANS increased stem focus R3 H 3 Y S B All the actions listed will function; however, the IRM I APRM will cause the trip, not the MS Radiation Monitors.

R4 F 2 Y S B R5 F 2 Y S B R6 H 3 Y E M State that the module has not been plugged back into the draw, or to determine which module is unplugged at the end of next to last sentence. This will make it clear that the module is still unplugged.

ANS Changed format for clarity.

R7 H 3 Y E M (a), (b) should start by saying a rod block exists, for symmetry.

ANS Added "a rod block exists" R8 H 3 Y S B R9 H 3 V S IN R10 H 2 _S N

7 1.

LOK 2.

LOD

3. Psychometric Flaws 4. Job Content Flaws Cred. Partial Job-MinutkI iher 6. 7. K (F/H) (1-5) Stem CuesI T/F I #/ Back- Q= TSRO UIEIS Explanation Focus Dist. Link Job InI 1nits ward I K/A I Oniv R FH 2 Y S N R12 F(H) 2 Y S N R13 F 2 Y S N R14 F 2 _Y S N R21 F 2 Y S N RIB F 2 Y S B R17 H 2 Y S N R18 H 2 Y S M R19 F(H) 3 Y S B R20 F 2 Y S N R21 F 3 Y S N R22 F 2 Y S N R23 F 2 Y S N R24 F 2 Y S N R25 F 12 Y S N