ML023290458

Draft - NRC Comments

ML023290458
Person / Time
Site:	Salem
Issue date:	08/21/2002
From:	Alan Blamey NRC/RGN-I/DRS/OSB
To:	Public Service Enterprise Group
Conte R
References
050272/OL03-301
Download: ML023290458 (18)

Text

August 21, 2002 MEMORANDUM TO:

Exam File PSEG RO / SRO - Salem Station FROM:

Alan Blamey 19,12-t/z,

/0 Z Senior Operations Engi eer

SUBJECT:

COMMENTS ON SALEM STATION REACTOR OPERATOR /

SENIOR REACTOR OPERATOR EXAM OUTLINE.

The purpose of this memorandum is to document comments on the Salem Station exam outline received on Friday, August 16, 2002. The exam outline was reviewed by A. Blamey, Senior Operations Engineer. The comments were provided to F. Kaminski, Examination Developer, PSEG on August 21, 2002. The comments are as follows.

Modified ES-201-2, Examination Outline Quality Checklist Section 1.

Written

c.

Overemphasizes of plant systems.

Control rod drive is overemphasized. (Also JPM on dropped control rod)

RO T1 / G1 inoperable or stuck control rod, T1 / G2 contentious rod withdraw, dropped control rod, T2 / G1 CRDS design features, T2 / G1 CRDS reactivity T3 control rod programing SRO T1 / G1 contentious rod withdraw, T1 / GI dropped control rod, T1 / G1 inoperable or stuck control rod, T2 / G1 CRDS design features, T2 / G1 CRDS reactivity T3 control rod programing Area Radiation Monitoring is overemphasized RO T1 / G2 accidental radwaste Rel. / ARM T2 / G1 ARM effect on fuel handling equipment T2 / G1 ARM effect on ARM with dose / distance T2 / G2 ARM effect on fuel pool cooling T2 / G3 Monitor radiation levels in CR during fuel handling Section 2.

Integrated Plant Operation (IPO)

Salem used Rev. 8 forms for ES-301-5 & ES-301-6, instead of the rev. 8, Supplement I forms. The ES-301-5 forms must be corrected; however, the ES-301-6 forms are the same in Supplement 1, so they do not have to redone.

b.

Must verify that: no scenarios will be duplicated from the applicants audit exam. This will be completed when the audit exam outline is completed. PSEG is aware of this condition and plans to prevent exam overlap.

c.

Scenario 1 has an instrument failure (Tavg) and Scenario 3 has a similar instrument failure (Tc). Both failures result in the same plant response and conditions. These scenarios are run on different days; therefore, one of these failures will be modified to ensure exam integrity.

C.

Section 3.

Walk Through - Job Performance Measures (JPM)

a. (3)

No JPMs are duplicated from the applicants audit exam.

Since the audit exam has not been developed this must be verified prior to administering the exam. PSEG is aware that the walk through (JPMs) exams can not be a duplicated from the audit exam.

d.

No items are duplicated on successive days.

These two JPMs appear to be nearly identical, one of the JPMs will be changed or they will be done on the same day to ensure exam integrity JPM Day one RO "Shift to Hot Leg Recirculation with 21 RHR Pump Unavailable.

JPM Day two SRO "Shift to Hot Leg Recirculation with 22 RHR Pump Unavailable."

Section 4.

General

d.

Check for duplication and overlap among exam sections.

The outline appears acceptable; however, the NRC can not make a final determination until the exam is drafted and the individual questions can be reviewed.

-3

e.

Check the entire exam for balance of coverage.

The outline appears balanced ; however, can not make a final determination until the exam is drafted and the individual questions can be reviewed.

SALEM SAMPLE PLAN POST-NRC REVIEW

SUMMARY

OF CHANGES SIMULATOR To address NRC comment regarding similarity of actions for Scenario 1, Event 2 and Scenario 3, Event 3: revised Scenario 1 by changing Event 2 from RCS Loop Tavg fails high to an Intermediate Range NIS Channel fails high. This change requires the RO to manually re-instate the Source Range NIS channels after the reactor trip.

As a result of the previous change: revised the Spare Scenario by deleting Under-compensated Intermediate Range Channels (Event 6), adding the controlling Pressurizer Level Channel fails low and moving that up to be Event 2.

There is a Pressurizer Level Channel fails high malfunction in Scenario 1 but the actions are different for high vs. low failure.

SIMULATOR JPM To address the NRC Reviewer comment regarding similarity of JPM's from one day to another: changed RO JPM S2 from "Shift to Hot Leg Recirculation with 21 RHR Pump unavailable" to "21 SJ44, CNTMT Sump Suction Valve, fails during the shift to Cold Leg Recirculation". However, it is worthy of note that the approved JPM Outlines on a previous NRC exam (ECHO Group) had similar tasks from one day to the next (Shift to Cold Leg Recirculation with different vital busses available).

WRITTEN SRO Tier 1, Group 1

° Changed 058 AA1.02 (Accidental Liquid Radwaste Release) to 068 AA1.28 (Control Room Evacuation). The UFSAR does not consider Accidental Liquid Waste Release to be credible.

Changed 001 AA2.04 (Continuous Rod Withdrawal) to 057 AA2.18 (Loss of Vital AC Electrical Instrumentation Bus) to increase examination coverage by reducing the number of control rod and rod control questions.

SRO Tier 3 Changed 2.2.33 (Knowledge of control rod programming) to 2.2.13 (Knowledge of tagging and clearance procedures) to increase examination coverage by reducing the number of control rod and rod control questions.

RO Tier 1, Group 2 Changed 001 AA2.04 (Continuous Rod Withdrawal) to 025 AA2.07 (Loss of RHR) to increase examination coverage by reducing the number of control rod and rod control questions.

Changed 058 AA1.02 (Accidental Liquid RadWaste Release) to E16 EAI.1 (High Containment Radiation). The UFSAR does not consider an Accidental Liquid Waste Release to be credible.

RO Tier 2, Group 1 Changed 068 G 2.4.31 (Liquid Radwaste) to 068 K5.04 to increase examination coverage by reducing the number of questions tied to radiation monitoring interlocks. There is only one alarm (RMS related) tied to the Liquid Radwaste System.

Changed 072 K5.02 (Area Radiation Monitoring) to 072 G 2.4.31 to maintain K/A selection distribution in Tier 2. Unlike the Liquid Radwaste System, there are numerous alarms and procedure based situations associated with Area Radiation Monitoring.

RO Tier 2, Group 2 Changed 033 A1.02 (Spent Fuel Pit Cooling) to 033 A1.01 to increase examination coverage by reducing the number of questions tied to radiation monitoring interlocks.

RO Tier 2, Group 3 Changed 034 A4.01 (Fuel Handling) to 034 A4.02 to increase examination coverage by reducing the number of questions tied to radiation monitoring interlocks.

RO Tier 3 Changed 2.2.33 (Knowledge of control rod programming) to 2.2.13 (Knowledge of tagging and clearance procedures) to increase examination coverage by reducing the number of control rod and rod control questions.

All summarized changes resulted from NRC Reviewer comments and/or were facility findings after written examination question development started.

9/16/02 K/A CHANGES

" SRO Question 61 - changed from E04 EAI.1 to E04 EA2.1. Two E04 EA1.1 K/A's had been randomly selected but question opportunities are limited because E04 (Salem LOCA LOCA OUTSIDE CONTAINMENT) is only a one page procedure.

"* SRO Question 67 - changed from 055 AA2.01 to 055 AA2.02. Unable to develop/validate an SRO level question against 055 AA2.01.

"* SRO Question 70 - 068 AA2.01 is very similar to an In-Plant JPM exam task. Randomly selected 001 AA2.03 from an unselected topic in the same Tier and Group (001 or 059).

"* SRO Question 78 - 028 AA2.1.12 very similar to T2G2 011 A3.03.

Manually selected E13 G2.4.6, an unselected topic from the same Tier and Group

MEMORANDUM TO:

Exam File PSE&G Salem Units 1 & 2 DATE:

October 29, 2002 FROM:

Alan Blamey Senior Operations Engineer

SUBJECT:

COMMENTS ON SALEM UNITS 1 & 2 EXAM.

The purpose of this memorandum is to document comments on the Salem exam received on Tuesday, September 24, 2002. The written exam was reviewed by A. Blamey, Senior Operations Engineer, P. Bissett, Senior Operations Engineer, and J. Laughlin, Operations Engineer. The comments are as follows.

ES-401-7 Written Examination Quality Checklist "A

See ES 401-9, "Written Exam Review Worksheet," for comments.

ES-301-3 Operating Test Quality Checklist Cat. A Admin Section General Comments Very that the JPMs have the required reviews as per the PSE&G program.

Very that the audit test has no overlap with the NRC exam.

Reactor Operator Administrative JPM Comments Clearing and Tagging and MOV needs to have a FORM 6 Resolution:

Changed to Que the applicant that the work control center will get form 6. In addition, added information to ensure that the operator tags the valve closed.

Independent verification of an ECP. Add additional steps to allow this JPM to be performed in a class room setting.

Resolution:

This JPM was acceptable as is; however, added additional information to allow it to be performed in a class room setting.

Senior Reactor Operator Administrative JPM Comments Classify the event in scenario ESG-3. The event should have been classified as a Site Area Emergency, not an ALERT.

Resolution:

The classification was changed to a Site Area Emergency.

Job Performance Measures (JPM)

General Comments Should alignment for CL recirculation and shift to HL be done on the same day?

They do not have to be performed on the same day. The JPMs are two distinct modes of operation and different enough that they can be performed on different days.

Resolution:

Shift to HL Recirculation ECCS mode with 22 RHR pump OOS.

This is not an alternate path JPM, there is no equipment failure that the applicant must identify. The 22 RHR pump is OOS, it is not a malfunction as required in NUREG-1021, appendix C, page 4 or 10.

Resolution:

Modified the applicants initial conditions to remove the 22 RHR pump OOS. The applicant must now identify that the 22 RHR pump is not running and take appropriate actions.

Re-energize pressurizer heaters in accordance with 2-EOP-TRIP-2 This is not an alternate path JPM, there is no equipment failure that the applicant must identify.

Resolution:

Accept - This is an alternate path JPM. 2E6D & 2G6D breakers are normally opened during an ATWS to trip the reactor. Therefore, the alternate path is the operator verifying the breakers and taking appropriate actions to close these breakers and re-energize the pressurizer heaters.

Initiate AFW Flow - Steps 3 & 8.1 should be critical steps Resolution:

Steps 3 & 8.1 are not critical tasks, they are verification steps.

Containment Spray Failure during a LBLOCA Are the CS pumps suppose to be on and they have not started? If yes then OK as alternate path.

Resolution:

This JPM is NOT an alternate path and has been reclassified as a normal JPM. The JPM on a "Dropped Rod in MODE 2, "

Cat. B was modified to become an alternate path JPM to preserve the integrity of the outline.

SStart and EDG from the control room This is not an alternate path JPM, there is no failure that the operator must identify.

Resolution:

This is an Alternate Path JPM. The applicant must identify that the Service Water Pumps did not start and shutdown the EDG after starting it.

_Take Local Control of the Pressurizer Backup Heaters This JPM had discrepancies between the procedure and JPM standard.

Resolution:

This JPM was rewritten to more clearly coincide with the limited procedure guidance that was available.

ES-301-4 Simulator Scenario Quality Checklist General Comments

_Very that the scenarios have the required reviews as per the PSE&G program. The original scenarios that were sent with the approved exam was not approved by the supervising manager, as required.

Verify that the simulator modeling is not altered for the exam.

Scenarios

_Scenario ESG1 and spare both have a failure of the pressurizer level controller. One of the level controller should be changed to another component that requires action, or ensure that if the spare scenario is used that it is not used in conjunction with ESGI.

Resolution:

This is acceptable because one scenario has the level instrument fail high and the other has the level instrument fail low. There are different and distinct actions that the operators must take as a result of the two failures.

The rate of power reduction in the spare scenario should be increased to 1% per minute so that the scenario does not take an hour to reach the desired power level.

Resolution:

Accept as is. The scenario can continue without a complete load reduction.

ES-401 Salem Written Examination Form ES-401-9 (R8, SI)

Review Worksheet

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Cred. lPartial Job-Minutia

1. /

Back-Q=

SRO U/E/S Explanation

- Focusl II j Dist.

Link units ward K/A Only I

F 2

X Y

U N

Implausible distractors: A. Startup P/B? any interlock for S/U of RC? D. Manual reactor trip would not cause urgent failure.

Removed interlock from stem and changed destractors.

2 H

3 Y

S N

3 F

3 Y

S N

Is this the thermal barrier return valve? Required knowledge?

The applicants must know valve number and this is a separate line.

4 F

2 Y

S N

5R H

2 Y

S N

6 H

3 Y

S N

7 H

3 Y

S N

8 F

3 Y

S N

9 H

3 Y

S N

10 F

2 Y

S N

Instructions (Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1.

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2.

Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3.

Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

More than one distractor is not credible.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4.

Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5.

Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6.

Based on the reviewer's judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

7.

At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

I

ES-401 2

Form ES-401-9 (R8, S)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD S

T I

P J

t (F/H)

(1-5)

StemCueslTIFlCred. Partial Job-Minutia

1. /

Back-Q=

SRO U/E/S Explanation Focusl j

I Dist.

Link units ward K/A Only 11 H

3 Y

S B

12 F

2 X

Y U

B "D" length & no ONLY is a giveaway. Distractor D shortened and only removedfrom distractors.

13 H

3 Y

S B

14 F

3 Y

S B

15 F

3 Y

S N

16 H

3 Y

S B

17 H

3 Y

S N

18 H

3 Y

S N

Will charging initially rise? Two correct answers? Charging will not initially increase, letdown will isolate. NOT2 correct ans.

19 F

2 Y

E B

Add RTB. RTB added to the distractors.

20 F

2 Y

S N

21 H

2 X

Y U

N Too easy. 3 implausible distractors. Question IDistractors reworded to increase level of difficulty.

22 H

3 Y

S B

23 F

2 Y

S B

24 H

3 Y

S N

25 F

2 Y

E N

Put distractors in stem. Moved manually started from the control room to the stem.

26 F

2 X

N U

N Is answer B correct?, what is HRPS? Question was changed to match the K/A.

27 F

2 Y

S N

28 H

3 Y

S N

29 F

2 Y

S N

30 H

3 Y

S N

ES-401 3

Form ES-401-9 (R8, SI)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/IF Cred. Partial Job-Minutial #/ IBack-Q=-SRO U/E/S Explanation Focusj I

Dist. __ink nts ward

,1,,-__On_

31 F

2 Y

E N

IAW the procedure Added in accordance with... to stem 32 H

2 Y

E B

lAW the procedure Added in accordance with... to stem 33 R F

2 Y

S N

34 H

3 Y

S B

35 H

2 Y

E N

lAW the procedure Added in accordance with... to stem 36 F

2 Y

S N

37 F

2 Y

S N

38 H

3 Y

S N

39 F

2 Y

S B

40 H

3 Y

S B

41 H

3 Y

E B

Stem format. Changed wording in stem.

42 R H

I Y

U M

LOD Removed reference from applicant to increase LOD 43 F

2 Y

S B

Is B also correct? NO, does not maintain fullflow.

44 F

2 Y

S N

Do you expect your operators to memorize this procedrue?

Yes, as per the learning objective.

45 F

2 Y

S B

46 F

3 Y

S N

47 F

2 Y

S N

48 H

3 Y

S N

49 H

3 Y

S N

50 H

3 N

S N

ES-401 4

Form ES-401-9 (R8, Si) 1

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD I

I I-II I

(F/H)

(1-5)Stem Cues T/F Cred. Partial Job-Minutia

1. /

Back-Q= SRO U/E/S Explanation Focus IIDist. Link Li

___I units I ward K/A On 51 F

3 X

Y U

N Length and phrasing of "B" answer is different than distractors.

Modify "B" length.

52 F

2 Y

S N

53 R H

2 Y

S B

54 H

2 Y

E B

Change to LO-LO. Changed 55 H

3 Y

S B

56 H

2 Y

S B

57 R H

2 Y

S N

58 H

4 Y

S N

59 R H

2 Y

S N

30 R H

2 Y

S N

31 R H

3 Y

S B 2001 NRC Exam' 32 R H

3 Y

S B

53 R F

2 Y

S N

34 R H

3 Y

S B

5 R H

3 Y

S B 2001 NRC Exam 56 R F

2 Y

S B

37 R F

2 Y

S M

68R F

2 Y

S N

69 R H

2 Y

S B

0 R F

2 Y

S N

ES-401 5

Form ES-401-9 (R8, SI)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Cred. Partial Job-Minutia

1. /

Back-Q=

SRO U/E/S Explanation Focus IIIDist.

Link units ward K/A Onl 71 R H

2 Y

S B

72 R H

3 Y

S N

73 R F

2 Y

S B

74 R F

3 Y

S N

Do you expect your operators to know from memory? YES 75 R H

2 Y

S M

76 R F

2 Y

S B

Added If any to stem.

77 R F

2 Y

S N

78 R H

2 Y

S N

79 R F

4 N

U N

Does not address flow portion of K/A. Change question to match K/A.

0 R F

2 Y

E B

Revise distractor "B" Does not need to be revised because no one in area of discharge.

81 R F

3 X

X N

U N

There is no discussion on isolation and would you expect your RO to know this information w/o TS or procedure / True false list. Change question to match K/A.

82 R F

4 N

U B

There is no action to mitigate as per the K/A. Changed question to match K/A.

83 R F

3 Y

S B

84 R H

2 Y

E N

Add "RMS TRBL" Added to stem.

85 R H

3 Y

S M

86 R H

3 Y

S N

87 R F

3 Y

S B

88 R H

3 Y

S N

89 R H

4 Y

S B

0 R H

3 Y

S B

ES-401 6

Form ES-401-9 (R8, SI)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD (F/H)

(1-5)

Stem IoCues T*F Cred. Partial Job-,Minutia

1. /

Back-0=

SRO UJEIS Explanation Focus I Dist. __ Link__

units ward WJ I Onl I

91 R F

2 Y

S N

92 R H

3 X

Y U

N "B" could also be correct. Change "B" to be incorrect.

93 R F

2 Y

S N

94 R F

3 Y

S N

95 R H

3 Y

S N

96 R F

2 Y

S N

97 R H

2 Y

E B

Noun names should be added to valve numbers? Added 98 R H

3 Y

S N

98 R F

3 X

Y E

B Reword distractors. Distractors reworded IOOR H

3 Y

S N

4 11.L.11J....J.

.L.........J

.L.1....1L....J.

i i

i 1

1 1

i i

i i

i i

i i

i

ES-401 7

Form ES-401-9 (R8, S 1)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F ICred. Partial Job-Minutia

1. /

Back-Q=

SRO U/E/S Explanation Focus Dist.

Link units ward K/A Only 5S H

3 Y

Y S

N Originally questioned K/A match. K/A match OK but revised the question and increased the level of difficulty.

33S H

3 Y

Y S

N 42S H

2 Y

Y E

B Explanation should state purple path not red, is step 21 RCS temp. control before faulted S/G evaluation? Explanation IIchanged.

53S H

2 Y

Y S

B 57S H

2 Y

Y S

N 59S H

2 Y

Y S

B 60S F

3 Y

N S

N 61iS H

3 Y

Y S

M 2001 NRC Exam 62S F

3 Y

N S

B 63S F

3 Y

N S

B 64S H

3 Y

Y S

N 65S H

3 Y

Y S

N 66S H

2 X

Y N

U N

Add "of the 3 CCW pumps" to A. Changed distractors to prevent more than one correct answer.

67S H

2 Y

Y E

N Condenser bullet is missing information. Reworded stem.

68S H

3 Y

N S

N 69 S H

3 Y

N S

N 70S H

2 Y

Y S

B 71S F

3 Y

N S

N 72S H

4 Y

Y S

M 73S F

3

-Y N

S N

ES-401 8

Form ES-401-9 (R8, Si)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD (F/H)

(1-5)

Stem ICuesj T/F J Cred. Partial Job-Minutia

1. /

Back-Q= 1SRO U/E/S Explanation Focus Dist.

Link units ward K/A I Only 74S H

3 N

Y U

N The K/A states EOP, the Q address an AB procedure.

Changed K/A to to reflect abnormal procedures instead of EOPs.

Numerous EOP questions in exam.

75S H

3 Y

Y S

M 76S F

2 Y

N S

N 77S F

2 Y

N S

N 78S H

3 Y

Y S

N 79S F

2 Y

Y S

N 80S F

2 N

N U

N K/A requires Q to determine and interpret conditions, not how to monitor parameters. Question changed to match K/A.

81S F

2 Y

N S

N 82S H

3 Y

Y E

N Add "if any" to stem. If any was added to stem.

83S H

3 Y

Y E

B have "C" include lAW TS 4.0.3? Added to "C" 84S F

2 X

Y N

U N

Description of channels in "C" could provide clues to ans.

Distractor "C" changed.

85S F

2 Y

N E

N Should "A" have only to ensure only one correct answer? Only added to "A" 86S F

2 Y

N S

N 87S H

3 Y

Y E

N Be more clear on what assembly is where. Changed 88S F

2 Y

N S

N 89S F

2 X

Y N

U N

A & B are nearly same and reword D for clarity. Wording changed 90S H

3 Y

Y E

B Add "if any" Ifany added 91S H

3 N

N U

N The Q is silent on mitigating consequences. Changed Question to more closely match K/A.

92S H

3 Y

Y S

B 2000 NRC Exam

ES-401 9

Form ES-401-9 (R8, SI)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD I/ H) j1-5) Stem Cues [TF Cred.

Jaroia i-inutiaI 1

Back-0= SRO UIEIS Explanation Focus j Dist.

Link units ward K/A Only I 93S F

2 Y

N S

N 94S F

2 Y

Y S

N 95S F

3 Y

Y S

N 96S F

2 Y

Y S

B 97S F

2 Y

Y S

N 98S F

2 Y

N S

B 99S F

2 Y

N S

N 100S H

2 Y

Y S

N