ML021690502

Draft - NRC Comments

ML021690502
Person / Time
Site:	Peach Bottom, Limerick
Issue date:	05/24/2002
From:	Wasong A Exelon Nuclear
To:	Conte R NRC/RGN-I/DRS/OSB
References
50-277/02301, 50-278/02301, 50-352/02301, 50-353/02301
Download: ML021690502 (10)

Text

ES-401 PBAPS / LGS LSRO Written Examination Form ES-401-9 (R8, SI)

Review Worksheet

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.7.

Q#LOK LOD [1 (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= JSRO U/E/S Focus Dist. Explanation Link I Junits ward K/A Only 1-1 F 2 Y S B 1-2 H 4 Y S B 1-3 F 2 X Y U B Answer B could also be correct. The stem will be reworded to prevent 2 correct answers.

1-4 H 2 Y S N 1-5 F 2 Y S B 1-6 H 2 Y S B 1-7 F 2 Y SN 1-8 F 3 Y SB Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g.,

unclear intent, more information is needed, or too much needless The stem or distractors contain cues (i.e., clues, specific determiners, information).

phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

• More One orthan moreone distractor is not credible.

distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

• The question is not linked to the job requirements (i.e., the question The question requires the recall of knowledge that is too specific has a valid K/A but, as written, is not operational in content).

The question contains data with an unrealistic level of accuracy for the closed reference test mode (i.e., it is not required to be known from memory).

or inconsistent units (e.g., panel meter in percent with question The question requires reverse logic or application compared to the in gallons).

job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-onl, (K/A and license level mismatches are unacceptable).

6. Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

7. At a minimum, explain any U" ratings e.., how the Appendix B psychometric attributes are not being met).

I

ES-401 2 Form ES-401-9 (R8, SI)

Q# 1.

LOK 2.

LOD 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

(F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ 1Back- Q= SRO U/E/S Explanation Focus j Dist. Link units ward K/A Only 2-2 F 3 Y S N 2-5 H 3 Y S N 2-3 H 2 Y S N 2-12 H 3 Y S N 2-6 F 2 Y S N 2-8 H 2 Y S N 2-10 H 2 X Y U N Anti Siphon is on the FPC System and not RR.

2-13 H 3 Y S N 2-15 F 3 Y S N 3-2 H 3 Y S N 3-7 F 2 Y S N 3-15 H 2 Y S N 2-1 F 2 Y S N 2-4 H 2 Y S N 2-7 H 3 Y S N 2-9 F 2 Y S N 2-11 H 3 Y S N 2-14 F 3 Y S N 3-1 H 3 Y S N 3-3 H 3 Y I LS N

ES-401 2 Form ES-401-9 R8, $

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) te Cues TIF Ce ~Partial Job- Miui #/ Bac-0 S0UIS Explanation

_1 Focus j Dist. Link lunits ward K/A Only 1 3-4 F 2 Y S N 3-6 H 3 Y S B 3-11 H 3 Y S N 3-12 F 2 Y S N 3-13 H 3 Y S N 3-14 H 3 N U N A3. Ability to monitor automatic operations of the FUEL HANDLING EQUIPMENT A3.02 tlnterlock operation. The question deals with manual operation. The question should be changed to reflect automatic operation.

3-5 H 3 Y S N 3-16 F 4 Y S N 3-18 F 3 Y S N 3-17 H 3 Y S N 3-21 H 4 Y E N THE PROCEDURE IS FOR 20C222, THE QUESTION IS FOR 1OC222. This does not effect the answer.

3-8 F 2 Y E N last sentence in the stem change th to a.

3-10 F 2 Y S N 3-19 H 2 Y S N 3-20 H 3 Y S N 3-9 F 2 Y E N Last sentence in the stem change where a the to where the.

4-1 H 3 Y S N 4-2 F 3 Y S N 4-3 F 3 Y S N 4-4 F 2 Y S N 4-5 F 2 Y S N 4-6 F 2 N S N

March 20, 2002 MEMORANDUM TO: Exam File Exelon LSRO - Peach Bottom / Limerick Station FROM: Alan Blarney .. 12_,

Senior Operations Engineer

SUBJECT:

COMMENTS ON PEACH BOTTOM / LIMERICK STATION LIMITED SENIOR REACTOR OPERATOR EXAM OUTLINE.

The purpose of this memorandum is to document comments on the Peach Bottom / Limerick Station exam outline received on Monday, March 18, 2002. The exam outline was reviewed by A. Blarney, Senior Operations Engineer. The comments were provided to J. Stevens, Examination Developer, Exelon on March 20, 2002. The comments are as follows.

Modified ES-201-2, Examination Outline Quality Checklist Section 1. Written A -5/a

' b. The Exelon modified form ES-201-2, does not verify that the outline has been randomly and systematically selected. This is required to prevent inadvertent biasing of the exam.

This issue was discussed with J. Stevens and the following method was used to obtain a random outline, with slight bias toward the Fuel Handling Equipment.

Exelon has performed a job task analysis (JTA) for the LSRO position. This K/A base is used to randomly select K/As in the following manor. (This forms the basis for deselecting K/A in the BWR catalog - NUREG 1123).

1. The Exelon developed JTA - K/As are characterized into the four subject area listed in ES-701section D.1.

2. System and procedures are randomly selected within these categories.

3. Once the system or procedure(EOP or Abnormal) is selected the final K/A is randomly selected within the system or procedure.

S h/ c. The modified form ES-201-2, does not verify that the deselected topics were acceptable. This was performed by a JTA that was

performed for the LSRO position. Exelon JTA was NOT reviewed as part of this exam outline review.

Section 2. Integrated Plant Operation (IPO)

b. Must verify that:

that no scenarios will be duplicated from the applicants S(2) audit exam. This will be completed when the audit exam outline is completed. Exelon is aware of this condition and

.S~- will plans to prevent exam overlap.

The scenarios will not be repeated on successive days.

The schedule can not be finalized at this time, it will be finalized during the preparation week. (Use of the refueling bridge)

Section 3. Walk Through - Job Performance Measures (JPM)

TO-1507-- a. (3) No scenarios are duplicated from the applicants audit exam.

Since the audit exam has not been developed this must be verified prior to administering the exam. Exelon is aware that the walk through (JPMs) exams can not be a duplicated from the audit exam.

Section 4. General Ensure the K/A importance ratings are at least 2.5.

The facility walk-through exam dose not have K/A importance assigned to the tasks. This includes section A and B. (ES-301-2, Section D.1.c)v- V s.a.

Check for duplication and overlap among exam sections.

The outline appears acceptable; however, the NRC can not make a final d6termination until the exam is drafted and the individual questions can be reviewed.

M-C. Check the entire exam for balance of coverage.

The outline appears balanced; however, can not make a final determination until the exam is drafted and the individual questions can be reviewed.

Comment:

Peach Bottom Section A, JPM A.3, "Drywell Access Restriction During Fuel Handling," (page 9 of 15) is very similar to the first component failure in the Limerick scenario No. 1 (page 12 of 15).

This scenario requires inspection of the drywell upper seal plate and because of access to this area fuel moves must be stopped.

NRC Action: In the past we have issued a dual license for Limerick and Peach Bottom LSROs.

We plan to continue this process in Region I; however, Exelon has requested to change the LSRO exam. In the past each applicant was given a common written exam and a full operating test at each site. Exelon has requested to change this test methodology as listed below.

1. A 50 question written examination that covers both Peach Bottom and Limerick. No Change.

2. In the past a total of 10 administrative JPMs were performed. Five administrative JPM at Limerick and five administrative JPMs at Peach Bottom.

REQUESTED CHANGE Exelon requested a change to have a total of five administrative JPMs combined for Limerick and Peach Bottom.

BASIS: The administrative procedures are very similar between the sites. In addition, there is one administrative JPM that specifically looks at differences between the sites.

3. A total of 10 JPMs will be performed. Five JPM at Limerick and five JPMs at Peach Bottom. No Change.

4. A total of 4 IPOs will be tested. There will be 2 at Limerick and 2 at Peach Bottom. No Change. No Change.

April 26, 2002 MEMORANDUM TO: Exam File Exelon LSRO - Peach Bottom / Limerick Station FROM: Alan Blamey 0-- \* " 4/2 (-,/0oz Senior Operations Engineer

SUBJECT:

COMMENTS ON PEACH BOTTOM / LIMERICK STATION LIMITED SENIOR REACTOR OPERATOR EXAM.

The purpose of this memorandum is to document comments on the Peach Bottom / Limerick Station exam received on Tuesday, April 16, 2002. The exam was reviewed by A. Blamey, Examiner / Inspector and C. Sisco, Examiner / Inspector. The comments were provided to J.

Stevens, Examination Developer, Exelon on April 26, 2002. The comments are as follows.

Section 1. Written

!/42..i' "* Comments documented on ES-401-9, "Written Examination Review Worksheet."

Section 2. Integrated Plant Operation (IPO)

All scenarios do not denote critical tasks.

Section 3. Walk Through - Job Performance Measures (JPM)

£iCategqoA JPM A2, should use detector locations other than the "A" detector to ensure that the applicants know the actual location of the detectors.

-. j gh. 0 Questions A3-1 is a direct look up if allowed to use reference material.

Based on ES-301, Attachment 1, this question should be closed reference questions.

Category B Limerick

_____.________- JPM LG-1, "Movement of the Dummy Fuel Bundle in the Spent Fuel 5,. - I..J1=_ pt-4 ,-*PooI," contains procedure S97.1 .A, but for the purpose of this JPM the doYcXC 4PM. procedure should be completed so the candidate can perform the fuel movement.

5/14/o2 S JPM LG-2, "LPRM Removal," should have procedure M-C-774-010 S4d. completed to the step that the applicant will be required to perform.

-2 S5iI,-/.-* JPM LG-3, "Movement of Fuel From the Fuel Prep. Machine," should

_5nf. have the prerequisites of FH-106 signed off as per the initial condition of the JPM.

6 /4/o2 l JPM LG-5, "Defeat Unit 1 & 2 Refuel Floor HVAC High Radiation Isolation

-J,. Signals," should have the first step of the procedure completed, suspending fuel handling activities.

Peach Bottom 5

."40z* JPM PB-2, "Full Automatic Fuel Transfer Within the Spent Fuel Pool,"

. I_ should have a completed copy of procedure FH-35 to allow fuel movement.

,*JPM PB-3, "Control Blade Removal Using the Combined Grapple," does

.54. not have a core map that is required in JPM step 2.

f_/.,F- JPM PB-5, "Abnormal WRNM Response During Core Alterations," should S;4. have procedure FH-6C with the prerequisites completed.

Section 4 General Comments Punch List Items S&642-, Complete the following items during the week of May 12, 2002 during the prep.

week.

S Verify no overlap on the operating test with the Audit Exam.

SVerify that no significant overlap with the last NRC LSRO Exam.

r Verify that the above changes have been made.

2 Complete checklist signatures when verification complete.

May 21, 2002 MEMORANDUM TO: Exam File Exelon LSRO - Peach Bottom / Limerick Station FROM: Alan Blarney Senior Operations Engineer

SUBJECT:

VALIDATION COMMENTS ON PEACH BOTTOM / LIMERICK STATION LIMITED SENIOR REACTOR OPERATOR EXAM.

The purpose of this memorandum is to document comments / corrections on the Peach Bottom I Limerick LSRO exam that occurred during the exam validation week (May 13, 2002). The exam was validated by A. Blamey, Examiner / Inspector, J. Caruso, Examiner / Inspector, and J.

Stevens, Exelon Exam Developer. The comments were provided to J. Stevens, Examination Developer, during the review. The comments were incorporated and are listed below.

Section 1. Written None Section 2. Integrated Plant Operation (IPO)

Peach Bottom IPO 1 Event 1, step 1 should be a critical task.

IPO 2 Add the Peach Bottom EALs for the applicant Section 3. Walk Through - Job Performance Measures (JPM) -'T "n;m Category A None Category B Limerick JPM LG-1, More clearly specified the actions that are expected when the target location is obstructed and added more detail on revising the CCTAS.

JPM LG-3, Steps 19a, 19e are not critical tasks they are verify steps.

JPM LG-4, Added the alarm window description.

-2 Peach Bottom JPM PB-I, Removed the critical task from step 5. This is a verification step. Remove the aids that were used to perform the in-office review.

JPM PB-2, Step 5 & 27 are not critical steps, they are verifications only.

Delete step 21c because it is a duplicate verification.

JPM PB-3, Include a core map in the JPM for the candidate to identify the proper location.

JPM PB-4, Changed the initial location of the Tri-Nuc filter and added a critical step to determine if movement could occur while secondary containment was not operating.