ML063060191

NRC Comments on Draft Licensee - Developed Exam (Written & Operating Tests) (Folder 2)

ML063060191
Person / Time
Site:	Oyster Creek
Issue date:	08/14/2006
From:	D'Antonio J Operations Branch I
To:	AmerGen Energy Co
Sykes, Marvin D.
References
Download: ML063060191 (33)

Text

NRC Written Exam Comments - SRO Exam 8/14/06 E = Enhancement G = General U = Unsat Question # Comment Resolution Type 1 Did not meet CFR 43.5 Rewrote the question to meet U CFR J Answer selection c was not Handout deleted. Rewrote the U credible; also wanted to question stem and several delete the handout answer selections.

Wanted a bullet in the Deleted the appropriate bullet E question stem deleted and reworded the remaining part of the stem Wanted TS handout deleted Deleted the TS handout and E reworded the question stem 9 Wanted the TS handout Deleted the TS handout E deleted 11 Wanted handouts deleted Deleted handouts; Modified E one answer choice (d) 15 They thought this was a K/A No changes G mismatch; After discussing it, they realized that it was OK as written.

25 They wanted handouts We dont believe that the  ?

deleted candidates have the common knowledge without the handouts; Well discuss this further. We may provide a larger handout.

Total Unset: 2/25 = 8%

NRC Written Exam Comments - RO Exam 8/15/06 E = Enhancement G = General U = Unsat Question Comment Resolution Type Complete 1 Check to see if the given The plant condition was not in a E J plant conditions put the rod block condition. But power plant into a rod block was reduced from 45% to 42% to condition. get just a bit further from the rod block line.

3 Wanted another comment Comment added to the question. E J added to state that vacuum Comment added to the question. E J that the cooldown rate remains the same.

5 Wanted the reference handout deleted.

Handout was deleted.

J J

6 Wanted comment added Comment added to the question. E that the design basis release occurred.

7 They had some concern No changes. G J that the Candidates could not understand direct/indirect 8 They wanted us to verify The associated lesson plan has an G J that answer selections c objective to state the PC and d were expected temperature and pressure limits.

knowledge for ROs.

12 Wanted comment added Comment added to the question. E J that DW pressure given was the peak DW pressure. ~

16 K/A Mismatch. Generated new question. J 18 They commented that the Added new reference which  %- J RAP reference provided one correct and one showed correct answer. Y incorrect answer. Wanted another reference to show the correct answer.

20 Wanted the word Comment added to the question. E J 1 additional added to stem.

22 They thought that answer Created new answer selection d. Id J selection d was not

plausible.

23 Wanted comment added no Comment added to the question. E J operator action.

25 Wanted a small correction Justification corrected. E J in the justification.

26 They felt that the reference Wrote new question. U J could point to another correct answer than originally thought.

29 They felt that both answers Modified the stem and answers. U J b and c were correct.

31 Wanted comment added no Comment added to the question. E J operator action.

33 They wanted to see a na G J reference that was not urovided them.

34 Wanted not bolded. Bolded the not. E J 39 Wanted comment added Added comment that DW pressure E J that DW pressure peaked at had risen to 2.2 psig and steady.

about 2 psig (LT 3) 41 Wanted comment added no Comment added to the question. E J operator action.

44 WA mismatch. New question written. J 46 Wanted comment added no Comment added to the question. J operator action.

47 Wanted reactor power Added that reactor power was 8%. E J value included.

51 Wanted typo corrected. Corrected two. E J 53 Wanted us to verify the Correct answer was verified. @U J correct answer. 190 Lc~flai 54 They felt that answer Modified the question stem and J selections c and d were answers.

direct lookup.

58 They felt the question was Wrote a new question. U J too easv.

59 They wanted to know if No changes. Let him understand G J there were any failures that 2 EMRVs open at one set which effected just 2 pressure, and 3 open at another EMRVs. uressure set uoint. OK.

61 They felt that answer Wrote new answer selection. J selection b was not plausible.

64 They felt that the expected Wrote new question. J actions for the correct answer did not match the referenced ABN.

A b 66 They asked if the CC Flow Yes. All CCW RRP alarms are to LOW alarm for RRP A individual pumps.

applied to only that RRP.

68 They felt that answer Created new answer selection, selection a was not credible.

They felt that answer Made new answer selection.

selection b was not Total Unsat: 11/75 = 14.7%

2 ES-401 Written Examination Review Worksheet Form ES-401-9 Instructions

[Refer to Section D of ES-401 and Appendix 6 for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamentalor (H)igher cognitive level.

2. - -

Enter the level of difficulty (LOD) of each question using a 1 5 (easy - difficult) rating scale (questions in the 2 4 range are acceptable).

3. Check the appropriate box if a psychometricflaw is identified:

• The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (Le., clues, specific determiners, phrasing, length, etc).

. The answer choices are a collection of unrelated true/false statements.

- The One or more distractors are is not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e& if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified The question is not linked to the job requirements (Le., the question has a valid WA but, as written, is not operational in content).

The question requires the recall of knowledge that IS too specific for the closed reference test mode (Le., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

. The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved WA and those that are desianated SRO-only (WA and license level mismatches are unacceptable).

f. Based on the reviewer'sjudgment, is the question as written (U)nacceptable(requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

I 7. At a minimum, explain any "U" ratings (e.@,how the Appendix 8 psychometric attributes are not being met).

orm FS-401 -9

? 4 G W JZ4, E

3 5 Q 5 3 X Control room evac KA; evac has no bearing on the answer to this question.

5 5 7 5 3- +?- D 40-

7. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

t Form FS-401-9

2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

LOD Explanation Stem Cues T/F Cred. Partial Job- Minutia #! Back- Q= SRO Focus Dist. Link units ward WA Only 5

5 5

5 no 5 could make a JPM out of this question 7

h I

6 Handout necessary? Just need to know EMRV opens due to fire, trip plant. Also, do plant conditions absent fire require trip? Also second ailed EMRV question. 7 4L?5 ow,/ ofL /UZ*GL~

[Refer to Section D of ES-401 and Appendix B for additional information regardingeach of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamentalor (H)igher cognitive level.

2. -

Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 4 range are acceptable).

3. the appropriate box if a psychometric flaw is identified:

.Check The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

. The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelatedtrue/false statements.

The One or more distractors are is not credible: single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (Le.,the question has a valid WA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory),

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved WA and those that are desianated SRO-only (WA and license level mismatches are unacceptable).

f. Based on the reviewer's judgment, is the question as written (U)nacceptable(requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

I 7. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 Written Examination Review Worksheet OC 2006 Form ES-401-9 Notes:l. Bolded Question numbers have lad a detailed technical reviev , The remaining questions were 1 didated against attached references.

y the licensee to address exa liner comments 7.

1. / Back- Q= SRO U/US Explanation units ward WA On1 Initial conditions should include rod block if applicable Add Condenser vacuum continues to degrade Without this one might conclude the condenser is intact. This would make b also correct.

Need to add Assuming cooldown is continued at this rate until 1200.

Otherwise question is incomplete.

bemove TS handout Add to stem: A design basis release from the dropped fuel bundle occurs. This ensures actions in c will occur.

OK from validation verify 35# vs 48# is expected RO knowledge

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOO (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/US Explanation Focus Dist. Link units ward WA Only

- 2 6 L 2 X a instead of d appears to be correct IAW step 7.3.1.4 of procedure fd 341 p~ B,V 4,v, 27 L 2 Y N S 28 L 3 Y N S

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (FIH) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward WA Only 46 H 3 y N E Add Assuming no operator action...

47H 2 X N N E Need to add APRM above downscale or a power level 48 L 3 Y N S

ES-401 2 Form ES-401-9 I I I r IN I

I JE I

I Renumber a-d (out of sequence)

I I

I

( N S

( N S knowledge that fuses remove power not discriminatory b seems implausible with only basic, generic knowledge of what an f N E EDG is for. Suggest slow start, output breaker does NOT close V N S Y N S No correct answer per procedural direction in ABN-35. cwould only be done for a Service Air Pressure <80 psig (step 3.4). D is procedurally directed (step 3.5) for an Instrument Air Pressure i75 psig but would NOT fix the problem. Q wants operator to deviate from procedure.

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/WS Explanation Focus Dist. Link units ward WA Only S-11 H 2 Y Y S 5-12 H 2 Y Y S 5-13 H 2 Y Y S .

ES-401 2 Form ES-401-9 ES-401 Written Examination Review Worksheet OC 2006 Form ES-401-9 Notes:l . B o l d 4 question numbers have had a detailed technical review. The remaining questions were validated against attached references.

I

ES-401 2 Form ES-401-9 I. 2. 3. Psychometric Flaws I 4. Job Content Flaws I 5. Other I 6. I 7. II Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/WS Explanation Focus Dist. Link units ward WA Only 26 L 2 X Y N a appears to be ftkmorrect IAW step 7.3.1.4 of procedure 341 +fPe.1 27 L 2 Y N S II 28 L 3

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward WA Only 29 H 2 X Two correct answers (band c) Per TS the act of going to IDLE is a required ACTION - 2 30 L 2 Y N S 31 H 3 y N E Add Assuming no operator action 32 L 3 Y N S 33 H 3 y N E Need procedure 645.6.031 to complete Technical review 34 L 2 y N E ...is NOT read...

35 H 2 Y N S 36 L 3 Y N S 37 L 3 Y N S 38 H 2 Y N S 39 H 2 y N E Add drywell pressure has peaked at 2.0 psig This will ensure a is wrong 40 H 3 Y N S 41 H 3 y N E Add Assuming no operator action... before What is the status ...

42 H 2 Y N S 43 H 3 Y N S- PSP is not related to drywell pressure (in WA). It is a function of torus 44H 2 X pressure.

J 45 H 2 Y N S

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5.Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/WS Explanation Focus Dist. Link units ward WA Only 46 H 3 y N E Add Assuming no operator action...

47H 2 X Need to add APRM above downscale to stem to satisfy WA.

48 L 3 Y N S

ES-401 2 Form ES-401-9

3. Psychometric Flaws I 4. Job Content Flaws I 5. Other I 6. 7.

Explanation Also need to add two bullets:

X EMG-3200.02 is entered Torus water level is being reduced Renumber a-d (out of sequence)

No correct answer. #2 only requires opening V-11-36, not (also) V-11-34 X

? Need to confirm the term reactor isolation will always describe a condition that will result in MSlV closure.

Verbally told yesby exam develwer Justification for Ddoes not address manual operation. This is NOT ir caution statement.

b seems implausiblewith only basic, generic knowledge of what an EDG is for. Suggest slow start, output breaker does NOT close

4. Job Content Flaws 5. Other 6. 7.

Job- Minutia #/ Back- Q= SRO U/WS Explanation Link I I I I I units ward WA Only I No correct answer per procedural direction in ABN-35. "C" would only be done for a Service Air Pressure e80 psig (step 3.4). " D is procedurally directed (step 3.5) for an InstrumentAir Pressure e 75 psig but would NOT fix the problem.

~~ ~~ ~

Y N S y N E Must confirm that only Recirc Pumps have "CCW FLOW L O alarms I I I I I I I lY IN IS I I I I I I y N E "a" does not appear sufficiently credible. Recommend "is past indexer"

ES-401 2 Form ES-401-9 T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/US Explanation Dist. Link units ward WA Only Y N S Y N S I I I I I I I I I I IY IY Is Y Y S N Y S

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO UIWS Explanation Focus Dist. Link units ward WA Only -

S-11 H 2 Y Y S 5-12 H 2 Y Y S 5-13 H 2 Y Y S .

I m

Oyster Creek Generating Station NRC Exam NRC 2006-1 ChanPes from the Original Exam Submission

1. Simulator JPMs

a. NRCSiml 0 Deleted Step 2 (obtain a copy of the procedure) and re-numbered remaining steps 0 Annotated CUEs from the floor evaluator as FLOOR CUE 0 Enhanced the simulator set-up section for clarity

b. NRCSim2 0 Deleted Step 2 (obtain a copy of the procedure) and re-numbered remaining steps 0 Annotated CUEs from the floor evaluator as FLOOR CUE and BOOTH CUE as those cues from the simulator booth operator 0 Enhanced the simulator set-up section for clarity 0 Added light indications on pump start on step 6

c. NRC Sim3 0 Correct typographical error in the set-up 0 Deleted Step 2 (obtain a copy of the procedure) and re-numbered remaining steps

d. NRC Sim4 0 Changed initial condition cue from a small break LOCA resulted in a high Drywell pressure condition to the reactor was manually scrammed 0 Deleted initial condition cue that said there was a hydraulic lock on the scram discharge volume 0 Deleted in the initiating cue which specific section of Support Procedure 21 to perform (section 5.0) 0 Inserted the hydraulic ATWS malfunction in the simulator set-up section and added to initiate ARI 0 Deleted Step 2 (obtain a copy of the procedure) and inserted step for Candidate to diagnose a hydraulic lock on the scram discharge volume 0 Annotated CUEs from the floor evaluator as FLOOR CUE and BOOTH CUE as those cues from the simulator booth operator

e. NRC Sim.5 0 Deleted Step 2 (obtain a copy of the procedure) and re-numbered remaining steps Page 1 of 6

Oyster Creek Generating Station NRC Exam NRC 2006-1

f. NRC Sim6 0 Deleted Step 2 (obtain a copy of the procedure) and re-numbered remaining steps 0 Corrected annunciator (to match that used in the validation) in the simulator set-up section and deleted remote operator action to trip the EDG locally (this part was erroneously left in) 0 Annotated CUEs from the floor evaluator as FLOOR CUE and BOOTH CUE as those cues from the simulator booth operator 0 Added a NOTE which says that procedure step 6.8 is only a notekaution.

Tried to add CUE that the SM wants the EDGl loaded to a specific value so that the Candidate must adjudt load; but, it works better with a FLOOR CUE that EDGl load is 2850 KW and after a small adjustment from the Candidate, the FLOOR CUE is that load is 2750 KW.

g. NRC Sim7 0 Deleted Step 2 (obtain a copy of the procedure) and re-numbered remaining steps 0 Added the required valve malfunction to within the body of the JPM.

0 Enhanced the simulator set-up section.

h. NRC Sim8 Deleted Step 2 (obtain a copy of the procedure) and re-numbered remaining steps 0 Annotated CUEs from the floor evaluator as FLOOR CUE and BOOTH CUE as those cues from the simulator booth operator Modified the FLOOR CUE in step 6 to say that another Operator has restarted the RB normal ventilation system, instead of will restart the ventilation system.

2. Plant JPMs

a. NRCPlantl Deleted Step 2 (obtain a copy of the procedure) and re-numbered remaining steps Corrected wording in the standard of the final step.

b. NRCPlant2 Deleted Step 2 (obtain a copy of the procedure) and re-numbered remaining steps Added plant component labels for those components that are labeled different than used in the procedure (see steps 4 , 5 and 7)

Added the standard that the Candidate simulate changing charger output voltage (see step 6)

Corrected step 6 to refer to the correct charger (C2) instead of charger C1 Page 2 of 6

Oyster Creek Generating Station NRC Exam NRC 2006-1

c. NRC Plant3 Deleted Step 2 (obtain a copy of the procedure) and re-numbered remaining steps

3. Admin JPMs

a. NRCAdminR01 Deleted Step 2 (obtain a copy of the procedure) and re-numbered remaining steps Deleted several conditions and replaced with all prerequisites to perform this procedure have been met.

Since this JPM is performed in the simulator, annotated CUEs from the floor evaluator as FLOOR CUE and BOOTH CUE as those cues from the simulator booth operator Changed the critical tasks in the following manner: considered a critical task as any step which required a reading be noted or calculation performed, instead of just the final outcome of the procedure as a single critical task Included procedure step 5.3 as a performance step, which had been cued as being performed by another Operator

b. NRCAdminR02 Deleted Step 2 (obtain a copy of the procedure) and re-numbered remaining steps, and placed Step 2 CUE into Step 1 Since this JPM is performed in the simulator, annotated CUEs from the floor evaluator as FLOOR CUE (and FLOOR NOTES) and BOOTH CUE as those cues from the simulator booth operator The NRC wanted to change this JPM with the following: Leave LPRM 20-25B BYPASSED into the APRM 7, but do NOT bypass this LPRM in the alarm circuit. This would prevent the LPRM 20-25B white light from being illuminated on the LPRM-APRM Auxiliaries drawer. This was expected to be noted by the Candidate in the performance of this JPM.

Unfortunately, the procedure does not require the Candidate to interface with the LPRM-APRM Auxiliaries drawer. Therefore, this will not be recognized by the Candidate, as expected. Instead, this JPM has been changed to include a third LPRM BYPASSED in the APRM drawer, but it will lack the proper documentation in procedure 403 Attachment 403-2.

This issue will be seen and noted by the Candidate, according to the procedure 202.1.

c. NRCAdminR03 The NRC wanted to change this JPM with the following: Include a proceduralized tag request, as the Candidate would actually see in the Plant. Attachment 3, Clearance Requester Checklist, of procedure OP-MA-109-101, Clearance and Tagging, has been added to the original handout to enhance realism.

Page 3 of 6

Oyster Creek Generating Station NRC Exam NRC 2006-1 Enhanced the Required Materials section to include a completed Attachment 3 of procedure OP-MA-109-101.

Added a note in the Initiating Cue which stipulates that this clearance will not be considered an Exceptional Clearance.

Modified the tagout request to only include the mechanical and electrical boundaries.

d. NRCAdminR04 0 The NRC wanted to see the Candidate calculate their total dose after having performed the simulated task within the JPM. The Initiating Cue has been changed to reflect this request:

Calculate your total dose you would have if you complete the Action Plan as written (consider only the dose received in the SDC Room).

Determine if you can actually complete the Action Plan. If you cannot complete the Action Plan, state what administrative requirements must be completed first.

This additional step to calculate dose, includes a small +/- in the allowance for the correct answer for total dose.

e. NRC Admin SROl 0 The procedure reference has been deleted from the Initiating Cue.

Added several items in the initial conditions:

0 DW pressure is 2.9 psig CHRRMS indicates 1000 R/hr No effluent radiation monitors are in alarm RPV water level is unknown. The original initiating condition stated that RPV Flooding EOP had been entered. This has been deleted.

f. NRC Admin SR02 This P M was new and replaced an original JPM sent as the submitted material. This JPM was reviewed on the NRC Prep Week.

Deleted Step 2 (obtain a copy of the procedure) and re-numbered remaining steps

g. NRC Admin SR03 0 Revised the Task Cue to: Review the completed procedure and determine if the Acceptance Criteria are met, and make any comments.

h. NRC Admin SR04 Revised to have the Candidate complete Section 1 of the Dose Control Level Extension Form (in RP-AA-203). The Initiating Cue has been changed to reflect this, as has the body of the JPM.

Page 4 of 6

Oyster Creek Generating Station NRC Exam NRC 2006-1

i. NRCAdminSR05 Added a second error in the calculated heat balance.

Deleted unnecessary information in the initial conditions.

4. Simulator Scenarios

a. NRC1 Added in Event 1 Description that Continuous Withdrawal is allowed.

Deleted the original Event 3 (SRO TS action) and replaced with a new Event 3 (SRO TS action). This also required deletion of a procedure cue related to the original Event 3, in the Initial Conditions Added NLO Role Play on Event 4 Added control rod drift to Event 5 Deleted Event 6 and re-numbered the remaining Events appropriately Added Booth actions to close the CRD charging water valve V-15-52 Added Emergency Plan Classification on the last Event

b. NRC2 Added ROLE PLAY to Event 2 for Operations Management Changed the CUE in Event 3 and required changes in the BOP actions for the changed CUE Made minor enhancements in Event 4 Added TS reference due to increased DW unidentified leakage in Event 5 and reference to ABN-59 Corrected error in Event 6 Modified Event 819 Terminating Cue Reduced steam leak rate in Event 8/9 Added Emergency Plan Classification on the last Event

c. NRC3 Enhanced Role Play actions during air compressor swap in Event 1 Modified the Scenario Summary for Event 3 (scrammed rod which stuck at about 04 and loss of a single RPS power supply). Added to Event 3 that the Crew may attempt to manually insert the scrammed control rod which did not fully insert, Procedure guidance on stuck rods in this condition in ABN-6 is not directive. Simulator Operator CUE has been added to deletd the stuck rod malfunction if the RO attempts to manually insert the stucl control rod.

Added a NOTE to place a CUE on the CRD temperature recorder which says that control rod 02-19 indicates 260" F and steady. (Event 4)

Added in Event 5 that the SRO may direct RPV pressure be changed (increased) after the down power, with the EPR.

Event 6 (Seismic Event) has been deleted, and the remaining Events re-numbered.

Modified the Initiating Cue in Event 8 to the ARM annunciator (1OFlf)

Page 5 of 6

Oyster Creek Generating Station NRC Exam NRC 2006-1 Added Emergency Depressurization anticipated actions Deleted Role Play of NLO contacting the Control Room of visible steam in the CU area (Event 7)

Added to initiate the Cleanup when the reactor is manually scrammed Added Emergency Plan Classification on the last Event Page 6 of 6