ML051320271

Draft - Nrc/Facility Comments (Folder 2)

ML051320271
Person / Time
Site:	Beaver Valley
Issue date:	02/08/2005
From:	Hynes C FirstEnergy Nuclear Operating Co
To:	Conte R NRC/RGN-I/DRS/OSB
Conte R
References
50-334/05-301, 50-412/05-301, ES-401, ES-401-9 50-334/05-301, 50-412/05-301
Download: ML051320271 (21)

Text

ES-401 Beaver Valley Unit 1 Written Examination February 2005 Form ES-401-9 Written Examination Review Worksheet NOTE : 7. BOLD & Italics reflect resolution of comments

2. Reviewed by Don Jackson I 1 12.1 3. Psychometric Flaws I 4. Job Content Flaws I 5. Other I 6. I 7.

LOD (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO u/E/s Explanation Focus Dist. Link units ward WA Only 2 X E Modified Q- Need Changes or Parent, Add More To "Constant Speed" 2 1 H 2 S H 2 S Check Valve Line Up At Facility H 2 X U Stem States No Other Failures, D Only Answer Indicating No Failure/

- Also How Does "C"Pump Auto Start If Already Running?

H 2 S

-H 2 S Modified Q- Need Changes or Parent F 2 S Use "Remotely Open" vs. "Manually Open", Remove Brackets Around

- N-1 -RC-519 H 2 X E "B"Not Plausible- Internal System Leak H 3 X E "C" and "D" Are Not Plausible- Times Too Long 7

H 2 S 11 1 X U Rule Out CIB Since Ctmt Pressure Not Mentioned , A Must Be Correct.

Too Simple, SI in All 4, and CIB In 3 2 S Explaination"B" Should Say CIA 13 2 S

~ ~ ~ ~~ ~

14 H 2 S 15 H 2 S 16 H 2 S 17 F 2 S Modified Q- Need Changes or Parent, Answer Doesn't Match Explaination

6. 7.

JIWS Explanation E How Are S/Gs At 20# If RCS At 165 Deg F U PORVs Credited or Not Credited, Number Irrelevant E Credited As Higher Cog, Actually Memory Level, Only YZ of WA Addressed S Credited As Higher Cog, Actually Memory Level, Not Discriminatory At The SRO Level S Need References S Credited As Higher Cog, Actually Memory Level U Not DiscriminatoryAt The SRO Level

qefer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. - -

Enter the level of difficulty (LOD) of each question using a 1 5 (easy difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometricflaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (Le., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated trudfalse statements.

One of more +lymnedistractors is not credible.

One or more distractors IS(are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (Le., the question has a valid WA but, as written, is not operational in content).

The question requires the recall of knowledge that IS too specific for the closed reference test mode (Le., it IS not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or applicationcompared to the job requirements.

5. Check questions that are samDled for conformance with the approved WA and those that are desianated SRO-only (WA and license level mismatches are unacceptable).

6. Based on the reviewer's judgment, is the question as written (U)nacceptable(requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

7. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

NUREG-1021, Revision 9

b/ j@ h~~-~\d*

$8 tnl 7 tt,tr: ill';

j)W %4lw*r'c (,z%:

ES-401 Beaver Valley Unit 1 Written Examination February 2005 Form ES-401-9 Written Examination Review Worksheet NOTE : 1. BOLD & Italics reflect resolution of comments

2. Reviewed by Don Jackson I 1. I 2. I 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

I I I I 1 ~ I 2. I 3.Psychometric Flaws I 4. Job Content Flaws I 5. Other I 6. 7.

Explanation Add "Directly Auto Trip" To Stem- "A" and "D" May Lead To Auto Trip Direct Answer- "What Color Is George Washington's White Horse!

Ensure "A" Is Not Correct- Can Residual Magnetism Self Excite Field?

~ ~~

Second Bullet Missing EDG #

t Rework To Show %ffect On System Not Valves Double JeoDardv With Number 11 Wrong WA Statement On Form, and Answer Proposed Is Wrong, "C" Is Correct "D" Not Plausible If ISFSI Under Discussion or Construction, CAF .

WA Resolved In Stem and Is Not Tested "A" Not Plausible, Why Would It Open?, and If S/G Drain Tank Pump Trips On High Pressure Then "B","C","D" Correct N/fi

'B" and "C" Not Plausible As They Obviously Lower Flow. "A" and "D" Are Also Correct, LOD=l Is This P&L Required RO Knowledge?

If Pzr Level Dropped To 10% SI Would Actuate, No ES-0.1, and No Letdown Restoration, Change Pzr Level.

c c

!??

m n

b ln a

C 0, 0 c c (II c

E m

0 B z W 2 u

l3 5

2 v) v) v) W I f

cn

w 3 v) v) i X

I I I I I I I I

Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. -

Enter the level of difficulty (LOD) of each question using a 1 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (Le., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated trudfalse statements.

One or more -istractors is not credible.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (Le.,the question has a valid WA but, as written, is not operational in content).

The question requires the recall of knowled e that is too specific for the closed reference test mode (Le., it is not required to be known from memory).

The questlon contains data with an unrealisyic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved WA and those that are desianated SRO-only (WA and license level mismatches are unacceptable).

6. Based on the reviewer's judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

17. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

NUREG-1021, Revision 9

BV1 FEB 2005 OPERATING EXAM REVIEW COMMENTS Examiners: S. Barr, D. Jackson, P.Presby General Comment:

Please make all applicant hand out material a different color paper including all cue sheets and reference materials.

General JPM Comment Ensure that the JPM cover sheet and the ES-301-1 & 2 WAS are correctly cross-referenced.

General Scenario Comments There is NO lower power scenario. Low power is defined ES-301, D.5.c. (footnote 2

+&%

, thereto) as criticality to 5 percent power). Given that we have four scenarios here and on the other unit, we should demand at least one low power scenario.

.e(. p 61 Will the crews have STAs? Surrogates? Is the STA continuously stationed in the Control Room or should we delay the STA and require a brief when dhe does report to the Control Room?

Admin. JPMS RO.A.l Task standard is that cooldown RATE is calculated and verified NOT to be within acceptable limits. However, cooldown rate is within limits. The problem to be determined by the applicant is that plant is operating in unacceptable region of pressure-temperature curve. Recommend task standard state that applicant determines not operating within acceptable pressure-temperature limits.

w71 /yca;k J7m tp 2.

F*F

/q/z R0.A.2 / SR0.A.2 Should we change the title to Perform an Estimate Critical Boron Concentration from Perform an Estimated Critical Position?

Title change not needed. Applicant is implementing the procedure titled petform ECP.

the signature block names provided in handouts in the JPMs in 1OM-50.4.F pg 13 of 29, SRO.A.2- U.R.Fine and I.C.Clearly in May become public record and, as such, should be more formal, without J

ak4// dL7d 1

& h / #$&Z &c- p & 7 ak&hL1 6 % IC A m~ .P a g e l o f 7 -5 e

BV1 FEB 2005 OPERATING EXAM REVIEW COMMENTS Any difference between these JPMs? None that I can tell. The RO title is PERFORM. The SRO title is REVIEW. I suspect thats what you wanted but the initiating cue for both is to REVIEW. The handouts for both support REVIEW because they are both filled out.

RO JPM provides current boron conc, SRO does not. Shouldnt both applicants get the same initial condition information?

Provide more detail for the answer:

i o What is the ECP or ECB? The ES-C-1 form or the Data Sheef 1 Answer Sheet should list the required minimum and maximum critical boron conc values (Example, for RO, 1834 5 CriticalBoron5 1844) 0 What is the minimum and maximum acceptable value?

$ o May want to specify tolerances around each subsidiary value just in case . . .

do What errors is the Applicant expected to identify?

R0.A.2 There is no Data Sheet 2. Should there be? &6 ~eeJd -

Where does 1 10 on CBD come from? Step IV.A.2.e. indicates 100.

Part B, Line 1 - Verified that Data Sheet value of 1720 is correct.

Part B, Line 2 key shows 875, with 675 in grey below. Assuming 875 is one of the errors to be located by the applicant due to use incorrect of 245 vice 24A, shouldnt the greyed correct value be 625 at a bank position of 1 10 steps? 675 is the correct value for a BOL ECP of 100 steps, not 1 10 steps. Fix the associated carry through error to the key korrect values.

Part C, Column II, Line 1 shows -7.2 pcm/ppm for DBW. However, using 1300 m (Part A Column V) and 1000 MWD/MTU Curve on CB20 yields a DBW of&m/ppm.

Error carries forward. Part C, Column 111, Line 1 should be +4 18 ppm. -7#1 Part C, Column IV, Line 1 shows 1337ppm. However, correct number from Part A, C Column V should be 1300 ppm.

Part C, Column V, Line 1 shows 1772 ppm. Should be 1718 ppm.

Part C, Column 11, Line 2 should be -7.18 pcm/ppm.

Part C, Column Ill, Line 2 should be +461 ppm.

Part C, Column IV, Line 2 should be 1300 ppm.

Part C, Column VI Line 2 should be 1761 ppm.

Part C, Columns VI thru VI11 are not completed. Applicant should fill these in to determine --C5C.

Lists task importance rating as 3.9. The SRO rating for 2.1.23 is 4.0.

Part C, Column 11, Line 1 should be -

8 pcm/ppm. Error carries forward.

Page 2 of 7

Lp BV1 FEB 2005 OPERATING EXAM REVIEW COMMENTS

&$fpzC, Column 11, Line 2 should be -7.18 pcm/ppm. Error carries forward.

o Part C, Column IV, Lines 1 and 2 should be 1300 ppm. Error carries forward.

R0.A.3 I SR0.A.3 Applicant is to review a tagging request. Should the Prepared By block be filled in?

Alternatively, make it one of the errors the Applicant is to find.

UA 1 I. ES-C- 1 form lists importance factor as 3.8. Should be 3.6.

• ch e !

b4.n .fip Both. JPM Step 1 states that OS-16 is not a correct DISCHARGE isolation point. Should 0 say this is not a correct SUCTION isolation point.

b\lI f l SR0.A.3. Only apparent difference is the addition of the breaker tagged ON instead of

& p%@OFF. Can we I should we throw in an administrative process error? Perhaps no second

$bC& review or Preparer and Reviewer (or Second Reviewer) are the same person. This may I require changing the cue from REVIEW to APPROVE.

e Both. Is it possible / reasonable to expect the Applicant to request electrical prints? --& 3755 SR0.A.3. Is there a TS issue or Risk Mgmt issue the SRO should or may address?

, R0.A.4 0

\\ k?? The RWPs should be arranged numerically to preclude a potential cue.

Will the current TEDE of 150 mR plus the planned 100 mR exceed any administrative limits?

ES-301- 1 sample plan calls for 2.3.2, Knowledge of ALA RA. However, RO.A.4 is tied to L

2.3.70, Ability to perform procedures to reduce excessive levels of radiation and guard against personnel exposure.

What basis does the applicant have for selecting RWP 1003 when you dont provide him/her 9 .&

.with the area survey map until after RWP selected? Would need to know general area dose p rate of room before you could select the RWP.

Seems too simple a JPM. Would expect any rad worker to be able to perform this task.

Not sufficiently discriminating for a license applicant. Recommend setting up task and survey map such that there are hot spots in room - but not in area of work - that would introduce error into the stay time calculation if inappropriately used. This would require the applicant to determine which dose rate to use instead of just choosing the highest reading.

bc- Recommend including a high surface contamination reading also (away from the area of work) to get applicant to show understanding of the different symbols on the map.

p SR0.A.4 The errors to be identified may be inconsequential with respect to the radiological implications. The missing signature in itself certainly is. The late start date may be - may even be a benefit because more time for the tank to decay before release. How about an above limit activity due to a miscalculation?

Page 3 of 7

BV1 FEB 2005 OPERATING EXAM REVIEW COMMENTS Should procedure 1/20M-19.4A.B be initialed through to completion?

SR0.A.5 Too easy. The equivalent of making an EAL determination. In this case, the Applicant is expected to recognize that the EAL can be terminated. Can we add some termination or reporting activities to beef it up?

Concept okay, but too much of a simple, direct lookup as written. Initial conditions state that RCS is depressurized. Should instead provide a list of plant parameters (temperature, pressure, pzr level, cntmt conditions, dose rates, etc) and let the applicant evaluate these conditions against all of the EAL tabs. Will then have to determine that current RCS i pressure meets the criteria *depressurized Simulator JPMs Gen:

None of the Alternate Paths have much Alternate meat. Just a single corrective step or implementation of the RNO column. None require the Applicant to determine what alternate path to follow.

Several JPMs listed as E (meaning emergency or abnormal in-plant), but are instead control room, not in-plant (JPMs S2,S3, S4 and S5). Total number of actual E JPMs (2 for RO/ISRO and 1 for USRO) still meet minimum criteria of 2 1).

No Lor low-power JPM for USRO. Minimum criteria is at least 1 for USRO.

s1:

is there enough of the alternate path to call it an alternate PATH?

any chance theyll get an automatic trip first? (SUR Trip - B&W, West??)

note preceding performance step 1 tells me to provide the Applicant with a copy of 1/M plot and the ECP. Shouldnt we also give them the procedure to use? The cover sheet does indicate that the procedure is to be provided.

AJkL Admin JPM A2 should be administered before this JPM to ensure ECP data from S1 does syi&fJ <e&

/,I not affect the integrity of A2.

s2: kc j \ L . 3 - - e;,,

t Should / could s t e p d a n d H be critical?

xip ut( - fi potentially not much PATH after the alternate route is taken - similar to comment to S1 Should the standard for performance step 24 be more specific?

WA reference not from one of the Safety Function sections of the WA catalog. To demonstrate or prove adequate safety function coverage, the referenced WA has to come

BV1 FEB 2005 OPERATING EXAM REVIEW COMMENTS from the associated sections (3.1 thru 3.9). Consider a WA from section 3.2

/f .r( Simulator setup for Unit 2 equivalent JPM includes guidance to the booth to increase leak size after applicant takes some action. Should the same guidance be included in the Unit 1 ve c i JPM?

L,?

/

J I- Must be tied to system on ES-301-2. Form shows S2 as E02 system and 5 3 as E03 system. JPMs need to be tied to specific systems of Sections 3.1 through 3.9 of the WA catalog.

WA reference not from one of the Safety Function sections of the WA catalog. However, this JPM applies to a Reactor Coolant System Inventory Control or a Reactor Pressure Control system (ECCS). Should be re-categorized on ES-301-2 form as a SF2 or SF3 JPM.

Looks like there are two JPMs tied to the same system (SZ and S3). Dont think ES-301 will allow this as it requires each of the RO/ISRO JPMs to evaluate a different safety function and same system shouldnt be used to evaluate more than one safety function (ES-30 1. D.4.a).

Step 12, include the common noun name for HCV-1MS-104 yfi fl LLL( C G L A .*A8- ~ C I P L ~ 06+~4fi;*;4 ii idlo dg<zdi. ,k .

• 6 h e alternate path begins at s t e t {. Theres nothing alternate about it - just a simple IF, . . .

THEN . . . step. Can we start them out at an earlier step?

A c.,-;

S6: { t w I ( l!: 4 cj :, w I M* a 4 r f , . t I auletlsUc &c I ort~

.c (h.4 Ir ketty straightforward - can we add a normal or emergency EDG shutdown?

s7:

U Again, straightforward - only 2 critical steps and theyre both easy - how about starting them with four operable PRNls. Then let them respond to the failed NI and go on to defeating the failed channel.

S8:

Why end it when flow is established? Its -300 gpm batch add at -90 gpm. Let the Applicant complete the add normally.

In-plant JPMs P1:

Poor WA match:

0 Al.01 - Ability to predict and/or monitor changes in parameter (to prevent exceeding 4kccx/A 1 design limits) associated with operating the HRPS controls including: Hydrogen concentration. There is no predicting or monitoring here.

M d i ..

Page 5 of 7

BV1 FEB 2005 OPERATING EXAM REVIEW COMMENTS

/

0 A4.01 - Ability to manually operate and/or monitor in the control room: HRPS controls. I dont think this is happening in the Control Room.

P2:

No comments

,h ~

5 Should Performance Step 6 be critical?

6 5 For Performance Step 7,include the site specific electrical safety requirements in the JPM AND make compliance or simulated compliance with those requirements a critical steD.

, r J-2 L J Does BV have a breaker that can be used for ACTUAL racking? Is it worth the effort?

w Cover sheet references system 055. One of the referenced WAS should come from Section

/

3.6 - perhaps 062.A2.05 (Ability to (a) predict the impacts of the following malfunctions or operatinos on the ac distribution system; and (b) based on those predictions, use I -

v ~ 1 1 procedures to correct, control, or mitigate the consequences of those malfunctions or operations: Methods for energizing a dead bus).

Scenario #1 Event 1:

/ Have the Applicant crew conduct their pre-job brief BEFORE arriving in the simulator to save time.

Event 6:

Scenario says to open RTB A & B when directed to do so. We need to ensure there is some delay if the RO or PO take the initiative and call for this immediately as the scenario unfolds.

Should the critical task #I have a time component, such as do the action prior to exiting a certain procedure or prior to equipment operator opening the TCBs locally?

Event 7:

Should the critical task #2 have a time component, such as do the action prior to exiting E-2?

On page 23, the crew checks and finds that PRT conditions are normal. But on Event 3 (PZR PT failure) the PORVs were opened. Any chance the crew will do something unexpected here?

Scenario #2 Event 1:

Page 6 of 7

BV1 FEB 2005 OPERATING EXAM REVIEW COMMENTS Have Applicant crew conduct their pre-job brief BEFORE arriving in the simulator to save time.

Potential redundancy. This reactivity manipulation is the same as Scenario 1's reactivity manipulation.

Scenario #3 Have Applicant crew conduct their pre-job brief BEFORE arriving in the simulator to save time.

Should the critical task #7 have a time component, such as do the action prior to exiting ES-O.l?

Scenario #4 Event 1:

Have Applicant crew conduct their pre-job brief BEFORE arriving in the simulator to save time.

Potential redundancy. This reactivity manipulation is the same as Scenario 1's reactivity manipulation.

Event 7:

Critical Task #1: How much time does this give Applicants to recognize & respond to the need to manually close MSIVs?

Page 7 of 7