ML101270325

2010-03 Draft OP Test Comments

ML101270325
Person / Time
Site:	Arkansas Nuclear
Issue date:	03/08/2010
From:	NRC Region 4
To:
References
50-313/10-03, 50-368/10-03
Download: ML101270325 (5)

Text

Attachment 10 Page 1 of 5 OBDI 202 - IOLE Process AN1 - 2010 - 03 DRAFT OPERATING TEST COMMENTS ADMIN JPMS JPM#

1.

Dyn (D/S)

2.

LOD (1-5)

3. Attributes

4. Job Content Errors

5.

U/E/S

6.

Explanation (See below for instructions)

IC Focus Cues Critical Steps Scope (N/B)

Over-lap Job-Link Minutia RO (A1)

S 4

N U

E S

JPM needs more than two steps (both critical) such as 1. examinee obtains ARP, 2. determines valid alarm for RIL using COLA Fig-4-A, 3. Use chart xx to determine yy, 4 Use LCO to determine zzz, etc.

Exam key is not adequate. Where is the mark up for drawing (X marks the spot), TS, etc?

Made changes as requested. Still needs a key for RCP map vs rods. Several steps not included as requested.

Modified JPM received 2/23/10 is Sat.

RO (A2)

S 3

X B

E E

S Too many critical steps, not all of these are critical steps.

JPM could not be validated correctly by several licensed operators, Licensee recommended a boundary/ tag out JPM for this one and NRC team agreed. Key (P and ID) needs more clearly marked valves and that X is closed and circle is opened.

Modified JPM received 2/23/10 is Sat.

RO (A3)

S 3

X B

E S

Need to clarify that 4 of 6 errors should include one or two that are critical to identify correctly: pump discharge pressure and vibration out of range are critical steps to make the OD call in order to pass the JPM. If they get these two right and the pump Inop call is critical, all others are not. This has a good mark up key for grading and standard purposes. In content of JPM standard, each item that is an error in package should also be a step to ID in the JPM standard.

Licensee agreed that the two mentioned errors are the two crit errors, all others non-critical. Received update with these changes on 2/23/10.

RO (A4)

S 2

X X

E E

S Is this the way your actual maps look like for RWPs now? Seems very easy and direct look up, which is not allowed.

ANO stated that they could not write a good one for this topic so chief examiner wrote one and ANO accepted it. It is no longer a direct look-up, however ANO changed the numbers to make it more easy.

The concept of the two numbers being close is lost if one stay time is on hours and the other in minutes. Licensee must change this back to 1950mR TEDE, 105mR/hr field, and dose rate for RWP is set to 20 mR/hr, with approx stay time of 12 minutes vs 30 minutes based on 2000 mR annual TEDE.

Modified JPM as requested. Received 2/23/10 is Sat.

SRO (A1)

S 2

N E

E S

Need better clarification for steps in JPM standard.

Still needs more steps in JPM std since more items are performed than the JPM standard dictates as indicated by the key and discussions during prep week. These items must be included in the JPM std, however these items are not to be marked critical.

Modified JPM as requested. Received 2/23/10 is Sat.

SRO (A2)

S 2

N E

E Reword steps 1 and 2 for clarity.

Still needs more steps in JPM std since more items are performed than the JPM standard dictates as indicated by the key and discussions during prep week. These items must be included in the JPM std, however these items are not to be marked critical.

0 Page 2 of 5 OBDI 202 - IOLE Process AN1 - 2010 - 03 DRAFT OPERATING TEST COMMENTS ADMIN JPMS JPM#

1.

Dyn (D/S)

2.

LOD (1-5)

3. Attributes

4. Job Content Errors

5.

U/E/S

6.

Explanation (See below for instructions)

IC Focus Cues Critical Steps Scope (N/B)

Over-lap Job-Link Minutia S

Modified JPM as requested. Received 2/23/10 is Sat.

SRO (A3)

S 3

X B

E S

Same as RO A3 with exception of formal declaration of inop of pump and completion of section 4.

Modified JPM as requested. Received 2/24/10 is Sat.

SRO (A4)

S 3

X E

S Same as RO A4. See its comments.

Modified JPM as requested. Received 2/23/10 is Sat.

SRO (A5)

D 2

X E

U S

Step 3 is marked as step 2, step 3 and 4 should be critical. Remaining steps not critical for this JPM.

Change cue sheet to remove notification as part of standard.

No steps are marked as critical now. What happened to JPM?

JPM Received 2/24/10 is Sat.

Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1.

Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.

2.

Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.

3.

Check the appropriate box when an attribute weakness is identified:

The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.

The JPM does not contain sufficient cues that are objective (not leading).

All critical steps (elements) have not been properly identified.

Scope of the task is either too narrow (N) or too broad (B).

Excessive overlap with other part of operating test or written examination.

4.

Check the appropriate box when a job content error is identified:

Topics not linked to job content (e.g., disguised task, not required in real job).

Task is trivial and without safety significance.

5.

Based on the reviewer=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

6.

Provide a brief description of any U or E rating in the explanation column.

7.

Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form.

0 Page 3 of 5 OBDI 202 - IOLE Process AN1 - 2010 - 03 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS JPM#

1.

Dyn (D/S)

2.

LOD (1-5)

3. Attributes

4. Job Content Errors

5.

U/E/S

6.

Explanation (See below for instructions)

IC Focus Cues Critical Steps Scope (N/B)

Over-lap Job-Link Minutia S1 (a)

D 2

E E

S Cues can never state that a valve is open as opposed to red light on, green light off, etc!

We discussed this during validation with licensee and the changes they made were incorrect (the wrong direction) and cue the entire step. Items in the simulator that provide positive response should never be called cues because they are stimuli for operator response as opposed to a cue from an examiner. Additionally, there should always be a note or break in the JPM indicating where the JPM has shifted to the Alternate path, which this JPM doesnt have between steps 4 and 5.

Changes made and submitted 2/24/10 S2 (b)

D 2

S Also, I did not see the addition we asked for in directing him/her to section 2 of this procedure in the initial cue of this JPM?

S3 (c)

D 2

U S

Not an Alternate path JPM. Licensee agreed to change outlines since there are two extra Alternate path JPMs for RO and SRO-I and still the minimum of 2 for the SRO-U outlines.

S4 (d)

D 2

S S5 (e)

D 3

S This JPM has clearly marked Alt path at step 4 as opposed to JPM a which has none.

S6 (f)

D 3

S S7 (g)

D 2

X E

S Step 3 is not critical since no actions are performed for this step.

Licensee made changes and resubmitted on 2.24.10, now Sat.

S8 (h)

D 2

S Inform applicant that JPM is over versus complete?

P1 (i)

S 2

S P2 (j)

S 3

X E

S Shift manager directs you (not requests you). Step 1, cue is breakers are open (off) which is incorrect. It should state that the breaker handles are in the down position but should be cued by examiners with a pen/pencil. Same applies for Steps 2 (breakers up not closed),

Step 3 (up), Step 4 (up) Step 5 (up), Step 6 (up) etc. Also, when do the alarms come in in the CR for these breaker manipulations? Answer, step 14.

JPm submitted 2/24/10 is now Sat.

P3 (k)

S 2

X E

S Cues cant state valve open/closed (must be green light off, red light on, etc). What is the standard for verification that CZ-47A valve is closed? Also, step3 should be split into two steps for closing the valve and tagging the valve. This is a manual chain driven valve in the overhead. Also, how is the release terminated? Is this when the valve is closed or what?

Licensee made changes and now the JPM is Sat.

0 Page 4 of 5 OBDI 202 - IOLE Process AN1 - 2010 - 03 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS JPM#

1.

Dyn (D/S)

2.

LOD (1-5)

3. Attributes

4. Job Content Errors

5.

U/E/S

6.

Explanation (See below for instructions)

IC Focus Cues Critical Steps Scope (N/B)

Over-lap Job-Link Minutia Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.

2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.

3. Check the appropriate box when an attribute weakness is identified:

$ The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.

$ The JPM does not contain sufficient cues that are objective (not leading).

$ All critical steps (elements) have not been properly identified.

$ Scope of the task is either too narrow (N) or too broad (B).

$ Excessive overlap with other part of operating test or written examination.

4. Check the appropriate box when a job content error is identified:

• Topics not linked to job content (e.g., disguised task, not required in real job).

• Task is trivial and without safety significance.

5. Based on the reviewer=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

6. Provide a brief description of any U or E rating in the explanation column.

7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form.

0 Page 5 of 5 OBDI 202 - IOLE Process AN1 - 2010 - 03 DRAFT OPERATING TEST COMMENTS SCENARIOS Scenario Set

1.

ES

2.

TS

3.

Crit

4.

IC

5.

Pred

6.

TL

7.

L/C

8.

Eff

9.

U/E/S

10. Explanation (See below for instructions) 1 E

S Verify TS calls are correct one last time. 3.3.1A, 3.3.13, and 3.3.15? Did staff run this scenario again to ensure IC issues were fixed? We were getting voltage issues on the grid that were not part of the scenario. Answer, resolved by staff.

Final review confirmed TS are correct 2/24/10 2

E S

Verify TS calls are correct one last time. 3.3.11, 3.3.2A, and 3.3.15? Cue page 8 for examiner note states loss of Pleasant Hill line and it should state loss of Mablevale line.

Final review confirmed TS are correct 2/24/10 3

U S

Verify TS calls are correct one last time. 3.1.4C does not apply according to Hopkins, TRM cant be used (event 1), and so we need a TS call prior to major event.

Received updated scenario 2/24/10 and is now Sat.

4 E

S Spare scenario.

Final review confirmed TS are correct 2/24/10 Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating test scenario sets. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1.

ES: ES-301 checklists 4, 5, & 6 satisfied.

2.

TS: Set includes SRO TS actions for each SRO, with required actions explicitly detailed.

3.

Crit: Each manipulation or evolution has explicit success criteria documented in Form ES-D-2.

4.

IC: Out of service equipment and other initial conditions reasonably consistent between scenarios and not predictive of scenario events and actions.

5.

Pred: Scenario sequence and other factors avoid predictability issues.

6.

TL: Time line constructed, including event and process triggered conditions, such that scenario can run without routine examiner cuing.

7.

L/C: Length and complexity for each scenario in the set is reasonable for the crew mix being examined, such that all applicants have reasonably similar exposure and events are needed for evaluation purposes.

8.

Eff: Sequence of events is reasonably efficient for examination purposes, especially with respect to long delays or interactions.

9.

Based on the reviewers judgment, rate the scenario set as (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory.

10. Provide a brief description of problem in the explanation column.

11. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form.