ML033430193

08/2003 Draft Operating Exam Comments

ML033430193
Person / Time
Site:	South Texas
Issue date:	09/09/2003
From:	Gody A Operations Branch IV
To:	Sheppard J South Texas
References
50-498/03-301, 50-499/03-301 50-498/03-301, 50-499/03-301
Download: ML033430193 (9)

Text

STP 2003 Initial Exam - Admin JPMs APPENDIX E - REGION IV OPERATING TEST JOB PERFORMANCE MEASURE QUALITY REVIEW MATRIX

1. 2. 3. Attributes 4. Job Content 5. 6.

JPM# Dyn LOD Errors U/E/S Explanation (D/S) (1-5) (See below for instructions)

IC Cues Critical Scope Over- Job- Minutia Focus Steps (N/B) lap Link RO A.1 S 2 S RO A.2 S 3 S JPM indicates location as simulator - can be done anywhere?

STP Comment: Yes, it can be done anywhere. JPM edited to remove simulator as location.

RO A.3 S 3 S In the Initiating Cue, delete - .....two (2) Critical and one (1) Non-Critical. As a MINIMUM, you are to IDENTIFY both critical errors. Per your cue, there are only 3 errors, but you answer key seems to show 4 - Data Sheet 1 addition error, Step 5.4.6, Step 5.4.14.3, and Step 5.4.14.4. I believe all of the errors are critical. Suggest changing cue to say - FOUR errors have been inserted into the surveillance. .... For grading, suggest passing if 3 of 4 errors are identified. For Step 5.4.8 of the key, insert correct value of 53.8 into the key. Will probably take longer than 20 minutes.

STP Comment: The formatting used regarding the # of critical or non-critical errors and the passing criteria is based on NRC feedback from previous STP exams. It was intended to provided specific guidance to the examiners for consistency of grading and to provide clarification to the students to ensure they didnt finish the JPM prematurely. Ok - not necessary to tell the candidates grading criteria in the cue - will leave as is.

We considered there to be 3 errors as we counted all of the initialing errors on page 25 as one error with critical and non-critical portions. Whether they are counted separately or not is academic.

However, we feel the error for step 5.4.14.3.a is non-critical because if the its been noted in steps 5.4.14.3.d and 5.14.4 that pump DP is within the required action range, the proper actions will be performed. OK.

The comment on step 5.4.8 asks that the correct value of 53.8 be inserted rather than the error value.

The error value is more appropriate as it reflects an error carried forward. There would be no basis for having the correct value if the component parts used to reach that value were erroneous.

This JPM may take longer than 20 min. however, its scheduled to be performed with one other for SROs during a time when a scenario will also be conducted. As such, there will be approx. 2 hr. to complete these 2 JPMs. OK, but check this time during validation week.

No changes were made to this JPM RO A.4 S 2 S SRO A.4

- Page 1 -

APPENDIX E - REGION IV OPERATING TEST JOB PERFORMANCE MEASURE QUALITY REVIEW MATRIX

1. 2. 3. Attributes 4. Job Content 5. 6.

JPM# Dyn LOD Errors U/E/S Explanation (D/S) (1-5) (See below for instructions)

IC Cues Critical Scope Over- Job- Minutia Focus Steps (N/B) lap Link SRO A.5 S 3 S In the initiating cue, delete - ...,one (1) Critical and two (2) N0n-Critical. As a MINIMUM, you are to IDENTIFY the Critical error. For the key, add the values at the bottom of pg 2 of Data Sheet 1 and in the remarks section of Data Sheet 1 the comment about the T.S limit being exceed is not true - need to remove this from the key. The cue needs to be changed to 4 errors and the grading criteria to include the two critical tasks - TS applicability and VCT level error calculation.

STP comment: Regarding the format for identifying critical/non-critical errors, see comment on JPM A3 above. Ok - not necessary to tell the candidates grading criteria in the cue - will leave as is.

Values added to bottom of page 2 of Data Sheet 1 on the Key. The comment on Data Sheet 1 regarding exceeding TS limits is based on the faulted calculation of 1.22 gpm. In realty, the limit is not exceeded. This is a determination the applicant is supposed to make at step 5.11. The statement at the bottom of page 1 of Data Sheet 1 is considered a carried-forward error based on the results of step 5.11. Added a note to page 1 of Data Sheet 1 to indicate this. The only critical error is on VCT level as all other calculational errors are based on that, including the Tech Spec applicability. No other changes were made to this JPM. OK.

SRO A.6 S 2 S SRO A.7 S 3 S SRO A.8 S 2 S Be sure and note on cover page of the JPM and the schedule that JPM is to be done with Simulator Scenario 1.

On the examiner cue sheet, change the statement, The 15 minute time limit..... to This JPM is time critical. The time limit starts when you (the applicant) understand the initial conditions and initiating cue.

STP comment: Added note to cover page referencing Scenario 1. Revised Initiating cue as recommended. OK.

Note: Original comments in black font. Licensees response in red font. Our response to their response in blue font.

- Page 2 -

Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically an system reconfiguration or realignment.

2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.

3. Check the appropriate box when an attribute weakness is identified:

• The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.

• The JPM does not contain sufficient cues that are objective (not leading).

• All critical steps (elements) have not been properly identified.

• Scope of the task is either too narrow (N) or too broad (B).

• Excessive overlap with other part of operating test or written examination.

4. Check the appropriate box when a job content error is identified:

• Topics not linked to job content (e.g., disguised task, not required in real job).

• Task is trivial and without safety significance.

5. Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

6. Provide a brief description of problem in the explanation column. Provide conclusion on whether JPM SET criteria satisfied (i.e., number/distribution of safety functions, A.3 and A.4 integrated with parts B/C, Admin topics per section meet ES).

- Page 3 -

STP 2003 Initial Exam - Control Room and In-Plant Systems JPMs APPENDIX E - REGION IV OPERATING TEST JOB PERFORMANCE MEASURE QUALITY REVIEW MATRIX

1. 2. 3. Attributes 4. Job Content 5. 6.

JPM# Dyn LOD Errors U/E/S Explanation (D/S) (1-5) (See below for instructions)

IC Cues Critical Scope Over- Job- Minutia Focus Steps (N/B) lap Link

a. D 3 S For Step 9 - assume that the candidate will need to call the simulator operator in order to get this done. May want to add a note to this effect, so the examiner realizes this. Normally, the examiner will do all communications with the candidate, so if some action needs to be taken by the simulator operator, please include in Step of JPM. Include expected annunciators for all steps/actions.

STP comment: added comment to step 9 indicating the student will have to contact the booth instructor for this step. Alarm info will have to be added during NRC validation week. Ok - verified changes.

b. S 2 S

c. D 3 E Step 8 should be a critical task. Since the DG failed to automatically load sequence, the candidate should understand integrated plant response and should know that the load centers have to be re-energized. Change Comment 1 to say, the required response is to re-energize the load centers and MCCs, but the candidate may stop the DG. When an automatic load sequencer fails, is there a procedure that directs the operators to manually start loads once the bus is energized?

STP comment: the evolutions in this JPM were deemed to be too similar to some of those in Scenario

1. 2. Per discussion with the Chief Examiner, this JPM was changed by adding a cue at step 8 to ensure the student follows the intended JPM path rather than secure the DG. Removed JPM step 8A as this no longer applies. Because of these changes, step 8 would not be a critical step. Ok - verified changes.

d. S 2 S

e. D 3 E Recommend editing initiating cue to: You are the Primary Reactor Operator. The Unit Supervisor directs you to respond to alarms located on Control Panel ? When the individual is ready to start the JPM, the examiner can prompt the simulator operator to input the failure. Step 9 does not appear to be critical. If Step 8 is completed correctly, I assume, the indicators would be in the correct state?

STP comment: Recommend leaving the cue as-is. This JPM is to be run concurrently with S2 which is performed on an adjacent Control Room Panel. Discussed with Bernie about panel locations - most actions for S5 are done on other panels away from S2 JPM done on Panel 4. So need to verify no interference with JPMs S2 and S5. Initiating this JPM from the alarm condition would be a distraction to the other student on the adjacent panel as well as prolong the time the students are in each others vicinity. This JPM was not intended to include diagnostics of a failed instrument as this is done extensively on the scenarios. As it is, the student must determine the power range instrument state and determine the appropriate procedural actions. Ok.

Step 9 is considered critical as it entails verifying Tech Spec required conditions. The student must determine if the status window state is correct for the existing plant conditions (power level). No changes were made to this JPM. Ok.

- Page 1 -

STP 2003 Initial Exam - Control Room and In-Plant Systems JPMs APPENDIX E - REGION IV OPERATING TEST JOB PERFORMANCE MEASURE QUALITY REVIEW MATRIX

1. 2. 3. Attributes 4. Job Content 5. 6.

JPM# Dyn LOD Errors U/E/S Explanation (D/S) (1-5) (See below for instructions)

IC Cues Critical Scope Over- Job- Minutia Focus Steps (N/B) lap Link

f. D 3 E Step 10 - add the purge exhaust fans switch name plate label information. Step 11 - remove the examiner cue, unless this is a simulator fidelity issue.

STP comment: title of switch added to JPM step 10. Cue for step 11 is necessary as the simulator ICS may not indicate the expected increase in flow cited in the procedure. Reviewed Step 11 and it seems to indicate that flow bounces around and is hard to read. Does this also occur in the plant? If this does occur in the plant, then no cue to be given. The candidate will have to determine on his own. If this is a simulator fidelity issue, would like to see simulator discrepancy report. Verified change to Step 10. Discussed this with Mike DeFrees, on 7/14, and he indicated that the simulator flow indication did not match the procedure and was not consistent with the plant. He also indicted that he would initiate a simulator DR. OK

g. D 3 E Step 3 - can we change the font so the tables can fit on 1 page? This may make it easier for the examiner to give the cue. For the * - add a note so the examiner knows these valves will not change position - these valves are normally open. Need to correct cue table and Attachment 2 table last column for Rows 2 - RCFC 11B/12B RE ICIV MOV-0147* and Row 3 - RCFC 11C/12C RET ICIV MOV-0208*. For Row 4 of the table, are the RCFC flows correct? I didnt think you had power (or is it just indication power) for Train C CCW valves, so how could they open and wouldnt the chill water still be lined up for Train C? Are the RCFC flows the same for both chill water and CCW?

For Step 7, cue table Row 2, Column 4 (Discharge Valve) - should the indication for MOV-0001C be Red - ON and Green - OFF or is this valve powered from E1(2)C2? If it is powered from E1(2)C2, then CSS pump flow would not be as high as the other 2 trains - it would be a minimum flow value?

STP comment: Cue for step 3 placed on its own separate page. Added note to step 3 that (*) valves do not reposition and are normally open. Corrected typos for RCFC fans. Cooling flow is provided for all trains because per the intitial conditions, power isnt lost to the Train C Load Centers until 10 minutes after the LOCA. This is sufficient time for the RCFC valves to reposition. The CCW Pumps are powered from 4160 V buses, thus they will be running and supplying RCFC cooling water. The flows provided are representative of RCFC cooling flow from CCW. On step 7, the valve indication for MOV-0001C represents a loss of power from E1(2)C2. The CSS Pump flow for C train will be the same as the others as the loss of power did not occur until 10 minutes following the LOCA. This is sufficient time for the discharge valves to open. OK - verified changes.

h. D 2 S Step 10 - For the normal purge dampers, at this time, why would the supply breakers be locked open?

Should the operators do this as part of Procedure FRZ3, Step 1.d.? Step 12, last sentence of cue -

add Train A - Fan 11A(21A) HC-VFN029 or Fan 12A(22A) HC-VFN030 and Train B Fan 111B(21B)

HC-VFN031 or Fan 12B(22B) HC-VFN032.

STP comment: Supply breakers for Normal Purge Dampers are locked open after dampers are closed to comply with Tech Spec requirements. The operators would check both Supplementary and Normal Purge Dampers closed at FRZ3, step 1.d. They can take the clearance as evidence the Normal Dampers are closed. Added additional info to cue on step 12. OK - verified changes to JPM.

i. S 2 S Step 3 and 6 cues need to be modified - valve handwheel stops turning. Step 5 cue, add - You hear the pump running.

STP comment: additional cue information added as suggested. OK - verified changes to JPM.

j. D 3 S

- Page 2 -

APPENDIX E - REGION IV OPERATING TEST JOB PERFORMANCE MEASURE QUALITY REVIEW MATRIX

1. 2. 3. Attributes 4. Job Content 5. 6.

JPM# Dyn LOD Errors U/E/S Explanation (D/S) (1-5) (See below for instructions)

IC Cues Critical Scope Over- Job- Minutia Focus Steps (N/B) lap Link

k. D 2 S For Step 6 - does the potentiometer have an increase/decrease marking on the controller?

STP comment: potentiometer does not have increase/decrease markings. Basically, they are dial potentiometers that are turned clockwise (increasing numbers) to increase output. OK Note: Original comments in black font. Licensees response in red font. Our response to their response in blue font.

Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically an system reconfiguration or realignment.

2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.

3. Check the appropriate box when an attribute weakness is identified:

• The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.

• The JPM does not contain sufficient cues that are objective (not leading).

• All critical steps (elements) have not been properly identified.

• Scope of the task is either too narrow (N) or too broad (B).

• Excessive overlap with other part of operating test or written examination.

4. Check the appropriate box when a job content error is identified:

• Topics not linked to job content (e.g., disguised task, not required in real job).

• Task is trivial and without safety significance.

5. Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

6. Provide a brief description of problem in the explanation column. Provide conclusion on whether JPM SET criteria satisfied (i.e., number/distribution of safety functions, A.3 and A.4 integrated with parts B/C, Admin topics per section meet ES).

- Page 3 -

S:\DRS\OB\EXAMS\stp\2003_IN_EX\Forms\STP-Aug-2003-AppN-Rev1.wpd STP 2003 Initial Exam APPENDIX N - REGION IV OPERATING TEST SCENARIO REVIEW MATRIX Scen 1 2 3 4 5 6 7 8 9 10 Explanation (See below for instructions)

Set ES TS Crit IC Pred TL L/C Eff U/E/S 1 E Event 3 - is the DA storage tank level controller shifted back to manual?

STP comment: in discussion with the Chief Examiner it was determined he meant Auto rather than manual. Clarified that the DA Level control would remain in manual as the signal inputs cannot be de-selected. No further action needed. OK Event 6 - no credit for this component failure, since the operator actions provide little or no insight into the applicants competence. The operator performs no actions and the identification is done during the isolation of the S/G.

STP comment: Because an additional malfunction was previously added to this scenario, loss of this item will have no impact on the minimum transient/events each candidate will receive. In discussion with the Chief Examiner, this was verified and the Chief Examiner annotated the applicable checklists. No further actions needed.

This comment is for all scenarios - the lead examiner for each scenario (examiner assigned to SRO candidate) will tell the simulator booth operator when to initiate the next event for all reactivity changes and when the SRO needs to make a T/S entry, as per the scenario Operator Actions list (FormES-D-2).

2 E Missing Event 4 operator actions STP comment: Event 4 actions added to scenario. OK - verified changes

- Page 1 -

S:\DRS\OB\EXAMS\stp\2003_IN_EX\Forms\STP-Aug-2003-AppN-Rev1.wpd APPENDIX N - REGION IV OPERATING TEST SCENARIO REVIEW MATRIX 3 E Event 6 needs to be changed or modified. This event is VERY similar to JPM S3 - where #12 DG started but did not automatically connect to the 4160v ESF Bus 1B due to a sequencer failure on Train B. JPM S3 does have an alternate path aspect to it and the output bkr does not shut, so the bus is dead which is different from Event 6, but very similar in other aspects.

STP comment: In discussion with the Chief Examiner it was decided to revise JPM S3 to ensure the intended path is taken rather than a potential alternate path. This change will make the two evolutions noted in this comment materially different. OK BU E Event 3 - ES-D-1 credit for this event needs to be RO, per ES-D-2.

STP comment: incorporated on form ES-D-1 for the Backup Scenario.

OK - verified changes.

- Page 2 -

APPENDIX N - REGION IV OPERATING TEST SCENARIO REVIEW MATRIX Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating test scenario sets. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1. ES: ES-301 checklists 4, 5, & 6 satisfied.

2. TS: Set includes SRO TS actions for each SRO, with required actions explicitly detailed.

3. Crit: Each manipulation or evolution has explicit success criteria documented in Form ES-D-2.

4. IC: Out of service equipment and other initial conditions reasonably consistent between scenarios and not predictive of scenario events and actions.

5. Pred: Scenario sequence and other factors avoid predictability issues.

6. TL: Time line constructed, including event and process triggered conditions, such that scenario can run without routine examiner cuing.

7. L/C: Length and complexity for each scenario in the set is reasonable for the crew mix being examined, such that all applicants have reasonably similar exposure and events are needed for evaluation purposes.

8. Eff: Sequence of events is reasonably efficient for examination purposes, especially with respect to long delays or interactions.

9. Based on the reviewers judgment, is the scenario set as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

10. Provide a brief description of problem in the explanation column Note: Original comments in black font. Licensees response in red font. Our response to their response in blue font.

- Page 3 -