ML031970806

From kanterella
Jump to navigation Jump to search
CNS - Init Exam - 06/2003 - Draft Operating Exam Comments
ML031970806
Person / Time
Site: Cooper Entergy icon.png
Issue date: 06/30/2003
From: Gody A
NRC Region 4
To: Jason Christensen
Nebraska Public Power District (NPPD)
References
50-298/03-301 50-298/03-301
Download: ML031970806 (8)
v • d • e


Text

S:\DRS\OB\EXAMS\CNS\2003_IN_EX\ADAMS INPUT\Comments\Draft Operating Exam Comments\Appdx_E.wpd APPENDIX E - REGION IV OPERATING TEST JOB PERFORMANCE MEASURE QUALITY REVIEW MATRIX

1. 2. 3. Attributes 4. Job Content 5. 6.

JPM# Dyn LOD Errors U/E/S Explanation (D/S) (1-5) (See below for instructions)

IC Cues Critical Scope Over- Job- Minutia Focus Steps (N/B) lap Link 1-1 D 2 S For Step 6, what is the indication or lack of indication that the rod is NOT latched that the operator is looking for? No changes required.

1-2 D 2 S Step 6 - what is an acceptable load rate? No changes required.

1-3 D 3 S Step 2 - what are the indications - annunciator #s, flow indicators, etc.. Step 9 - recommend using time compression - ensure candidate knows the 5 min requirement - after approx 1 minute, indicate 5 minutes have passed. Will make enhancements as requested.

1-4 D 3 S 1-5 D 3 S Should Step 8 read - reset light off? Is the button for Step 7 RFPT B Trip Reset button the same as Step 10 - RFPT B Overspeed Trip Reset? Licensee indicated JPM was correct as written.

1-6 D 3 S 1-7 S 2 E Edit Step 12 standard - believe it should only say, The operator tightens the screws. Will modify as requested.

2-1 S 3 E Edit the cue for Step for to read -....and the engine does not start.. JPM steps differ from procedure steps. What happened to Procedure Step 8.3.1? Appears that diesel would NOT start without this switch in start. What is the significance of Note 2 in Step 8.3? Looked at licensee comments and based on additional information, concur with their comments. They will add, ...and the engine does not start, to Step 4 - verify.

2-2 S 3 E Step 1 describes a special wrench - where is this wrench located? For the cues for Step 2 and subsequent steps, modify the examiner cues - basically tell the candidate the handwheel has stopped moving (either open or shut) and then when asked by the candidate, tell them the position of the valve stem and limit switches. Looked at licensee comments - Add a note for the examiner that the wrench is yellow and is on a lanyard at the location. For Step 2 and subsequent steps, modify the cue as described - this is realistic. Licensee agreed and will make changes - verify.

2-3 S 2 S Is LI-70 a local indication or control room? No changes required.

- Page 1 -

Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically an system reconfiguration or realignment.
2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3. Check the appropriate box when an attribute weakness is identified:
  • The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
  • The JPM does not contain sufficient cues that are objective (not leading).
  • All critical steps (elements) have not been properly identified.
  • Scope of the task is either too narrow (N) or too broad (B).
  • Excessive overlap with other part of operating test or written examination.
4. Check the appropriate box when a job content error is identified:
  • Topics not linked to job content (e.g., disguised task, not required in real job).
  • Task is trivial and without safety significance.
5. Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6. Provide a brief description of problem in the explanation column. Provide conclusion on whether JPM SET criteria satisfied (i.e., number/distribution of safety functions, A.3 and A.4 integrated with parts B/C, Admin topics per section meet ES).

- Page 2 -

S:\DRS\OB\EXAMS\CNS\2003_IN_EX\ADAMS INPUT\Comments\Draft Operating Exam Comments\Appdx_E_RO_ADMINR1.wpd APPENDIX E - REGION IV OPERATING TEST JOB PERFORMANCE MEASURE QUALITY REVIEW MATRIX

1. 2. 3. Attributes 4. Job Content 5. 6.

JPM# Dyn LOD Errors U/E/S Explanation (D/S) (1-5) (See below for instructions)

IC Cues Critical Scope Over- Job- Minutia Focus Steps (N/B) lap Link A.1-1 D 3 E The JPM must have a clearly identified task standard against which the task will performance will be measured (NUREG 1021, App C, B.3). Change IC to get rid of LOCA, for stable containment parameters. Be sure to put range of acceptable values based on stable state conditions. License will modify as requested - verify A.1-2.1 S 2 S Open Reference. Change outline description to match JPM. Is there a written exam question like this? Not Staffing Requirements but Equipment Control A.1-2.2 S 2 S Open Reference. Change outline description to match JPM. Not Staffing Requirements but Equipment Control A.2 D 3 S A.3 S 2 S A.4 S 3 S Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically an system reconfiguration or realignment.
2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3. Check the appropriate box when an attribute weakness is identified:
  • The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
  • The JPM does not contain sufficient cues that are objective (not leading).
  • All critical steps (elements) have not been properly identified.
  • Scope of the task is either too narrow (N) or too broad (B).
  • Excessive overlap with other part of operating test or written examination.
4. Check the appropriate box when a job content error is identified:
  • Topics not linked to job content (e.g., disguised task, not required in real job).
  • Task is trivial and without safety significance.
5. Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6. Provide a brief description of problem in the explanation column. Provide conclusion on whether JPM SET criteria satisfied (i.e., number/distribution of safety functions, A.3 and A.4 integrated with parts B/C, Admin topics per section meet ES).

- Page 1 -

S:\DRS\OB\EXAMS\CNS\2003_IN_EX\ADAMS INPUT\Comments\Draft Operating Exam Comments\Appdx_E_SRO_ADMIN.wpd APPENDIX E - REGION IV OPERATING TEST JOB PERFORMANCE MEASURE QUALITY REVIEW MATRIX

1. 2. 3. Attributes 4. Job Content 5. 6.

JPM# Dyn LOD Errors U/E/S Explanation (D/S) (1-5) (See below for instructions)

IC Cues Critical Scope Over- Job- Minutia SRO ADMIN EXAM Focus Steps (N/B) lap Link A.1-1 S 3 S A.1-2 S 3 E The JPM must have a clearly identified task standard against which the task will performance will be measured (NUREG 1021, App C, B.3).

Change IC to get rid of LOCA, for stable containment parameters. Be sure to put range of acceptable values based on stable state conditions.

Step 7 - What/how is data bad (instrument failed low)? Will modify JPM as requested - verify A.2 S 3 S A.3 S 2 S Add a cue (if the candidate asks), Radioligical Control Manager recommends KI distribution. Why wouldnt KI be recommended for injured persons? Why isnt KI as important for the injured persons as for the rescue workers? Understand licensee explanation for KI distribution, but add cue if asked by candidate, the Radiological Control Manager Recommends KI distribution.

A.4 S 3 E Shouldnt the critical time be 15 minutes? Proc 5.7.6, Notification, requires notifications to be made to state and local officials within 15 minutes. Change time critical standard to 15 minutes. Will modify JPM as requested - verify.

S:\DRS\OB\EXAMS\CNS\2003_IN_EX\ADAMS INPUT\Comments\Draft Operating Exam Comments\Appdx_F_RO.wpd APPENDIX F - REGION IV OPERATING TEST QUESTION QUALITY REVIEW MATRIX RO Admin Exam Ques.

1. 2. 3. Psychometric Errors 4. Job Content Errors 5. 6.

Q# LOK LOD Explanation (F/H) (1-5) Stem Direct One Min Scope Job- Minutia SRO Back- U/E/S (See below for instructions)

Focus L/U Ans Resp (TS?) Link Only wards A1- H 3 S 2.1 A1- H 3 S 2.2 Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it.

The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1. Classify level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.
2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3. Check the appropriate box when a psychometric error is identified:
  • Stem lacks sufficient focus to solicit only the answers listed (e.g., unclear on intent, answer needed, or unnecessarily negatively phrased)
  • Direct lookup (e.g., desired answer contained in obvious reference),
  • Question does NOT solicit single demonstrably correct answer,
  • Minimum response for passing credit NOT described in key,
  • Scope of question outside guidance of NUREG (e.g., why not how, TS bases not system, emergency not emergency plan),
4. Check the appropriate box when a job content error is identified:
  • Topics not linked to job content (e.g., disguised task, not required in real job, invalid K/A).
  • Recall of too specific knowledge,
  • RO test items test at the SRO job level or vice versa,
  • Reverse logic or application compared to job.
5. Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6. Provide a brief description of problem in the explanation column.

- Page 1 -

S:\DRS\OB\EXAMS\CNS\2003_IN_EX\ADAMS INPUT\Comments\Draft Operating Exam Comments\Appdx_NR1.wpd APPENDIX N - REGION IV OPERATING TEST SCENARIO REVIEW MATRIX Scen 1 2 3 4 5 6 7 8 9 10 Explanation (See below for instructions)

Set ES TS Crit IC Pred TL L/C Eff U/E/S x x e See attached comments.

1 x x e See attached comments 2

x x e See attached comments 3

4 x x e See attached comments Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating test scenario sets. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1. ES: ES-301 checklists 4, 5, & 6 satisfied.
2. TS: Set includes SRO TS actions for each SRO, with required actions explicitly detailed.
3. Crit: Each manipulation or evolution has explicit success criteria documented in Form ES-D-2.
4. IC: Out of service equipment and other initial conditions reasonably consistent between scenarios and not predictive of scenario events and actions.
5. Pred: Scenario sequence and other factors avoid predictability issues.
6. TL: Time line constructed, including event and process triggered conditions, such that scenario can run without routine examiner cuing.
7. L/C: Length and complexity for each scenario in the set is reasonable for the crew mix being examined, such that all applicants have reasonably similar exposure and events are needed for evaluation purposes.
8. Eff: Sequence of events is reasonably efficient for examination purposes, especially with respect to long delays or interactions.
9. Based on the reviewers judgment, is the scenario set as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
10. Provide a brief description of problem in the explanation column

- Page 1 -

Draft Operating Exam Comments Scenarios: General Comment: No ES 301-6s sent, called facility on 4/25.03.

Also, for example on Scenario 2: Corrected by modified scenarios you sent Tom Events such as event 1, 2 take credit for malfunctions and operator actions but no specific actions shown other than dispatching an outside operator. Need to add more specific actions with name/nomenclature labeling of key switches manipulated or key meters monitored by the operator or pnl designations. Similarly, for actions of perform 5.3 EMPWR, Restore Service Water, or Manually Start DG1 & DG2, there needs to be actions with key specifics, pnl designators, or you need to attach the applicable steps to that section of the scenario event sheets for exam administration by the examiners.

Comparison of the 301-5s and the scenarios is confusing because one has to compare the schedule rotation with the form to ascertain whether the evolution types are satisfied. For example, SRO#1 has to rotate into the BOP PSN in order to satisfy the count on scenario 3. When this factor is accounted for the numbers appear accurate. Same comment applies to the 301-6s; you need the schedule to verify counts. NRC verified all competencies will be met based on schedule and scenarios as written.

For all scenarios - brief the crew approximately 30 minutes before starting their scenario. Give the crew the turnover sheet and let them do any prep work - i.e., review procedures for upcoming evolutions. When the crew enters the simulator, the only thing they will need to do is walkdown the boards and take the shift.

Scenario 1

  • Will Event 1 take more than 10 minutes? Recommend that Event 2 start 2 minutes after the condensate pump is restarted (minimize dead time). Change time line based on changes made. No time change recommended - ok
  • Event 5 - will any control board indications be impacted (motor amps, flows, etc)? No - Based on field report only
  • For event termination - When is the reactor considered depressurized? Add depressurization criteria in the Scenario Scenario 2

- Page 2 -

  • Event 2 - what are the indications that the RO will look for to identify this condition? Just the annunciator? yes
  • Event 4 - what is the associated MCB annunciator and when does it actuate? Will add annunciators to activities sheet
  • For critical tasks - define RPV level for TAF and spell out PSP. Will make recommended changes.

Scenario 3

  • Event 1 - per the scenario description, the BOP should get this evolution. Change the Position or description to be consistent with each other. Will change position to BOP
  • Event 3 - Position is blank - assume this is the RO. Will enter RO for Event 3.
  • Critical Task 1 - specify the minimum RPV pressure for the required scram. Will specify safety limit value for pressure.

Scenario 4

  • No comments.

Overall, the scenarios look very good!

- Page 3 -

Retrieved from "https://kanterella.com/w/index.php?title=ML031970806&oldid=2651956"