ML050480177
| ML050480177 | |
| Person / Time | |
|---|---|
| Site: | San Onofre  |
| Issue date: | 12/30/2003 |
| From: | Gody A Operations Branch IV |
| To: | Ray H Southern California Edison Co |
| References | |
| 50-361/03-301, 50-362/03-301 50-361/03-301, 50-362/03-301 | |
| Download: ML050480177 (15) | |
Text
ES-401 Written Examination Form ES-401-9 (R8, S1)
Review Worksheet Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E /S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only R1 F
2 T
U KA mismatch; stem wording (lit twice procedure name needed)
R2 H
4 S
R3 H
4 T
U KA mismatch R4 F
3 S
R5 H
4 T
U KA mismatch R6 H
3 E
KA CE/A11 EK2.2 ???
R7 F
2 S
R8 F
2 S
R9 F
2 S
R10 F
3 T
U KA mismatch Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following c oncepts.]
1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
More than one distractor is not credible.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
5.
Check questions that are sampled for conform ance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
6.
Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
7.
At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being m et).
ES-401 2
Form ES-401-9 (R8, S1)
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E /S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only R11 F
3 S
R12 F
2 S
R13 H
3 T
E 69 EK1.2 ???; C credible ?; clarify responses R14 H
4 T
E Choice C correct???
R15 H
4 T
E Trend information on parameters needed?
R16 F
3 S
R17 H
3 S
R18 H
4 T
T T
U Power increase on drop C EA?; Choice A correc t?; B, D c redible?
R19 H
4 T
U KA mismatch R20 H
4 S
R21 H
4 E
09 EA 3.26?; delete boldface R22 H
4 E
E04 EA 2.01?; delete boldface; Choice A credible?
R23 H
3 S
R24 H
4 E
Edit answer C to reflect long term effect R25 H
4 S
R26 H
4 T
U Same as question R2 R27 F
3 T
T E
Edit (remove at; replace plant with reactor); Specify which level R28 F
2 S
R29 F
3 E
Edit (remove redundant slight meter change indication)
R30 F
2 T
U KA mismatch R31 H
3 T
T T
U KA mismatch; No question asked in stem ; answer/stem disconnect; Xe?
R32 H
4 E
Edit (spell out iaw)
R33 F
2 S
ES-401 3
Form ES-401-9 (R8, S1)
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E /S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only R34 H
3 T
E Edit (is/actions); procedure name?; Choice B correct?; KA AK3.02?
R35 H
2 E
Edit (answer needs identified in key)
R36 F
2 E
Edit (KA designator)
R37 H
3 E
Edit (remove comm ent plant specific data)
R38 F
2 T
E KA 01 AA 3.06 does not exist?
R39 H
3 E
Edit (remove comm ent temps and pressures; add which of the to Q R40 F
2 E
Edit stem (add of the... R CS leak rate)
R41 H
2 T
E incom plete stem inform ation; No answer identified R42 F
2 S
R43 H
3 S
R44 F
2 E
Edit (remove boldface)
R45 H
3 E
Edit (remove boldface; include system 015 in KA designator)
R46 F
2 U
Same question as R38 R47 F
2 S
R48 F
2 T
U Stem cues answer of Question R47; KA mismatch R49 H
2 E
gpm or gallons in stem?
R50 F
3 S
R51 F
3 S
R52 H
3 S
R53 H
3 E
No answer identified R54 F
2 S
R55 F
2 E
Same KA as question R54; Q not related to liquid rad waste (generic)
R56 F
2 S
ES-401 4
Form ES-401-9 (R8, S1)
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E /S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only R57 F
2 S
R58 F
3 S
R59 H
3 S
R60 H
3 S
R61 H
4 S
R62 H
4 S
R63 H
3 T
U KA mismatch R64 F
2 T
U KA mismatch R65 H
3 T
U KA mismatch R66 H
3 S
R67 H
2 S
R68 H
2 T
U KA mismatch R69 F
2 S
R70 H
3 S
R71 F
2 T
U KA mismatch R72 F
3 E
Edit (remove bolface)
R73 H
3 E
Include procedure name R74 H
3 E
Edit KA 063 K2.01???
R75 F
2 S
R76 F
2 E
KA mismatch?
R77 F
2 S
R78 F
2 S
R79 F
3 S
ES-401 5
Form ES-401-9 (R8, S1)
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E /S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only R80 H
2 S
R81 H
3 S
R82 F
2 T
E SRO only question?
R83 H
2 S
R84 H
2 S
R85 H
3 S
R86 F
2 S
R87 H
3 S
R88 F
2 S
R89 H
4 T
E SRO only question?
R90 H
4 E
Edit (remove boldface)
R91 F
2 S
R92 F
2 S
R93 F
2 S
R94 F
2 T
E SRO only question?
R95 F
2 E
Edit (remove boldface)
R96 F
2 S
R97 H
3 S
R98 F
2 S
R99 F
3 S
R100 F
2 S
S1 F
3 T
U KA mismatch S2 F
3 S
ES-401 6
Form ES-401-9 (R8, S1)
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E /S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only S3 H
3 T
U RO system level; Edit (place question at end of stem)
S4 F
2 S
S5 F
3 T
U RO system only S6 H
3 S
S7 F
2 S
S8 F
2 S
S9 H
3 S
S10 F
3 S
S11 F
3 S
S12 F
2 S
S13 F
3 T
U RO system only S14 H
3 T
U RO system only S15 F
2 T
U KA mismatch S16 F
3 T
U RO system only S17 F
2 T
U RO system only S18 H
3 T
U RO system only S19 H
3 S
S20 H
3 T
U RO system only S21 F
2 S
S22 F
2 T
U RO system only S23 H
3 S
S24 F
2 T
U RO system only S25 F
3 S
S:\\DRS\\OB\\EXAMS\\SO NG S\\SON GS-INIT-EXAM 12-2003\\SONG S-12-2003-NRC DRAFT EXAM C OMMEN TS\\RO Admin JPM review.wpd
- Page 1 -
APPENDIX E - REG ION IV OPERA TING TEST JO B PER FO RM AN CE ME ASUR E Q UA LITY REVIEW MA TRIX JPM#
1.
Dyn (D/S) 2.
LOD (1-5)
- 3. Attributes
- 4. Job Content Errors 5.
U/E /S 6.
Explanation (See below for instructions)
IC Focus Cues Critical Steps Scope (N/B)
Over-lap Job-Link Minutia RO A1a S
3 S
RO A1b S
3 E
Step 9 of the JPM listed the ans wer as 8.8 D K/K. The key has the answer as 8.2 D K/K using Fig 4.3.
W e get 8.2 DK/K using Fig 4.3. W hich is correct? In addition, there is no tolerance for this answer -
W hy?
RO A2 S
3 E
Prints provided to applicant have different valve designations than those documented in the JPM. Step 2 refers to the casing vent valve - W hy?
RO A3 S
3 X
E IC does not provide designate drain valve to be opened. Step 1 listed as critical for applicant to determine applicable survey map from handouts provided. Only survey map provided is the applicable one.
RO A4 S
2 S
S:\\DRS\\OB\\EXAMS\\SO NG S\\SON GS-INIT-EXAM 12-2003\\SONG S-12-2003-NRC DRAFT EXAM C OMMEN TS\\RO Admin JPM review Resolution.wpd
- Page 1 -
APPENDIX E - REG ION IV OPERATING TEST JOB PE RFO RMANCE MEA SURE QU ALITY R EVIEW MAT RIX (COM M ENT RE SOLUTION)
JPM#
1.
Dyn (D/S) 2.
LOD (1-5)
- 3. Attributes
- 4. Job Content Errors 5.
U/E /S 6.
Explanation (See below for instructions)
IC Focus Cues Critical Steps Scope (N/B)
Over-lap Job-Link Minutia RO A1a S
3 S
RO A1b S
3 S
Will determine correct answer and revise JPM. Will also add a tolerance for this value.
RO A2 S
3 S
Will revise JPM to include valve numbers as shown on print. Valve designations in JPM are the designations that will be recorded in the tag out.
RO A3 S
3 S
Will check on the drain v alve (there may be only one). If ther are more than one, will designate what valve to go to. Will revise the JPM to state to find the correct page of the provided survey map.
RO A4 S
2 S
Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
1.
Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically an system reconfiguration or realignment.
2.
Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3.
Check the appropriate box when an attribute weakness is identified:
The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
The JPM does not contain sufficient cues that are objective (not leading).
All critical steps (elements) have not been properly identified.
Scope of the task is either too narrow (N) or too broad (B).
Excessive overlap with other part of operating test or written examination.
4.
Check the appropriate box when a job content error is identified:
Topics not linked to job content (e.g., disguised task, not required in real job).
Task is trivial and without safety significance.
5.
Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6.
Provide a brief description of problem in the explanation column. Provide conclusion on whether JPM SET criteria satisfied (i.e., number/distribution of safety functions, A.3 and A.4 integrated with parts B/C, Admin topics per section meet ES).
S:\\DRS\\OB\\EXAMS\\SO NG S\\SON GS-INIT-EXAM 12-2003\\SONG S-12-2003-NRC DRAFT EXAM C OMMEN TS\\SR O AD MIN JPM review.wpd
- Page 1 -
APPENDIX E - REG ION IV OPERA TING TEST JO B PER FO RM AN CE ME ASUR E Q UA LITY REVIEW MA TRIX JPM#
1.
Dyn (D/S) 2.
LOD (1-5)
- 3. Attributes
- 4. Job Content Errors 5.
U/E /S 6.
Explanation (See below for instructions)
IC Focus Cues Critical Steps Scope (N/B)
Over-lap Job-Link Minutia SRO A1a S
2 S
SRO A1b S
2 U
JPM task is to review Shutdown Margin Calculation and direct appropriate actions, JPM provided. SAME as RO JPM has applicant calculate SDM and nothing else. W hile there are some errors in the calculations, the result is the same - need to change. Low discrimination.
SRO A2 S
2 E
Couldnt find in procedure requirement for discharge valve to be shut before suction valve - does such a requirem ent exist?
SRO A3 S
3 X
E IC does not provide designate drain valve to be opened. Step 1 listed as critical for applicant to determine applicable survey map from handouts provided. Only survey map provided is the applicable one.
SRO A4 S
2 S
S:\\DRS\\OB\\EXAMS\\SO NG S\\SON GS-INIT-EXAM 12-2003\\SONG S-12-2003-NRC DRAFT EXAM C OMMEN TS\\SR O AD MIN JPM review Resolution.wpd
- Page 1 -
APPENDIX E - REG ION IV OPERATING TEST JOB PE RFO RMANCE MEA SURE QU ALITY R EVIEW MAT RIX (COM M ENT RE SOLUTION)
JPM#
1.
Dyn (D/S) 2.
LOD (1-5)
- 3. Attributes
- 4. Job Content Errors 5.
U/E /S 6.
Explanation (See below for instructions)
IC Focus Cues Critical Steps Scope (N/B)
Over-lap Job-Link Minutia SRO A1a S
2 S
SRO A1b S
2 S
Will revise the JPM such that the result is also incorrect.
SRO A2 S
2 S
Will check if a procedure requirement for the discharge valve to be shut before the suction valve is closed exists. If it does exist will insure that the JPM is correct.
SRO A3 S
3 S
Will check on the drain v alve (there may be only one). If ther are more than one, will designate what valve to go to. Will revise the JPM to state to find the correct page of the provided survey map.
SRO A4 S
2 S
Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
1.
Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically an system reconfiguration or realignment.
2.
Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3.
Check the appropriate box when an attribute weakness is identified:
The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
The JPM does not contain sufficient cues that are objective (not leading).
All critical steps (elements) have not been properly identified.
Scope of the task is either too narrow (N) or too broad (B).
Excessive overlap with other part of operating test or written examination.
4.
Check the appropriate box when a job content error is identified:
Topics not linked to job content (e.g., disguised task, not required in real job).
Task is trivial and without safety significance.
5.
Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6.
Provide a brief description of problem in the explanation column. Provide conclusion on whether JPM SET criteria satisfied (i.e., number/distribution of safety functions, A.3 and A.4 integrated with parts B/C, Admin topics per section meet ES).
APPENDIX E - REG ION IV OPERA TING TEST JO B PER FO RM AN CE ME ASUR E Q UA LITY REVIEW MA TRIX JPM#
1.
Dyn (D/S) 2.
LOD (1-5)
- 3. Attributes
- 4. Job Content Errors 5.
U/E /S 6.
Explanation (See below for instructions)
IC Focus Cues Critical Steps Scope (N/B)
Over-lap Job-Link Minutia S1 D
3 X
E Proc edure states insertion limit of 115"for PLCEAs, JPM cue only gives 112.5" Tech. Spec. limit. How do we know it is an alternate path?
S2 D
3 X
E Need prerequisite information (e.g., mode). Closing HV 9352 is not marked as c ritical. W hat is effect on system if left open?
S3 D
3 X
X E
JP M trans itions to Procedure SO23-3-2.18 and starts at step 6.3.7. The JPM does not provide this information for the examiner. Need procedure steps and number. A lso need termination point for this JPM (procedure step).
S4 D
2 X
E JP M has reactor shutdown, therefore why do they trip the reactor after the shaft sheer? If the reactor trip is required by the ARP, what about the 5 second limit imposed by the ARP?
S5 D
3 S
Do we need a note for examiner at beginning of JPM that keys # 8 and 23 required to complete task?
S7 D
3 S
Do we need a note for examiner at beginning of JPM that key 52 required to complete task? Need procedure steps in JP M because JPM starts at step 6.3.9 page 7 of procedure.
S6 (C1)
S 3
E of Procedure S O23-6-2 does not list a 2A06 bus to 2XR1 transformer connection as stated in the JPM.
P1 S
3 E
Procedure gives L-85 (L-86) vice L-45 (L-46) given in JPM. Procedure has step to open Neutral Breaker, this step is not in JPM.
P2 S
3 S
W hat is the purpose of the cue Atmospheric Dump Valve HV-8419 has been selected.?
P3 S
2 E
Need to add that steps completed up to 3.4 to I.C. JPM references different procedure for Diesel shutdow, procedure refers to SO23-2-13, JP M refers to SO 23-3-3.23. Steps for diesel shutdown are not consistent with the steps listed in Procedure SO-3-3.23.
APPENDIX E - REG ION IV OPERATING TEST JOB PE RFO RMANCE MEA SURE QU ALITY R EVIEW MAT RIX (COM M ENT RE SOLUTION)
JPM#
1.
Dyn (D/S) 2.
LOD (1-5)
- 3. Attributes
- 4. Job Content Errors 5.
U/E /S 6.
Explanation (See below for instructions)
IC Focus Cues Critical Steps Scope (N/B)
Over-lap Job-Link Minutia S1 D
3 S
Will revise the JPM to state the correct insertion limit.
S2 D
3 S
Leaving HV-9352 open will hav e no effect because it is in series with another v alve that is closed.
S3 D
3 S
Procedure steps will be added to the JPM. The candidate will not have Procedure SO23-3-2.18 (steps will be done by memory.
S4 D
2 S
This JPM is not a RCP Shaft Sheer event, only a thrust bearing overheating event. Tripping of the reactor is a requirement of the procedure for an overheated thrust bearing. Will check on the 5 second limit. If required by the ARP, they will revise the JPM accordingly.
S5 D
3 S
They will provide the keys to the examiner who will than give them to the candidate when requested.
S7 D
3 S
Will provide Key 53 to the examiner to give to the candidate when requested.
S6 (C1)
S 3
S The designation of 2XR1 in the JPM (should be 2XR2) is a typographical error that will be corrected.
P1 S
3 S
The L-85 (L-86) designation is for the MG sets while the L-45 (L-46) designation is the panel designation, therefore the JPM is correct. The candidates are not expected to open the Neutral breakers (it is not necessary).
P2 S
3 S
The cue will be deleted.
P3 S
2 S
The JPM will be revised to m atch the procedure. That will make the JPM steps consistent with the procedure.
GENERIC COMMENTS FOR ALL JPMS:
ADD PROCEDURE STEPS TO THE JPMS DESIGNATE ON THE JPM COVER SHEETS IF THE JPM IS AN ALTERNATE JPM.
APPENDIX E - REG ION IV OPERATING TEST JOB PE RFO RMANCE MEA SURE QU ALITY R EVIEW MAT RIX (COM M ENT RE SOLUTION)
JPM#
1.
Dyn (D/S) 2.
LOD (1-5)
- 3. Attributes
- 4. Job Content Errors 5.
U/E /S 6.
Explanation (See below for instructions)
IC Focus Cues Critical Steps Scope (N/B)
Over-lap Job-Link Minutia S1 D
3 S
Will revise the JPM to state the correct insertion limit.
S2 D
3 S
Leaving HV-9352 open will hav e no effect because it is in series with another v alve that is closed.
S3 D
3 S
Procedure steps will be added to the JPM. The candidate will not have Procedure SO23-3-2.18 (steps will be done by memory.
S4 D
2 S
This JPM is not a RCP Shaft Sheer event, only a thrust bearing overheating event. Tripping of the reactor is a requirement of the procedure for an overheated thrust bearing. Will check on the 5 second limit. If required by the ARP, they will revise the JPM accordingly.
S5 D
3 S
They will provide the keys to the examiner who will than give them to the candidate when requested.
S7 D
3 S
Will provide Key 53 to the examiner to give to the candidate when requested.
S6 (C1)
S 3
S The designation of 2XR1 in the JPM (should be 2XR2) is a typographical error that will be corrected.
P1 S
3 S
The L-85 (L-86) designation is for the MG sets while the L-45 (L-46) designation is the panel designation, therefore the JPM is correct. The candidates are not expected to open the Neutral breakers (it is not necessary).
P2 S
3 S
The cue will be deleted.
P3 S
2 S
The JPM will be revised to m atch the procedure. That will make the JPM steps consistent with the procedure.
GENERIC COMMENTS FOR ALL JPMS:
ADD PROCEDURE STEPS TO THE JPMS DESIGNATE ON THE JPM COVER SHEETS IF THE JPM IS AN ALTERNATE JPM.
S:\\DRS\\OB\\EXAMS\\SONG S\\SONG S-INIT-EXAM 12-2003\\SONG S-12-2003-NRC DRAFT EXAM COMMEN TS\\Scenario review.wpd
- Page 1 -
APPENDIX N - REGION IV OPERATING TEST SCENARIO REVIEW MATRIX Scen Set 1
ES 2
TS 3
Crit 4
IC 5
Pred 6
TL 7
L/C 8
Eff 9
U/E/S 10 Explanation (See below for instructions) 1 Yes Yes Yes Yes Yes No Yes Yes E
Needs a time line.
2 Yes Yes Yes Yes Yes No Yes Yes E
Needs a time line.
3 No Yes Yes Yes Yes No Yes Yes E
Needs a time line.
Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating test scenario sets. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
1.
ES: ES-301 checklists 4, 5, & 6 satisfied.
2.
TS: Set includes SRO TS actions for each SRO, with required actions explicitly detailed.
3.
Crit: Each manipulation or evolution has explicit success criteria documented in Form ES-D-2.
4.
IC: Out of service equipment and other initial conditions reasonably consistent between scenarios and not predictive of scenario events and actions.
5.
Pred: Scenario sequence and other factors avoid predictability issues.
6.
TL: Time line constructed, including event and process triggered conditions, such that scenario can run without routine examiner cuing.
7.
L/C: Length and complexity for each scenario in the set is reasonable for the crew mix being examined, such that all applicants have reasonably similar exposure and events are needed for evaluation purposes.
8.
Eff: Sequence of events is reasonably efficient for examination purposes, especially with respect to long delays or interactions.
9.
Based on the reviewers judgment, is the scenario set as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
10.
Provide a brief description of problem in the explanation column
S:\\DRS\\OB\\EXAMS\\SONG S\\SONG S-INIT-EXAM 12-2003\\SONG S-12-2003-NRC DRAFT EXAM COMMEN TS\\Scenario review Resolution.wpd
- Page 1 -
APPENDIX N - REGION IV OPERATING TEST SCENARIO REVIEW MATRIX (COMMENT RESOLUTION)
Scen Set 1
ES 2
TS 3
Crit 4
IC 5
Pred 6
TL 7
L/C 8
Eff 9
U/E/S 10 Explanation (See below for instructions) 1 Yes Yes Yes Yes Yes No Yes Yes E
Will provide a time line.
2 Yes Yes Yes Yes Yes No Yes Yes E
Will provide a time line.
3 No Yes Yes Yes Yes No Yes Yes E
Will provide a time line.
Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating test scenario sets. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
1.
ES: ES-301 checklists 4, 5, & 6 satisfied.
2.
TS: Set includes SRO TS actions for each SRO, with required actions explicitly detailed.
3.
Crit: Each manipulation or evolution has explicit success criteria documented in Form ES-D-2.
4.
IC: Out of service equipment and other initial conditions reasonably consistent between scenarios and not predictive of scenario events and actions.
5.
Pred: Scenario sequence and other factors avoid predictability issues.
6.
TL: Time line constructed, including event and process triggered conditions, such that scenario can run without routine examiner cuing.
7.
L/C: Length and complexity for each scenario in the set is reasonable for the crew mix being examined, such that all applicants have reasonably similar exposure and events are needed for evaluation purposes.
8.
Eff: Sequence of events is reasonably efficient for examination purposes, especially with respect to long delays or interactions.
9.
Based on the reviewers judgment, is the scenario set as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
10.
Provide a brief description of problem in the explanation column