ML041830281
| ML041830281 | |
| Person / Time | |
|---|---|
| Site: | Davis Besse  |
| Issue date: | 05/03/2004 |
| From: | Lanksbury R NRC/RGN-III/DRS/OLB |
| To: | |
| Shared Package | |
| ML041600289 | List: |
| References | |
| 50-346/04-301 | |
| Download: ML041830281 (16) | |
Text
VARIOUS CHECKLISTS FOR THE DAVIS-BESSE INITIAL EXAMINATION - MAY 2004
ES-201 Examination Security Agreement Form ES-201-3 date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be admin-istered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC. Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensees procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.
- 2.
Post-Examination To the best of my knowledge,,I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of5/?b~+S~~/?From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.
PRINTED NAME JOB TITLE / RESPONSIBILITY SIGNATURE (1)
DATE SIGNATURE (2)
DATE NOTE NOTES:
25 of 25 NUREG-1021, Draft Revision 9
&Lp / 3 - 3 E--5JE Examination Security Agreement Form ES-201-3
- 1.
Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of dsby date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be admin-istered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC. Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensees procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.
4 5qi.I.
as of the
- 2.
Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of!&bY+5/ickYFrom the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.
I L.
- 13.
- 14.
- 15.
NOTES:
25 of 25 NUREG-1021, Draft Revision 9
t, Facility: Davis Besse NPS Date of Examination: 5/3/04 - 5/7/04 Operating Tes
- 1. GENERAL CRITERIA
- a.
- b.
C.
- d.
- e.
The operating test conforms with the previously approved outline; changes are consistent with sampling requirements (e.g., 10 CFR 55.45, operational importance, safety function distribution).
There is no day-to-day repetition between this and other operating tests to be administered during this examination.
The operating test shall not duplicate items from the applicants' audit test(s)(see Section D.l.a).
Overlap with the written examination and between different parts of the operating test is within acceptable limits.
It appears that the operating test will differentiate between competent and less-than-competent applicants at the designated license level.
- 2. WALK-THROUGH CRITERIA
- a.
Each JPM includes the following, as applicable:
0 initial conditions 0 initiating cues 0 references and tools, including associated procedures 0 reasonable and validated time limits (average time allowed for completion) and specific 0
specific performance criteria that include:
designation if deemed to be time critical by the facility licensee
- detailed expected actions with exact criteria and nomenclature
- system response and other examiner cues
- statements describing important observations to be made by the applicant
- criteria for successful completion of the task
- identification of critical steps and their associated performance standards
- restrictions on the sequence of steps, if applicable
- b.
Repetition from operating tests used during the previous licensing examination is within acceptable limits (30% for the walk-through) and do not compromise test integrity.
C.
At least 20 percent of the JPMs on each test are new or significantly modified.
- 3. SIMULATOR CRITERIA
- a.
The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES-301-4 and a copy is attached.
Printed Name I Signature
- a. Author
- b. Facility Reviewer(*)
Uumber: 1 Initials Date NOTE:
The facility signature is not applicable for NRC-developed tests
- Independent NRC reviewer initial items in Column "c;" chief examiner concurrence required NUREG-1021, Draft Revision 9 24 of 27
Facility: Davis Besse NPS Date of Examination: 5/3/04 - 5/7/04 I. GENERAL CRITERIA Operating Tes
- a.
- b.
C.
- d.
- e.
The operating test conforms with the previously approved outline; changes are consistent with sampling requirements (e.g., 10 CFR 55.45, operational importance, safety function distribution).
There is no day-to-day repetition between this and other operating tests to be administered during this examination.
The operating test shall not duplicate items from the applicants' audit test(s)(see Section D.l.a).
Overlap with the written examination and between different parts of the operating test is within acceptable limits.
It appears that the operating test will differentiate between competent and less-than-competent applicants at the designated license level.
- 2. WALK-THROUGH CRITERIA
- a.
Each JPM includes the following, as applicable:
R initial conditions 0
initiating cues 0 references and tools, including associated procedures 0 reasonable and validated time limits (average time allowed for completion) and specific 0 specific performance criteria that include:
designation if deemed to be time critical by the facility licensee
- detailed expected actions with exact criteria and nomenclature
-system response and other examiner cues
-statements describing important observations to be made by the applicant
- criteria for successful completion of the task
- identification of critical steps and their associated performance standards
- restrictions on the sequence of steps, if applicable
- b.
Repetition from operating tests used during the previous licensing examination is within acceptable limits (30% for the walk-through) and do not compromise test integrity.
C.
At least 20 percent of the JPMs on each test are new or significantly modified.
- 3. SIMULATOR CRITERIA
- a.
The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES-3014 and a copy is attached.
- a. Author
- b. Facility Reviewerr)
- c. NRC Chief Examiner (#)
- d. NRC Supervisor
\\lumber: 2 Initials NOTE:
The facility signature is not applicable for NRC-developed tests
- Independent NRC reviewer initial items in Column "c;" chief examiner concurrence required NUREG-I 021, Draft Revision 9 24 of 27
W W
Facility: Davis Besse NPS Date of Examination: 5/3/04 - 5/7/04
- 1. GENERAL CRITERIA Operating Tee
- a.
- b.
C.
- d.
- e.
The operating test conforms with the previously approved outline; changes are consistent with sampling requirements (e.g., 10 CFR 55.45, operational importance, safety function distribution).
There is no day-today repetition between this and other operating tests to be administered during this examination.
The operating test shall not duplicate items from the applicants audit test(s)(see Section D.l.a).
Overlap with the written examination and between different parts of the operating test is within acceptable limits.
It appears that the operating test will differentiate between competent and less-than-competent applicants at the designated license level.
- 2. WALK-THROUGH CRITERIA Each JPM includes the following, as applicable:
0 initial conditions 0 initiating cues 0 references and tools, including associated procedures 0 reasonable and validated time limits (average time allowed for completion) and specific 0 specific performance criteria that include:
designation if deemed to be time critical by the facility licensee
- detailed expected actions with exact criteria and nomenclature
-system response and other examiner cues
- statements describing important observations to be made by the applicant
- criteria for successful completion of the task
- identification of critical steps and their associated performance standards
- restrictions on the sequence of steps, if applicable
- b.
Repetition from operating tests used during the previous licensing examination is within acceptable limits (30% for the walk-through) and do not compromise test integrity.
C.
At least 20 percent of the JPMs on each test are new or significantly modified.
- 3. SIMULATOR CRITERIA
- a.
The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES-301-4 and a copy is attached.
Printed Name / Sianature
- a. Author
- b. Facility Reviewerr)
- c. NRC Chief Examiner (#)
- d. NRC Supervisor Vumber: 3 Initials NOTE:
The facility signature is not applicable for NRC-developed tests
- Independent NRC reviewer initial items in Column c; chief examiner concurrence required NUREG-1021, Draft Revision 9 24 of 27
ES-301 Simulator Scenario Quality Checklist Form ES-30 1-4 TARGET QUANTITIATIVE ATTRIBUTES (PER SCENARIO; SEE SECTION D.5.d)
- 1.
Total malfunction (5-8)
- 2.
- 3.
Abnormal events (2-4)
Malfunctions after EOP entry (1 -2)
- 4.
Major transients (1-2)
- 5.
- 5.
EOPs enteredrequiring substantive actions (1 -2)
EOP contingencies requiring substantive actions (0-2)
- 7.
Critical tasks (2-3)
Facility: Davis-Besse NPS Date of Examination: 5/3/04 Scenario Numbers: 1 I 2 Operating Actual Attributes 719 2 1 1 411 1 1 1 112 211 212 QUALITATIVE ATTRIBUTES
'est No.: 1 Initials a -
P I
I
_yx/ I
+
sL3 25 of 27 NUREG-1021, Draft Revision 0
U TARGET QUANTITIATIVE ATTRIBUTES (PER SCENARIO; SEE SECTION D.5.d)
- 1.
Total malfunction (5-8)
- 2.
Malfunctions after EOP entry (1-2)
- 3.
Abnormal events (2-4)
- 4.
Major transients (1 -2)
- 5.
- 6.
EOPs enteredlrequiring substantive actions (1-2)
EOP contingencies requiring substantive actions (0-2)
L, Actual Attributes 71915 2 / 1 / 1 41112 1 / 1 / 1 11212 2 / 1 / 0 ES-301 Simulator Scenario Quality Checklist Form ES-301-4
~~
Facility: Davis-Besse NPS Date of Examination: 5/3/04 Scenario Numbers: I I 2 / 3 Operatini QUALITATIVE ATTRIBUTES
- 7.
Critical tasks (2-3)
I 2 / 2 / 1
'est No.: 2 Initials 33 25 of 27 NUREG-1021, Draft Revision 0
U TARGET QUANTITIATNE ATTRIBUTES (PER SCENARIO; SEE SECTION D.5.d)
- 1.
Total malfunction (5-8)
- 2.
- 3.
Abnormal events (2-4)
Malfunctions after EOP entry (1-2)
- 4.
Major transients (1-2)
- 5.
- 5.
EOPs enteredrequiring substantive actions (1-2)
EOP contingencies requiring substantive actions (0-2)
- 7.
Critical tasks (2-3)
ES-301 Simulator Scenario Quality Checklist Form ES-301-4 Actual Attributes 71915 21111 411 1 2 11111 11212 2 1 110 21211
'.? - -,
Facility: Davis-Besse NPS Date of Examination: 5/3/04 Scenario Numbers: 1 1 2 / 3 Operatin1 QUALITATIVE ATTRIBUTES
'est No.: 3 Initia
-qT
+
S k
+
C# -
25 of 27 NUREG-1021, Draft Revision 0
ES-301 I
Transient and Event Checklist 1
6 Reactivity 0
Normal I
- SRO-U Instrument /
2,3, 0
0 1,2, 39 4, 5, 7, Component 2*
4,5, 8 8 7 3 6
10 Major 1
OPEMTING TEST NO.: 1 Form ES-301-5 W
Reactivity I 0
I I
I I
I I
Normal I
- 2*
Instrument /
Component Major I
Author:
NUREG-I 021, Draft Revision 9 26 Of 27
tr t,
Reactivity Normal SRO-U Instrument /
Component Major ES-301 0
I*
2*
1 Transient and Event Checklist Form ES-301-5 OPERATING TEST NO.: 2 SRO-I As SRO Instructions: (1)
(2)
Enter the operating test number and Form ES-D-1 event numbers for each evolution type.
Reactivity manipulations may be conducted under normal or controlled abnormal conditions (refer to Section D.5.d) but must be significant per Section C.2.a of Appendix D.
- Reactivity and normal evolutions may be replaced with additional instrument or component malfunctions on a one-for-one basis.
Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to the applicants competence count toward the minimum requirement.
(3)
Author:
'4 NRC Reviewer:
NUREG-I021, Draft Revision 9 26 Of 27
ES-301 I
I I
I I
Transient and Event Checklist Reactivity Normal Form ES-301-5 0
I
- OPERATING TEST NO.: 3 Instrument /
Component Major As RO SRO-I 2*
1 SRO-U Reactivity Normal Instrument /
Component Major 0
I
- 2*
1 Instructions: (1)
(2)
(3)
Author:
NRC Reviewer:
Enter the operating test number and Form ES-D-1 event numbers for each evolution type.
Reactivity manipulations may be conducted under normal or controlled abnormal conditions (refer to Section D.5.d) but must be significant per Section C.2.a of Appendix D.
- Reactivity and normal evolutions may be replaced with additional instrument or component malfunctions on a one-for-one basis.
Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to the applicants competence count toward the minimum requirement.
NUREG-I 021, Draft Revision 9 26 Of 27
L, InterpretlDiagnose Events Understand Plant and and Conditions System Response Comply With and Use Procedures (1)
ES-301 Competencies Checklist FO~TYI ES-301-6 z:::
6 23, 4#5, 6
12, 3,4, 5.6, 7,8 SI SCEI -
I Operate Control Boards (2)
Communicate and Interact With the Crew Demonstrate Supervisory Ability (3)
Use Tech. Specs. (3)
Comply With and Competencies 2 1,2,
- 334, 5,6 i.31 3,5 0
ARIC 3
6
+
(I)
Includes Technical Specification compliance for an RO.
(2) Optional for an SRO-U.
(3) Only applicable to SROs.
instructions:
Circle the applicants license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.
Author:
27 of 27 NUREG-1021, Draft Revision 9
ES-401 Written Examination Form ES-401-6 Quality Checklist
- 7.
Between 50 and 60 percent of the questions on the RO exam are written at the comprehension I analysis level; the SRO exam may exceed 60 percent if the randomly selected WAS support the higher cognitive levels; enter the actual question RO/SRO question distribution(s) at right
- 6.
Bank use meets limits (no more than 75 percent from the bank, at least 10 percent new, and the rest modified); enter the actual ROISRO-only question distribution at right Memory 3419 Bank Modified N~~
3515 1413 26117 CIA 41 116
- 8.
- 9.
IO.
- 11.
References/handouts provided do not give away answers Question distribution meets previously approved examination outline; deviations are justified Question psychometric quality and format meet ES, Appendix B, guidelines The exam contains the required number of one-point, multiple choice items; the total is correct and agrees with value on cover sheet Printed Na@ I Signatup/
Initial a
I I
Date
- a. Author
- b. Facility Reviewerr)
\\CA,ctln4
- c. NRC Chief Examiner(#)
-&dO+
q \\a04
- d. NRC Regional Supervisor J
Note:
- The facility reviewers initiaklsignature is not applicable for NRC-developed examinations;
- Independent NRC ieviewer initial items i n Column c; chief examiner concurrence required NUREG-1021, Draft Revision 9 42 of 46
ES-403 Written Examination Grading Form ES-403-1 Quality Checklist Facility:
Davis-Besse Date of Exam:
05/10/04 Exam Level: RO-Initials I
Item Description a
b C
- 1.
Clean answer sheets copied before grading puw &.-
- ad
- 2.
Answer key changes and question deletions justified and documented
- 4.
Grading for all borderline cases (80 +/- 2% overall and 70 +/-
4% on the SRO-only) reviewed in detail
- 3.
Applicants scores checked for addition errors (reviewers spot check > 25% of examinations)
Rxw h%
~
~
~
- 6.
Performance on missed questions checked for training questions missed by half or more of the applicants deficiencies and wording problems; evaluate validity of
- 5.
All other failing examinations checked to ensure that grades ll are iustified
~~
pu b-926-d
- a. Grader
(*)
The facility reviewers signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.
5 of 5 NUREG-1 021, Draft Revision 9
ES-403 Written Examination Grading Form ES-403-1 Quality Checklist Facility:
Davis-Besse Date of Exam: 05/10/04 Item Description
- 1.
- 2.
Clean answer sheets copied before grading Answer key changes and question deletions justified and documented
- 3.
- 4.
- 5.
- 6.
Applicants' scores checked for addition errors (reviewers spot check > 25% of examinations)
Grading for all borderline cases (80 +/- 2% overall and 70 +/-
4% on the SRO-only) reviewed in detail All other failing examinations checked to ensure that grades are justified Performance on missed questions checked for training deficiencies and wording problems; evaluate validity of questions missed by half or more of the applicants Printed Name / Signature ixam Level:mSRO Initials a
- d. NRC Supervisor (*)
Rad 2
Date b! 31 O't
(*)
The facility reviewer's signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.
5 of 5 NUREG-1 021, Draft Revision 9