IR 05000281/2002301
| ML021570240 | |
| Person / Time | |
|---|---|
| Site: | Surry  |
| Issue date: | 03/28/2002 |
| From: | NRC/RGN-II |
| To: | Virginia Electric & Power Co (VEPCO) |
| References | |
| 50-280/02301, 50-281/02301 | |
| Download: ML021570240 (67) | |
Text
SURRY EXAM 2002-301 50-280, 281/2002-301 MARCH 18 - 28, 2002 Administrative Documents (Yellow Paper) Exam Preparation Checklist....... Exam Outline Quality Checklist.... Exam Security Agreement........ Administrative Topics Outline (Final)
.............. ES-201-1 V/
..............
ES-201-2,/
.............. ES-201-3/
.............. ES-301-1 V/ Control Room Systems and Facility Walk-through Test Outline (Final)........................... Operating Test Quality Check Sheet... Simulator Scenario Quality Check Sheet Transient and Event Checklist........ Competencies Checklist.............
1 Written Exam Quality Check Sheet....
1 Written Exam Review Worksheet......
1 Written Exam Grading Quality Checklist 1 Post-Exam Check Sheet............
...
... ES-301-5
... ES-301-6
^/
'/
/
/
/
... ES-401-9
Facility:
Surry Power Station Date of Examination: Weeks of March18 & 25. 2002 Examinations Developed by:
Facility
/ NRC (circle one)
Target Chief Date*
Task Description / Reference Examiner's Initials-180 1. Examination administration date confirmed (C.l.a; C.2.a & b)
RSB-120 2. NRC examiners and facility contact assigned (C.l.d; C.2.e)
RSB-120 3. Facility contact briefed on security & other requirements (C.2.c)
RSB-120 4. Corporate notification letter sent (C.2.d)
RSB
[-90]
[5. Reference material due (C.l.e; C.3.c)]
12/01 RSB-75 6. Integrated examination outline(s) due (CAl.e & f; C.3.d)
RSB-70 7. Examination outline(s) reviewed by NRC and feedback provided RSB to facility licensee (C.2.h; C.3.e)
-45 8. Proposed examinations, supporting documentation, and RSB reference materials due (C.l.e, f, g & h; C.3.d)
-30 9. Preliminary license applications due (C.1.1; C.2.g; ES-202)
RSB-14 10. Final license applications due and assignment sheet prepared RSB (C.1.1; C.2.g; ES-202)
-14 11. Examination approved by NRC supervisor for facility licensee RSB review (C.2.h; C.3.f)
-14 12. Examinations reviewed with facility licensee (C.1.j; C.2.f & h; C.3.g)
RSB-7 13. Written examinations and operating tests approved by RSB NRC supervisor (C.2.i; C.3.h)
-7 14. Final applications reviewed; assignment sheet updated; waiver RSB letters sent (C.2.g, ES-204)
15. Proctoring/written exam administration guidelines reviewed with RSB-7 facility licensee and authorization granted to give written exams (if applicable) (C.3.k)
-7 16. Approved scenarios, job performance measures, and questions RSB distributed to NRC examiners (C.3.i)
Target dates are keyed to the examination date identified in the corporate notification lette They are for planning purposes and may be adjusted on a case-by-case basis in coordination with the facility license Applies only to examinations prepared by the NR ES-201 Examination Outline Form ES-201-2 (R8,S1)
Quality Checklist Facilit: SURRY POWER STATION Date of Examination: Weeks of March 18 & 25, 2002 Initials Item Task Description a
b c# a. Verify that the outline(s) fit(s) the appropriate model per ES-40 RLM RSB W
R b. Assess whether the outline was systematically and randomly prepared in accordance with RLM RSB I
Section D.1 of ES-401 and whether all K/A categories are appropriately sample T T
c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topic RLM Ras E
N d. Assess whether the justifications for deselected or rejected K/A statements are appropriat RLM RSB a. Using Form ES-301-5, verify that the proposed scenario sets cover the required number of RSB normal evolutions, instrument and component failures, and major transient S I
b. Assess whether there are enough scenario sets (and spares) to test the projected number RLM RSB M
and mix of applicants in accordance with the expected crew composition and rotation schedule without compromising exam integrity; ensure each applicant can be tested using at least one new or significantly modified scenario, that no scenarios are duplicated from the applicants' audit test(s)*, and scenarios will not be repeated over successive day c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative and g{,
RSB quantitative criteria specified on Form ES-301-4 and described in Appendix.
a. Verify that:
RLM RSB (1) the outline(s) contain(s) the required number of control room and in-plant tasks, W
(2) no more than 30% of the test material is repeated from the last NRC examination,
/
(3)* no tasks are duplicated from the applicants' audit test(s), and T
(4) no more than 80% of any operating test is taken directly from the licensee's exam bank b. Verify that:
RLM RSS (1) the tasks are distributed among the safety function groupings as specified in ES-301, (2) one task is conducted in a low-power or shutdown condition, (3) 40% of the tasks require the applicant to implement an alternate path procedure, (4) one in-plant task tests the applicant's response to an emergency or abnormal condition, and (5) the in-plant walk-through requires the applicant to enter the RC c. Verify that the required administrative topics are covered, with emphasis on performance-RLM RSB based activitie I d. Determine if there are enough different outlines to test the projected number and mix of RLM RSB applicants and ensure that no items are duplicated on successive day.
a. Assess whether plant-specific priorities (including PRA and IPE insights) are covered in the RLM RSB appropriate exam sectio G E
b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriat RLM RSB N
E c. Ensure that K/A importance ratings (except for plant-specific priorities) are at least RLM RSB R
A d. Check for duplication and overlap among exam section RLM RSB L
e. Check the entire exam for balance of coverag RLM RSB f. Assess whether the exam fits the appropriate job level (RO or SRO).
RLM RSB a. Author Robert L. Monk
, - b. Facility Reviewer (*)
N/A c. NRC Chief Examiner (#)
Richard S. Baldwin
.{-.
ýz"
d. NRC Supervisor Michael E. Ermstes *
e / /2 Note:
- Not applicable for NRC-developed examination # Independent NRC reviewer initial items in Column "c;" chief examiner concurrence require JOO3
__
/ / C-t'r ye 2.C)...
.9/
e£Sc r ES-201 Examination Security Agreement Form ES-201-3 Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of-S/z-4zidzfzas of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC.Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations andlor an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any Indications or suggestions that examination security may have been compromise.
Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.
PRINTED NAME 2. a*en Grovc 4.'
cS1 z
i 5. eRode,*- 4
,,
6. C0t5s0pA.E 6-.
- Wr/
7. c26AWEAbC6 C, fY4/t*,
8. w'fnT-r A-
7 L 10. \\<,
&:*
1 _wd/z\\ v e m.L.skpqI_
13.-T-*,
A.b~e,.
JOB TITLE / RESPONSIBILITY LeJ Lox# /uso lcrva5aL S2AU~4t f~f#'mr 1J01hi'jMAP,_ /CV Ass. S'aAtctj,ýro r c6cc_:iArco.
SIGNATURE (1)
~4 I
ikX-'c' rQA
-
fl-tsNr e..
tF4Cr-sygdrwt Ltctv$r c,-tS 14.DAv; s
T-- UM4/u C -,-.
15. k-r,",v W
- W-A-tV-Pt'
NOTES:
NUREC-1021, Revision 8 DATE SIGNATUYE (2)
DATE NOTE oz.-.
3/i4/o2 3j;4b?
Z.
24 pf§?
311P02
/-aCkz 14(f 7/at
ES-301 Administrative Topics Outline Form ES-301-1 (R8, S1)
Facility: Surry Power Station Date of Examination: Weeks of March 18 & 25. 2002 Examination Level (circle one):
RO / SRO Operating Test Number: 1 Administrative Describe method of evaluation:
Topic/Subject 1. ONE Administrative JPM, OR Description 2. TWO Administrative Questions Perform valve position verifications (KA2.1.29)(RO)
Obtain VCT trend from computer and determine estimated leak rate. (KA2.1.19)(RO&SRO) Perform component tag out (KA2.2.13)(RO) Perform Attachment 1 of O-OSP-RC-002, Primary to Secondary leak-rate using condenser air ejector rad monito (KA2.3.I 1)(RO) Stability Class Calculation lAW EPIP 2.01(KA2.4.39)(RO)
ES-301 Administrative Topics Outline Form ES-301-1 (R8, Sl)
Facility: Surry Power Station Date of Examination: Weeks of March 18 & 25. 2002 Examination Level (circle one): SRO (I & U)
Operating Test Number: 2 Administrative Describe method of evaluation:
Topic/Subject 1. ONE Administrative JPM, OR Description 2. TWO Administrative Questions Take actions lAW OPAP-0008 for mis-positioned locked valve that makes TS system inoperable. (KA2.1.33)(SRO)
Obtain VCT trend from computer and determine estimated leak rate. (KA2.1.19)(RO&SRO) Determine that work order cannot be released due to improper plant conditions for PMT. (KA2.2.17)(SRO) Determine pathway to and from work site that yields the lowest dose using a survey map. (KA2.3.10)(SRO) Notification determination lAW EPIP 1.01 (KA2.4.41)(SRO)
ES-301 Control Room Systems Form ES-301-2 (R8, S1)
and Facility Walk-Through Test Outline Facility: Surry Power Station Date of Examination: Weeks of March 18 & 25. 2002 Exam Level (circle one): RO / SRO-I Operating Test No.: 1 B.1 Control Room Systems System / JPM Title Type Safety Code*
Function a. Perform the Immediate Operator actions for Loss of Main S, D 4s Feedwater flow (Facility # L026.08)
(KA054AA2.02)
b. Recover dropped control rod (Facility # L065.03)
S, D
(KA001K4.09)
c. Verify Natural Circulation Flow w/ flow not occurring and S, A, M
Steam Dump failure (Facility #L038.03)
(Significant (KA002A4.02)
Modification)
d. Transfer to cold leg recirc w/ RMT functional and S,A,M
MOV1860B failing to open (Facility #LO52.01)
(Significant (KA011EA1.11)
Modification)
e. Loss of RHR at at mid-loop(Facility #L040.5)
S, D, L 4p (KA025AA2.07)
f. Respond to #3 EDG failure to load (Facility #L055.06)
(KA064A4.06)
g. Perform CLS Component Verification (E-0, attachment 1),
S,A,M
w/ A CTMT air recirc fan failing to trip (Facility #L067.01)
(KA022A3.01, A4.01)
B.2 Facility Walk-Through a. Cross-tie Unit 1 emergency busses to restore Instrument D
Air (Facility #L035.03)
(KA056AA1.37)
b. Locally start the Emergency SW Pumps and mitigate D,M,A 4s over-temperature condition. (Facility #13.02)
(KA076A2.01)
c. Makeup to the Unit 2 RWST from the Spent Fuel Pit D, R
(Facility #53.01 B)
(KA006A2.02)
- Type Codes: (D)irect from bank, (M)odified from bank, (N)ew, (A)lternate path, (C)ontrol room, (S)imulator, (L)ow-Power, (R)CA
ES-301 Control Room Systems Form ES-301-2 (R8, S1)
and Facility Walk-Through Test Outline Facility: Surry Power Station Date of Examination: Weeks of March 18 & 25, 2002 Exam Level (circle one): SRO(U)
Operating Test No.: 2-U B.1 Control Room Systems System / JPM Title Type Safety Code*
Function b. Recover dropped control rod (Facility # LO65.03)
S, D
(KA001 K4.09) d. Transfer to cold leg recirc w/ RMT functional and S,A,M
MOV1860B failing to open (Facility #L052.01)
(Significant (KA011EAi)
Modification) B.2 Facility Walk-Through a. Cross-tie Unit 1 emergency busses to restore Instrument D
Air (Facility #L035.03)
(KA056AA1.37)
b. Locally start the Emergency SW Pumps and mitigate D,M,A 4s over temperature condition. (Facility #13.02)
(KA076A2.01)
c. Makeup to the Unit 2 RWST from the Spent Fuel Pit D, R
(Facility #53.01 B)
(KA006A2.02)
- Type Codes: (D)irect from bank, (M)odified from bank, (N)ew, (A)lternate path, (C)ontrol room, (S)imulator, (L)ow-Power, (R)CA
ES-301 Operating Test Quality Checklist Form ES-301-3 (R8, S)
Facility: Surry Power Station Date of Examination: Weeks of March 18 & 25, 2002 Operating Test Number: 1 Initials 1. GENERAL CRITERIA a
b c# The operating test conforms with the previously approved outline; changes are consistent with
"SB sampling requirements (e.g., 10 CFR 55.45, operational importance, safety function distribution). There is no day-to-day repetition between this and other operating tests to be administered Rj A
RSS during this examinatio The operating test shall not duplicate items from the applicants' audit test(s)(see Section D.1.a).
RLM RSS Overlap with the written examination and between operating test categories is within acceptable RLM RSB limit It appears that the operating test will differentiate between competent and less-than-competent R-J RSS applicants at the designated license leve. WALK-THROUGH (CATEGORY A & B) CRITERIA
- Each JPM includes the following, as applicable:
RSS initial conditions I Initiating cues
- references and tools, including associated procedures reasonable and validated time limits (average time allowed for completion) and specific designation if deemed to be time critical by the facility licensee
- specific performance criteria that include:
- detailed expected actions with exact criteria and nomenclature
- system response and other examiner cues
- statements describing important observations to be made by the applicant
- criteria for successful completion of the task
- Identification of critical steps and their associated performance standards
- restrictions on the sequence of steps, if applicable The prescripted questions in Category A are predominantly open reference and meet the JR"'
criteria in Attachment 1 of ES-30 Repetition from operating tests used during the previous licensing examination is within RL-ASS acceptable limits (30% for the walk-through) and do not compromise test integrit At least 20 percent of the JPMs on each test are new or significantly modifie "'-'4 3. SIMULATOR (CATEGORY C) CRITERIA The associated simulator operating tests (scenario sets) have been reviewed in accordance with
+R"'
Form ES-301-4 and a copy is attache Printed Name / Signature Date a. Author Robert L. Monk/
A,,Y.e.-t
/a, z b. Facility Reviewer(*)
N/A c. NRC Chief Examiner (#)
Richard S. Baldwin/
-
/z-/-
t d. NRC Supervisor Michael E. Emstes/
r.-
6/0
"
NOTE:
- The facility signature Is not applicable for NRC-developed test # Independent NRC reviewer initial items in Column "c;" chief examiner concurrence required.
-n
ES-301 Simulator Scenario Quality Checklist Form ES-301-4 (R8, S)
Facility: Surry Power Station Date of Exam: 03/18-28/02 Scenario Numbers: 1 / 2 / Operating Test No.: 1 QUALITATIVE ATTRIBUTES Initials a
b*
c# The initial conditions are realistic, in that some equipment and/or instrumentation may be out RLM RSB of service, but it does not cue the operators into expected event.
The scenarios consist mostly of related event RLM RSB Each event description consists of RLM RSB the point in the scenario when it is to be Initiated the malfunction(s) that are entered to initiate the event the symptoms/cues that will be visible to the crew the expected operator actions (by shift position)
the event termination point (if applicable) No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario RLM RSB without a credible preceding incident such as a seismic even.
The events are valid with regard to physics and thermodynamic RLM RSS Sequencing and timing of events is reasonable, and allows the examination team to obtain RLM RSB complete evaluation results commensurate with the scenario objective.
If time compression techniques are used, the scenario summary clearly so indicate RLM RSB Operators have sufficient time to carry out expected activities without undue time constraint Cues are give.
The simulator modeling is not altere RLM RSB The scenarios have been validated. Any open simulator performance deficiencies have been RLM RSB evaluated to ensure that functional fidelity is maintained while running the planned scenario.
Every operator will be evaluated using at least one new or significantly modified scenario. All RLM RSB other scenarios have been altered in accordance with Section D.4 of ES-30.
All individual operator competencies can be evaluated, as verified using Form ES-301-6 RLM ASO (submit the form along with the simulator scenarios).
1 Each applicant will be significantly involved in the minimum number of transients and events RLM RSS specified on Form ES-301-5 (submit the form with the simulator scenarios).
1 The level of difficulty is appropriate to support licensing decisions for each crew positio RLM ASS TARGET QUANTITATIVE ATTRIBUTES (PER SCENARIO; SEE SECTION D.4.D)
Actual Attributes
..
.. Total malfunctions (5-8)
5/4/3 RLM RSS Malfunctions after EOP entry (1-2)
2/3/4 RLM RSB Abnormal events (2-4)
4/3/3 RLM RSB Major transients (1-2)
1/1/1 RLM ASS EOPs entered/requiring substantive actions (1-2)
1/1/2 RLM RSS EOP contingencies requiring substantive actions (0-2)
1/0/0 RLM RSB Critical tasks (2-3)
2/2/2 RLM RSB
ES-301 Transient and Event Checklist Form ES-301-5 (R8, Si)
Surrv Power Station OPERATING TEST NO.: 1 Ev._lution
- ype Reactivity Minimum Number Scenario Number ROl eaP
Ro2BoP IRo38P Io4o P
Normal
Instrument /
Component Major
S1 2
1 Reactivity
1 Normal
Instrument /
Component
13
I1
12
4
5 l 1,
2,3 i?
Major 61,0 i8 Reactivity
Normal
Instrument/
Component Major
89 Reactivity
Normal
SRO-U Instrument/
Component Major
Instructions:
Author:
(1)
Enter the operating test number and Form ES-D-1 event numbers for each evolution typ (2)
Reactivity manipulations may be conducted under normal or controlled abnormal conditions (refer to Section D.4.d) but must be significant per Section C.2.a of Appendix (3)
Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to the applicant's competence count toward the minimum requiremen Robert L. Monk, #1 & #2, R. Baldwin # 3 Richard S. Baldwin, Applicant I ype RO As RO SRO-[
As SRO
i, 3,4
13
a
1, 2,
3, S
7,
4 8,
,
__
NRC Reviewer:
Surry Power Station March 2002 Applicant #1 Applicant #2 Applicant #3 RO/SRO-I/SRO-U RO/SRO-I/SRO-U RO/SRO-I/SRO-U Competencies SCENARIO SCENARIO SCENARIO RO BOP
4 SRO SRO
4
2
2
2 Understand and Interpret 5?,6, 2,,3,7 1,3,7 4.2,,2,3 8,
8,9
,8 7,8,9, 56, Annunciators and Alarms
10 7,8 Diagnose Events
2,5,6,,7 1,3,7 425 123
8,9
.8 3,7,8, 5,6, and Conditions 9,
7,8
10 Understand Plant 4,5,6 2,5,6 213,7 1,3,6 412S, 1,2,3
,8,10
,8,9
.,8,9,
,,
and System Response
6,7,8 Comply W ith and 4,6 2,.,3,6 4,2,3, 1,,6
,7,8 7,8,9, 4,5, Use Procedures (1)
6,7,8 Operate Control 4.5,6 4,6 1., Boards (2)
Communicate and 4,5,6 2,.2,3 1,3,6 23,4 123 C9,10
.7.8,
5,7,8:
,45 Interact With the Crew
9, 6.7,8
Demonstrate Supervisory 512,3 8,, Ability (3)----
-
,
Comply With and d,6
Use Tech. Specs. (3)
Notes:
(1) Includes Technical Specification compliance for an R (2) Optional for an SRO- (3) Only applicable to SRO Instructions:
Circle the applicant's license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applican Author:
Robert L. Monk NRC Reviewer:
Richard S. Baldwin Competencies Checklist Form ES-301-6 (R8, $1)
Competencies Checklist Surry Power Station March 2002 Applicant #1 Applicant #2 Applicant #3 RO/SRO-I/SRO-U RO/SRO-I/SRO-U RO/SRO-I/SRO-U Competencies SCENARIO SCENARIO SCENARIO Ro SOP
4 BoP Ro
4 sRo sRO
4
2
2
2 Understand and Interpret 57,6, 2.5,6 2,3,7 1,3,7 4,213, 1,2,3,9
,
7,8,9, 5.6, Annunciators and Alarms
10 7,8 Diagnose Events.5,6 2,3,7 137 4,2.5, 1,23
'10
8 3,7,8, 5,6, and Conditions 9,
7,8
Understand Plant 4,.5,6 2,3,7 1,3,6 4,2,,2,3
,8,10
,8,9
,7,8 3,8,9,
,4,3, and System Response
678 Comply With and 4,6 245 137 1,36 42,3 1,2,3 678
45 Use Procedures (1)
6,7,8 Operate Control 4,5,6 4,6 123 1,36
,7,10
,9 Boards (2)
Communicate and 4,.5,6 1,2,3 136 234,,9, 10
,7.8,
5,7,8, A45, Interact With the Crew
9 678
Demonstrate Supervisory
1,2,3 p ry 4,5,7, 45, Ability (3)
v!;,
-
,
T Comply With and 6:
Use Tech. Specs. (3)
L Notes:
(1) Includes Technical Specification compliance for an R (2) Optional for an SRO- (3) Only applicable to SRO Instructions:
Circle the applicant's license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applican Author:
Robert L. Monk NRC Reviewer:
Richard S. Baldwin ES-301 Form ES-301-6 (R8, $1)
Written Examination Quality Checklist Form ES-401-7 (R8, Si)
Facility: Surry Power Station Date of Exam:
March 2002 Exam Level: SRO Initial Item Description a
b-c& Questions and answers technically accurate and applicable to facility
/
! a. NRC K/As referenced for all questions
/ b. Facility learning oblectives referenced as available RO/SRO overlap is no more than 75 percent, and SRO questions are appropriate
$4 per Section D.2.d of ES-401 tot Question selection and duplication from the last two NRC licensing exams appears consistent with a systematic sampling process Question duplication from the license screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate:
- the audit exam was systematically and randomly developed; or
- the audit exam was completed before the license exam was started; or L the examinations were developed independently; or
- the licensee certifies that there is no duplication; or
- other (explain)
1 Bank use meets limits (no more than 75 Bank Modified New L
percent from the bank at least 10 percent new,
,,b and the rest modified); enter the actual question
4'n"
distribution at right u
1 Between 50 and 60 percent of the questions on Memory C/A the exam (including 10 new questions) are r.ILA written at the comprehension/analysis level enter the actual question distribution at right l
J References/handouts provided do not give away answers Question content conforms with specific K/A statements in the previously approved examination outline and is appropriate for the Tier to which they are assigned; deviations are justified 1 Question psychometric quality and format meet ES, Appendix B, guidelines,.
1 The exam contains 100, one-point, multiple choice items; the total is correct and agrees with value on cover sheet Printed Name / Signatu re{)7 Date a. Author Robert L. Monk/Cz're-rad W. Laska
.,A.10 b. Facility Reviewer ()
N/A J
c. NRC Chief Examiner (#) M6 Richard S. Baldwin,-
3T- ____
d. NRC Regional Supervisor Michael E. Ernstes"
/4*,,-*ý f-41/o Note:
- The facility reviewer's initials/signature are not applicable for NRC-developed examination # Independent NRC reviewer initial items in Column "c;" chief examiner concurrence required.
Written Examination Quality Checklist Form ES-401-7 (R8, Si)
Facility: Surry Power Station Date of Exam: March 2002 Exam Level: RO Initial Item Description a
b-.
Questions and answers technically accurate and applicable to facility
&
&ý a. NRC K/As referenced for all questions I'V b. Facility learning oblectives referenced as available RO/SRO overlap is no more than 75 percent, and SRO questions are appropriate 'AL per Section D.2.d of ES-401 Question selection and duplication from the last two NRC licensing exams M
appears consistent with a systematic sampling process
,_ Question duplication from the license screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate:
- the audit exam was systematically and randomly developed; or
- the audit exam was completed before the license exam was started; or Dr X the examinations were developed independently; or
- the licensee certifies that there is no duplication; or other (explain) Bank use meets limits (no more than 75 Bank Modified New
/
percent from the bank at least 10 percent new, All and the rest modified); enter the actual question *)
,
i distribution at right 9I Between 50 and 60 percent of the questions on Memory C/A the exam (including 10 new questions) are written at the comprehension/analysis level; enter the actual question distribution at right References/handouts provided do not give away answers Question content conforms with specific K/A statements in the previously approved examination outline and is appropriate for the Tier to which they are assigned; deviations are justified 1 Question psychometric quality and format meet ES, Appendix B, guidelines I/I 1 The exam contains 100, one-point, multiple choice items; the total is correct and
agrees with value on cover sheet Printed Name / Signature I
Date a. Author Robert L. Monk/ /04ten e" rrad W. Laska e,,
b. Facility Reviewer (*)
N/A I
I I
c. NRC Chief Examiner (#)
Richard S. Baldwin 4A-e--,
d. NRC Regional Supervisor Michael E. Ernstes tic;
/
Note:
- The facility reviewer's initials/signature are not applicable for NRC-developed examination # Independent NRC reviewer initial items in Column "c;" chief examiner concurrence required.
(I K
(I ES-401 Written Examination Form ES-401-9 (R8, Si)
Review Worksheet.
3. Psychometric Flaws 4. Job Content Flaws 5. Other.
0#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- /
Back-0= SRO U/E/S Explanation Focus Dis Link units ward K/A Only Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive leve.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 -4 range are acceptable). Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statement More than one distractor is not credibl One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements Qi.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirement.
Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). Based on the reviewer's judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met)..
3. Psychometric Flaws 4. Job Content Flaws 5. Other.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- /
Back-0= SRO U/_E./S Explanation Focus Dis Link units I ward K/A Only RO/SRO COMBINED QUESTIONS
M 1-2 S
O01AA1.01, Modified, RLM States modified, no bank question attache Agree with Mike it is easy, will look at rest of test to determine what if any changes need to be made to this questio February 27, 2002 ADD STEADY STATE CONDITIONS.
Page 1 of 50
3. Psychometric Flaws Stem Focus Cues I T/F Cre Dist.
Partial (
4. Job Content Flaws Job Link Minutia
- /
Back-Q=
units ward KI A-k SRO Only U/E/S Explanation
H 2-3 S
001 K3.01, New RLM, Appears o /27/02 No facility comment
H
S 002K6.03, New, RLM Appears o /27/02 FACILITY COMMENTS, AT LEAST A 7 ON THE SCALE OF DIFFICULT WANT THE QUESTION REPLACE ASK MIKE 3/08/02 Replaced question with facility developed question
M
S 003A3.02, Modified, RLM Easy as Mike stated Appears ok, will review rest of test to determine if change is necessar /27/02 NO FACILITY COMMENT
F
S 003AK2.05, New, RLM Appears o /27/02 MEMORIZATION OF A NOTE, OUTSIDE OF RO KNOWLEDGE. ONL A HANDFUL OF PEOPLE KNOW THIS ANSWER. THE NORMAL PWF SUPPLY AND BACKUP ARE NOT REDUNDAN LOWER LEVEL ARP, NOT A NORMALLY RESPONDED TO AR WANT IT REPLACE /11/02 Replaced with facility generated question
F 2-3 S
004K2.04, Bank, GWL Appears o NO COMMENT Page 2 of 50 1.
LOK (F/H) LOD (1-5)
(
- Other
3. Psychometric Flaws Stem Focus Cues T/F Cre Dist.
Partial (
4. Job Content Flaws Job Link Minutia
- /
Back units ward
- Other 0=
SRO K/A Only 6.
U/EFS (
Explanation
F 2-3 S
005AK1.02, Bank, RLM Change stem to state WOOTF combinations of parameters will be.....
The stem was changed. Have BOB look at this to see if it is o Discus January 30, 2002 OK as change /27/02 UNDERLINED ADVERSELY. OK
H
S 006K5.10, New, RLM Appears o /27/02 added RCS penetration at end of distractor Ok
H
E 007G2.4.47, New, RLM Do we need to state in the stem what procedure we are in? ES-This may ensure we get the correct answe Discuss with Bob January 30, 2002, Added to the stem OK as i No Facility COMMENT
H 2-3 E
011EK3.03, New, RLM
- "**
A lot of window dressing. Does the applicant require all this information to answer the question. I am not sure he doe Discus February 1, 2002 Changed the stem to remove superfluous informatio S OK as change NO FACILITY COMMENT Page 3 of 50 I LOK (F/H) LOD (1-5)
3. Psychometric Flaws Stem Focus Cues I T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
- ,
Back-0=
units ward K/A 4'
SRO Onlv 6.
U/EFS S
(
7.
Explanation 011K3.02, Modified, RLM Original question not available with packag Could clarify the stem to let them know you are talking about Pressurzer pressure. Don Let Bob see how the stem was change January 30, 2002 OK as changed, GWL reviewe TYPOS MADE THIS A UNIT 2 QUESTION BY ADDING 2
H
E 013G2.4.4, Modified, GWL
Do we need to define what lowering rapidly means? Can we put a rate in here? The same goes for RCS pressure lowering. In the past we have had problems not providing this type of information. Discuss with Gerr Otherwise appears o February 1, 2002 Will have to see what the facility has to say. Would the plant be in these conditions. Are all the data points correct for the position we have. IE is this the correct PZR level for the condition NO FACILITY COMMENT H
S 014K3.02, New, GWL Appears o /27/02 NO CORRECT ANSWE Re-worded the stem and to make it clearer. Made the question concern B rods, see the ste ok as change H
S 015AK2.10, Bank GWL Appears o No Facility COMMENT Page 4 of 50
- Other Q#
11 LOK (F/H)
H LOD (1-5)
3. Psychometric Flaws Stem Focus Cues I T/F Cre Dist.
Partial (
4. Job Content Flaws Job Link Minutia Other
- /
Back-Q=
SRO units ward SRK/A Only U/E/S ( Explanation
H
S 015K4.08, Bank, GWL Appears o NO Facility COMMENT
H
E 017A3.01, Bank, GWL Distractor B is the only one that provides a reason why the generators have boiled dry. Is it necessary to have that in the answer? Probably can remove that part and still be a valid answe Not sure distractor D is valid, Discuss with Gerr January 30, 2002 Changed B to remove the feed water los OK as is, no need to change this question 2/27/02 Steam Tables, yes will provide steam table NO FACILITY COMMEN F
S 017K1.01, New, GWL
- " No answer is designated, it looks like C should be the answer but need to discuss with the facilit Appears o January 30, 2002 Need to ask the facility 2/27/02 OVER AND ABOVE THE REQUIREMENTS OF AN OPERATOR, THIS WOULD BE DONE BY THE IC TECH KEN WILL REWRITE THIS QUESTIO /08/02 Facility rewrote question Page 5 of 50 1.
LOK (F/H) LOD (1-5)
,i Q#
4. Job Content Flaws j.. Other Stem Focus Cues I T/F Cre Dist.
Partial Job Link Minutia units Back ward Q=
K/A SRO Only 6.
U/E/S S
7.
Explanation 022K4.05, New, GWL Changed the stem to add "directly" so that service water could not be an answe Discuss with Gerry.
January 30, 2002 OK as change /27/02 COULD JUSTIFY TWO ANSWERS, IF YOU ASSUMED THE INITIAL OR FINAL PHASES.
Added words to the stem. Ok
F
S 025AK1.01, Bank, RLM Appears ok. Changed the stem to add sets of parameters because there are set January 30, 2002 OK NO FACILITY COMMENT
H
X U?
024AK1.04, New, GWL OK I would not have selected a or d because of the low temperature. Is there some significance of the 45 degrees? I think these need to be replace Discuss with Gerr January 30, 2002 S
45 deg F. comes from the RWST temperature requirement OK as i NO FACILITY COMMENT Page 6 of 50 LOK (F/H)
F LOD (1-5)
2-3
3. Psychometric Flaws
3. Psychometric Flaws Stem Cues Focus T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
-. Other
- /
Back-0=
SRO units ward K/A Only 6.
U/E/S 7.
Explanation
F
E?
026A1.01, Bank, RLM Mikes question was never answered. I also think that capacity should be capability. Discuss with RLM January 30, 2002 S
Added ability to the stem vice the other suggestion OK as changed 2/27/02 UNDERLINED NO H
S 027AK3.01, New, GWL Appears ok. The attached reference material does not reflect this answer. The justifications of non correct answers stating they are not the correct answer is not a good justificatio Discuss with GWL Disagree with LOK determination. Not sure this is a high level H more a high level M January 30, 2002 Understand it is a Higher level questio No changes necessar /27/02 NO FACILITY COMMEN '23 H
S 028AK2.03, Modified, GWL Appears ok 2/27/02 MADE IT A UNIT 2 OK AS CHANGED.
Page 7 of 50 LOD (1-5) LOK (F/H)
3. Psychometric Flaws Stem Focus Cues] T/F Cred. Partial Job Dist. I ILink 4. Job Content Flaws Minutia units Back ward
- Other Q=
K/A SRO Only 6.
U/E/S S
7.
Explanation 029A3.01, NEW, RLM Not sure why anyone would pick C. Is this a misconception of the operators? We may need to seek a different distractor. Discuss with RLM. Is there another Red Monitor that can be substituted?
Not sure I agree with LOK, could be a high level Memor Otherwise appears ok.
February 1, 2002 February 5, 2002 Distractor C was changed to be a red monito Change is ok 2/27/02 CHANGED MODE 6 TO REFUELING SHUTDOW NO OTHER COMMENT
F 2-3 E
032AK2.01, Modified, GWL This question was modified such that the stem did not ask the reason for OK the answer. Since the question would have two answers (B&C) if the reasons were removed. Leave the reasons in and then add back to the S
stem the basis or reason or why it is correc Discuss with GWL January 30, 2002 OK as is, no change necessar NO FACILITY COMMEN H
S 033K1.05, New, RLM Made minor changes to stem and fixed wording error Discuss with Bob January 30, 2002 Discussed with GWL 2/27/02 CHANGED ONE OF THE BULLETS TO REWORD WHAT WAS ACTUALLY HAPPENIN OK AS CHANGED.
Page 8 of 50 F.
24 LOK (F/H)
H LOD (1-5)
I.
3. Psychometric Flaws 4. Job Content Flaws
.. Other.
O#
LOK LOD rr I
i I
(F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- /
Back-0=
SRO U/E/S Explanation
____Focus Dis Link units ward K/A On y
F 2-3 E
034G2.1.27, New, RLM This question appears to be ok but I am not sure they stem talks about the system the question is about. Is there a chance that there is another system that has the same switch? Would like to add to the stem something like:
WOOTF represents the purpose of the MCPLS in the fuel handling system?
S Discuss with BOB January 30, 2002 Discusses with GWL and added the recommended words to the ste OK as change /27/02 HAVE CHANGES TO QUESTION. DO NOT HAVE OPERATORS AS FUEL HANDLERS. NOT SURE THE APPLICABILITY TO THE OPERATORS. NOT REQUIRED TO HAVE LICENSED OPERATORS ON THE FUEL TEA AS WRITTEN 2 CORRECT ANSWERS. Answer D is also correct, there is more than one micro switc Changed distractor D. See exam for change OK as change H
S 035A1.01, New, RLM Made some minor changes, Appears ok January 30, 2002 Need to add the level transmitter and re arrange the stem initial condition February 5, 2002 S
Changed from channel 3 to the actual level transmitter. OK as change /27/02 Added 1 FW to mark numbers Added Channel 3, OK as changed.
Page 9 of 50
3. Psychometric Flaws Stem Focus Cues I T/F Cre Dist.
Partial 4.
Job Link Job Content Flaws I " Other Minutia units Back ward Q=
K/A SRO Only U/E/S S Explanation H
H
S 039A4.04, Bank, RLM Underlined sixty Also added the designation for the turbine driven afw pum Appears o /27/02 NO FACILITY COMMENT Page 10 of 50 1.
LOK (F/H)
3 LOD (1-5)
038EK3.02. New, RLM Added the noun name for E-3 and underlined the word NEXT to ensure answer d was not considered a plausible answe January 30, 2002 Ok as change /27/02 PORVS ARE SET AT 1035, NOW WE ARE TELLING THEM AT 102 IF YOU LOOK AT INSTRUMENT AIR DOING NOTHING IS AN ANSWE THERE IS NO CORRECT ANSWER AS IT SIT CHANGED 1020 TO 980, NOW NEED TO TAKE AN ACTIO /8/02 Added parenthesis statement that all controllers are at the proper setpoint and changed to a unit 2 questio.
3. Psychometric Flaws 4. Job Content Flaws
- . Other.
(F/H)
(1-5)
Stem CuesI T/F Cred. Partial Job-Minutia
- /
Back-Q=
SRO U/E/S Explanation Focus Dis Link units ward K/A Onlyl
F 2-3 X
U 040AK2.02, Bank, RLM Have a question about the plausibility of distractors A and C, since they
- k~
do not have remote indications any one that has this knowledge will be able to eliminate that particular distractor. May need to replace thes Why would D be wrong. Looks like it could be o Defer to Bob February 1, 2002 Distractor D is ok, no procedure to go to that would allow thi Going to look at changing either A or February 5, 2002 S
Changed distractor A, Change is ok 2/27/02 WANT TO CHANGE DISTRACTOR D. To outside service building, replace turbine buildin OK as change H
S 041K5.07, New, RLM Changed the stem to ask more of what the distractors were looking fo Discuss with Bob Otherwise appears o January 30, 2002 Discussed with GWL Changes ok 2/27/02 stm dmrps in steam pressure mode, auto is not functiona Operate it in manual only. If the transmitter failed would not have an effec /08/02 Change stem to indicate that operator made steam flow changes vice controller failure, acceptable to the facility.
Page I Iof 50
/
1.
LOK (F/H)
M LOD (1-5)
2-3 34 1 M 1 2-3 3. Psychometric Flaws Stem Focus Cues I T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
- Other
- /
Back-C=
SRO units ward K/A Onlv U/E/S S Explanation 045A3.11, Bank, RLM Should we change distractor C to say 3 seconds rather than a short delay? Ask Bob Underlined immediately in answer b Otherwise appears ok January 30, 2002 Ask Bob February 1, 2002 Changed distractor C to add a 3 second time delay to make it look like OK as change /27/02 UNIT I REPLACED LAST FALL, UNIT 2 WILL BE REPLACED NEXT MONTH. OPERATORS HAVE NOT BEEN TRAINED ON THI FACILITY WILL COME UP WITH A NEW QUESTION. BASED ON NE" MATERIA /11/02 Facility submitted new question 051AK3.01, Bank, RLM No answer is marked, looks like it should be A?
Appears o January 30, 2002 Need to look at distractor A again, it may be a correct answer als Discuss with Bob February 1, 2002 We determined this is ok. NO changes are require NO FACILITY COMMENT Page 12 of 50
L# 1. Psychometric Flaws 4. Job Content Flaws Other.
(F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- /
Bac U/E/S Explanation Focus Dis Link units ward KiA On_,
H
X E
054AA1.02, New, RLM Need the noun name for GOP-1.5. Done ok Do we need to add to the stem that the crew is in E-0. The stem sort of S
leads you to think you are in Ap-21. Discuss with Bob January 30, 2002 Added to the stem E-0 OK as changed 2/27/02 WANT TO MODIFY THIS QUESTION DELETED THE LAST BULLET BECAUSE THE PROCEDURE WE HAD WOULD NOT HAVE BEEN ENTERED BECAUSE OF THE RX TRI OK AS CHANGE F
?
OOOAA2.01, Be iA. RflL1 1t l
9aeettci h
RE T3... ?
.......
..
S February 1, 2002 055EK3.02, Bank, RLM The sheet had two questions on it, the correct one was below the first questio The question is ok as change NO FACILITY COMMENT.
Page 13 of 50
K PsychometricFlaws 4. Job Content Flaws Other.
LO#
LOD Pril Mntauiswr (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- /
Back-Q=
SRO U/F/S Explanation Focus Dis Link units ward K/A Only 1
H
X E
055G2.1.28, New, RLM Made minor typo change The stem of the question asks for the operation and purpose of the Steam Jet Air Ejector system. The question is really asking about one component, I think this could be clarified bette Based on the initial conditions WOOTF describes the operation and purpose of TSV-SV-1 -1 02A?
Stem focus S
January 30, 2002 Changed the stem and the distractors to be cleare Discussed with GW OK as change /27/02 DELETED THE WORD THROTTLE IN THE STE OK AS CHANGE H 2-3 S
056AK3.01, New, RLM Changed the stem slightly, More of a memory level question than a higher level questio Discuss changes with Bo January 30, 2002 Discussed with GWL ok as a base line Higher level questio OK as changed 2/27/02 MORE THAN ONE ANSWER. D IS ALSO A CORRECT ANSWE Changed the stem and distractors B and OK as change H
S 056G2.1.32, New, RLM Appears o No FaCILITY COMMENT Page 14 of 50
(.
3. Psychometric Flaws 4. Job Content Flaws
.. Other.
Q# LOK LODII I
(F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- /
Back-Q=
SRO U/E/S Explanation Focus Dis Link units ward K/A Ony
F 2-3 S
057AA1.06, New, RLM The reference material provided does not provide answer to this questio OK as i /27/02 DO NOT EXPECT THE OPERATORS TO KNOW THI DO EXPECT THEM TO KNOW IMPLICATIONS OF LOSS OF POWER TO RACKS WANT IT REPLACED WANTS TO ADD INFORMATION "TO A LOSS RESULTING IN LOSS OF POWER TO THE CONTROL RACK AND THE MCR CONTROL STATIO THE VALVE SHIFTS TO AUTO HOLD AND MANUAL CONTROL IS LOST - FOR D?
WILL ASK MIK /08/02 Rewrote question
F 2-3 E
059AK3.03, New, RLM
- i Distractor C does not make sense to me. Discuss with Bob Look to replace distractor UPON further review it appears that Bob took the wrong path and went to the RNO. In order to go to the RNO you had to have a low failur Discuss with Bob February 1, 2002 Seems as though the ARP 0-RM-P5, does not seem to work correctl What do you do on a failure as we do in this question. Does not seem to address the detector failur /27/02 NO FACILITY COMMENT Page 15 of 50
3. Psychometric Flaws Stem Focus Cues I T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
- /
Back units ward
.. Other Q=
SRO K/A Only U/E-S S Explanation H
Page 16 of 50 1.
LOK (F/H)
3 LOD (1-5)
059G2.4.31, New, RLM Changed the stem a little, have Bob look at i Do you expect the operators to know this from memory?? It is a stem in an ARP maybe beyond their scope of knowledg Appears ok January 30, 2002 Why would steam flow be an indication of an electric pump trip?
February 1, 2002 Changed steam flow to feed flo Yes, Bob thinks this is required knowledg OK as change Ask licensee if this is RO/SRO memory knowledge. ***
2/27/02 NO FACILITY COMMENT
,1
3. Psychometric Flaws Stem Focus Cues I T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
- Other
- /
Back-Q= 1SRO units ward I K/A Only U/E/S Explanation F
3/08/02 Facility rewrote question
H U
061 K2.02, Modified, RLM This is a repeat question, very similar to Common Number 3 Discuss with Bob. One of them needs to be replace February 5, 2002 RLM made a modified question, changed a H to a F level S
Question is ok as propose NO FACILITY COMMENT.
Page 17 of 50
/
1.
LOK (F/H)
2-3 LOD (1-5)
E?
/
060AK2.02, Bank, RLM Is this something the applicants should know from memory??
Minor typo change The answer is obviously correct when the procedure is in front of yo But distractor B looks to be the next thing that needs to be done. Do we need to clarify the stem to add ie. First or Next to make sure there is no doubt on what should be done?
Discuss with Bo January 30, 2002 Changed the stem to read more clearl New Bob's inpu February 1, 2002 Changed stem and changed distractor /27/02 WHAT IS THE FIRST ACTION AFTER THE IMMEDIATE ACTIONS IN AN AP. SAY IT IS NOT REQUIRED KNOW FROM MEMOR DISCUSS WITH MIKE
(.
3. Psychometric Flaws 4. Job Content Flaws
. Other.
0#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- /
Back- 0=
SRO U/E/S Explanation Focus Dis Link units ward K/A Only
S 063A2.01, NEW This question on the header indicates it is a common question, need to identify what test it is supposed to be o While the question indicates there are problems with grounds the answers deal with TS requirements associated with the loss of power. I am not sure this meets the intent of the KA. Discuss. OK not a problem Why is distractor D incorrect, it is not a fully correct answer but the valve has to be closed prior to the power being removed. May need to change the stem to illicit the TOTAL answe Not an answer. OK as is no changes necessar January 25, 2002 2/27/02 GETTING DOWN TO THE SOLENOIDS TO THE PORVS. YE DO NOT BELIEVE THERE IS ENOUGH INFORMATION TO DETERMINE THE PORV IS INOPERABL IN ADDITION DO NOT THINK THIS IS RO OR SRO KNOWLEDGE LEVE THIS IS NOT RESOLVED NEED TO EITHER REPLACE OR REWRIT /08/02 Facility rewrote Page 18 of 50
Off Stem Focus Cues T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
,. Other
- /I Back-Q=
units I ward IK/A SRO Only U/E/S S Explanation 062AA1.05, New, RL Is the correct terminology CCW or is it just CW? Need to determine thi Otherwise appears o January 30, 2002 Discussed with GWL and removed the Water from the C /27/02 SHOULD HAVE DEBORATING DEMINS IN. HEAT UP WILL CAUSE INCREASE IN BORO Changed the stem ok as changed.
F
S 063A4.02, Bank, GWL Appears ok 2/27/02 DEBATING VALIDITY OF THE QUESTION. TWO INDICATIONS ONE POSITIVE, OR NEGATIVE, IF A GROUND OCCURS IT GOES TO A BRIGHT LIGH HAVE A POSITIVE AND NEGATIVE LIGH NRC WILL WORK ON THI /08/02 GWL rewrote Page 19 of 50 3. Psychometric Flaws 46 LOD (1-5) LOK (F/H)
H
3. Psychometric Flaws Stem Focus Cues T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
- /
Back units ward
ý. Other Q=
SRO K/A Only U/E/S K Explanation
PROCEDURES.
NEED TO DISCUSS WITH MIKE 3/11/02 Facility rewrote question
F 2-3 S
067AK1.02, Bank, GWL Appears ok 2/27/02 CHANGED THE STEM. SEE EXAM O F 2-3 S
068AK2.03, Bank, GWL Appears o /27/02 A AND B ARE CORRECT ANSWER NOT SURE WHAT THE FIX IS, NEED TO GET RID OF A. BANK QUESTION IS IN ERRO /08/02 Modified answer A to be incorrect.
Page 20 of 50 LOK (F/H)
H LOD (1-5)
064A2.06, New, GWL Need to determine what needs to be done in accordance with Procedure XXX. Has to be added to the ste Also, the requirement only comes about when shutting down the D/ Need to add to the stem the DG is going to be shut down and what are the requirement Discuss with Gerry January 30, 2002 Changed the stem, see question to see the change /27/02 THIS INFORMATION COMES OUT OF A OP. DON'T EXPECT THE OPERATORS TO KNOW THE TIME LIMIT FROM MEMOR WANT THE OPERATORS TO KNOW BASIS NOT NUMBERS IN THE
3. Psychometric Flaws Stem Focus Cuesl T/F Cre Dist.
Partial (
4. Job Content Flaws Job Link Minutia units Back ward
,. Other C=
K/A SRO Only 6.
U/E/S (
7.
Explanation
H
S 068K6.10, Bank, GWL What is RS, do we need to spell it ou Is this a question that an RO is supposed to know?
Otherwise appears o January 30, 2002 Discussed with GWL No changes are necessary, will see if the licensee has a problem with i /28/02 NO FACILITY COMMEN H
S 071A4.07, New, RLM Appears o /28/02 VERY HIGH LEVEL QUESTION, GOOD ONE BECAUSE IT TESTS A LOT OF KNOWLEDGE NO FACILITY COMMENT, THINK IT IS A 5 LO H 3-4 E/U 072A2.02, New, RLM Is this something an RO or an SRO will have to know from memory? I
don't think s Discuss with BOB If it is appears o February 1, 2002 Ask Licensee if it is ok to test both. ***
2/28/02 BOTH ANSWER B AND C MAKE THE FAIL ALARM COME O VERIFY IF THE DETECTOR OPERATES PROPERLY AND WRITE A WORK REQUEST. THIS IS ALL THEY ARE REQUIRED TO D OPERATORS ARE NOT REQUIRED TO KNOW THIS LEVEL OF DETAIL. PROCEDURE REQUIRES JUST A WORK REQUEST TO BE GENERATE CHANGED DISTRACTOR C. OK AS CHANGED.
Page 21 of 50
/ LOK (F/H) LOD (1-5)
3. Psychometric Flaws Stem Focus CuesI T/F Cre Dist.
Partial (
4. Job Content Flaws Job Link Minutia
- /
Back units ward Other Q=
SRO K/A Only U/E/S ( Explanation
H
S 072K5.02, New, RLM Did some word smithing, appears ok. Have Bob look at the change January 30, 2002 ok as is 2/28/02 NO FACILITY COMMENT, TOUGH ON F
S 073G2.1.32, New, RLM This is a backward logic question, these are not supposed to be use Since there are not that many I think it is o However, I think the utility will have a problem with the requirement to know this information. It may be an SRO knowledge requirement but for an RO? We will see what they have to sa January 30, 2002 Will see what Surry has to say!
2/28/02 NO FACILITY COMMENT
F
S 075K1.08, New, RLM Will have to ask the licensee if the wording for the answer is correc Made some minor editorial changes to the questio Bob look a Otherwise appears ok January 30, 2002 Check at facility CHANGED ANSWER D TO MAKE IT MORE CORRECT changes are ok.
Page 22 of 50 C)#
1.
LOK (F/H) LOD (1-5).
3. Psychometric Flaws 4. Job Content Flaws
.. Other.
O#
LOK LOD I
(F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- /
Back-Q=
SRO U/E/S Explanation I Focusj Dis Link units ward K/A Onlyn1
F
S?
076AA1.04, Modified, GWL Changed the wording a little. Have Gerry look a Also, could we change the High Radiation Alarm to something like when detecting high radiation?
Discuss with Gerr Otherwise appears o January 30, 2002 Changes appear to be o /28/02 NO FACILITY COMMEN H
S?
079G2.4.49, Bank, RLM I am not convinced that there is not more than one answe AP -40 requires you to trip the reactor so entry into AP 40 performs the
trip the reactor par What would cause you to trip the reactor prior to getting into AP 40?
Discuss with RLM or license January 30, 2002 There is no correct answer based on how we think the procedures wor Discuss with Bob S
February 1, 2002 Changed all the distractors. OK as change /28/02 MINOR WORD CHANGES OK 3/8/02 Enhanced answer Page 23 of 50 I
3. Psychometric Flaws Stem Focus Cues T/F Cre Dist.
Partial 4. Job Content Flaws Job-Minutia
- /
Back Link I units I ward
- Other 0=
K/A SRO Onlv 6.
U/E/S (
7.
Explanation
F
S G2.1.1, Bank, GWL Wording could be better but appears to be ok as i January 30, 2002 With GWL reworded the stem to make it cleare OK as changed 2/28/02 MINOR WORD CHANGE, OK
H
S G2.1.12, Modified, GWL Changed the stem a little, Appears to be o Is this allowable RO knowledge level?
January 30, 2002 GWL thinks this is RO knowiedge OK as is 2/28/02 LESSON PLAN WAS NOT UP DATED TO A RECENT CHANGE TO TECH SPECS. HAVE TWO CORRECT ANSWER ON THE COMMON NOT A FAIR QUESTION FOR AN RO. NEED TO LOOK TO HAVE AN SRO ONLY QUESTION. *...
ANSWER D IS CURRENTLY CORRECT BUT FOR AN OLD REVISION TO TSS. WANTED TO CHANGE ANOTHER INCORRECT ANSWER WITH THE NEW CORRECT ANSWER. FACILITY STATES IT WOULD BE UNFAIR TO DO SO. NEED TO DISCUSS WITH MIKE 3/08/02 Left the "Old" correct as a distractor and added new correct answer per a recent Tech Spec change. Moved this question to the SRO only test and exchanged with G2.1.4 from the SRO only test.
Page 24 of 50 aQt 1.
LOK (F/H) LOD (1-5)
3. Psychometric Flaws Stem Focus Cues T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
- /
Back units ward
- Other 0=
SRO K/A Onlv 6.
U/E/S (
7.
Explanation
F 2-3 S
G2.2.12, New, RLM The question is ok, it appears to be pretty basi January 30, 2002 2128/02 DISTRACTOR 0 IS NOT INCORRECT CHANGED D TO MAKE IT INCORREC OK AS CHANGE M
S G2.2.13, Bank, RLM appears to be o January 30, 2002 2/28/02 NO FACILITY COMMENT F
S G2.2.30, New, RLM Appears to be o Another NOT questio January 30, 2002 2128/02 HAVE SOME CONFLICTING STUFF. TS REQUIRES CONTAINMENT TO BE SE CHANGED THE QUESTION TO NOT BE A NOT QUESTIO CHANGED DISTRACTORS C AND A AND THE STE OK AS CHANGED.
Page 25 of 50
/
1.
[OK (F/H) LOD (1-5)
(
3. Psychometric Flaws Stem Cues Focus T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
- /
Back units ward Other Q=
SRO K/A Only U/E/S E
S Explanation F
THAN ASK MIKE 3/11/02 Replaced question
H
S G2.3.2, Modified, GWL Word smithed Appears o January 30, 2002 GWL agrees 2/28/02 ADDED 20 FOOT, TO MAKE IT A VALID LINE SOURCE, OK AS CHANGED.
Page 26 of 50
/ LOK (F/H)
3 LOD (1-5)
G2.3.1, New, RLM The question is ok, however, could change the numbers to not be the same as the example in the lesson pla Discuss with Bob January 30, 2002 Changed the numbers see the questio OK as changed 2/28/02 TOUGH QUESTION LEVEL OF DETAIL AND THE JOB LINK DO NOT THINK THIS IS A TOPIC THAT WOULD DIFFERENTIATE BETWEEN A COMPETENT OR NON COMPETENT OPERATOR HAVE NEVER DONE IT HERE, FROM MEMORY THINK IT IS UNFAI THIS WOULD BE DONE BY HP AND THEY, THE OPERATORS WOULD BE WALKED THRU I CFR 55.45(10) REQUIRES THE OPERATORS ON THE OPERATING TEST TO BE DEMONSTRATE KNOWLEDGE OF SIGNIFICANT RADIATION HAZARDS, INCLUDING PERMISSIBLE LEVELS IN EXCESS OF THOSE AUTHORIZED......
FACILITY STATES THE KA IS 2.6 WHICH IS MARGINALLY HIGHER
3. Psychometric Flaws Stem Cues Focus T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
- /
Back units ward
,. Other Q=
K/A SRO Only 6.
U/E/S 7.
Explanation
F
S G2.3.4, Modified, GWL Appears o /28/02 CHANGE IN TERMINOLOGY OK AS CHANGE F 2-3 S
G2.4.39, Bank, GWL Can not tell what is the correct answer from material provide The question is not modifie Appears o January 30, 2002 2/28/02 CHANGED D TO MAKE IT MORE INCORREC OK AS CHANGE F
S G2.4.5, Bank, GWL Minor word smithing Appears ok January 30, 2002 2/28/02 CHANGED D TO MAKE IT MORE CORRECT AND CHANGED C TO MATCH THE LOOK OF OK AS CHANGE F
S G2.4.14, New, GWL Minor Word smithing Appears O January 30, 2002 2/28/02 REMOVED THE WORD OF IN THE STE OK AS CHANGED Page 27 of 50 0# LOK (F/H) LOD (1-5)
f
3. Psychometric Flaws Stem Focus Cues T/F Cred. Partial Dist.I 4. Job Content Flaws Job Link Minutia units Back ward
- Other 0=
K/A SRO Onlv 6.
U/E/S (
7.
Explanation
H
S G2.4.20, Modified, GWL Minor Word smithing Is this something an RO is required to know from memory??
Appears o January 30, 2002 Have facility determine RO applicability. **
2/28/02 NO FACILITY COMMENT
F
E G2.4.12, New, RLM Minor word smithin I do not understand the answer. I read the attached material but need this explained to m Distractor C on the hard copy is different from the one in LXR, which is correct?
Discuss with Bo January 30, 2002 Need to ask Bob and licensee, is this pay grade?
February 1, 2002 S
Changed the distractors to be equal to 7 or Appears o /02 NO FACILITY COMMENT.
Page 28 of 50 I LOK (F/H) LOD (1-5)
(
L1-.-1. Psychometric Flaws 4. Job Content Flaws
.*.Other.
Q#
LOD (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- /
Back-0=
SRO U/E/S Explanation Focus Dis Link units ward K/A Onlyl
H
E WE09G2.4.4, Modified, GWL Minor Word Smithing Two of the distractors have reasons why the action should be done and two do not. (A and C) They should be the sam Distractor B states "INITIATE" this procedure, should it say "Transition to"?
Should we use a RVLIS value in the stem, rather than just saying it is <
78%?
Do we need to put a rate at which pressurizer level is changing?
Is this RO knowledge level?
Discuss with Gerr January 30, 2002 Changed A and C, Changed B Need to see what RVLIS level points are. **
Need to check if this is RO knowledge. **
2/28/02 TOUGH QUESTION BUT CAN NOT FIND ANYTHING WRONG WITH I NO FACILITY COMMENT
H
S WE11EA1.1, New, GWL Minor word smithing Appears o January 30, 2002 ok as changed 2/28/02 CHANGED THE THIRD BULLET TO MAKE IT CLEARER AND ADDED THE STE OK AS CHANGED.
Page 29 of 50
3. Psychometric Flaws Stem Cues Focus T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia units
.. Other Back-Q= ISRO ward I K/A Only U/E/S S Explanation M
M S
WE16EK1.3, Bank, GWL Appears o /28/02 NEED TO CHANGE THIS IF WE GIVE THEM THE CSFT CHANGED THE STEM TO ADD AN, ALSO CHANGED C THE CORRECT ANSWER AND WILL ALLOW THE CSFTS TO BE USED ON THE EXA SAO ONLY QUESTIONS
H?
S 001AA2.03 The answer is a partially correct answer. You have to attempt to stop the rod motion before you trip the reactor. Is it necessary to add this to the answer?
This is a low CA level question at bes Answer:
Rods are in manual during start up. Question is O No change is necessary. January 25, 2002 Page 30 of 50 Q#
1.
LOK (F/H)
WE13G2.4.4, Modified, GWL Not modified, just rearrange The answer based on material should be B vice C. Discuss with Gerr Appears o January 30, 2002 GWL was correct, no change is necessar /28/02 NO FACILITY COMMENT, TOUGH QUESTION. TYPO IN DISTRACTOR OK AS CHANGED 3/08/02 Facility asked that question be changed due to a KA mismatch. The original question asked "exit" conditions. The KA is for "entry" conditions. The facility proposed a replacement question which is acceptable. LOD (1-5)
3. Psychometric Flaws Stem Focus Cues I T/F Cre Dist.
Partial (
4. Job Content Flaws I.,
Other Job Link Minutia units 0=
KIA SRO Onlv Back ward U/E/S S*
S Explanation 005A2.03 NEW Change the stem of the question to... have NOT been successful. I think it may read better. Capitalize NOT To ensure we expect the answer from AP-27, we need to change the question to In accordance with AP-27, Loss of Decay Heat Removal Capability," WOOTF is the correct course of action?
Changed the stem and the questio January 25, 2002 OK as changed This question is similar to the JPM so it was replace Facility Oked the replacement.
008A2.01 NEW Question is not necessarily an SRO only question. Not sure why an RO would not be required to be able to determine this information. Discuss OK as i January 25, 2002 Proposal on distractor D from facility Distractor D can be considered correc Will change distractor D to the proposed distractor Will also have to electrical change to the stem.
H
S 008AA2.19, Bank Should put the distractors in order of size. Place C as A and shift the first two down. This will change the answer to C but that should be O Swapped A and C and left B as the answe OK as is January 25, 2002 2127/02 Facility commen Add initial to the stem. OK Page 31 of 50 Q#
3 LOK (F/H)
H H LOD (1-5)
2
3. Psychometric Flaws Stem Focus Cues I T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
- /
Back units ward
.,. Other 0=
K/A SRO Only U/E/S S Explanation
1 ok as changed no further changes are necessary.
H
S 010A2.03, Bank Question appears to be o Mike had a comment that added to the stem, THE PRT remains intac Was this discussed and decided not to insert this?
Should not be necessar OK as i January 25, 2002
H 2-3 S
012A2.02, Bank, Reference material does not reflect why the answer is the one chose The question appears to be ok, could answer A also be a correct answer?
A is not an answer ok as i January 25, 2002 NEED TO LOOK AT THIS ONE TO SEE WHAT HAPPENED TO THE REPLACEMEN /08/02 Facility ok'd Page 32 of 50 1.
LOK (F/H)
H LOD (1-5)
009EA2.04 Bank Distractor C describes injection flow as ECCS flow however, distractor A only addresses injection flow. Should use the same terminolog Took out ECC Is D a viable answer? Would the procedure not have you isolate the accumulators during a small break LOCA? OK as is No further changes necessar January 25, 2002 2/27/02
'Concern that there are 2 correct answer Add to the stem a pressure that is greater than accumulator pressur This will ensure there is only one correct answer. Will add 750 psig to the ste. Psychometric Flaws Stem Focus X
Cues I T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
- /
Back units ward
,. Other Q=
KIA SRO Only U/E/S E
E S
(I Explanation 033G24.21, NEW The answer B is sort of confusing it starts out with a negative gamma like it was a negative startup rate. As far as I can tell those words are not necessary. Is this true?
In answer B removed the Decreasing and replaced it with trendin Added noun names to the procedur Ask Gerry if OK. ??
January 31, 2002 Changed the distractors and stem a little more Ok as changed.
026AA2.01, Bank Can the stem be changed to state for the last bullet, that the CC surge tank level has increase Other wise appears ok Changed to have the CCW surge tank trend u OK This question is the same/similar as 008A1.04 on the RO only exam need to change one or the othe February 1, 2002 Re-reviewed these and they are different enough to leave as is.
NO change necessary 10 1H 1 3 1 S 1029G2.4.21. Bank Appears to be OK Page 33 of 50 1.
LOK (F/H)
H H LOD (1-5)
3
9
3. Psychometric Flaws Stem Focus Cues I T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
- / IBack-QA units ward K/A
- Other SRO Only 6.
U/E/S
Explanation
H
E?
037AA2.10, Modified In the stem, you state in two of the bullets that they were previously evaluated as permissible, is this necessary or is it just window dressing and can be thrown out. Discus OK as is no changes necessar January 25, 2002 2/27/02 Facility requested to add TS in the stem of WOOTF. Agreed complete U 062A2.04, THIS IS A REPEAT OF 012A2.02 WHICH IS SRO NUMBER 7. ONE OF THESE NEEDS TO BE REPLACE NEED to look a February 5, 2002 S
New question developed, changed from memory to CA leve Question is ok as i /27/02 A & B are both correct as writte Will look at the proposed question.........
3/08/02 Enhanced distractors, facility satified Page 34 of 50 LOK (F/H) LOD (1-5)
(I 1 I-KO. Psychometric Flaws 4. Job Content Flaws
,*. Other.
(F/H)
(1-5) IStem Cues T/F Cred. Partial Job-Minutia
- /
Back-0=
SRO U/E/S Explanation IFocus J
Dis Link units ward K/A On_
H 3-4 S
065AA2.01, NEW Question appears to be ok. As stated in the question we still have some technical question about the way the system work Is it this question a SRO would be expected to know from memory?
Should we provide handouts? Discuss Look into giving them two ARPs to answer this questio January 25, 2002**......
2/27/02 Different pressure switches, have to re do this question. Also have to use Tubbine Building operator vice A /08/02 Jointly developed new question with facility
H
E 103A2.05, NEW GWL Received is misspelled on the test. FIXED I think it would be better to change distractor A to add the VPAP Attachment 1 to the station manager. It would be more correct but not the oxygen concentration and then change B to the confined space entry permi The Licensee may not like this question because it requires the SROs to know from memory the oxygen setpoint. Will have to see what they sa Review with Gerr January 31, 2002 S
Changed distractors B and OK as changed Page 35 of 50 (
/,. PsychometricFlaws 4. Job Content Flaws
.Other.
(F/H)
O(1-5) StemCues T/F Cred. Partial Job-Minutia
- /
Back-0=
SRO U/E/S Explanation Focus Dis Link units ward K/A Only
F 2-3 S
G2.1.4, BANK Change per to in accordance with. Done Place the parenthesis outside of the I Done Are they required to know this from memory? They may want the TS provided. If this is true then the question is kind of trivia Appears to be o See what the facility says about the use of T January 31. 2002 Will see what the utility has to say with or without Tech Specs. **
2/27/02 FACILITY STATES PUSHING THE MEMORY OF THE QUESTION. OK WILL LEAVE IT THER /8/02 Shifted this question to combined test with reference available to RO's only. This question was exchanged for G2.1.12 on common tes C 2-3 S
G2.1.34, Bank Appears to be o Not sure why anyone would take distractor A as an answe OK as is no change necessar January 25, 2002 2/27/02 Facility believes that A, B and C are all correct answers. Not sure of the procedure that would take care of this. Letdown Red Hi alarm. Have purification in service. Would expect chemistry to tell the operators to do something, but they do not expect operators to know the chemistry procedure Need to rewrite or develop a new questio /08/02 GWL developed new question based on facility comments Page 36 of 50
(.
3. Psychometric Flaws 4. Job Content Flaws
. Other.
Q# LOK LOD (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- /
Back-0=
SRO U/E/S Explanation Focus Dis Link unts ward K/A Ony
C
X E/U G2.2.6, NEW Is this something an SRO would be required to know from memory. I think it may not be. WE need to see what the facility has to say to determine if this is a fair question. I do not think they will like this questio OK as is January 25, 2002 2127/02 OPERATIONAL VALIDITY OF QUESTION IS NOT THERE. MINUTIA, APPROVAL LEVELS DRIVEN BY THE PROCEDURES. WOULD EXPECT THEM TO KNOW THE REQUIREMENT OF CHANGE OF INTENT OR NO NEED TO LOOK A C
S/E G2.3.10, Bank This question appears to be ok. Not sure it is an SRO only knowledg Is this something the facility expects the SRO to know from memory? It should b If we use this question we need to under line the words acknowledge and approve in order to key in on those words. Presently, it is hard to read and we don't want this to be a reading exercis OK as changed, January 25, 2002 2/27/02 RP DEPARTMENT HAS TO GET THE SROs SIGNATURE, FACILITY STATES THIS IS MINUTIA. NOT A STRONG JOB LIN WILL NEED TO DO SOMETHING 3/08/0 Replaced question with facility developed question Page 37 of 50
3. Psychometric Flaws Stem Cues Focus T/F Cre Dist.
IPartial 4. Job Content Flaws Job Link Minutia I units Back ward
-. Other Q=
SRO K/A Only U/E/S ( Explanation WE02G2.4.6, Modified, GWL The distractors B (the answer) and D are the only two that provide the basis for the answer. Distractors A and C have procedural actions that should be taken they do not contain basis for this action. Need to make them simila In the bullets PZR level would be pegged hi or off scale hi being in an SI situation. Should we change this?
Pressurizer is misspelled in distractor D.
IS and ARE are used in the distractors after Termination make them all "is met."
Criteria, need tc January 31, 2002 NO need to change distractors ok as i Changed is to are OK as changed 2/27/02 E-0 DO NOT HAVE CRITERIA FOR ADVERSE CONTAINMEN NEED TO CHANGE THE STEM TO PLACE THE PLANT IN E-1, LOSS OF REACTOR OR SECONDARY CONTAINMENT. WHY COMMIT FROM MEMORY THE ADVERSE NUMBER WILL ADD IN E-1 Done.
Page 38 of 50 Q#
19 LOK (F/H)
H LOD (1-5)
3. Psychometric Flaws Stem Focus CuesI T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia Other
- / IBack-0=
SRO units ward K/A Only U/E/S ( Explanation WE03G2.4.6, Bank GWL In two distractors you use secured but another you use stopped. Should be the same. Either is OK. STOPPED is ok Place the distractors in order of pumps secured, one two then three and the last being all three RCPs remain runnin Should we put in distractor A that it reduces the heat input? NO The reference material does not support produce effective heat transfer, where does this come from?
January 31, 2002 Changed to stoppe Decided to leave in same order Do not need to add to A This distractor was used because it was a bank question.
OK as changed.
H 3-4 E
WE04G2.4.5, Bank In the stem, add the noun name for AP-16.......
Is it better to rearrange the stem to say something like, If the Leak was determined to be unisolable, upon exiting E-0, WOOTF is the correct procedure transitions for the event in progres Changed, ok as i January 25, 2002
H
S WEOSEA2.1, Bank See changes on the exam. Already made Appears to be o January 25, 2002 Page 39 of 50 1.
LOK (F/H)
H LOD (1-5)
20 S
3. Psychometric Flaws Cred. Partial Job Dis Link 4. Job Content Flaws Minutia
,. Other
- /
Back-Q=
SAO units ward K/A Only U/E/S S
( Explanation H
H
S WE08EA2.1, BANK GWL This question while manipulative is not that difficult, you just have to plug the parameters given into the CSF status tree. I guess it is o January 31, 2002 OK Will not provide the CSFTS **
H
S WE14EA2.1, BANK, GWL Are the CSF status trees going to be provided? If so, this is sort of like question number 25. It appears to be o January 31, 2002 Will see what utility says with or without CSFSTs**
RO ONLY QUESTIONS Page 40 of 50 I LOK (F/H)
3 LOD (1-5)
Stem Cues Focus WE06EA2.1, Bank, GWL Is it necessary to describe the rate at which the containment pressure is increasin Will this require the use of the Safety Function Status Trees ? If it does this is a look up questio Otherwise appears o January 31, 2002 DO not need to add rat See what the facility says about the use of CSFSTs**
2/27/02 DO WE INTEND GIVING THEM STATUS TREES?
WILL NEED TO EVALUATE IF THESE ARE NECESSARY.
T/F
3. Psychometric Flaws Stem Focus Cues I T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
- Other
- /
Back-0=
SRO units ward IK/A Only U/E/S ( Explanation
F 2-3 E
001A101, Modified, GWL Should the stem be changed to state Insertion Limit Computer to be more specific then rod insertion limit circuitry?
Otherwise appears to be o January 31, 2002 Changed the stem to the above S
ok as change NO FACILITY COMMENT
F 2-3 S
003K4.07, Bank, GWL Appears to be o NO FACILITY COMMEN H
S 004K6.17, New, GWL Need to add the noun name for the breaker and the procedure Ap-Can you open 1350 from the control room? I think so, would this be wrong, need to look at the procedure. Discuss 0 Otherwise appears ok January 31, 2002 Noun name for breaker still need this info. ***
The breaker that is tagged out is the breaker for the mov in the control roo /27/02 ADDED NOUN NAM GWL WILL LOOK AT TO SEE IF CAN MAKE C NOT AN ANSWER 3/08/02 GWL fixed by making answer C to refer to FCV-1 113B Page 41 of 50 1.
LOK (F/H) LOD (1-5)
3. Psychometric Flaws Stem Cues T/F Focus Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
-. ther
- /
Back-Q=
SRO units ward K/A Ony 6.
U/E/S (
7.
Explanation
F
E 005K6.03, NEW, GWL The answer does seem obvious. IS there a failure mechanism in the CCW system that could be used vice stating manually. Discuss January 31, 2002 Discuss with facility to see if there is a valve that can be used to make it less obviou February 1, 2002 OK as is, no change necessar NO FACILITY COMMENT
H
S 007A2.02, NEW, RLM Underline the word and to ensure the readers realize it is a two part question, (DONE) Otherwise appears o January 31, 2002 ok NO FACILITY COMMENT
H 2-3 S
008A1.04, Bank, GWL In stem, should use numbers or spelling for the unit. Changed Unit one to Unit 1. (Done)
Appears o January 31, 2002 NO FACILITY COMMENT Page 42 of 50 LOK (F/H) LOD (1-5)
3. Psychometric Flaws Stem Focus Cues I TIF Cre Dist.
Partial 4. Job Content Flaws Other Job Link Minutia units Back ward Q=
K/A SRO Only U/E/S U
S (I Explanation
8 H
E 012K4.06, Bank, GWL Mike had a comment what was the resolution?
Change the stem to read, Which one of the following is the correct reactor protection system response to this event. DONE January 31, 2002 Need to change the last or first distractor, needs more work February 1, 2002 S
Changed the distractor ok as change NO FACILITY COMMENT Page 43 of 50 G#
H LOK (F/H)
3 LOD (1-5)
010A2.03, Modified, GWL This event appears on the First Scenario. WE need to change this question to avoid the overlap between these two sections of the tes This is more like a memory level question. If you don't know it, the answer can not be determine January 31, 2002 GWL will do this change February 5, 2002 New Question developed by GWL Changed to 1445 vice 144 Change is o D IS THE CORRECT ANSWER NO FACILITY COMMENT
(.
3. Psychometric Flaws 4. Job Content Flaws Other.
(F/H)
(1-5)
Stem Cues IT/F Cred. Partial Job-Minutia
- /
Back-Q=
SRO U/E/S Explanation Focus Dis Link units ward K/A Ony
F 2-3 S
013K5.02, Bank, GWL Changed the Stem to read, If a valid plant condition requiring an automatic safety injection (SI) was generated, which one of the following will occur?
Appears to be o January 31, 2002 OK as change /27/02 FACILITY HAS A PROBLEM WITH LEVEL OF DETAI WILL LOOK INTO THIS FURTHER. Testing the RO operator at instrument tech leve They expect the RO to know what input Need to add to the stem we are talking about SI and not CLS ASK MIKE...............
3/08/02 Tested as written
H 2-3 S
015A3.03, Modified, GWL, The test item is designated as a modified, not sure how this is the cas Discuss with Gerr Otherwise appears to be o January 31, 2002 This question was modified. Ok as i /27/02 FACILITY STATES BEYOND THE SCOPE. DO NOT EXPECT THE ROS TO KNOW THE OVERLAP OF THE SOURCE RANGE AND INTERMEDIATE RANG WILL CHANGE THIS TO A SHUT DOWN VICE A START U N-32 AT 1 E 5 IS A TRIP SIGNAL. CRITICAL IN THE IR AND THE SOURCE RANGES AUTOMATICALLY TURN OFF ABOVE P-6, 1E-1 /08/02 Left question as is and will provide NI overlap handout Page 44 of 50
/
3. Psychometric Flaws Stem Focus Cues I T/F Cre Dist.
Partial (
4. Job Content Flaws Job Link Minutia
w Sack-0=
units ward K/A SRO Only U/E/S S
E/U ( Explanation 016K4.03, Bank, RLM Appears to be o January 31, 2002 This question has been MODIFIE See changes on the exm OK as change /27/02 NO CORRECT ANSWE /3 OUT OF 2/3 S/Gs CHANGED THE STEM AND DISTRACTORS TO ADD 2/3 OR 1/ MADE THE ANSWER CORRECT.
022A4.05, Modified, GWL This question states that it is modified. It has just been rearranged,
- nothing has changed. Discuss with Gerry.
January 31, 2002 This question was modified, OK as i /27/02 TYPO IN STEM FIXE NO CORRECT ANSWERS OK AS CHANGED looked at the wrong initial questoin.
Page 45 of 50 LOK (F/H)
M Of
12 LOD (1-5)
2-3 M
1 2-3 13 1H 13-4 1S 1029G2.4.49, Modified, GWL, Appears to be o January 31, 2002
- Other
3. Psychometric Flaws Stem Focus Cues I T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
- /
Back units ward
,. Other Q=
K/A SRO Only 6.
U/E/S (
7.
Explanation
H
S 059K3.03, Modified, GWL The question states modified but the question it came from was not listed nor was it attached to the question. Could not determine if it was modified correctl January 31, 2002 OK as i /27/02 CHANGED THE STEM TO 90 % VICE 80 NOT SURE THE TRIP WOULD COME I H 3-4 S
062K1.02, Bank, GWL Appears to be ok January 31, 2002 OK 2/27/02 NO FACILITY COMMENT Page 46 of 50 Q# LOK (F/H) LOD (1-5)
3. Psychometric Flaws Stem Cues Focus T/F Cre Dist.
IPartial 4. Job Content Flaws Job Link Minutia units Back ward
-. Other Q=
SRO K/A Only U/E/S S
( Explanation M
H
S 071A2.09, Bank, GWL Appears to be o January 31, 2002 ok as i /27/02 NO FACILITY COMMENT
H
S 074EA2.01, Bank, RLM Appears to be o January 31, 2002 ok as i /27/02 NO FACILITY COMMENTS Page 47 of 50 LOK (F/H)
2-3 LOD (1-5)
069AA2.01, Bank, RLM Mike had a question about RO knowledge, should an RO know this information. Normally ye See what the facility has to sa Otherwise appears o January 31, 2002 ok as is 2/27/02 FACILITY, WHAT IS THE EXPECTATION OF THE R DO NOT THINK IT IS A VALID RO RLM WILL MAKE IT EVIDENT THAT IT IS OPEN FOR A NORMAL EXIT OR ENTRANCE. SOME TYPE OF ACTIVITY THAT IS ALLOWABLE WOULD MAKE IT CLEARE FIX THIS ON /08/02 Changed distractor to reflect a Maintenance entry, which is permissible.
Q#
(.
3. Psychometric Flaws 4. Job Content Flaws
. Other.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- /
Back-Q=
SRO U/E/S Explanation Focus Ds Link units ward K/A Only
H
?
076K3.01, New, RLM
Made some minor wording changes to the questio There is a question about using other indications to get the same answer has this been fully developed yet?
Discuss with Bob February 1, 2002 S
OK as is, see what licensee says. May need to provide the print we used to develop the question. ***
2/27/02 C COULD BE AN ANSWER. NEED TO LOOK AT THIS ONE TO CHANGE C WILL PROVIDE THE ONE LINE DIAGRAM DURING THE EXAMINATIO RLM WILL FI /08/02 Changed answer C to be backpressure vice air bound. Facility accepted. Also allowed the use of a handou F 2-3 E
103A4.09, Bank, GWL Appears to be o All the answers have the action to be taken and the reason that action was taken, however, the stem only asks the action to be taken. Should change the stem to state something like - WOOTF is the action and reason for that action following a High Alarm Trip of Process Vent Red Monitor RI-GW-101/102?
January 31, 2002 Changed the stem, rearranged the ste OK as change /27/02 NO FACILITY COMMENT Page 48 of 50
/
3. Psychometric Flaws Stem Focus CuesI T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia;
- Other
- /
Back-O=
SRO units ward IKA Onlv U/E/S ( Explanation
F 2-3 S
WE01G2.4.5, Bank, GWL Appears to be ok January 31, 2002 ok as i /27/02 NO FACILITY COMMENT
H
S WE03G2.4.4, Bank, GWL Are RO's required to have this knowledge memorized? Discuss with GW Minor editorial changes. Done See what the facility says about this questio January 31, 2002 As licensee if Ros have to know this. **
2/27/02 NO FACILITY COMMENT
H 3-4 E
WEO4G2.4.9, Bank, GWL This question states it is a bank question, but the bank question attached is quite a bit different. I think this is more modifie Need to add back in the noun names for the valves being manipulated in the ste Is this an RO knowledge level question. It may not be. May be more like an SRO. Need to discuss with GWL and license I made minor editorial and typographical change S January 31,2002 Ask licensee if RO knowledg It is modified as stated in the question distrators.'
OK as i ADDED ECA NO FACILITY COMMENT Page 49 of 50 LOK (F/H) LOD (1-5)
3. Psychometric Flaws Stem Focus Cues T/F Cre Dist.
Partial 4. Job Content Flaws Job Link Minutia
- /
Back units ward
-. Jther Q=
SRO K/A Only U/E/S S
S Explanation WE05EK2.1, NEW, GW Some additions to the stem that are don Need to add the noun name for ES-Otherwise appears o January 31, 2002
- Need to ask facility if they do this locally or from the control roo /27/02 TYPOS CORRECTE CHANGED DISTRACTOR D. SEE EXAM FOR CHANGE.
WE08EK3.3, Bank, GWL Need to understand why answer A is not an additional correct answer. It seems that the left column of the procedure would allow this to be o Discuss this with Gerr Otherwise appears o January 31, 2002 Need to ask the licensee what to do in this case. Not sure an RO needs to know this information. ASK 2/27/02 FACILITY BELIEVES THE LEVEL OF DETAIL IS MEMORIZATION OF THE PROCEDURE STEP WILL ASK MIK NEED TO REEVALUATE THIS 3/08/02 Replaced Question with one more acceptable to facility Page 50 of 50 LOK (F/H)
F 24 LOD (1-5)
2-3
25 1 H f
Written Examination Grading Quality Checklist Form ES-403-1 (R8, Si)
Facility: Surry Power Station Date of Exam: March 14, 2002 Exam Level: RO/SRO Initials Item Description a
b c Clean answer sheets copied before grading RLM
/. Answer key changes and question deletions justified and RLM documented Applicants' scores checked for addition errors RLM (reviewers spot check > 25% of examinations)
Am Grading for all borderline cases (80% +/- 2%) reviewed in RLM detail All other failing examinations checked to ensure that grades N/A NA are justified Performance on missed questions checked for training RLM deficiencies and wording problems; evaluate validity of questions missed by half or more of the applicants Printed Name / Signature Date a. Grader Robert L. Monk d,&A.e#' pfn,'
4/3/02 b. Facility Reviewer(*)
N/A c. NRC Chief Examiner (*) Richard S. Baldwin d. NRC Supervisor (*)
Michael E. Ernstes (*)
The facility reviewer's signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.
Post-Examination Check Sheet Surry Power Station Task Description Date Complete Facility written exam comments or graded exams received and 4/3/02 verified complete Facility written exam comments reviewed and incorporated and 4/3/02 NRC grading completed, if necessary Operating tests graded by NRC examiners 4/12/02 NRC Chief examiner review of written exam and operating test 4/12/02 grading completed Responsible supervisor review completed
/1 Management (licensing official) review completed 2/ Y A License and denial letters mailed
-//12lo Facility notified of results
<I4 411Z Examination report issued (refer to NRC MC 0610)
1 Reference material returned after final resolution of any appeals Form ES-501-1 (R8, $1)