ML14233A328
| ML14233A328 | |
| Person / Time | |
|---|---|
| Site: | Quad Cities  |
| Issue date: | 04/28/2014 |
| From: | Palagi B NRC/RGN-III/DRS/OLB |
| To: | Exelon Generation Co |
| Shared Package | |
| ML13093A385 | List: |
| References | |
| Download: ML14233A328 (36) | |
Text
2014 QUAD CITIES NUCLEAR POWER STATION INITIAL LICENSE EXAMINATION ADMINISTRATIVE FILES
ES-201 Examination Preparation Checklist Form ES-201-1 Facility: QtA. "ol C./.J.,'e.s Date of Examination: A,orlJ../Cf I
Developed by: Written-Facility '[g( NRC D II Operating-Facility D NRCO Target Chief Date*
Task Description (Reference)
Examiner's Initials
-180
- 1.
Examination administration date confirmed (C.1.a; C.2.a and b) 6f?
-120
- 2.
NRC examiners and facility contact assigned (C.1.d; C.2.e)
Pf
-120
- 3.
Facility contact briefed on security and other requirements (C.2.c)
(3(
-120
- 4.
Corporate notification letter sent (C.2.d)
~/
[-90]
[5.
Reference material due (C.1.e; C.3.c; Attachment 3)]
i!A
{-75}
- 6.
Integrated examination outline(s) due, including Forms ES-201-2, ES-201-3, (3f ES-301-1, ES-301-2, ES-301-5, ES-D-1's, ES-401-1/2, ES-401-3, and ES-401-4, as applicable (C.1.e and f; C.3.d)
{-70}
{7.
Examination outline(s) reviewed by NRC and feedback provided to facility
~
licensee (C.2.h; C.3.e)}
{-45}
- 8.
Proposed examinations (including written, walk-through JPMs, and
&f scenarios, as applicable), supporting documentation (including Forms ES-301-3, ES-301-4, ES-301-5, ES-301-6, and ES-401-6, and any Form ES-201-3 updates), and reference materials due (C.1.e, f, g and h; C.3.d)
-30
- 9.
Preliminary license applications (NRC Form 398's) due (C.1.1; C.2.g; Of ES-202)
-14
- 10.
Final license applications due and Form ES-201-4 prepared (C.1.1; C.2.i; K-?
-14
- 11.
Examination approved by NRC supervisor for facility licensee review eP (C.2.h; C.3.f)
-14
- 12.
Examinations reviewed with facility licensee (C.1.j; C.2.f and h; C.3.g)
(?f
-7
- 13.
Written examinations and operating tests approved by NRC supervisor (5P (C.2.i; C.3.h)
-7
- 14.
Final applications reviewed; 1 or 2 (if >10) applications audited to confirm ;J?
qualifications I eligibility; and examination approval and waiver letters sent (C.2.i; Attachment 5; ES-202, C.2.e; ES-204)
-7
- 15.
Proctoring/written exam administration guidelines reviewed f'f with facility licensee (C.3.k)
-7
- 16.
Approved scenarios, job performance measures, and questions
~~
distributed to NRC examiners (C.3.i)
- Target dates are generally based on facility-prepared examinations and are keyed to the examination date identified in the corporate notification letter. They are for planning purposes and may be adjusted on a case-by-case basis in coordination with the facility licensee.
[Applies only] {Does not apply} to examinations prepared by the NRC.
ES-201 &vllb C/71C"L Examination Security Agreement Form ES-201-3
- 1.
Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) or#iJ1rl'U7' as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.
- 2.
Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of tlttbt.f. From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.
PRINTED NAME JOB TITLE I RESPONSIBILITY SIGNATURE (1 )
DATE SIGNATURE (2)
DATE NOTE
- -;*...... K'"""' -,.,,,
E"MNJ AvzM?tZ
/()/l'Pr ~
~--
...,,~o e,._., -<vrH"
~
?L= -
¥/Lt __
cl<&"'
........,.,,,,. 2, oi?.~~J.~
- t.
C3 tv S *
.£!..:btfl f"vl<<~
_w *
~ "6 c:;?/.
EJ' It?.
~.............,.
XTSA-
'l
~
~--
t.t;fo.JTI'~I.CIO!L
~*
~/e/~;lht;n~
H~~~~
1
\\
r~~
t V.~;UDAm 1 /., '/<
- $;I~. t==
tAIJI.. A.r-i*l*/'f'.
f'"f.&.*t~{
lf.., t.rla =trv:
y:FPl
.6'"" a-.2d... I
',c-t-wr..,_ ~v*;,u*-r_
u&i..t?
I'
- It
~;
f lt.fl tDIWl lll tJ3 tt;,t,~
I T /<
7.7.Zf.--
Va ~;;~;
r I~ I'/
11.¥-
ES-201, Page 27 of 28 RECEIVED MAY 3 0 2014
ES-201 c:PUA-0 e./716->
Examination Security Agreement Form ES-201-3
- 1.
Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of ot,/zf/l'i as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from.this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.
- 2.
Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of ~zs/t't. From the date tl:lat I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.
PRINTED NAME JOB TITLE I RESPONSIBILITY 1......-::.' ~* **-
'::::;:;:::u ~
L
~
t
~~28--;'1
.,. /<J;~ _.*jc.k.Cu~
1'i. t: 'klo/'L-..._
,... 'lh~ fj.._~k.,,_, S"lli/t'1 1 /tft/L..o,.fL f4<:MVk ?' V.~
1
- _
lleyo~.u r !'"*
~Me~!
t~tv*rr
- "'2'~~ f-7-4'(~
~
,-;;;;;:;::-::=-1T"""'U
.. **'.** ~
~
- ~~=---5]?5~
-~-..w*.. -,.~Ht VAt.-tpez-roa
-~~
FJjJ.~_
.~c..>[.(._~.. !+~
v~~
~
~a-~1--
5"-Zo ~
rSJ~~~*:-:
~~
~~
=,rv'"¥¥ w-~-**-
---~
~-f_
~.......!J_~
_5'-"tr (d~~~Cilc-vh a..(
- r
~
ES-201, Page 27 of 2e
ES-201 Qcnfo C(7t~s Examination Security Agrea_ment Form ES-201-3
- 1.
Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of 1( V'6\\ 1'1 as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.
- 2.
Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of 4-f/z,llf;'f. From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.
PRINTED NAME JOB TITLE I RESPONSIBILITY SIGNATURE (1)
DATE DATE NOTE
- 1.
~
- s. o
..... l-!1/
s-<-3*t~
- 2.
~<?
<r31-.i!i
~21-'f
- 3.
S>
3-J HcJ il rzcc=
- 4.
,._ ) %?
't.+/-1.l.
~::{f::_
- 5.
E p M
'H.:l!L S-?"""Y---
6..T;u."'
L.e.c~t..-...~ *.,..
5-i........ t.t-...- (A.o,_J.,-~
-.'*U*J<t
..;.;&JY
- 7. ( C.INTON Wlf.!.IAflQ 5(;G.IJ~Te1L L/J1i/'f
'J'I==
- 8. OR1J r
~o<Sr~~
If-
.~
~17/1'1 __
- 9..J t Vtok.(
, ~ I IJto.re.,. 1M...
1 If.. __
10.&t; -(;(!,._,_,
s-wrro,::; <<v ><<r fif., __
- 11.
~
~C'1'"u\\!c
'1/J.cll'f
~--
- 12. ~
- 2. H4'Z/I
((,M(=f&"l
~/,.,/,'(
J/1/i:t--
- 13.
- 14.
- 15.
NOTES:
ES-201, Page 27 of 28
ES-401 Record of Rejected K/As (RO)
Form ES-401-4 Tier 1 Group 1 Tier/
Randomly Reason for Rejection Group Selected KIA T1/G1 295025 EA 1.08 Quad Cities does not have RRCS. Randomly selected an alternate A1 Became statement.
295025 EA 1.04 T1/G1 295001 AA 1.04 Quad Cities does not have Rod control and information system. Randomly Became selected an alternate A 1 statement.
295001 AA 1.07 T1/G1 295006 2.2.4 2.2.4 is not relevant for 295006. Randomly selected an alternate generic Became KIA from the field of unused Generic Equipment Control KIAs for this 295006 2.2.42 system.
T1/G1 295024 EK1.02 Quad Cities does not have Mark Ill containment. Selected the only other Became available K1 statement.
295024 EK1.01 T1/G1 295003 AK2.05 Quad Cities does not have Isolation Condensers. Randomly selected an Became alternate K2 statement.
295003 AK2.06 T1/G1 295016 AK3.02 K3.02 (Reasons for turbine trip during control room abandonment) is not Became applicable to Quad Cities. Randomly selected an alternate K3 statement.
295016 AK3.03 T1/G1 295038 EK2.06 Overlap with walk through. 295038 EK2.06 already selected and used for Became SRO Admin JPM 4.
295038 EK2.03 Randomly selected an alternate K2 statement.
T1/G1 600000 AA2.04 Unable to write a psychometrically sound question for AA2.04.
Became Randomly selected an alternate A2 statement.
600000 AA2.03 ES-401, Page 27 of 33
- ----------------------c---------------
ES-401 Record of Rejected K/As (RO)
Form ES-401-4 Tier 1 Group 2 Tier/
Randomly Reason for Rejection Group Selected KIA ES-401, Page 27 of 33
ES-401 Record of Rejected K/As (RO)
Form ES-401*4 Tier 2 Group 1 Tier/
Randomly Reason for Rejection Group Selected KIA T2/G1 203000 K6.07 203000 K6.07 is not applicable to Quad Cities. Randomly selected an Became alternate K6 statement.
203000 K6.02 T2/G1 209002 A4.05 209002 (HPCS) was used as a place-holder for SSMP during the initial Became random selection process. Replaced HPCS with SSMP. A4.05 did not SSMP apply to SSMP so randomly selected a new A4 from the RCIC statements.
(217000) A4.04 T2/G1 262002 2.4.30 2.4.30 is not applicable to the RO license level at Quad Cities. Randomly Became selected an alternate generic KIA from the field of unused Generic 262002 2.4.31 Emergency Procedures/Plan KIAs for this system.
T2/G1 209001 A2.07 A2.07 is not relevant to the RO license level at Quad Cities. Randomly Became selected an alternate A2 statement.
209001 A2.03 T2/G1 209001 K6.02 2009001 K6.02 (loss/malfunction of EDGs effect on Core Spray)
Became overlapped with 264000 K5.06 (EDG load sequencing with Core 209001 K6.04 Spray/RHR). Randomly selected an alternate K6 statement.
T2/G1 263000 K1.03 263000 K1.03 is not applicable to Quad Cities. Randomly selected an Became alternate K1 statement.
263000 K1.02 T2/G1 300000 K3.01 Quad Cities does not have Containment Air. Randomly selected an Became alternate K3 statement.
300000 K3.02 ES-401, Page 27 of 33
ES-401 Record of Rejected K/As (RO)
Form ES-401*4 Tier 2 Group 2 Tier/
Randomly Reason for Rejection Group Selected Kl A T2/G2 230000 K1.08 203000 K1.08 is not applicable to Quad Cities. Randomly selected an Became alternate K1 statement.
230000 K1.01 T2/G2 245000 2.2.39 2.2.39 is not applicable to 245000 at Quad Cities. Randomly selected an Became alternate generic KIA from the field of unused Equipment Control KIAs.
245000 2.2.12 T2/G2 204000 K5.07 204000 K5.07 (RWCU conductivity measurement) is not operationally valid Became for the RO license level. Randomly selected an alternate K5 statement.
204000 K5.08 T2/G2 214000 A4.03 214000 A4.03 (Control Rod Drive temperature measurement) is not Became operationally valid for the RO license level. Randomly selected an alternate 214000 A4.02 A4 statement.
T2/G2 268000 2.1.27 268000 2.1.27 (Knowledge of system purpose and/or function) is not Became operationally valid for a license level exam. Randomly selected an 268000 2.1.23 alternate generic KIA from the field of unused Conduct of Operations KIAs.
ES-401, Page 27 of 33
ES-401 Record of Rejected K/As (RO)
Form ES-401-4 Tier 3 Tier/
Randomly Reason for Rejection Group Selected Kl A T3 2.1.27 2.1.27 was already selected in RO Tier 2*Group 2. Randomly selected an Became alternate generic KIA from the field of unused Equipment Control KIAs.
2.1.21 T3 2.3.11 2.3.11 was already selected in SAO Tier 3. Randomly selected an Became alternate generic KIA from the field of unused Radiation Control KIAs.
2.3.12 T3 2.4.23 Unable to write a psychometrically sound question related to 2.4.23.
Became Randomly selected an alternate generic KIA from the field of unused 2.4.47 Emergency Procedures/Plan KIAs.
T3 2.3.4 Overlap. 2.3.4 selected for RO Admin JPM 4. Randomly selected an Became alternate generic KIA from the field of unused Radiation Control KIAs.
2.3.13 T3 ES-401, Page 27 of 33
ES-401 Record of Rejected K/As (SRO)
Form ES-401-4 Tier 1 Group 1 Tier/
Randomly Reason for Rejection Group Selected KIA T1/G1 295016 2.2.3 2.2.3 is not related to 10 CFR 55.43(b) in accordance with NUREG 1021, Became D.1.c. Randomly selected an alternate generic KIA from the field of 295016 2.4.34 unused generic KIAs for this system.
T1/G1 295021 2.1.30 2.1.30 is not related to 10 CFR 55.43(b) in accordance with NUREG 1021, Became D.1.c. Randomly selected an alternate generic KIA from the field of 295021 2.2.40 unused generic KIAs for this system.
T1/G1 295019 2.4.6 2.4.6 was already selected in RO Tier 2 Group 1. Randomly selected an Became alternate generic KIA from the field of unused generic KIAs for this system.
295019 2.2.22 T1/G1 295028 EA2.02 Overlap with RO 295028 EK1.01. Randomly selected an alternate EA2 Became statement.
EA2.04 T1/G1 295025 EA2.05 Unable to write a psychometrically sound question to EA2.05. Randomly Became selected an alternate EA2 statement.
295025 EA2.03 ES-401, Page 27 of 33
ES-401 Record of Rejected K/As (SAO)
Form ES-401-4 Tier 1 Group 2 Tier/
Randomly Reason for Rejection Group Selected KIA T1/G2 295022 2.1.28 295022 system has been sampled on the Certification written exam and on Became the Operating exam. 2.1.28 is not related to 10 CFR 55.43(b) in 295017 2.1.25 accordance with NUREG 1021, D.1.c. Randomly selected an alternate evolution from the field of unused SAO Tier 1 Group 2 evolutions.
Randomly selected an alternate generic KIA from the field of unused generic KIAs for this svstem.
ES-401, Page 27 of 33
ES-401 Record of Rejected K/As (SRO)
Form ES-401-4 Tier 2 Group 1 Tier/
Randomly Reason for Rejection Group Selected Kl A T2/G1 300000 A2.01 Unable to write an SRO license level question to 300000 A2.01. Since Became there are no other A2 statements for the 300000 system, an alternate 217000 A2.13 system and A2 statement was randomly selected from the field of unused systems in SRO Tier 2 Group 1.
T2/G1 215005 2.4.46 215005 was selected twice in RO Tier 2 Group 1.
Randomly reselected Became an alternate system from the field of unused systems in SRO Tier 2 Group 206000 2.4.46
- 1.
T2/G1 218000 2.4.34 2.4.34 was already selected in SRO Tier 1 Group 1. Randomly selected Became an alternate generic KIA from the field of unused generic KIAs for this 218000 2.4.1 system.
T2/G1 217000 A2.13 Unable to write a SRO license level question to 217000 A2.13 (RCIC loss Became of room cooling). Randomly selected an alternate A2 statement.
217000 A2.05 T2/G1 218000 2.4.1 2.4.1 (Knowledge of EOP entry conditions and immediate action steps.) is Became not SRO license level. Randomly selected an alternate generic KIA from 218000 2.4.9
- the field of unused generic KIAs.
ES-401, Page 27 of 33
ES-401 Record of Rejected K/As (SRO)
Form ES-401-4 Tier 2 Group 2 Tier/
Randomly Reason for Rejection Group Selected KIA T2/G2 241000 2.4.2 241 000 already selected in RO Tier 2 Group 2. Randomly selected an Became alternate system from the field of unused systems in SRO Tier 2 Group 2.
223001 2.4.45 2.4.2 already selected in RO Tier 3. Randomly selected an alternate generic KIA from the field of unused generic KIAs for this system.
T2/G2 201001 2.2.38 2.2.38 already selected in RO Tier 3. Randomly selected an alternate Became generic KIA from the field of unused generic KIAs for this system.
21 0001 2.2.25 ES-401, Page 27 of 33
ES-401 Record of Rejected K/As (SRO)
Form ES-401*4 Tier3 Tier/
Randomly Reason for Rejection Group Selected KIA T3 2.1.7 2.1.7 was already selected in SRO Tier 2 Group 1. Randomly selected an Became alternate generic KIA from the field of unused Conduct of Operations KIAs.
2.1.37 T3 2.2.43 Unable to write a SRO license level question to 2.2.43 (Knowledge of the Became process used to track inoperable alarms) 2.2.14 ES-401, Page 27 of 33
ES-403 Written Examination Grading Form ES-403-1 Quality Checklist Facility: Quad Cities Date of Exam: 4/28/14 Exam Level: RO ~ SRO ~
- 1.
- 2.
- 3.
- 4.
Item Description Clean answer sheets copied before grading Answer key changes and question deletions justified and documented Nohe. Mule,(
Applicants' scores checked for addition errors (reviewers spot check >25% of examinations)
Grading for all borderline cases (80% +/- 2% overall and 70 or 80, as applicable,+/- 4% on the SRO-only) reviewed in detail
- 5.
All other failing examinations checked to ensure that grades are justified
" *I ul"(;.$
- 6.
Performance on missed questions checked for training deficiencies and wording problems; evaluate validity of questions missed by half or more of the applicants Printed Name I Signature
- a. Author
- b. Facility Reviewer(*)
- c. NRC Chief Examiner(*)
- d. NRC Supervisor(*)
Initials a
b c
Date 5/6/14 5/6/14
~'f-
!M
(*)
The facility reviewer's signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.
ES-403, Page 5 of 5
ES-401 QUAD CITIES 2014-Written Examination Review Worksheet Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- I Back-Q=
SRO 8/M/N UIE/S Explanation Focus Dist.
Link units ward KIA Only 1
H 2
y 8
e NRC: Edit the table provided to provide the procedure header info and the page number. Also edit the stem to say something like, "Using the table below which of the following is closest to the actual water level."
s Resolution: Stem changed to address comment 2
H 3
y 8
s 3*
L 3
y 8
s 4
L 3
X y
N e
NRC: Distractor "C" not credible now about something like "Prevent s
the vessel pressure Safety Limit from being exceeded."
Resolution: Change made 5
H 3
y N
s
~
Instructions (Refer to Section D of ES-401 and Appendix 8 for additional information regarding each of the following concepts.]
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy-difficult) rating scale (questions in the 2-4 range are acceptable).
- 3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid KIA but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).
- 6.
Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7.
Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any "U" ratings (e.g., how the Appendix 8 psychometric attributes are not being met).
30 QUESTION FOR WHICH AN INITIAL AND DETAILED REVIEW WAS PERFORMED
ES-401 2
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- I Back-Q=
SRO B/M/N U/E/S Explanation Focus Dist.
Link units ward KIA Only 6*
F 3
y B
s 7
H 3
y N
s 8
H 3
y N
e NRC: Consider rewording distractor "A" to "Give ALL outboard s
MSIVs a CLOSED signal leave the inboard MSIVs as is."
Resolution: Distractor reworded 9*
H 3
X y
N S-+
NRC: What is the setting on the E-15 alarm?
s Resolution: Alarm is set such that it will be cleared at 100 psig.
10 H
3 y
B s
11 L
3 X
X y
N v
NRC: Exceeding a low pressure limit would be "... due to exceeding s
the Primary Containment Pressure Limit" ie no negative pressure.
Consider changing stem to "... containment failure due to over pressurization."
Resolution: Stem reworded 12*
H 3
X y
N tJ.'?.
NRC: Can HPCI provide 5600 gpm at 1100 psig if not there is no correct answer. Consider changing the pressure rise to "... from 900 psig to 950 psig" s
Resolution: HPCI is surveillance tested up to 1250 psig therefor it should perform as assumed in this question.
13 H
2 y
N s
14 H
3 y
B s
15*
L 3
y B
s 16 H
2 y
M e
NRC: Why is it given that "All But four control rods are fully s
inserted... ". If it is not needed in the stem please delete it.
Resolution: Stem revised to remove unnecessary information.
17 L
3 y
N s
18 L
3 y
B s
19 H
2 y
N s
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- I Back-Q=
SRO B/M/N UIE/S Explanation Focus Dis!.
Link units ward KIA Only 20*
L 3
y N
s 21 H
3 y
B s
22 H
4 y
N s
23*
H 2
y N
s 24 L
3 y
B s
25 H
4 y
N s
26*
L 3
y N
s 27 H
2 y
N s
28 H
4 y
N s
29*
H 3
y N
s 30 L
2 y
N s
31 H
2 y
N s
32 L
3 y
N s
33*
H 5
X X
y N
e NRC: I don't think we can expect operators to remember the action to s
isolate the air supply for hours and then re-adjust it. Why does this work?
Resolution: Discussed why the correct answer is right and made a small change to the correct answer.
34 H
3 y
N s
35 L
3 y
N s
36*
H 3
y B
s 37 H
3 y
N s
38 L
1 y
N y
NRC: To get the correct answer all you need to know is the each s
LPRM in putting into an APRM is used to produce the APRM reading.
Resolution: The question was replaced.
39*
H 3
y B
s
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- I Back-Q=
SRO B/M/N U/E/S Explanation Focus Dist.
Link units ward KIA Only 40 H
3 X
y N
y NRC: During a station blackout the condenser is unavailable so one s
would choose "A" or "B".
Resolution: After discussion with site staff one of the distractors was accepted and one replaced.
41 H
3 y
N e
NRC: Change "D" to "ONLY the inboard MSIVs close" s
Resolution: Distractor reworded.
42 H
3 X
y N
e NRC: As is "A" and "B" both partly answer the question but nether s
says the reason the valves are open is the open logic is complete.
The stem asks the reason for the position. You may want to reverse the quest logic by having the blowdown inhibited.
Resolution: Two of the distractors were reworded.
43*
L 3
y N
s 44 L
2 y
B s
45 H
3 y
N s
46*
L 3
y N
s 47 L
3 y
N s
48 L
2 y
B s
49*
L 3
y B
s 50 H
3 y
N s
51 H
3 y
N s
52 H
3 y
N s
53*
H 3
X y
N e
NRC: Change distractor "A" such that ACB 152-3102 will not close.
s Maybe trip ACB 152-3104 then close 152-31 02. Consider adding to the stem that smoke is reported near SWGR 14-1.
Resolution: This question was replaced due to it being the same as a JPM action.
54 H
2 y
B s
55 H
2 y
N s
56*
H 3
y M
s
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- I Back-Q=
SRO B/M/N U/E/S Explanation Focus Dist.
Link units ward KIA Only 57 L
2 y
N s
58 H
4 y
N s
59*
H 3
y N
s 60 L
4 y
N e
NRC: Consider adding the control valve positions that are verified to s
the stem (CV #1:.::, 45% open, CV#2..... )
Resolution: Stem reworded 61 H!
3 y
N e
NRC: This is a high cognitive level question. Consider changing s
stem to be 100 % power and increasing.
Resolution: Recommended change made.
62 L
2 y
N s
63*
L 2
y N
e NRC: Distractors and answer B, C, and D must be rewritten to s
explain how to quantify both identified and unidentified leakage.
Resolution: Distractors reworded.
64 H
3 y
N s
65 H
3 y
N s
66*
L 2
N B
y NRC: This question does not test the ability to interpret and execute s
procedure steps.
Resolution: Stem changed to ask about bypass valve closure.
67 L
4 y
N s
68 H
4 y
N s
69*
L 3
y B
s 70 L
4 N
B y
NRC: This not a radiological safety question.
s Resolution: Distractors changed to include a reactor power level limit set for radiological reasons.
71 L
3 y
B s
72 L
2 y
B s
73*
L 3
y N
s
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- I Back-Q=
SRO B/M/N UIE/S Explanation Focus Dist.
Link units ward KIA Only 74 H
3 y
N s
75 H
3 y
N s
76*
H 3
y y
N s
77 H
3 y
y M
s 78 H
3 y
y M
s 79*
H 3
y y
N s
80 H
3 y
y N
s 81 H
3 y
y B
s 82 H
3 y
y B
s 83 H
2 y
y N
s 84 H
2 y
y N
s 85 H
2 y
y N
s 86 H
2 y
y N
s 87 H
2 y
y N
s 88 H
3 X
y y
N Y+
NRC: IN the stem remove the line "There was NO rod motion as a s
result of the scram" There is a later bullet that 50% of the rods are withdrawn. ?if all rods get inserted should the answer be different?
Should power level be supplied? Would TWO pumps be correct because there are no pressure concerns? Can "B" be a correct answer.
Resolution: The stem was changed to clarify the reactor conditions and the time of those conditions.
89 H
3 y
y N
s 90 H
3 y
y B
s
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- I Back-Q=
SRO B/M/N UIE/S Explanation Focus Dist.
Link units ward KIA Only 91 L
2 y
y N
s 92 H
2 y
y B
s 93 H
3 y
y N
s 94 H
3 y
y N
s 95 L
2 y
y B
s 96 L
2 y
y B
s 97 H
3 y
y N
s 98 L
2 y
y B
s 99 H
3 y
y N
s 100 H
3 y
y N
s
THE LICENSEE SUMITTED OUTLINE WAS EXCEPTED AS SUBMITTED.
AND OPERATING TEST WAS ACCEPTED AS SUBMITTED WITH ONLY MINOR CHANGES MADE TO EASE ADMINISTRATION.
May 7, 2014 Regional Administrator, Region III U.S. Nuclear Regulatory Commission 2443 Warrenville Road, Suite 210 Lisle, IL 60532-4352 ATTN: Mr. Bruce B. Palagi
Subject:
NRC Initial License Written Examination The Initial License Written Examination was administered at Quad Cities Nuclear Power Station (QCNPS) on May 5, 2014. The candidates were briefed on the examination rules and no clarifying comments on any of the questions were given by the proctors. In accordance with NUREG-1021, Section ES-501, comments which resulted from the post-examination review and the examination analysis have been forwarded to the Chief Examiner, Bruce B. Palagi.
This letter provides the notification that QCNPS will not be challenging any questions pertaining to the Initial License Examination, administered May 5, 2014.
In addition, the Station requests this examination and associated materials be withheld from public disclosure for a period of 2 years to allow its use as an evaluation tool for subsequent initial license candidates.
Should you have any questions concerning this letter, please contact Mr. Christopher J.
Burkhart at (309) 227-4057.
Respectfully, Martin R. De Vault Site Training Director Quad Cities Nuclear Power Station cc:
Chief, Operations Branch - NRC Region III NRC Senior Resident Inspector-Quad Cities Nuclear Power Station RECEIVED MAY 0 8 21M
U.S. Nuclear Regulatory Commission 2014 RO Written Exam (Quad Cities)
Changes made while administering the examination Question #9 - Clarified to all applicants, wrote on the white board, and annotated in the exam key:
"The 2A RHR pump§ trips... "
Should read:
"The 2A RHR pump trips... "
- e~M:U~ATtON Pd\\J$WER KEl';.,; >. j:;~z),
u.s~ Nl!f¢tear.R$gutatory Oom.iOn.aOi4 RQ WrittSn Exam* (QuadQiti~sk.,*~ i;;.,i,,j,~!;~~iMz:
Unit 2 is shutdown and decay heat is being removed via the 2A RHR Loop in Shutdown Cooling (SOC).
The 28 loop of RHR is tagged out and drained.
The 2A RHR pumriifrips, and the 28 RHR pump will NOT start.
The problem with fh~ 2A RHR pump has been corrected and the 2A RHR pump is now available for service.
The following conditions are reported:
MO 2-1001-47, SOC HDR DOWNSTREAM SV, is CLOSED MO 2-1001-50, SOC HDR UPSTREAM SV, is CLOSED Annunciator 902-3 E-15, SHUTDOWN COOLING LOW PRESS PERM, is CLEAR Reactor water level is +31 inches and rising slowly Which of the following lists the MINIMUM actions, if any, that must be taken PRIOR to re-opening the SOC Isolation valves and starting the 2A RHR pump?
A.
REDUCE reactor pressure to less than 100 psig AND RESET the SOC Isolation logic B.
REDUCE reactor pressure to less than 100 psig ONLY C.
RESET the SOC Isolation logic ONLY D.
NO action is required Answer:
A With no RHR pumps running, decay heat will cause reactor pressure to rise. When pressure exceeds 100 psig, the Shutdown Cooling Low Pressure Permissive alarm clears, indicating that an RHR pump CANNOT be started while aligned for Shutdown Cooling. Reactor pressure must be reduced to less than 100 psig and the isolation logic manually reset before a RHR pump is permitted to start.
Annunciator 902-3 E-15 actual setpoint is 118 psi g. Reducing reactor pressure to less than 100 psig will cause the annunciator to alarm.
Distractor 1 Is Incorrect: Plausible if assumed that the isolation logic need only be reset if there were a Group 2 Isolation signal. There is no indication of a Group 2 Isolation from the stem.
Distractor 2 is incorrect: Plausible because the SOC Isolation logic must be reset and if assumed that the SOC low press perm alarm being clear means that a RHR pump start is permitted.
Distractor 3 is incorrect: Combination of distractors 1 and 2.
Reference:
QCOA 1000-02 Rev 19, QCOP 1 000-05 Rev 49, QCAN 901 (2)-3 E-15 Rev 8.
Reference provided during examination: None Cognitive level: High Level (RO/SRO): RO Tier: 1 Group: 1 KIA: 295021 AA2.06 QDC OPS ILT EXAM Page: 17 of 159 08April2014
- eXAMINATtON*.ANSWER.* KE't**
u.ls. Nllclear RegutatoryOommi$slon 20.14*Rown*n ex$n(QuadCitttsF Ability to determine and/or interpretihe following as they apply to LOSS OF SHUTDOWN COOLING:
Reactor pressure (CFR: 41.10 /43.5/45.13)
IMPORTANCE RO 3.2 SRO 3.3 SRO Justification: N/A Question Source: New Question History: N/A Comments: Per NRC: added clarification to answer explanation for annunciator 902-3 E-15 setpoint.
Associated objective(s):
295021.AA2.06 Reactor pressure (R0=3.2/ SR0=3.3)
SR-1000*1<24 (Freq: LIC=B)
Given an RHR system operating mode and various plant conditions, PREDICT how each supported system will be impacted by the following RHRIRHRSW system failures:
- a.
RHR pump trip
- b.
RHRSW pump trip
- c.
Loss of 125vdc to RHR initiation and/or loop select logic
- d.
Abnormal RHR discharge header pressure QDC OPS ILT EXAM Page: 18 of 159 08April2014
_,.,, Exelon Generation Post Examination Test Item Analysis Individual Exam Analysis Lesson Plan I Course:
ILT 12-1 TQ-AA-224-F200 Revision 4 Page 1 of 2 Examination No./
Title:
U.S. Nuclear Regulatory Commission 2014 RO Written Exam (Quad Cities)
- 1. Check the results of the examination.
a) D Small class ($6) and some questions were missed by three (3} or more students. Analysis required for these questions.
b) [8J Large class (>6) and some questions were missed by more than 30% of the class. Analysis required for these questions.
c) D Analysis of questions requested by individual.
d) D None of the above apply and no further analysis is necessary.
- 2. If an analysis is to be performed, list the question number(s) on the question analysis worksheet and review each using the following guidelines. Record the results on the worksheet.
a) Review each question to ensure it is worded clearly and is not ambiguous. Ensure that it is properly constructed. Refer to JA-AA-222-0020, "Test Item Design and Approval Job Aid".
b) Review each question to ensure that it ties properly to the learning objectives' condition, performance, and standard.
c) Review each question to ensure that it is technically accurate.
d) Review each question to ensure that adequate material was presented to allow for achievement of the associated learning objective.
- 3. Determine if action is required for problem questions (e.g. delete from examination bank; revise question, revise lesson plan, etc.). When action is required, ensure an assignment is initiated for documentation and tracking purposes.
Comments: None Completed By: Christopher Burkhart I Date: 51612014 Assignment Tracking Number(s): NIA Answer Key Change I Examination Re-grade NIAINIA Authorized by:
Group Lead I Date Retention Requirement per SRRS ID 3D.125 Training Documentation Required for Accreditation 2 Yrs
QUESTION ANALYSIS WORKSHEET Vision KIA Construction Objective Technically TQ-AA-224-F200 Revision 4 Page 2 of 2 LP Content Change Question Number Obj ID Category OK?
Based?
Accurate?
Acceptable?
Needed?
38 45 63 66 69 (Y-N)
(Y-N)
(Y-N)
(Y-N)
(Y-N) 954162 215005 y
y y
y Al.01 933519 216000 y
y y
y Al.04 977401 268000 y
y y
y 2.1.23 953227 2.1.20 y
y y
y 953226 2.2.38 y
y y
y Retention Requirement per SRRS 10 30.125 Training Documentation Required for Accreditation 2 Yrs N
N N
N N
Question Statistics (On This Test Only)
U.S. Nuclear Regulatory Commission 2014 RO Written Exam (Quad Cities)
IDENTIFICATION DIFFICULTY VARIABILITY RESPONSES Point Avg
%Avg (0- 6)
Std QSYSID QUID Type Value Score Score Diff Disc lndex 2 Dev Count Right Wrong Partial Blank Invalid MC Responses 1
952174 1
MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao 87 Co Do 2
947326 2
MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao Bo C1 Do 3
953276 3
MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 A1 Bo Co Do 4
966984 4
MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao 87 Co Do 5
939353 5
MC 1.00 0.71 71 2.4 0.50 0.49 7
5 2
0 0
0 As 82 Co Do 6
947654 6
MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 A1 Bo Co Do 7
953307 7
MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 A7 Bo Co Do 8
953312 8
MC 1.00 0.86 86 1.2 0.00 0.38 7
6 1
0 0
0 A1 86 Co Do 9
967054 9
MC 1.00 0.71 71 2.4
-1.00 0.49 7
5 2
0 0
0 As 82 Co Do 10 924938 10 MC 1.00 0.86 86 1.2 0.50 0.38 7
6 1
0 0
0 A1 86 Co Do 11 979043 11 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao Bo Co D7 12 953315 12 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao Bo C1 Do 13 954139 13 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao Bo C1 Do 14 953316 14 MC 1.00 0.86 86 1.2 0.00 0.38 7
6 1
0 0
0 As Bo C1 Do 15 946919 15 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao Bo C1 Do 16 974296 16 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao 87 Co Do 17 953362 17 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao 87 Co Do 18 977328 18 MC 1.00 0.86 86 1.2 0.00 0.38 7
6 1
0 0
0 As Bo C1 Do 19 947914 19 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao Bo C1 Do 20 967615 20 MC 1.00 0.86 86 1.2 0.50 0.38 7
6 1
0 0
0 Ao Bo C1 De 21 953423 21 MC 1.00 0.86 86 1.2 0.50 0.38 7
6 1
0 0
0 Ao 81 Co 06 22 973368 22 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao 87 Co Do 23 926304 23 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao 87 Co Do 24 953444 24 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao Bo C1 Do 25 953445 25 MC 1.00 0.86 86 1.2 0.00 0.38 7
6 1
0 0
0 A1 Bs Co Do 26 953463 26 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao Bo C1 Do 27 953488 27 MC 1.00 0.86 86 1.2 0.50 0.38 7
6 1
0 0
0 A1 Bo Cs Do 28 972777 28 MC 1.00 0.71 71 2.4 0.50 0.49 7
5 2
0 0
0 Ao 82 Co Ds 29 973366 29 MC 1.00 0.71 71 2.4 0.50 0.49 7
5 2
0 0
0 Ao Bs Co D2 30 973367 30 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao Bo Co D7 31 953489 31 MC 1.00 0.86 86 1.2 0.50 0.38 7
6 1
0 0
0 Ao 86 C1 Do 32 953502 32 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 A7 Bo Co Do 33 973369 33 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao Bo C1 Do 34 944753 34 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 A1 Bo Co Do 35 953567 35 MC 1.00 0.71 71 2.4 0.00 0.49 7
5 2
0 0
0 As Bo C2 Do 36 976684 36 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 A7 Bo Co Do 37 953569 37 MC 1.00 0.86 86 1.2 0.50 0.38 7
6 1
0 0
0 Ao Bs Co D1 38 954162 38 MC 1.00 0.43 (43) 4.3 0.50 0.53 7
3 4
0 0
0 A3 82 C2 Do 39 954182 39 MC 1.00 1.00 wo 0.0 0.00 0.00 7
7 0
0 0
0 A7 Bo Co Do 40 954183 40 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao Bo Co D7 41 954573 41 MC 1.00 0.86 86 1.2 0.50 0.38 7
6 1
0 0
0 As 81 Co Do 42 976738 42 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 A7 Bo Co Do 43 954575 43 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 A7 Bo Co Do 44 946063 44 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 A1 Bo Co Do QDC OPS IL T EXAM Page: 2 of 3 06 May 2014
2 Question Statistics (On This Test Only)
U.S. Nuclear Regulatory Commission 2014 RO Written Exam (Quad Cities)
IDENTIFICATION DIFFICULTY VARIABILITY RESPONSES Point Avg
%Avg (0. 6)
Std QSYSID QUID Type Value Score Score Diff Disc lndex 2 Dev Count Right Wrong Partial Blank Invalid MC Responses 45 933519 45 MC 1.00 0.57 (57) 3.4 1.00 0.53 7
4 3
0 0
0 Ao S1 C2 D4 46 943898 46 MC 1.00 0.86 86 1.2 0.00 0.38 7
6 1
0 0
0 As S1 Co Do 47 944158 47 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao So C1 Do 48 933171 48 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 A1 So Co Do 49 944262 49 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao So Co D7 50 953446 50 MC 1.00 0.86 86 1.2 0.00 0.38 7
6 1
0 0
0 Ao 86 Co D1 51 972767 51 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 A1 So Co Do 52 967213 52 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao So C1 Do 53 941032 53 MC 1.00 0.86 86 1.2
-1.00 0.38 7
6 1
0 0
0 A1 So Co D6 54 944699 54 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao 87 Co Do 55 940205 55 MC 1.00 0.86 86 1.2 0.50 0.38 7
6 1
0 0
0 A1 86 Co Do 56 940666 56 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao So Co D7 57 976904 57 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao 87 Co Do 58 967152 58 MC 1.00 0.86 86 1.2 0.50 0.38 7
6 1
0 0
0 Ao 86 Co D1 59 942116 59 MC 1.00 0.86 86 1.2 0.50 0.38 7
6 1
0 0
0 A1 86 Co Do 60 966988 60 MC 1.00 0.71 71 2.4 0.00 0.49 7
5 2
0 0
0 As So C2 Do 61 979297 61 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao So Co D1 62 941402 62 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao 81 Co Do 63 977401 63 MC 1.00 0.43 (4:Y 4.3
-0.50 0.53 7
3 4
0 0
0 Ao So C4 D3 64 966862 64 MC 1.00 0.86 86 1.2 0.00 0.38 7
6 1
0 0
0 A1 So Cs Do 65 946659 65 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao 87 Co Do 66 953227 66 MC 1.00 0.57 (57) 3.4 0.50 0.53 7
4 3
0 0
0 A4 S3 Co Do 67 966980 67 MC 1.00 0.71 71 2.4 0.50 0.49 7
5 2
0 0
0 A1 S1 Co Ds 68 966975 68 MC 1.00 0.86 86 1.2 0.00 0.38 7
6 1
0 0
0 Ao So C6 D1 69 953226 69 MC 1.00 0.43 ~ 4.3 0.50 0.53 7
3 4
0 0
0 A1 S3 Co D3 70 977410 70 MC 1.00 0.71 71 2.4 0.50 0.49 7
5 2
0 0
0 A1 So Cs D1 71 952178 71 MC 1.00 1.00 100 0.0 0.00 0.00 7
7 0
0 0
0 Ao So C1 Do 72 977787 72 MC 1.00 0.71 71 2.4 1.00 0.49 7
5 2
0 0
0 As So C1 D1 73 953223 73 MC 1.00 0.86 86 1.2
-1.00 0.38 7
6 1
0 0
0 As So C1 Do 74 966931 74 MC 1.00 0.86 86 1.2 0.50 0.38 7
6 1
0 0
0 A1 So C6 Do 75 967122 75 MC 1.00 0.86 86 1.2 0.00 0.38 7
6 1
0 0
0 Ao So C1 06 Difficulty: Merely an interpretation of the % average score. Diff = 6 X (1 - min), where n = % avg score+ 100.
Discrimination Index: A measure of how well the test item distinguishes the knowledgeable students from the unknowledgeable.
The "knowledgeable" students are those whose scores on this exam rank in the top 27% among all students on this exam; the "unknowledgeable" are those whose scores rank in the bottom 27%.
The index is the difference between how well the knowledgeable and unknowledgeable students scored on the item. Specifically, it is the difference in percent average score for the item, divided by 100, between these two student groups. For example, if the top-scoring students for the whole exam scored an average of 83% for a test item, while the lowest-scoring students averaged 60% for that same test item, then the discrimination index for the item equals 0.23.
Thus, values for the index can range from -1.00 to +1.00. An index of 0.30 or higher means the item is considered quite good at discriminating the knowledgeable students from the unknowledgeable, and is therefore a good test item--on this exam, at least. A negative index means there is probably something wrong with the test item, or with how the material was taught.
The discrimination index is computed only for exams with more than 5 students.
QDC OPS ILT EXAM Page: 3 of 3 06 May 2014
~, Exelon Generation Post Examination Test Item Analysis Individual Exam Analysis Lesson Plan I Course:
ILT 12-1 TQ-AA-224-F200 Revision 4 Page 1 of 2
~-------------
U.S. Nuclear Regulatory Commission 2014 SRO Written Exam Examination No./
Title:
(Quad Cities)
~~----~-------------------------------------------
- 1. Check the results of the examination.
a) D Small class (:$;6) and some questions were missed by three (3) or more students. Analysis required for these questions.
b) D Large class (>6} and some questions were missed by more than 30% of the class. Analysis required for these questions.
c) D Analysis of questions requested by individual.
d) ~ None of the above apply and no further analysis is necessary.
- 2. If an analysis is to be performed, list the question number(s) on the question analysis worksheet and review each using the following guidelines. Record the results on the worksheet.
a) Review each question to ensure it is worded clearly and is not ambiguous. Ensure that it is properly constructed. Refer to JA-AA-222-0020, "Test Item Design and Approval Job Aid".
b) Review each question to ensure that it ties properly to the learning objectives' condition, performance, and standard.
c) Review each question to ensure that it is technically accurate.
d) Review each question to ensure that adequate material was presented to allow for achievement
. of the associated learning objective.
- 3. Determine if action is required for problem questions (e.g. delete from examination bank; revise question, revise lesson plan, etc.). When action is required, ensure an assignment is initiated for documentation and tracking purposes.
Comments: Exam analysis was performed on all questions that were missed by half or more of the applicants IA W NUREG 1021 Form ES-403-1.
Completed By: Christopher Burkhart I Date: 5/6/2014 Assignment Tracking Number(s): N/A Answer Key Change I Examination Re-grade NIA IN/A Authorized by:
Group Lead I Date Retention Requirement per SRRS ID 3D.125 Training Documentation Required for Accreditation 2 Yrs
QUESTION ANALYSIS WORKSHEET Vision KIA Construction Objective Technically TQ-AA-224-F200 Revision 4 Page 2 of 2 LP Content Change Question Number Obj ID Category OK?
Based?
Accurate?
Acceptable?
Needed?
76 (Y-N)
(Y-N)
(Y-N)
(Y-N)
(Y-N) 951996 295004 y
y y
y 2.2.37 Retention Requirement per SRRS ID 30.125 Training Documentation Required for Accreditation 2 Yrs N
2 Question Statistics (On This Test Only)
U.S. Nuclear Regulatory Commission 2014 SAO Written Exam (Quad Cities)
IDENTIFICATION DIFFICULTY VARIABILITY RESPONSES Point Avg
%Avg Std QSYSID QUID Type Value Score Score (0- 6)
Diff Disc lndex 2 Dev Count Right Wrong Partial Blank Invalid MC Responses 1
951996 76 MC 1.00 0.33 33 4.8 N/A 0.58 3
1 2
0 0
0 Ao Bo C2 D1 2
951997 77 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao Bo Co D3 3
952127 78 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao Bo Co D3 4
952630 79 MC 1.00 0.67 67 2.7 N/A 0.58 3
2 1
0 0
0 Ao 82 Co D1 5
946774 80 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao 83 Co Do 6
946848 81 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao 83 Co Do 7
934382 82 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao Bo C3 Do 8
952649 83 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao Bo C3 Do 9
952562 84 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 A3 Bo Co Do 10 952656 85 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao 83 Co Do 11 966602 86 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao 83 Co Do 12 978946 87 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao Bo Co D3 13 979049 88 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 A3 Bo Co Do 14 967316 89 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao Bo C3 Do 15 977844 90 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 A3 Bo Co Do 16 974295 91 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao 83 Co Do 17 943596 92 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao 83 Co Do 18 945667 93 MC 1.00 0.67 67 2.7 N/A 0.58 3
2 1
0 0
0 Ao 81 C2 Do 19 951782 94 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao 83 Co Do 20 951902 95 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao 83 Co Do 21 938853 96 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao 83 Co Do 22 979103 97 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao Bo Co D3 23 939467 98 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao 83 Co Do 24 951926 99 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao Bo Co D3 25 967604 100 MC 1.00 1.00 100 0.0 N/A 0.00 3
3 0
0 0
0 Ao 83 Co Do Difficulty: Merely an interpretation of the% average score. Diff = 6 X (1 - n-vn), where n = % avg score+ 100.
Discrimination Index: A measure of how well the test item distinguishes the knowledgeable students from the unknowledgeable.
The "knowledgeable" students are those whose scores on this exam rank in the top 27°/o among all students on this exam; the "unknowledgeable" are those whose scores rank in the bottom 27%.
The index is the difference between how well the knowledgeable and unknowledgeable students scored on the item. Specifically, it is the difference in percent average score for the item, divided by 100, between these two student groups. For example, if the top-scoring students for the whole exam scored an average of 83% for a test item, while the lowest-scoring students averaged 60% for that same test item, then the discrimination index for the item equals 0.23.
Thus, values for the index can range from -1.00 to +1.00. An index of 0.30 or higher means the item is considered quite good at discriminating the knowledgeable students from the unknowledgeable, and is therefore a good test item--on this exam, at least. A negative index means there is probably something wrong with the test item, or with how the material was taught.
The discrimination index is computed only for exams with more than 5 students.
QDC OPS ILT EXAM Page: 2 of 2 06 May 2014
QUAD CITIES 20141LT NRC WRITTEN EXAM SEATING CHART R03 JEREMIE STOUT SRO 1 CHAD BRITION R04 JOHN VINCENT FRONT OF CLASSROOM RO 1 ANDY BELZER SR03 MATISEELEY RO 2 BEN UREK SR02 DAVID PRATI