ML093280324

From kanterella
Jump to navigation Jump to search
Initial Exam 2009-301 Final Administrative Documents
ML093280324
Person / Time
Site: Vogtle  Southern Nuclear icon.png
Issue date: 11/18/2009
From:
NRC/RGN-II
To:
Southern Nuclear Operating Co
References
50-424/09-301, 50-425/09-301, ES-201-1
Download: ML093280324 (30)
v • d • e


Text

ES-201 Examination Preparation Checklist Form FOTIn ES-201-1 Facility: Vogtle 2009-301 Date of Examination: 6/1-12/2009 6/1-1212009 Facility Examinations Developed by:

Written 1 / Operating Test Chief Target Task Description (Reference) Examiner's Date**

Initials

-180 l.

1. Examination administration date confirmed (C.1.a; C.2.a and b) Ld

£d

-120 2. NRC examiners and facility contact assigned (C.l.d; (C.1.d; C.2.e) ~~

-120 3. Facility contact briefed on security and other requirements (C.2.c) :ecf

~~

-120 4. Corporate notification letter sent (C.2.d) LL L£

[-90] [S. Reference material due (C.l.e; (C.1.e; C.3.c; Attachment 2)] LJ.-

Lc:L

(( -7S} 6. Integrated examination outline(s) due, including Forms ES-201-2, ES-201-3, ES-

~~

.£cf.,

301-1, ES-301-2, ES-301-S, ES-D-1's, ES-401-1I2, ESAOl-3, ES-D-l's, ES-401-1/2, ES-401-3, and ES-401-4, as applicable (C.l.e (C.1.e and f; C.3.d)

{-70} {7. Examination outline(s) reviewed by NRC and feedback provided to facility licensee (C.2.h; C.3.e)} ~;;i

(( -4S} 8. Proposed examinations (including written, walk-through JPMs, and scenarios, as applicable), supporting documentation (including Forms ES-301-3, ES-301-4, 4~

ES-301-S, ES-301-6, and ES-401-6), and reference materials due (C.l.e, (C.1.e, f,g f, g and h; C.3.d) 4;[

-30 9. Preliminary license applications (NRC Form 398's) due (C.l.l;(C.Ll; C.2.g; ES-202) /tJ-

/t;L

-14 10. Final license applications due and Form ES-201-4 prepared (C.l.l; (C.Ll; C.2.i; ES-202) 1"-

~L

-14 11. Examination approved by NRC supervisor for facility licensee review (C.2.h; C.3.+/-)

C.3.f) ~;i L,/-

-14 12. Examinations reviewed with facility licensee (C.l.j; (C.1.j; C.2.f and h; C.3.g) ~L

-7 13. Written examinations and operating tests approved by NRC supervisor (C.2.i; C.3.h) ~c!-

~;i

-7 14. Final applications reviewed; 1 or 2 (if >10) applications audited to confirm qualifications 1/ eligibility; and examination approval and waiver letters sent (C.2.i;Attachment (C.2.i; Attachment 4; ES-202, C.2.e; ES-204) 4~

~J

-7 IS. Proctoring/written exam administration guidelines reviewed with facility licensee (C.3.k) L,t.:£

~~

-7 16. Approved scenarios, job performance measures, and questions distributed to NRC examiners (C.3.i) 4'L

  • Target dates are generally based on facility-prepared examinations and are keyed to the examination date identified in the corporate notification letter. They are for planning purposes and may be adjusted on a case-by-case basis in coordination with the facility licensee.

[Applies only] {Does not apply} to examinations prepared by the NRC.

ES-201 Examination Outline Quality Checklist Form ES-201-2 6na.1 Facility: VOGTLE Date of Examination: 6-1-2009 Initials Item Task Description a b* c#

1. a. Verify that the outline(s) fit(s) the appropriate model, in accordance with ES-401.

W I R

R b. Assess whether the outline was systematically and randomly prepared in accordance with I Section D.1 of ES-401 and whether all KIA categories are appropriately sampled.

T T c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics.

E N d. Assess whether the justifications for deselected or rejected KIA statements are appropriate.

2. a. Using Form ES-30i -5, verify that the proposed scenario sets cover the required number ES-301-5, of normal evolutions, instrument and component failures, technical specifications, S and major transients.

I M b. Assess whether there are enough scenario sets (and spares) to test the projected number U and mix of applicants in accordance with the expected crew composition and rotation schedule L without compromising exam integrity, and ensure that each applicant can be tested using A at least one new or significantly modified scenario, that no scenarios are duplicated T from the applicants' audit testes), and that scenarios will not be repeated on subsequent days.

o c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative R

R and quantitative criteria specified on Form ES-301-4 and described in Appendix D.

3. a. Verify that the systems walk-through outline meets the criteria specified on Form ES-30i-2:

ES-301-2:

(1) the outline(s) contain(s) the required number of control room and in-plant tasks W

W distributed among the safety functions as specified on the form 1 (2) task repetition from the last two NRC examinations is within the limits specified on the form T (3) no tasks are duplicated from the applicants' audit testes)

(4) the number of new or modified tasks meets or exceeds the minimums specified on the form (5) the number of alternate path, low-power, emergency, and RCA tasks meet the criteria on the form.

b. Verify that the administrative outline meets the criteria specified on Form ES-30i-1:

ES-301-1:

,I;~~ rAJ ,1;1-(1) the tasks are distributed among the topics as specified on the form (2) at least one task is new or significantly modified (3) no more than one task is repeated from the last two NRC licensing examinations

c. Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days.
4. a. Assess whether plant-specific priorities (including PRA and IPE insights) are covered in the appropriate exam sections.

G E b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate.

N

c. Ensure that KIA importance ratings (except for plant-specific priorities) are at least 2.5.

E R d. Check for duplication and overlap among exam sections.

A L. e. Check the entire exam for balance of coverage.

f. Assess whether the exam fits the appropriate job level (RO or SRO).

.Date "

a. Author e.

,I:; - iJ -0'1

b. Facility Reviewer (*) &;-20-C>~
c. NRC Chief Examiner (#) .$I.2@?
d. NRC Supervisor iifltk~1 Note: # Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.
  • Not applicable for NRC-prepared examination outlines

ES-201 ES*201 Examination Outline Quality Checklist ES-201-2 Form ES*201*2 Fill ;a. \

Facility: Vogtle Date of Examination: 06-26-2009 Initials Item Task Description a b* c#

1. a. Verify that the outline(s) fit(s) the appropriate model, in accordance with ES-401. AlIA rJ/~ ~t w

W R b. Assess whether the outline was systematically and randomly prepared in accordance with I 0.1 of ES-401 and whether all KIA categories are appropriately sampled.

Section D.1 N/~ rJ/ A li:J.

T T c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics. NIIt NiAl-lt

,~ A1 Lt.

E N d. Assess whether the justifications for deselected or rejected KJA KIA statements are appropriate.

v

2. a. Using Form ES-301-5, verify that the proposed scenario sets cover the required number S

of normal evolutions, instrument and component failures, technical specifications, and major transients. NIA- '" JA "/A I

M b. Assess whether there are enough scenario sets (and spares) to test the projected number I U and mix of applicants in accordance with the expected crew composition and rotation schedule L

A without compromising exam integrity, and ensure that each applicant can be tested using at least one new or significantly modified scenario, that no scenarios are duplicated AIM Ii/A T from the applicants' audit test(s), and that scenarios will not be repeated on subsequent days.

o0 R

c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative and quantitative criteria specified on Form ES-301-4 and described in Appendix D. AI/It fll"
3. a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2:

(1 ) the outline(s) contain(s) the required number of control room and in-plant tasks (1)

W distributed among the safety functions as specified on the form wl~ IJ/A 1I (2) task repetition from the last two NRC examinations is within the limits specified on the form T (3) no tasks are duplicated from the applicants' audit test(s)

(4) the number of new or modified tasks meets or exceeds the minimums specified on the form

( altemate path, low-power, emergency, and RCA tasks meet the criteria (5) the number of alternate on the form.

b. Verify that the administrative outline meets the criteria specified on Form ES-301-1:

(1 ) the tasks are distributed among the topics as specified on the form (1)

(2) at least one task is new or significantly modified IV/A ,,'"

W/A #/"

(3) no more than one task is repeated from the last two NRC licensing examinations

c. Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days. I'IJA- IV/A 'It
4. a. Assess whether plant-specific priorities (including PRA and IPE insights) are covered in the appropriate exam sections. <11 I1j ~.t G

E b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate. <~' III .eX N

E

c. Ensure that KIA importance ratings (except for plant-specific priorities) are at least 2.5. <~ ~;
A .£::1' R d. Check for duplication and overlap among exam sections. (~~ f{/ ~.f.

A AA L e. Check the entire exam for balance of coverage. ,,<I(;!Y- 41 ,l;t.

f. Assess whether the exam fits the appropriate job level (RO or SRO). W ~. I~j.

v

~~ rr:'IS":J:atureL

-lrintAl4 '!T:'~~aturek

~ ..(Jyv

{p Date

a. Author
b. Facility Reviewer (*)

T. N. Thompsonl D. Scukanecl ~ J~~'L

~.

{/

A V "1-¥)

6~/1-

c. NRC Chief Examiner (#) JidlAJ'" Le 2 ;Jr'. /~..1 .>.. ~ /~ L~ ,,-gS/olj I 7
d. NRC Supervisor .uLI ,-.~ ..... "'C. W(O~fAI / ...~ 1: d"tflb'

'(!!I1.u,,'-'!TJ1J.Jy Note: # Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.

  • Not applicable for NRC-prepared examination outlines

ES-201 Examination Security Agreement Form ES-201-3

1. Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of 6/1/2009 as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as~s a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in.cancellation in cancellation of the examinations andlor and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.
2. Post-Examination To the best of my knowledge, I did not divulge to any unauthorized unauthorized persons any information concerning conceming the NRC licensing examinations administered during the week(s) of 6/1/2009 . From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME JOB TITLE 1 / RESPONSIBILITY

1. Thad N. Thompson Operations Instructor r

-:J.-.~ '7

.r-._

f ,.."._

.. v,....... ..,..- .. -~-. -- . ' ,"".-.~

_A _ n " .

~ ,

() . ." ..,/-[ ". ~.""" ~~ ~

j _ * *

  • _ __ .* , #~

0 .

ES-201, Page 26 of 27

ES*201 Examination Security Agreement Form ES*201*3 ES-201-3

1. Pre*Examination Pre-Examination knowledge about the NRC licensing examinations scheduled for the week(s) of 6 I acknowledge that I have acquired specialized knowledge' :z~",o<l as of the date It / Qcr 6/1'; to of my signature. I agree that I Willwill not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. IJ understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and feedback),

understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2. Post-Examination To the best of my know!~dge, know!*;d~e,l,did unauthori~ed persons I did not divulge to any unauthorized p~rsons any co~cerning the NRC a.ny information concerning ~RC licensing licensi~g examinations

~xamina~i~ns a?minist.ered administered 6/1/1. 0 ~'f. From the date that I entered into during the week(s) of 6!1/J,o Into this security secunty agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME JOB TITLE / RESPONSIBILITY SIGNATURE (1) DATE SIGNATU~E (2)

$IGNATU2.E DATE NOTE

- - - - - - - - - -----------,.-------- ~-~----------------------------..,}~'.,.'------ ---

~+---------

~---..::...---------

ES-201, P 27 of 28

ES-201 Examination Security Agreement Form ES-201-3

1. Pre-Examination I acknowledge that I have acquired specialized knowledgeabout the NRC licensing examinations scheduled for the week(s) of 6-/-0 T

'1 as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if ifthe the individual does not select the training content or provide direct or indirect feedback). Furthermore, Ilam am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2. Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered of6'" 1- 0 during the week(s) of6"/- 1. From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME SIGNATURE (1) DATE DATE NOTE fP3L

~:1. ~~ ~p7: ~~~~~~~~~~_ _-ri~~~~~~~_ _~~~~_6:;{gr 6-a~",_

2. 2~,i~~ -xiUty ~~

~e1.~

3'1::1L~~ J~f\.{

ri,~:g(

,,-;1Q -Oq

~:%h*~~t/iiti£

6. ___ _ _ _ __

--=c.-_ ____ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _-'-_ _ _ _ _ _ _ - - - - - - - - / - - - - - _ _ __

~~~t;--g(

I ,I

7. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ __ ____ ___ __ __ __ __ _ __ ___
8. ___ __ ___ ______ __ __ ___ __ __ __ __ __ _ __ ____ ____ ___ ____ __ __ - ____ ______ _____ __ _ __ _-_-_________
9. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ __ _ ____

10 10.. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ __ _ ____

11 . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __

11. __ _ _ ___

12

12. . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

13 . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __

13. __ ____ __

14.

14 . _____ _ __ ____ __ _ __ ____ __ _ ____ __ __ _ __ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __

15.,_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

15.

NOTES:

ES-201, P 27 of 28

ES-301-3 Operating Test Quality Checklist P,,,,

P,4 s..

a.. I Facility: Vogtle Date of Examination: 6-1-2009 Operating Test Number: 2009-301 Initials

1. General Criteria
a. The operating test conforms with the previously approved outline; changes are consistent with sampling requirements (e.g., 10 CFR 55.45, operational importance, safety function distribution).
b. There is no day-to-day repetition between this and other operating tests to be administered during this examination.
c. The operating test shall not duplicate items from the applicants' audit test(s). (see Section D.1.a.)
d. Overlap with the written examination and between different parts of the operating test is within acceptable limits.
e. It appears that the operating test will differentiate between competent and less-than-competent applicants at the designated license level.
2. Walk-Through Criteria
a. Each JPM includes the following, as applicable:

initial conditions initiating cues references and tools, including associated procedures reasonable and validated time limits (average time allowed for completion) and specific designation if deemed to be time-critical by the facility licensee operationally important specific performance criteria that include:

detailed expected actions with exact criteria and nomenclature system response and other examiner cues statements describing important observations to be made by the applicant

( criteria for successful com completion pletion of the task

.. identification of critical steps and their associated performance standards restrictions on the sequence of steps, if applicable

b. Ensure that any changes from the previously approved systems and administrative walk-through outlines (Forms ES-301-1 and 2) have not caused the test to deviate from any of the acceptance criteria (e.g., item distribution, bank use, repetition from the last 2 NRC examinations) specified on those forms and Form ES-201-2.
3. Simulator Criteria The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES-301-4 and a copy co is attached.

Date

a. Author 5~ "e,.()" (}j
b. Facility Reviewer(*) ..!D~a![n~ie~I-,S~c<bul.!SkE.afrne~c;"'/L.c.~~~~~~.dJ!.~~~~::'::::::::=-_ _ __

5' -2<J-o tl

c. NRC Chief Examiner (#) 5iJ70,pJ)7 I ~7
d. NRC Supervisor ~s1Z1-;;'7 NOTE:
  • The facility signature is not applicable for NRC-developed tests.
  1. Inde en dent NRC reviewer initial items in Column "c"; chief examiner concurrence re Independent uired.

required.

ES-301-4 Simulator Scenario Quality Checklist r- I NIV "" LJ J L.!

Facilty: Vogtle Date of Exam: 6-1-2009 Scenario Numbers: 1 /2/3 ~T O(2erating Test No.: 2009-301

"""" 301 QUALITATIVE ATTRIBUTES Initials a b*

b' c#

~~

.~. pf tJ

1. The initial conditions are realistic, in that some equipment and/or instrumentation may be out of service, but it does not cue the operators into expected events. tj
2. The scenarios consist mostly of related events. ~~

~ ~. ~ lJ III

3. Each event description consists of
  • the point in the scenario when it is to be initiated
  • the malfunction(s) that are entered to initiate the event dpi

(~ J/ Jt

  • the symptoms/cues that will be visible to the crew
  • the expected operator actions (by shift position)
  • the event termination point (if applicable) '#

r~~ I~ jj

~~

4. No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario without a credible preceding incident such as a seismic event. "\~ ~" I
5. The events are valid with regard to physics and thermodynamics. ~~

~ ~ IJ;J-- 1,J--

~~ pJ d

~.~

6. Sequencing and timing of events is reasonable, and allows the examination team to obtain complete evaluation results commensurate with the scenario objectives.
7. If time compression techniques are used, the scenario summary clearly so indicates.

Operators have sufficient time to carry out expected activities without undue time constraints.

Cues are given.

W~:

~~\

<~r\

...\'

d "

A

'I ~J

,tJ

~~ .~

" r~ llt

8. The simulator modeling is not altered. tJ¥~ l,j

( I

( 9. The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator performance deficiencies or deviations from the referenced plant have been evaluated to ensure that functional fidelity is maintained while running the planned scenarios. \~~

~~

\

\

4 );;f

);j.

10. Every operator will be evaluated using at least one new or significantly modified scenario.

All other scenarios have been altered in accordance with Section 0.5 of ES-301. ~~'

~ M It)

--:~ ,tJ

11. All individual operator competencies can be evaluated, as verified using Form ES-301-6

,~~ ~ 1i

,#~ t;! 1.1

,r~K (submit the form along with the simulator scenarios).

12. Each applicant will be significantly involved in the minimum number of transients and events specified on Form ES-301-5 (submit the form with the simulator scenarios). ~ ~~; '((f

\ \ ,\ '?(f );j

  • jJ
13. The level of difficulty is appropriate to support licensing decisions for each crew position. ~~ 'At

~~: 'At III Target Quantitative Attributes (Per Scenario; See Section D.S.d) Actual Attributes -- -- --

1. Total malfunctions (5-8) 10/9/8 -ti
2. Malfunctions after EOP entry (1-2) 4/5/3 11-

.4:1

3. Abnormal events (2-4) 5/3/3 Ll;J-2::£
4. Major transients (1-2) 1 /1

/ 1 /2 I%J

£J

5. enteredlrequiring substantive actions (1-2)

EOPs entered/requiring 2/3/2 ~;l

,t;f

6. EOP contingencies requiring substantive actions (0-2) 0/2/0 ~1.
7. Critical tasks (2-3) 3/6/4 A:J

~

ES-301-4 Simulator Scenario Quality Checklist ~/#"4. ,::'/111""

II Facilty: Vogtle ~

Date of L .L ....

Exam:. 6-1-2009

" 1 " ...

. Scenario Numbers: 4 / / 0 O[!erating .~ .

Testt No.: 2009-301 QUALITATIVE ATTRIBUTES Initials a b* c#

1. The initial conditions are realistic, in that some equipment and/or instrumentation may be out of service, but it does not cue the operators into expected events. # i;;J

~

"Ii~ ,-
'{tl\

~{

~~ I{f

(- 'jJj l/qj /4J

2. The scenarios consist mostly of related events. £fJ 1.1 It
3. Each event description consists of
  • the point in the scenario when it is to be initiated
  • the malfunction(s) malfunction( s) that are entered to initiate the event

'~

4.

  • the symptoms/cues that will be visible to the crew
  • the expected operator actions (by shift position)
  • the event termination point (if applicable)

No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario

'\'<

it

~

~: ~~ IJ)

<,\\1\

,\\1' Jj

);j without a credible preceding incident such as a seismic event.

5. The events are valid with regard to physics and thermodynamics. ~01\' Pf

~~\' pr IJf j,J

6. Sequencing and timing of events is reasonable, and allows the examination team to obtain complete evaluation results commensurate with the scenario objectives. ~~

~ w-~

\~

\)1 a tI~ tl1 iJj jj

~,.. q HI"'. 11)

7. If time compression techniques are used, the scenario summary clearly so indicates.

Operators have sufficient time to carry out expected activities without undue time constraints.

Cues are given. <,/, tJ

8. The simulator modeling is not altered. ~:@W~I

~ ~~(

I4r );J Illi j;J

~\ t1I ¢(/

9. The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator performance deficiencies or deviations from the referenced plant have been evaluated to ensure that functional fidelity is maintained while running the planned scenarios. ef' I\~

1,,1 1.j

~ ~~ ('\It(!

.fJf );J

~

10. Every operator will be evaluated using at least one new or significantly modified scenario.

All other scenarios have been altered in accordance with Section D.5 0.5 of ES-301.

< ," (\'

j;J

<~~ 41

~~~ 4! 11; lJ

11. All individual operator competencies can be evaluated, as verified using Form ES-301-6 (submit the form along with the simulator scenarios).

?f~ QI W'~ QJ....... I~J

,I'

/1'

12. Each applicant will be significantly involved in the minimum number of transients and events specified on Form ES-301-5 (submit the form with the simulator scenarios). ~~~~\

'<~\ lJ

13. The level of difficulty is appropriate to support licensing decisions for each crew position. ~ I~

<~

~i I~

Target Quantitative Attributes (Per Scenario; See Section D.S.d) Actual Attributes -- -- --

1.

i. Total malfunctions (5-8) 9/ / 11

.,It

2. Malfunctions after EOP entry (1-2) 5/ / ~i

.II

3. Abnormal events (2-4) 4/ / ./,1

~L

4. Major transients (1-2) 2/ / -l.:i L:i
5. EOPs entered/requiring substantive actions (1-2) 2/ / Lbt

,l;f-.

6. EOP contingencies requiring substantive actions (0-2) 0/ / l.ej

.cJ

7. Critical tasks (2-3) 2/ / ~;/

.1;1

05/28/2009 07:3l 07:3L FAX 706 826 3953 VEGP TRAINING CENTER 141 003 141003

""j.

--,,:.' ES-a01-4 ES-:101-4 Simulator Scenario Quality Checklist .F ~.4 L.J.

F //HALl

===----~===-===================------===========~--=======-=9 I.E!£\~

~~!!l;.!S:.;;V.;:.09!1,;tI;;;;e;,...,J!D~a~te1:i0~f:SE~x~am~;6-~1.;;;2,;OO:;;9;......3S~ce~na~rI~o~N~u~m!!!!be~rs~:==4~/~/=~O~pe~ra~ti~*n~g!=~~8!;1lSlt;;t~N~o*iio:~20~O::;9~-3~0~1r=======o====l1 Vogtle Date of Exam: 6-1-2009 Scenario Numbers: 4 I I Operating Test No.: 2009-301 QUALITATIVE ATTRIBUTES Initials a b* 0#

c#

~'--------------------------------------------------~~~-r~~1

1. The initial conditions are realistic, in that some equipment and/or Instrumentation may be out ~~" r /JJ. ~~.t of service, seNice, but bl,lt it il does not OI.Ie we the operators into expeeted expected events. ,,0:. I YO r-------------------------~--------~~--------------------------~~~~~~~~I

~. T_h_e_s_~

2. __Ti'le __ na_ri_o_s_eo_n_s_~_t_m_o_s~~y_~

scena~ios consist mostly of ~_I_a~

__related

___ events. __________________________________4~~~~-~4'~~~~~~~~.J e_ve_n_ts_.

3. Each event description consists of
  • the point in the scenario when It Is to be Initiated belnltlaled
  • the ti'le malfunetion(s) malfunction(s) that are a~ entered to initiate the event
  • the symptoms/cues that will be visible to the crew
  • Ifle the expected operator actions (by shift Si'lift position) i" te~mlnatlon paint the event termination point (if<if appliclilble) applicOible) r--'
4. No mOre than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario without a credible preceding Incident such suci'l as a selsmie event

~.

5. The events are valid with regard to physics and thermodynamics. thennodynamics.
6. Sequenclng and timing of events Is reasonable. and allows Ifle the examination teem teOim to obtain IJOITlplete evaluatioll resull$

complete evaluation results oommensurate commensurate with the scenario objectives.

7. If time compression techniques are used, the scenario summary dearly clearly so Indicates.

Operators have sufficient time to carry out expected activities without undue time constralnt:s. constraints.

Cues are given.

/ "

.:1.11

8. The simulator modeling Is not altered altered..

f----

( /'

I'

9. The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator performance deficiencies or deviations devialions fromft'om the!;he referenced plant planl have i'lave beell I;Jeen evaluated to ensure that functional fidelity is maintained while running the planned scenarios.

-10. Every operator will wlll be evaluated using at least one new or slgnlflcandy significantly modified SOElnario.

scenario.

All other scenarios have been altered in accordance with Section D.5 0.6 of ES-301.

ES-.301.

-11. All individual opera~or operator competencies can be evaluated, as verified using Fom'l ~on'1'\ 5S-301-6 eS-301-6 (submit the funn folTT1 IiIlollg 0I10n9 with the ti'le simuletor simulOitor scenariOS).

r-

12. Each applicant will be significantly involved in the minimum number of transients and events sp$CIfied spacltied on Fonn Fol'I'I'\ ES-301-5 (submit !;he the form with the simulator scenarios).

r-

13. The level of 01' difficulty diffioulty is appropriate to support lioensing decisions for each crew position.

i---

Target Quantitative Attributes (Per Scenario; SCflnarlo; See SectionS8ctlon D.5.d) Actual Attributes - ..- ---

-2.:2.1. Total malfunctions (5-8) g/ /

91 ~II¥. I_/!Ii. l£tX I~;t:

Malfunctions after afief EOP entry (1-2) 51 J 6/1 ~~ 1A¥

~ l/{f ~/ ~;

~

3. Abnormal events (2-4)

Abnonnal 41 I/ ~~ l,q

(~ 1#1' ~..t ~~
4. Major transients (1-2) 2/1I 2/ 44~~ ~~Iq Ig .b h I--

~.

5. EOPs enteredJrequiring entered/requiring substantive actions aotions (1-2) 2 J /I 2/ ,~~

1 'AY ..H i:trtt '/tF ~

6. eop EOP contingencies I'EIql,liring requiring $ubstantive substantive actions (0-2) oI I 011 ~

.oC ~~ 9 ,q ~,;t. ~;.

1--.

=.

7, Critical tasks (2~3) (2-3) 2/ /

21 :q.

~ '(If/. Ib g.

05/28/2009 07:3l 07:3L FAX 706 826 3953 VEGP TRAINING CENTER III 002 141002 ES-::01-4 Simulator Scenario Quality Checklist Fnv44 p" N4 4

~==========================----~==========--------~

r==="- '

.~~ ill ffi Vogtle VogUe Date of Exam: 6-1-2009 Data 6" 1"2009 Number!:!: 1 12/3 Scenario Numbers: Operating 2009-301 O~ratlng Test No.: 2Q09-301 QUALITATIVE QUAl.ITATIVE ATTRIBUTES AlTRIElUTES Initiais Initials a b" 0#-

C#

~------------------------------~r_~~~~~~I

.i:-t

~ IJf ~-t

1. The Inllial In!tlal c:ondltlons condilions are realistic:,

realistic. in that some equiprnent equipment end/or andlor instrumentation in5trumentation may be out ~~

of service, service. but it does dOe:! not cue the operators into expected events. ~.

r--

~------~------------~------~------------------------~~~~~~I r

2.

The seenariQ$ consist mostly of related events, sceMrios consl$t events.

~------------------~~--------------------------------------------~~_4~~+~~,~,_~1

~~. IJf ~:l If .J1 fi7

3. Each event description consists of
  • the point in the scenario when it is to be initiated
  • 6 the malfunction(l;)

malfunction(s) that <Ire the symptoms/cues that wI!!

Olre entered to initililte initiate the event will be visible to the crew d tJ-.

the expected operator actions actlon$ (by shift posl~on)

$hlft posl1:lon) the event termination point (if applicable)

'#IP r.~

4. No more than one non*mechanlstlc non-mechanistic failure (e.g .* pipe break) Is incorporated Into InlO the sc:enar'lo scenado without iiia credible preceding incident such as iiia 5eismic 5eisrnic event. ~ ~.:f rr----- A 5 ..

r---

I---

The events are valid with regard to physics and thermodynamics. ~~ .~ L:f-

~~ ,oJ It.J

6. Sequencing and timing of events is i5 reasonable, and allows the examination team to obtain

-- , complete evaluation results ccmmensurate commensurate wIth with the scenario objectives.

~~ at It

7. If tim timee compression techniques are used, the scenario summary clearly so Indicates. ,\.'

Operators have sufficient time to carry out expected activities without undue time ccnstraints. constraints.

Cues are given.

~

A

.~ -,

-BB.. , The simulator modeling is i5 not altered.

altered, ~~ .~

L.i

"!  :- L

,@* Ii ~J-

9. The Thl'! scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator performance deficiencies !;Ir or deviations from the referaneed referenced plant have ha.ve been evah.Jated evaluated to ensure enSure that functional fidelity Is maintained while running the planned scenarios.

r----

10.

r----

Every operator will be evaluated I,lslng using at least one new or significantly modified scenario.

All other scenarios have been altered in accordance with Section D.5 0.5 of eS*301.

scenario,

~~ fJI' ~;t

11. All individual operator c:ompetencles competencies can be (submit the form along with .the simulator scenarios).

b@ evaluated. as verified using Form ES*301*6 ES"301*6

~

~,; rII, ~cf w:

r----'

12.

r----

~-

Each applicant will be significantly Involved involved In the minimum number of transients and events specified on Form ES*301-5 (submit the form with the 5irnl,llOltor simulOltor 5Ceflarl05).

5cenarios). ~ 'klf lt~

=

13.

The level of dlfflClJlty difficulty Is appropriate to support licensing decisions for each crew position. ~~ 'A'.! k,),,1

~.

Target Quantitative Attribl,ltes Attribl,lte$ (Per (Pet Scenario:

Seen;'lfio; See Saetion

~--~------------~--------~----------~----+_------------_4~~~~~--~1 Seotion D.S.d)

D.5.d) Actual Attributes --

~

-, 1. Total malfunctions (5-8) 10/9/8 ~ AI' R lu 11/

2. Malfunctions after eop eop entry (1-2)

~----------------~~~------------------~--+-------------~~~~~~~I

~,

4/513 4/5/3 ~~ra f)/

/lI'lQ 0

~~A~bn~o~rm~al~e~ve~n~ts~(~2_-4~)~

-, 3. Abnormal events (2-4) ________________________--____ -+________ ____ ~tii 5/3/3 ~ ~

~~~~~~~

pI.J ~ Id l.d ~~

/ 4.

r----

~

Major transients (1-2) 1 11/2 1/1/2 ,~1~

V"........ ,pp I2fI M r--

5. eops eops entered/requiring substantive actions (1-2)

~----------~~~----------~~--~--------~+-~~--~----~~~S6~~~

2/3/2 ~ tJ/

,~ d ~ § 6.

r----

contlngencies requiring substantive actions (0-2)

EOP contingencies

~--~~~~----------~--~------------+-------------~~~~~~~~I 0/2/0 ~~ ~. ,.1# "Ifj

~ A..l ~~ It.~

=

=

7. CritiC<lI Critic",1 task&

tasks (2-3) 3/6/4 ~fJl!)J" t'i1i!Jl( dd.

(

ES-301-5 Transient and Event Checklist

=================================================================9

Vogtle Nuclear Plant Date of Exam: June 1-5,2009 Operating Test No.: 2009-301 A E Scenarios P V V 1 2 3 4 T M P E o I L N N CREW CREW CREW CREW T N I T POSITION POSITION POSITION POSITION A I

M C S A B S A B S A A B S A A B L U U

A T R T o R T o R T o R T o M(*)

N Y o C P o C P o C P o C P R U T P E

RX 4 o SRO-I xx NOR 2 SRO-U I/C 23 4 4 2 x

MAJ 6 2 2 TS 11 0 2 2 RX 4 o RO NOR xx SRO-I IIC 13 4 4 2 x

MAJ 6 2 2 TS o o 2 2 RO RX 4 o x

NOR SRO-I x I/C 11 4 4 2 MAJ 6 2 2 TS o o 2 2 Instructions:

1. Check the applicant level and enter the operating test number and Form ES-D-1 event numbers for each event type; TS are not applicable for RO applicants. ROs must serve in both the "at-the-controls (ATC)" and "balance-of-plant (BOP)"

positions; Instant SROs must serve in both the SRO and the ATC positions, including at least two instrument or component (I/C) malfunctions and one major transient, in the ATC position. If an Instant SRO additionally serves in the BOP position, one IICI/C malfunction can be credited toward the two IIC malfunctions required for the ATC position.

2. Reactivity manipulations may be conducted undernormal under normal or controlled abnormal conditions (refer to Section D.5.d) but must be significant per Section C.2.a of Appendix D. (*) Reactivity and normal evolutions may be replaced with additional instrument or component malfunctions on a 1-for-1 basis.
3. Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to the applicant's competence count toward the minimum requirements specified for the applicant's license level in the right-hand columns.

ES-301-6 Competencies Checklist r/#A-£...

r01/A-L..

Facility: Vogtle Date of Examination: 6-1-2009 Operating Test No.: 2009-301 APPLICANTS Competencies RO X SRO-I X SRO-U X SCENARIO SCENARIO SCENARIO 1 2 3 4 1 2 3 4 1 2 3 4 Interpret/Diagnose InterpretlDiagnose 12 12 1 2 12 12 12 12 12 12 12 12 123 12 12 12 12 12 Events and Conditions 34 34 34 34 34 34 34 34 456 34 34 34 56 56 56 56 56 56 56 56 789 56 56 56 78 78 7 78 78 78 7 78 78 7 78 9 9 9 9 9 Comply With and 12 12 1 2 12 1 2 12 12 12 123 12 12 12 12 12 Use Procedures (1) 34 34 34 34 34 34 34 34 456 34 34 34 56 56 56 56 56 56 56 56 789 56 56 56 78 78 7 78 78 78 7 78 78 7 78 9 9 9 9 9 Operate Control 12 1 2 12 12 12 Boards (2) 34 34 34 34

( 56 56 56 56

\ 78 78 7 78 9 9 Communicate 12 1 2 12 12 1 2 12 12 12 123 12 12 12 12 and Interact 34 34 34 34 34 34 34 34 456 34 34 34 56 56 56 56 56 56 56 56 789 56 56 56 78 78 7 78 78 78 7 78 78 7 78 9 9 9 9 9 Demonstrate 12 12 12 12 12 12 123 12 12 12 Supervisory Ability (3) 34 34 34 34 456 34 34 34 56 56 56 56 789 56 56 56 78 78 7 78 78 7 78 9 9 9 Comply With and 12 12 12 12 12 12 123 12 12 12 12 Use Tech. Specs. (3) 34 34 34 34 456 34 34 34 56 56 56 56 789 56 56 56 78 78 7 78 78 7 78 9 9 9 Notes:

(1 ) Includes Technical Specification compliance for an RO.

(2) Optional for an SRO-U.

(3) Only applicable to SROs.

Instructions:

Check the applicants' license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.

ES-401 Record of Rejected KI As KlAs Form ES-401-4 Tier / Group Randomly Reason for Rejection Selected KIA T2/G1 061A2.02 SRO T2/Gl Vogtle design does not have air operated steam supply valves for the AFW system. 061A2.03 was randomly selected as a replacement.

SRO T2/G2 071A2.03 Vogtle design does not use rupture disks for the Waste Gas Disposal System. 071A2.09 was randomly selected as a replacement.

ROT3 G2.4.41 Emergency classification thresholds are not RO duties at Vogtle.

G2.4.43 was randomly selected as a replacement.

(

ES-401, Page 27 of 33

ES-401-6 Written Examination Quality Checklist F! N t:I~ L./.J fi Facility: Vogtle Date of Exam: 6-26-2009 Exam Level: RO X SROX SRO X Initial Item Description on a b* c#

1. Questions technically accurate and a ns and answers are technicall applicable icable to the facility.

facili

~;:

"'1~~ .; I(( l--t.t

2. a.

b.

NRC K/As are referenced for all questions.

Facility learning objectives ectives are referenced as available.

~i d 1,j

3. SRO questions are appropriate in accordance with Section D.2.d of ES-401 ~ _(2i ltt 1-
4. The sampling process was random and systematic (If more than 4 RO or 2 SRO questions were repeated from the last 2 NRC licensinglicensi exams, consult the NRR OL program ram office).

office 10;" : ~l, t

5. Question duplication from ihe the license screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate:

_ the audit exam was systematically and randomly developed; or d

W

_ the audit exam was completed before the license exam was started; or

_ the examinations were developed independently; or XX the licensee certifies that there is no duplication; or .~ );1

_ other (explain)

6. Bank use meets limits (no more than 75 percent Bank Modified New .. PI );1

~

from the bank, at least 10 percent new, and the rest t' new or modified); enter the actual RO / SRO-only question distribution(s) at right.

estion distributi ht.

11 /2 10/ 1 54/22 I,r

7. Between 50 and 60 percent of the questions on the RO Memory CIA q} -4JI exam are written at the comprehension/ analysis level;

~

~1 the SRO exam may exceed 60 percent if the randomly I:

( selected K/As support the higher cognitive levels; enter 33/2 42/23 t..~"

the actual RO / SRO question uestion d distribution(s) at right.

8. References/handouts provided do not give away answers or aid in the elimination of distractors. {)/ 4' J;.1
J~ W
9. Question content conforms with specific KIA statements in the previously approved examination outline and is appropriate for the tier to which they are assigned; deviations are justified. it
10. Question psychometric n<"",nnmetric quality and format meet the guidelines in ES Appendix ix B. @ .?if v1;J
11. The exam contains the required number of one-point, multiple choice items; the total is correct and agrees with the value on the cover sheet.

~: ¢if )i Printed ~nat~~l~ Date

a. Author Thad N. Thompson /

I rI;,'J '2'L,,-r.;~I '-19 "()1

b. Facility Reviewer (*) Daniel Scukanec / Po-e::k" ,11'" ;<_ {/ 6-;1;.)~<'.}1

.edu..*.n he.a -dV' /~ ~'><..-::J 2'~ th~

c.

d.

NRC Chief Examiner (#)

NRC Regional Supervisor LUJ,~ -r: WiTJAUIiJ,J / ~ ~

/ ~'qlMff ~

/ t$Bt; fJ' ,

Note:

  • The facility reviewer's initials/signature are not applicable for NRC-developed examinations.
  1. Independent NRC reviewer initial items in Column "c"* "c"; chief examiner concurrence required. uired.

ES-401 Written Examination Review Worksheet -- Vogtle SRO Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back- Q= SRO B/M/N u/EiS U/E/S Explanation Explanation Focus Dist.

Dis!. Link units ward KIA Only Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. . Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
3. Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (Le.,(i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

The question is not .linked linked to the job requirements (Le.,

(i.e., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (Le., (i.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any "U" "u" ratings (e.g., how the Appendix B psychometric attributes are not being met).
1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem I Cues Stein, Focus Cues I T/F c~ed'l T/F I Credo Partial Dlst.

Dis!.

Partial JOb-I Job- Minutia Link Minutia I #/,

  1. / I Back-Back-units ward Q= ISRO Q= SRO B/M/N U/EiS KIA Only U/E/S Explanation

Q#

1.

1.

I LOK I 2.

2.

LOD

3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

(F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 1 H X I X X X X N U Supporting documentation provided in package is incomplete. Underlining and bolding in stem provides cue. Charging line flow fluctuating between 30 and 130 gpm will not result in charging line HIILO FLOW alarm. Based on the information given, explain what would cause the VCT HI/LOW LEVEL alarm? Based on the information provided, explain why one would conclude there was a loss of letdown flow. If that conclusion can not be reached based on information given, then distractors C and Dare not plausible.. It also appears that this question can be answered based on RO knowledge alone.

Not SRO. REVISED -- MODIFED STEM AND DISTRACTORS (OK) 2 I H I 1 I I X I I N I U U IBased on the information provided, this question

§.§. appears to be a collection of true false statements. The questions can be answered with general knowledge. Based on note associated with TS and procedures concerning PORV operation. It is also general knowledge, that when a peace of TS equipment is not operable, proceeding to a higher mode is not allowable.

Very low level of difficult. SRO only. ---

CONVENCED US THAT THIS WAS AN SRO QUESTION AND WAS ACCEPTABLE. NOT A U. (OK AS WRITTEN) 3 I I xX I N I u IThe question can be answered with RO only knowledge. Determining that an ATWS exist is passed on system knowledge and the LCO entered is based on the applicants understanding of what LCO must be entered when an ATWS occurs. This is RO knowledge. CHANGED STEM AND DISTRACTORS (OK)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only I 4 N S MADE A CHANGE TO THE STEM AND DISTRACTORS (OK) 5 H N S MADE CHANGES TO THE DISTRACTORS (OK) 6 M S DID REWORD STEM (Ok) 7 X N U Not SRO only. Question can be answered with system only knowledge REWORDED THE STEM AND DISTRACTORS (OK) 8 H 2 X E E May be two correct answers. Both B & & D.

REWORDED DISTRACTORS TOOK OUT RETURN TO LEVEL. UNDERSTANDING LCO REQUIREMENT ASSURES KIA MATCH (OK) 9 F 1 X N u LOD. Simple Memory. Which one describes the .... Not SRO only WROTE A NEW QUESTiON (OK) 10 H 2 X N U Question can be answered using RO knowledge only WROTE NEW QUESTiONS (OK) 11 H 3 N sS ADDED WORDS TO THE DISTRACTOR (OK) 12 H 3 N s REWORDED DISTRACTORS AFTER REVIEWING QUESTION. (OK) 13 2 N sS (OK) 14 H 2 X N U Not SRO only. Question can be answered with RO/systems knowledge. For the given conditions wilen should feed and bleed be terminated? I would think that the RO would know that SG level must be restored before terminating. NEW QUESTION (OK) 15 H 3 N sS REWORDED THE STEM (OK)

1.

1. I 2.
2. 3. Psychometric Flaws 4. Job Content Flaws , 5. Other , 6.

, 7.

, 8.

Q# I LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 16 ,

16 H , 2 X N U Question can be answered with system knowledge only. What actions should be taken if a loss of CV is occurring? Reduce power. How is this achieved? What should be observed while reducing power and what actions are required to keep ROD BANK LO-LO LIMIT alarm clear?

What was the initial power level? WROTE NEW QUESTION (OK) JOHN DOES NOT LIKE THE QUESTION, THINKS IT IS BEYOND SRO KNOWLEDGE .... I DISSAGREE. WE DISCUSSED THIS AGAIN AND CAME TO AN AGREEMENT.

17 I F? I I XX , N

, uU Not SRO only. What steps are marked N/A?

'Not Could not locate in the procedure where recirculation of -~ 1hour is required prior to sampling. Is this a general knowledge question?

I do not see where procedures were used to correct/control ......

correct/control. ..... CORRECTED INFORMATION IN THE STEM. QUESTION SHOULD NOT HAVE BEEN UNSAT. IT IS THE SRO RESPONSIBILITY TO REVIEW/APPROVE RELEASE PERMITS. {OK} (OK}

18 18 , H H I I xX I N I U INot SRO only. Can be answered with system knowledge and knowledge of expected actions after a component/equipment problem has been identified. WROTE NEW QUESTiON (OK) 19 I F 2 N N S?

, 8? &

I Need to make sure there are not two correct (B &

D) answers. Reference procedure was not provided. REMOVED PART OF THE DISTRACTORS. CRAIG WAINTED TOO ....

HAD TO CHANGE QUESTION BACK TO INCLUDE WHAT CRAIG SUGGESTED DID NOT WORK. ...... .... QUESTION IS OK

10

1. 20
2. 30 Psychometric Flaws
3. 40
4. Job Content Flaws 50
5. Other 60
6. 70
7. 80 8.

Q# LOK LOD j (F/H) (1-5) Stem Cues T/F til Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N U/EIS U/E/S Explanation Focus Dist.

Dis!. Link units ward KIA Only 20 H 2 X X N s (OK) 21 H 2 N s (OK) 22 F 2 X X N E Consider changing distractors such that a higher value is give for one of the other limits identified in the emergency exposure guideline ADDED INFORMATION TO THE STEM AND ORGNIZED DISTRACTOR (OK) 23 F N s ADDED WORDS TO THE STEM (OK) 24 X X X X u The stem of the question provides part of the note. The answer provides the rest rest. Memory question. Are RO not responsible for the notes/cautions? REWORDED QUESTIION (OK) 25 H 2 X X X X N U Not SRO only. As written the question only requires RO knowledge to answer. Although the question was used on a Wolf Creek exam, the way it was modified made it RO only. As written the question appears to be a T/F' T/F type question.

ES-401 Written Examination Review Worksheet -- Vogtle RO Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem JcuesJ Stem Focus Cues T/F Credo Partial Job-T/F lcredjpartial Dist.

Dis!.

JOb-j Minutia Link

~Ck-Minutil #/ Back-units ward Q= jSROSRO B/M/N U/E/S KIA Only Explanation Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.) concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
3. Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e.,(Le., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e.,(Le., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., (Le., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any "u" "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).
1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other .6.
6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist.

Dis!. Link units ward KIA Only 1 F 2 S (OK) 2 H 4 S? Is this an RO question? LICENSEE SAID IT WAS SRO. WE REMOVED #s FROM DISTRACTORS (OK)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 3 H 2 X uU Possible two correct answers (B &C).

Identify the failures that a multiplexing failure could result in (failures within the rod control system) EXPLAINED WHY THERE WERE NOT TWO CORRECT ANSWERS (QUESTION IS OK) 4 F 2 MlB M/B S MADE MINOR CHANGES (OK) 5 H 3 N S CHANGED 32 TO 35 MINOR CHANGEES (OK) 6 3 N S (OK) 7 F 2 M (OK) 8 H 3 N S (OK) 9 H 1 N S? Added clarity to stem and identified testing on A RT Breakers. Changed distracter wording.

SATLOD 10 2 B S (OK) 11 N S (OK) 12 H 3 X X M/B U Based on the information given, I do not consider A & B plausible. You also give a reason in two distractors (cue). What is "indicating properly?" If the applicant concludes indication properly based on conditions, there could be two correct answers.

Question replaced with Farley 2003 question.

SAT 13 H 1 S (OK)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

B.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist.

Dis!. Link units ward KIA Only 14 H S (OK) 15 F 2 s (OK) 16 H 2 s (OK) 17 H X X s Distractor C does not appear to be plausible.

Both CCPs have a common suction. Not possible to have trains alignment for suctions sources.

Evaluated drawings, procedures and determined distractors are minimally acceptable. SAT 18 1B F s (OK) 19 2 S (OK) 20 2 N s (OK) 21 H 2/3 s (OK) 22 H 2 N s (OK) 23 F N s (OK) 24 H X X E Rearrange information in the stem (order of events - unit shut down ..... )

Stem rearranged SAT 25 F X SI?

Sf? Please provide documentation where you discuss simi-auto swapover of RHR References provided - SAT

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 26 F 1 X u U KIA K/A asked for a reason. Asking for a reason might increase LOD Question replaced and edited - LOD 2 SAT 27 F X X M u U KIA failed to address operational implications.

Stem gives the reason why.

Discussed operational implications - reason why is in stem but is not required to be tested by KA.

28 H 2 s (OK) 29 H 2 X X N u U If CNTM is in service for respirable air quality control, would one expect any dampers to be closed? How couidcould a system operate properly I with one set of dampers closed?

Added containment pressure to stem to increase plausibility of distractors. System normally operated with only exhaust open.

SAT 30 F xX u Given the fact that you have a power supply failure for a detector, why would you not enter a procedure to repair/correct the problem?

Distractors B & D 0 are not plausible Question altered to include core alterations. SAT

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N UlE/S Explanation Focus Dist.

Dis!. Link units ward KIA Only 31 F 1 X UlE u/E LOD. May be able to reword stem/distractors.

Discussed makeup water requirements (normal vs requirement to use borated water).

32 H 3 5 S (OK) 33 F N N un I Please Axpla explain the plausibility of distractors C&

C&

10.

D. Unable to locate information in reference material Imai I Repl, Replaced distracotrs C&D with basis for ARV's.

34 F 1 X u U ILl LOD. I! do not see the plausibility of A& A & C. Are Itthere I any scenarios which would allow rod withdrawal would be done with out the permission Iwi )n of the S8?

lof Replaced second half of distractor with I en lei !:jO! iCY L0ratilr emergency boration termination criteria.

35 F 2 51?

SI? Need to review additional information on excessive hydrogen production.

Plausable due to hydrogen production during charging but not a concern during discharging.

36 H 2 5 S (OK) 37 H 3 5 S (OK) 38 H 3 5 S CHANGED WORDING IN STEM AND DISTRACTORS (OK)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Sack-Back- Q= B/M/N U/E/S SRO S/M/N Explanation Focus Dist.

Dis!. Link units ward KIA Only 39 H N 8s (OK) 40 H M/B MIS Sat - Higher order - level systems knowledge 41 H X X E Why do we need the alarm information in the stem. Consider stating "all systems functioned as designed following the trip ... "We would not expect the RO to identify TS Alarm indicates where in the UOP the shift is and is required to correctly answer question.

TS has been removed 42 H X N E Explain the plausibility of A && B. I do not know/nor could I locate an cases where power is required to be produced by 100 MW increments.

MW reference removed - distractors modified to enhance clarity 43  ? 1 X X X u LOD. KIA ask for knowledge of reason. This question failed to address the KIA. Possible two answers as written. Do not see in the procedure where it is required to verify NSCW discharge valves are closed.

Valve closure is a required interlock.

Reason for valve closure added to distractor.

44 F 2 B S 8s (OK) 45 H 2 N 8s (OK) 46 H 1 N s?

8? LOD MODIFITED QUESTION. (OK)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

I Q# LOK LOD I (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= U/E/S SRO B/M/N UlE/S Explanation Focus Dist.

Dis!. Link units ward KIA Only 47 H 2 N S (OK) 48 H 2 N S? What are Hagen controllers? REMOVED CONTROLLER NAME - ADDED AUTO MANUAL CONTROLLER (OK) 49 F X N WE There could be two correct answers. Look at IThere l~iE requirement identified in ODCM. CHANGED STEM AND DISTRACTORS. (OK) 50 F X OKiS  ???? need additional information on the detector.

Need more information to determine the plausibility of distractors B & D. DECIDED TO USE ORIGINAL QUESTION. PROVED THAT DISTRACTORS WERE OK BASED ON DESIGN OF MONITORS. (OK) 51 H 3 X WE IF!For the condi"uul conditionsj:::; given why would one expect t:Xj.Jt:l;t to perform a rapid power reduction? If this is true, A Iperla;

& C would not be plausib!e. WROTE NEW

&C lou, QUESTION. (OK) 52 H 3 M S (OK) 53 F 2 X N u U IRad Rad Chern Lab monitors are disable. Two Iimplausible il )1;Hl~ible distractors

'QvlUI;::'".. WROTE NEW 10UI QUESTION. (OK) 54 F N S (OK) 55 H 2 s S (OK) 56 H S (OK) 57 F S (OK) 58 H 2 S (OK)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N U/E/S Explanation Focus Dist.

Dis!. Link units ward KIA Only 59 F Y, y" X X N E Consider rewording the stem/distractors to have the applicant identify the window. It appears that as written the applicant only need to choose from two correct answers (A & & B)

8) WROTR NEW QUESTION (OK) 60 F 1 s LOD (OK) 61 F 1 s LOD (OK) 62 F 1 X E LOD Distractor A is not plausible. Review def of local control. Appears acceptable for in-plant JPM .. ACCEPTED QUESTION AS WRITTEN.

{OK} NOT A U MINOR CHANGES (OK}

u

~OD implausible distractors A & B. WROTE 63 F 1 X B LOD.

NEW QUESTION 64 H 2 s (OK) 65 H 3 s (OK) 66 F X X u Where are the preaccess filter units. Distractors do not appear to be plausible. CHANGED STEM.AND DISTRACTORS DlSTRACTORS (OK} QUESTION SHOULD NOT HAVE BEEN A U 67 H 2 s (OK) 68 H 1 X u As written the question has a very LOD. Two distractors are implausible. Inadequate supporting documentation and wording in distractors. PROVIDED JUSTIFICATION AS THE WHY THE QUESTION IS ACCEPTABLE (OK}

{OK} NOT A AUU

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= u/E/S SRO B/M/N UlE/S Explanation Focus Dist. Link units ward KIA Only 69 H 2  ?? Is this an RO question? AGREED THAT IT WAS NOT AN RO QUESTION. SELECT A DIFFERENT KIA AND WRITE A NEW QUESTION 70 H 1 S (OK) 71 F 2 S (OK) 72 H X X X B/M E/S ElS Based on the information/lack of information there may be two correct answers. The level in the other SGs are not provided. Take a look at the procedure and answer questions concerning SGs levels and work through procedure. Prove that there are not two correct answers. REWROTE QUESTION (OK) ..

73 F 2 X E Based on the question asked, distractor D 0 is not plausible. What dose higher priority of ORANGE vs RED have to do with the question asked?

Check to see if this is similar to an SRO question.

WROTE NEW QUESTION.

74 H 2 S (OK) 75 F 1 X X X E LOD. More information could be provided in the stem. Ask the applicant to identify the procedure to be used and the bases for using the procedure.

ACCEPTED QUESTION AS IS (OK)

Green Unsat.

Red - Should not have been identified as Unsat.

Question that had to be replaced that were Identified as E or had a ?

11/13/2009 14:21 FAX 706 826 3953 VEGP TRAINING CENTER l4J 001 141001 ES-40:1 Written Examination Grading FOrm ES-403-1 Quality Checklist I Facil  : Vogtle Date of Exam: 06/26/09 Exam Level: RO[ZJ SRO[Z]

Initials Itsm Descri tion 1.

2. Answer key changes and question deletions justified and documented 3.

3- Applicants' scores checked for addition errors reviewers Ss ot check> 25% of examinations

4. Grading for all borderline cases (80 +/-21% overall and 70 or 80, as a licable, +/-4% on the SRO-onl reviewed in detail 5, All other failing examinations checked to ensure that grades are 'ustified
6. Performance on missed questions checked for training deficiencies and wording problems; evaluate validity of uestions missed b half or more of the a plicants

.\......./.

Printed Name/Signature Date

a. Grader "-z...~, 0;

(.. "-z...l-,

b. F ~cility Reviewer(*)

Reviewer(") 6-;?6-0?

C-;?6-0?

"1-12--

"1-15..- 00:1 :\

c. N ~c Chief Examiner (*) o//¢l:Jt:I'j o//¢?L:J4?'j
d. NRC Supervisor (*)

(*) reviewers signature is not applicable for examinations graded by the NRC; The facility reviewer's two independent NRC reviews are re uired.uired_

ES-403, Page 5 of 5

Retrieved from "https://kanterella.com/w/index.php?title=ML093280324&oldid=2560804"