ML101520098

301 Final Administrative Documents ES-201-2 (Section 7)

ML101520098
Person / Time
Site:	Vogtle
Issue date:	03/28/2010
From:	NRC/RGN-II
To:
References
ES-201-2
Download: ML101520098 (32)

Text

Ii!

ES-201 Examination Outline Quality Checklist Form ES-201-2

--,c,,JAt... -

Facility:

"o~ Ii-Ie Date of Examination: 7/Sc> - 8)'11'"

""I Item

1.

W R

I T

T E

N

2.

S I

M U

L A

T 0

R

3.

W 1

T Task Description

a. Verify that the outline(s) fit(s) the appropriate model, in accordance with ES-401.

b. Assess whether the outline was systematically and randomly prepared in accordance with Section D.1 of ES-401 and whether all KiA categories are appropriately sampled.

c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics.

d. Assess whether the justifications for deselected or rejected KiA statements are appropriate.

a:i Using Form ES-301-5, verify that the proposed scenario sets cover the required number of normal evolutions, instrument and component failures, technical specifications, and major transients.

b. Assess whether there are enough scenario sets (and spares) to test the projected number and mix of applicants in accordance with the expected crew composition and rotation schedule without compromising exam integrity, and ensure that each applicant can be tested using at least one new or significantly modified scenario, that no scenarios are duplicated from the applicants' audit test(s), and that scenarios will not be repeated on subsequent day§.

c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative and quantitative criteria specified on Form ES-301-4 and described in Appendix D.

a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2:

(1) the outline(s) contain(s) the required number of control room and in-plant tasks distributed among the safety functions as specified on the form (2) task repetition from the last two NRC examinations is within the limits specified on the form (3) no tasks are duplicated from the applicants' audit test(s)

(4) the number of new or modified tasks meets or exceeds the minimums specified on the form (5) the number of altemate path, low-power, emergency, and RCA tasks meet the criteria on the form.

b. Verify that the administrative outline meets the criteria specified on Form ES-301-1:

(1) the tasks are distributed among the topics as specified on the form (2) at least one task is new or significantly modified (3) no more than one task is repeated from the last two NRC licensing examinations

c.

z+..

Determine if there are enough different outlines to test the projected number and mix of applicants and ensureJhat n()Jtems are duplicated on subsequent days.

Assess whether plant-specific priorities (including PRA and IPE insights) are covered in the appropriate exam sections.

G

b.

E Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate.

N

c. Ensure that KiA importance ratings (except for plant-specific priorities) are at least 2.5.

E

d.

R Check for duplication and overlap among exam sections.

A

e. Check the entire exam for balance of coverage.

Initials a I b* I c#

1/

1\\ 1 ~I 'II

/~

II

~~I~t IJ.~.* I~t

~,

~IVI~~

IJ*"

.... t-- ~

1 W,

1~~I~I.Ji W

~~~I~i

~l~l~j

~(l...U1 l~f~1 L i'<.:.2-," _. ~

~i~lA.i.

~r~L~fl L

f.

Assess whether the exam fits the appropriate job level (RO or SRO).

l~i~~:t

a. Author

o. NRC Ch;.f E>mm;o" (#)

b F Tt ac" y Reviewer (*)

d. NRC Supervisor

-y----

Note:

1. Indep~ndent NRC reviewe!initial items in Column "c n

chief examiner concurrence required.

'* SU~ V'Y\\,"i"( c:.-o S~~ A a.A T. Lv}

ES-201, Page 25 of 27 Date 1 'c3-0J 7/n/o,

¢¥,()7 7jL6/rJ 1 ES-201 Examination Outline Quality Checklist Form ES-201-2 Facility:

"O~ Ii-Ie Date of Examination: 7/Sc> - 8)'11<1"1 Item Task Description a

1.

a. Verify that the outline(s) fit(s) the appropriate model, in accordance with ES-401.

W R

b. Assess whether the outline was systematically and randomly prepared in accordance with I

Section D.1 of ES-401 and whether all KiA categories are appropriately sampled.

T Assess whether the outline over-emphasizes any systems, evolutions, or generic topics.

T

c.

/

E

d. Assess whether the justifications for deselected or rejected KiA statements are appropriate. /

N

2.

a:i' Using Form ES-301-5, verify that the proposed scenario sets cover the required number (1<

of normal evolutions, instrument and component failures, technical specifications, S

and major transients.

I M

b. Assess whether there are enough scenario sets (and spares) to test the projected number

,\\

U and mix of applicants in accordance with the expected crew composition and rotation schedule vP L

without compromising exam integrity, and ensure that each applicant can be tested using A

at least one new or significantly modified scenario, that no scenarios are duplicated f..,.;;'

T from the applicants' audit test(s), and that scenarios will not be repeated on subsequent days. "

0

c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative

~

R and quantitative criteria specified on Form ES-301-4 and described in Appendix D.

3.

a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2:

(1) the outline(s) contain(s) the required number of control room and in-plant tasks W

distributed among the safety functions as specified on the form i

1 (2) task repetition from the last two NRC examinations is within the limits specified on the form T

(3) no tasks are duplicated from the applicants' audit test(s)

(4) the number of new or modified tasks meets or exceeds the minimums specified on the form

/.

(5) the number of altemate path, low-power, emergency, and RCA tasks meet the criteria

~

on the form.

b. Verify that the administrative outline meets the criteria specified on Form ES-301-1:

,\\

(1) the tasks are distributed among the topics as specified on the form iJ~

(2) at least one task is new or significantly modified (3) no more than one task is repeated from the last two NRC licensing examinations

c. Determine if there are enough different outlines to test the projected number and mix i;)~

of applicants and ensure that no items are duplicated on subsequent days.

4.

a. Assess whether plant-specific priorities (including PRA and IPE insights) are covered

1. J<

in the appropriate exam sections.

G

b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate.

~

E N

c. Ensure that KiA importance ratings (except for plant-specific priorities) are at least 2.5.

~

E

d. Check for duplication and overlap among exam sections.

~~

R A

e. Check the entire exam for balance of coverage.

if ~

L

f.

Assess whether the exam fits the appropriate job level (RO or SRO).

W

~.

.\\

I*tiff cd) lIra-'"

Printe I.

/1/

I hQdl Nr Tho/WIl1S(lAi tiP:*

'OI-v 'YI~ II'

a. Author

b. Facility Reviewer (*)

R. J..t::6111,a-1tI.rtf;;/ELO / nL?').". /1 & ~

£/1/

c. NRC Chief Examiner (#)

EJL,'" ~

1l"'.::1. ;{r /. A"!..L.,1.,.

C7:~ (Ji

d. NRC Supervisor K. b \\.; T t-Ic A A ({ / /t.,./J-c.if~

~.,

I

/"

Note:

1. Independent NRC reviewer initial items in Column "c n

chief examiner concurrence required.

'* SU~ V'Y\\,"i "(<::::Q S~~ A a.A T. Lv}

ES-201, Page 25 of 27 Initials b*

c#

1/

~l~ 'II r'

~

j,J..

rv ~J.

V A:J.

t,... ~1

~.Ji

~ I~i t.. ~j t-1A!1

~.A,-J a.... A.i

~.1:3 4-i-/;:J Date 1 'ci-OJ 7/n/o,

¢¥~7 7.j-1:.61LrJ I

I ES-201 Examination Outline Quality Checklist Form ES-201-2 F,tJAL...-

Facility:

VO~TU Date of Examination: 7/~o -8/n/07!

Item Task Description a

1.

a. Verify that the outline(s) fit(s) the appropriate model, in accordance with ES-401.

~

w R

b. Assess whether the outline was systematically and randomly prepared in accordance with

~

I Section D.1 of ES-401 and whether all KIA categories are appropriately sampled.

T

c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics.

~

T

~"

E

d. Assess whether the justifications for deselected or rejected KIA statements are appropriate.

~

N

2.

a.: Using Form ES-301-5, verify that the proposed scenario sets cover the required number

~

of normal evolutions, instrument and component failures, technical specifications, 1*£','"

S and major transients.

1f'Ir;o. '1-I M

b. Assess whether there are enough scenario sets (and spares) to test the projected number U

and mix of applicants in accordance with the expected crew composition and rotation schedule L

without compromising exam integrity, and ensure that each applicant can be tested using

~

A at least one new or significantly modified scenario, that no scenarios are duplicated T

from the aQPlicants' audit test(s), and that scenarios will not be repeated on subsequent days.

0

c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative R

and quantitative criteria specified on Form ES-301-4 and described in Appendix D.

3.

a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2:

(1 ) the outline(s) contain(s) the required number of control room and in-plant tasks W

distributed among the safety functions as specified on the form 1

(2) task repetition from the last two NRC examinations is within the limits specified on the form

~

T (3) no tasks are duplicated from the applicants' audit test(s)

(4) the number of new or modified tasks meets or exceeds the minimums specified on the form (5) the number of alternate path, low-power, emergency, and RCA tasks meet the criteria on the form.

b. Verify that the administrative outline meets the criteria specified on Form ES-301-1:

(1 ) the tasks are distributed among the topics as specified on the form (2) at least one task is new or significantly modified (3) no more than one task is repeated from the last two NRC licensing examinations

c. Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days.

4.

a. Assess whether plant-specific priorities (including PRA and IPE insights) are covered W

in the appropriate exam sections.

G

b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate.

~

E N

c. Ensure that KIA importance ratings (except for plant-specific priorities) are at least 2.5.

~

E

d. Check for duplication and overlap among exam sections.

~

R A

e. Check the entire exam for balance of coverage.

~

L

f.

Assess whether the exam fits the appropriate job level (RO or SRO).

a. Author

-f,.,Q N rJ, "nte~"r9l A 1!2f}-¥ I

a I

OM'\\ Dc...'

/

b. Facility Reviewer (*)

R.~~ tvlAI'ISPNlh.. () / :JC/.l./Z

.L~£/

c. NRC Chief Examiner (#)

£d.:v,n L~*~dr / /,.J.." 1" --'.4~ Lh* J

d. NRC Supervisor 1<, b '-r\\ H A A r:.. I ('<....Arc). lH

.~

I L

Note:

1. Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.

ES-201, Page 25 of 27

'* 'ES - 2..0 I - 2...

fV(t. T ~ OP~"'T \\ ~

""reS -C w4S

<!.-OMPl-Gt:n 4-tJ'O O1...IVE'Yt..EO SE(,)AM-r£L-~.

Initials b*

c#

~ itJ

<?- 11:t

~

1. j!

N/A AlA!

>Is X I

~ X I

I I

~ X I

)J:-

X

¥-' X

"* X

~ i;t

,... /':'J' (1.... l-li' 0,...- 11:£ a... Ld

"~ 1-t1 DatS-7-'2.

~()i 7L2.~Lo7 f//O/?A'7 irjl..j( > 1 X ltc-vleW,d.. UndcY' ES...,2ol... 2 fbY" Dp~'I"atj fcsi.);.l ES-201 Examination Outline Quality Checklist Form ES-201-2 Facility:

VO~TU Date of Examination: 7/~o-8111/o' Item Task Description a

1.

a. Verify that the outline(s) fit(s) the appropriate model, in accordance with ES-401.

t#

w R

b. Assess whether the outline was systematically and randomly prepared in accordance with

~

I Section D.1 of ES-401 and whether all KIA categories are appropriatel),sampled.

T Assess whether the outline over-emphasizes any systems, evolutions, or generic topics.

~

T

c.

~"

E

d. Assess whether the justifications for deselected or rejected KIA statements are appropriate. #'

N

2.

a.: Using Form ES-301-5, verify that the proposed scenario sets cover the required number

~

of normal evolutions, instrument and component failures, technical specifications, 1'£.'

S and major transients.

I 119r;t. ~!-

M

b. Assess whether there are enough scenario sets (and spares) to test the projected number U

and mix of applicants in accordance with the expected crew composition and rotation schedule L

without compromising exam integrity, and ensure that each applicant can be tested using

~

A at least one new or significantly modified scenario, that no scenarios are duplicated T

from the applicants' audit test(s), and that scenarios will not be repeated on subsequent days.

0

c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative R

and quantitative criteria specified on Form ES-301-4 and described in Appendix D.

3.

a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2:

(1 ) the outline(s) contain(s) the required number of control room and in-plant tasks W

distributed among the safety functions as specified on the form 1

(2) task repetition from the last two NRC examinations is within the limits specified on the form

~

T (3) no tasks are duplicated from the applicants' audit test(s)

(4) the number of new or modified tasks meets or exceeds the minimums specified on the form (5) the number of alternate path, low-power, emergency, and RCA tasks meet the criteria on the form.

b. Verify that the administrative outline meets the criteria specified on Form ES-301-1:

(1 ) the tasks are distributed among the topics as specified on the form i~i at least one task is new or significantly modified no more than one task is repeated from the last two NRC licensing examinations

c. Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days.

4.

a. Assess whether plant-specific priorities (including PRA and IPE insights) are covered

~

in the appropriate exam sections.

G

b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate.

~

E N

c. Ensure that KIA importance ratings (except for plant-specific priorities) are at least 2.5.

~

E

d. Check for duplication and overlap among exam sections.

~l R

A

e. Check the entire exam for balance of coverage.

~

L

f.

Assess whether the exam fits the appropriate job level (RO or SRO).

~

Il *

a. Author

-f, ~ N rJ, ir:lnte~fr9l., AIiifr-¥ I

a I

OM'\\Jlc...:

,ANI

/

b. Facility Reviewer (*)

R.~~ IYIAI'ISp~e... o I :Jt7.l-//"..

Lf5.£/'

c. NRC Chief Examiner (#)

/;du.l.n LL' *~d;'- /,~j F_ - )~-£_~ Lh. I

d. NRC Supervisor J<,bo.-d,J.AAC / l'<...ArJ I'H

.~

I Note:

1. Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.

ES-201, Page 25 of 27

'* 'ES - 2..0 I - 2...

fV(t. T ~ OP~"'T \\ ~

""reS -C w4S

<!.-OMPl-Gt:n 4-tJ'O O1...IVE'Yt..EO SE(,)AM-r£L-~.

Initials b*

c#

~ Itl

<?- 11:<1-

~

If';

N/A lolA

>Is X

~ X

~ X

)J:-

X

¥-' X

"* X

~ t;(

,... ~J' (1.... 1:..1 0,...- 10£ a... J:.d

"~ )::f DatS-7-'2.

~() (

7L2.~Lo7 f'//obA,7 idl'1l> 1 X ltc-vleW,d.. UndcY' ES...,2ol... 2 fbY" Dp~'I"atj fcsi.);.l

ES-201, Rev. 9E Examination Preparation Checklist Form ES-201-1 Facility:

Vogtle Date of Examination:

7/30 - 8/15 Examination Prepared By (Circle):

Facility NRC Written / Operating Test Target Chief Date*

Task Description (Reference)

Examiner's Initials

-180

1.

Examination administration date confirmed (C.1.a; C.2.a and b)

~f-

-120

2.

NRC examiners and facility contact assigned (C.1.d; C.2.e)

~.1

-120

3.

Facility contact briefed on security and other requirements (C.2.c)

£L

-120

4.

Corporate notification letter sent (C.2.d)

./oc!-

[-90]

[5.

Reference material due (C.1.e; C.3.c; Attachment 2)]

.~;j

{-75}

6.

Integrated examination outline(s) due, including Forms ES-201-2, ES-201-3, ES-301-1, ES-301-2, ES-301-5, ES-D-1's, ES-401-1/2, ES-401-3, and A;L ES-401-4, as applicable (C.1.e and f; C.3.d)

{-70}

{7.

Examination outline(s) reviewed by NRC and feedback provided to facility

~L licensee (C.2.h; C.3.e)}

{-45}

8.

Proposed examinations (including written, walk-through JPMs, and scenarios, as applicable), supporting documentation (including Forms ES-301-3, ES-301-4, ES-301-5, ES-301-6, and ES-401-6), and reference

.~oL materials due (C.1.e, f, g and h; C.3.d)

-30

9.

Preliminary license applications (NRC Form 398's) due (C.1.1; C.2.g;

~ci ES-202)

-14

10. Final license applications due and Form ES-201-4 prepared (C.1.1; C.2.i; L;L ES-202)

-14

11. Examination approved by NRC supervisor for facility licensee review

£;1 (C.2.h; C.3.f)

-14

12. Examinations reviewed with facility licensee (C.1.j; C.2.f and h; C.3.g)

~f

-7

13. Written examinations and operating tests approved by NRC supervisor

~1 (C.2.i; C.3.h)

-7

14. Final applications reviewed; 1 or 2 (if >10) applications audited to confirm qualifications / eligibility; and examination approval and waiver letters sent

~ci (C.2.i; Attachment 4; ES-202, C.2.e; ES-204)

-7

15. Proctoring/written exam administration guidelines reviewed A!ci!-

with facility licensee (C.3.k)

-7

16. Approved scenarios, job performance measures, and questions

~~

distributed to NRC examiners (C.3.i)

• Target dates are generally based on facility-prepared examinations and are keyed to the examination date identified in the corporate notification letter. They are for planning purposes and may be adjusted on a case-by-case basis in coordination with the facility licensee.

[Applies only] {Does not apply} to examinations prepared by the NRC.

ES-201, Rev. 9E Examination Preparation Checklist Form ES-201-1 Facility:

Vogtle Date of Examination:

7/30 - 8/15 Examination Prepared By (Circle):

Facility NRC Written / Operating Test Target Chief Date*

Task Description (Reference)

Examiner's Initials

-180

1.

Examination administration date confirmed (C.1.a; C.2.a and b)

~f-

-120

2.

NRC examiners and facility contact assigned (C.1.d; C.2.e)

~.1

-120

3.

Facility contact briefed on security and other requirements (C.2.c)

LL

-120

4.

Corporate notification letter sent (C.2.d)

/!cI-

[-90]

[5.

Reference material due (C.1.e; C.3.c; Attachment 2)]

L;i

{-75}

6.

Integrated examination outline(s) due, including Forms ES-201-2, ES-201-3, ES-301-1, ES-301-2, ES-301-5, ES-D-1's, ES-401-1/2, ES-401-3, and

~;L ES-401-4, as applicable (C.1.e and f; C.3.d)

{-70}

{7.

Examination outline(s) reviewed by NRC and feedback provided to facility

~L licensee (C.2.h; C.3.e)}

{-45}

8.

Proposed examinations (including written, walk-through JPMs, and scenarios, as applicable), supporting documentation (including Forms ES-301-3, ES-301-4, ES-301-5, ES-301-6, and ES-401-6), and reference

.~oL materials due (C.1.e, f, g and h; C.3.d)

-30

9.

Preliminary license applications (NRC Form 398's) due (C.1.1; C.2.g;

£'ci ES-202)

-14

10. Final license applications due and Form ES-201-4 prepared (C.1.1; C.2.i; L;L ES-202)

-14

11. Examination approved by NRC supervisor for facility licensee review

£;1 (C.2.h; C.3.f)

-14

12. Examinations reviewed with facility licensee (C.1.j; C.2.f and h; C.3.g)

~:f

-7

13. Written examinations and operating tests approved by NRC supervisor

~J (C.2.i; C.3.h)

-7

14. Final applications reviewed; 1 or 2 (if >10) applications audited to confirm qualifications / eligibility; and examination approval and waiver letters sent

~ ci (C.2.i; Attachment 4; ES-202, C.2.e; ES-204)

-7

15. Proctoring/written exam administration guidelines reviewed

~ci!-

with facility licensee (C.3.k)

-7

16. Approved scenarios, job performance measures, and questions

~1 distributed to NRC examiners (C.3.i)

• Target dates are generally based on facility-prepared examinations and are keyed to the examination date identified in the corporate notification letter. They are for planning purposes and may be adjusted on a case-by-case basis in coordination with the facility licensee.

[Applies only] {Does not apply} to examinations prepared by the NRC.

ES-201 Examination Security Agreement.

Form ES-201-3

1.

Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of F!>{J hJ <5". t 7 as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2.

Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of 1/3o --!l!n/o? From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME

1. Ti"""la.& N*Ti,c""10>ON

2. 1? f.-££. It{ A'" S ;:: / et4 ES-201, Page 26 of 27 ES-201 Examination Security Agreement.

Form ES-201-3

1.

Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of F!>{J hJ <5". t 7 as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2.

Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of 1/3o --!l!n/o? From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME

1. Ti"""la.& N*Ti,c""10>ON

2. 1? f.-££. It{ A'" S ;:: / et4 ES-201, Page 26 of 27

Checklist, FNL Checklist, FNL

ES-301 Operatil19Iest Quality Checklist Form ES-301-3 Ft,J A L.-

Facility:

VOG-r(,~

~J~Ojco" ~

Date of Examination:.e. fa 0 7 OperatinQ Test Number: I Initials

1. General Criteria a

b*

c#

a.

The operating test conforms with the previously approved outline; changes are consistent with

@~ ~ --tJ samplinQ requirements (e.g., 10 CFR 55.45, operational importance, safety function distribution).

.I,~

b.

There is no day-to-day repetition between this and other operating tests to be administered

~

~.-l:1 during this examination.

~

c.

The operatinQ test shall not duplicate items from the applicants' audit test(s). (see Section D.1.a.)

ilP ~,tj

d.

Overlap with the written examination and between different parts of the operating test is within

{1f:~ ~ ",j acceptable limits.

~~

v

~

e.

It appears that the operating test will differentiate between competent and less-than-competent f7J; f-,c.1 applicants at the desiQnated license level.

2. Walk-Through Criteria

a.

Each JPM includes the following, as applicable:

initial conditions initiating cues references and tools, including associated procedures reasonable and validated time limits (average time allowed for completion) and specific

~

designation if deemed to be time-critical by the facility licensee

~<

r"'"

operationally important specific performance criteria that include:

~1 detailed expected actions with exact criteria and nomenclature

~\\.

system response and other examiner cues statements describing important observations to be made by the applicant criteria for successful completion of the task identification of critical steps and their associated performance standards restrictions on the sequence of steps, if applicable

b.

Ensure that any changes from the previously approved systems and administrative walk-through W 4-1 outlines (Forms ES-301-1 and 2) have not caused the test to deviate from any of the acceptance

~

criteria (e.g., item distribution, bank use, repetition from the last 2 NRC examinations) specified on those forms and Form ES-201-2.

3. Simulator Criteria The associated simulator operating tests (scenario sets) have been reviewed in accordance with

~~ (l...- 41 Form ES-301-4 and a copy is attached.

,,f P'"ted N,m. 'Jrf:/!:J ~

Dol.

a.

Author I hq& N, I horm05cJAi rI.

(

7-e.3-07

b.

Facility Reviewer(*)

~. J..E£ rYJJlNJ/{6UJ /,7~J~

_1 /f/

7/~J/07

c.

NRCChiefExaminer(#).EJWJ'" i,t'!:...1i.;:rIC", /~)~ ~ (hI 7!..e¥t2 /)/)?

!S>lo"'rJ-I~AArrL/~ ~/

{I I

d.

NRC Supervisor Z/-1..."',t.. ~7 NOTE:

The facility signature is not applicable for NRC-developed tests.

I Independent NRC reviewer initial items in Column "CO; chief examiner concurrence required.

I ES-301, Page 24 of 27 ES-301 Operating Test Quality Checklist Form ES-301-3 Ft,J A L.-

Facility:

VOG-r(,~

~J~Oj40" ~

Date of Examination: l!!., fa 0 7 Operating Test Number: I Initials

1. General Criteria a

b*

c#

a.

The operating test conforms with the previously approved outline; changes are consistent with W'\\ ~ --tJ sampling requirements (e.g., 10 CFR 55.45, operational importance, safety function distribution).

.I,~

b.

There is no day-to-day repetition between this and other operating tests to be administered

1. ~.-l:1 durina this examination.

~

c.

The operating test shall not duplicate items from the applicants' audit test(s}. (see Section D.1.a.)

1Jt' ~ "j.

v

d.

Overlap with the written examination and between different parts of the operating test is within

~~ ~ ",j acceptable limits.

e.

It appears that the operating test will differentiate between competent and less-than-competent W. f-,c.1 applicants at the designated license level.

2. Walk-Through Criteria

a.

Each JPM includes the following, as applicable:

initial conditions initiating cues references and tools, including associated procedures reasonable and validated time limits (average time allowed for completion) and specific i,

designation if deemed to be time-critical by the facility licensee f-operationally important specific performance criteria that include:

~1 detailed expected actions with exact criteria and nomenclature

~\\.

system response and other examiner cues statements describing important observations to be made by the applicant criteria for successful completion of the task identification of critical steps and their associated performance standards restrictions on the sequence of steps, if applicable

b.

Ensure that any changes from the previously approved systems and administrative walk-through tI outlines (Forms ES-301-1 and 2) have not caused the test to deviate from any of the acceptance

~ 4-J criteria (e.g., item distribution, bank use, repetition from the last 2 NRC examinations) specified on those forms and Form ES-201-2.

3. Simulator Criteria The associated simulator operating tests (scenario sets) have been reviewed in accordance with

~~ (l...- 41 Form ES-301-4 and a copy is attached.

,,f Pri"too N,m71iJ;J.d) ~

D,Ie h

r/J Nr/~

a.

Author T q& N, Thorm05cJAi

-ot'.

""'v. '7-e.3-07

b.

Facility Reviewer(*)

~. J..E£ rYJJJIJ.J/{f:.U) /.7~J~ i IV 7/~J/o7

c.

NRC Chief Examiner (#) EJAL.)JA LH"_1i.;f". IJ:}~ {i--JL (h /

7!..e¥t2 /)/)?

!S>lo"'rJ-I~AArr L/ ~

~/

{I I

d.

NRC Supervisor Z/-1..."',t.. ~7 NOTE:

The facility signature is not applicable for NRC-developed tests.

Independent NRC reviewer initial items in Column "CO; chief examiner concurrence required.

ES-301, Page 24 of 27

Quality Checklist, FNL Quality Checklist, FNL

ES-301 Simulator Scenario Quality Checklist Form ES-301-4

-FIJkC- -

IOF 2.

Facility:

Vo"L-6 7.[",,)01 ~ '3/,0/.07 Date of xam:

Scenario Numbers: 1/ 2./ 3 Operating Test No.:

I QUALITATIVE ATTRIBUTES Initials a

b*

c#

1.

The initial conditions are realistic, in that some equipment and/or instrumentation may be out {I ~ k:t of service, but it does not cue the operators into expected events.

2.

The scenarios consist mostly of related events.

.:.~ a.- lJ:.j

3.

Each event description consists of the point in the scenario when it is to be initiated the malfunction(s) that are entered to initiate the event

,qJ ~

the symptoms/cues that will be visible to the crew

.tt the expected operator actions (by shift position) the event termination point (if applicable)

4.

No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario #' t.... l-tj without a credible preceding incident such as a seismic event.

5.

The events are valid with regard to physics and thermodynamics.

~

". --t.!

~

6.

Sequencing and timing of events is reasonable, and allows the examination team to obtain

(

" ~ l..tl complete evaluation results commensurate with the scenario objectives.

7.

If time compression techniques are used, the scenario summary clearly so indicates.

~#

Operators have sufficient time to carry out expected activities without undue time constraints.

?- ",,1 Cues are given.

8.

The simulator modeling is not altered.

~J L

.A:-J.

9.

The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator

~

performance deficiencies or deviations from the referenced plant have been evaluated

~.J;t to ensure that functional fidelity is maintained while running the planned scenarios.

10.

Every operator will be evaluated using at least one new or significantly modified scenario.

IlJ i- -k.!

All other scenarios have been altered in accordance with Section 0.5 of ES-301.

11.

All individual operator competencies can be evaluated, as verified using Form ES-301-6

~

(t.- );J (submit the form along with the simulator scenarios).

( \\I

~l

12.

Each applicant will be significantly involved in the minimum number of transients and events

~~., t-.lj specified on Form ES-301-5 (submit the form with the simulator scenarios).

13.

The level of difficulty is appropriate to support licensing decisions for each crew position.

~

~.-t:l Target Quantitative Attributes (Per Scenario; See Section D.S.d)

Actual Attributes

1.

Total malfunctions (5-8) 7

/7/ 1 W"

~ 1:..1.

2.

Malfunctions after EOP entry (1-2) 2-

/ I /

I

~: ~ -ti

3.

Abnormal events (2-4) 5 / ! / S ~

~ 2J.

4.

Major transients (1-2)

, / '2. /

I ~

t.. );,j

5.

EOPs entered/requiring substantive actions (1-2) 2-

/ 2. / 1- "# <l.-. J:d.

6.

EOP contingencies requiring substantive actions (0-2)

I

/,

/

0 ~

~ I"tJ

7.

Critical tasks (2-3) 3

/ J / 3 ?lP tz-IbJ 8C£NA~/~

2 J ES-301, Page 25 of 27 ES-301 Simulator Scenario Quality Checklist Form ES-301-4 IOF 2. -

Facility:

Vo"L-6 7.[",,)01 ~ '3/,0/.07 Date of xam:

Scenario Numbers: 1/ 2./ 3 Operating Test No.:

I QUALITATIVE ATTRIBUTES Initials a

b*

c#

1.

The initial conditions are realistic, in that some equipment and/or instrumentation may be out {I ~ k:t of service, but it does not cue the operators into expected events.

2.

The scenarios consist mostly of related events.

.:.~ a.- 1J:..;1

3.

Each event description consists of the point in the scenario when it is to be initiated the malfunction(s) that are entered to initiate the event

,qJ ~

the symptoms/cues that will be visible to the crew

.tt the expected operator actions (by shift position) the event termination point (if applicable)

4.

No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario #' t.... l-tj without a credible preceding incident such as a seismic event.

5.

The events are valid with regard to physics and thermodynamics.

~

". --t..t

~

6.

Sequencing and timing of events is reasonable, and allows the examination team to obtain

(

" ~ l..tl complete evaluation results commensurate with the scenario objectives.

7.

If time compression techniques are used, the scenario summary clearly so indicates.

~#

Operators have sufficient time to carry out expected activities without undue time constraints.

?- ",,1 Cues are given.

8.

The simulator modeling is not altered.

~J L

.A:-J

9.

The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator

~

performance deficiencies or deviations from the referenced plant have been evaluated

~.J;t to ensure that functional fidelity is maintained while running the planned scenarios.

10.

Every operator will be evaluated using at least one new or significantly modified scenario.

IlJ i- -k.!

All other scenarios have been altered in accordance with Section 0.5 of ES-301.

11.

All individual operator competencies can be evaluated, as verified using Form ES-301-6

~

(t.- );J (submit the form along with the simulator scenarios).

( \\I

~l

12.

Each applicant will be significantly involved in the minimum number of transients and events

~~., t-.lj specified on Form ES-301-5 (submit the form with the simulator scenarios).

13.

The level of difficulty is appropriate to support licensing decisions for each crew position.

~

.<il-.-t:l Target Quantitative Attributes (Per Scenario; See Section D.S.d)

Actual Attributes

1.

Total malfunctions (5-8) 7

/7/ 1 W"

~ 1:..1.

2.

Malfunctions after EOP entry (1-2) 2-

/ I /

I

~: ~ -ti

3.

Abnormal events (2-4) 5 / ! / S ~

~ 2J.

4.

Major transients (1-2)

/ '2./

I ~

t.. );,j

5.

EOPs entered/requiring substantive actions (1-2) 2-

/ 2. / 1- "# <l.-. J:d.

6.

EOP contingencies requiring substantive actions (0-2)

I

/,

/

0 ~

~ I"tJ

7.

Critical tasks (2-3) 3

/ J / 3 ?lP tz-IbJ 2 J ES-301, Page 25 of 27

~~

ij ES-301 Facility:

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

Simulator Scenario Quality Checklist Form ES-301 ~4

-FIIVAL-

Lop Z VOGTC£ 7/BoT01;;;e/lo/07

.J Date of !:xam:

Stenario Num~ers: T / S- / (. Operating Test No.:

QUALITATIVE ATIRIBUTES The initial conditions are realistic, in that some equipment and/or instrumentation may be out of service, but it does not cue the operators into expected events.

The scenarios consist mostly of related events.

Each event description consists of the point in the scenario when it is to be initiated the malfunction(s) that are entered to initiate the event the symptoms/cues that will be visible to the crew the expected operator actions (by shift position) the event termination point (if applicable)

No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario without a credible preceding incident such as a seismic event.

The events are valid with regard to physics aT1clthe~mocJynamics.

Sequencing and timing of events is reasonable, and allows the examination team to obtain complete evaluation results commensurate with the scenario objectives.

If time compression techniques are used, the scenario summary clearly so indicates.

Operators have sufficient time to carry out expected activities without undue time constraints.

Cues are given.

The simulator mod eli Q.£L is not altered.

The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator performance deficiencies or deviations from the referenced plant have been evaluated to ensure that functional fidelity is maintained while running the planned scenarios.

Every operator will be evaluated using at least one new or significantly modified scenario.

All other scenarios have been altered in accordance with Section 0.5 of ES-301.

All individual operator competencies can be evaluated, as verified using Form ES-301-6 (submit the form along with the simulator scenarios).

Each applicant will be significantly involved in the minimum number of transients and events specified on Form ES-301-5 (submit the form with the simulator scenarios).

The level of difficu~is appropriate to support licensing decisions for each crew position.

Initials a I b* I c#

~~I~_llJ

~1<?--IM W1f.- ~.t

~il (l; ltJ

~I <to ~j

~~~j I~;

~~ {l.-~

~~_~J.&l

~1~~j

~

ez-[lJ Wi~lU

~~Jtj

~(I (L,/ j;;:)

Target Quantitative Attributes (Per Scenario; See Section D,5.d)

Actual Attributes

1.

Total malfunctions (5-8)

B / (, / 5"

~\\~I1-I.J;j

2.

Malfunctions after EOP entry (1-2) 2

/ Z I ( J@i~ltJ

3.

Abnormal events (2-4) t.j /'/1 'f t~ ~1.tJ

4.

Major transients (1-2)

2.

/ I / ( ~l~LtJ

5.

EOPs enteredlrequiring substantive actions (1-2)

2.

/'2./ z 'W4 ~Il;i

6.

EOP contingEln_cLes requiring substantive actions (0-2) o /0/ o ~j o...-~ol

7.

Critical tasks (2-3) 2.'/.3/ 2.~1 (J....~J 8CGItIAR-lo

'-I (' "s ES-301, Page 25 of 27 ES-301 Simulator Scenario Quality Checklist Form ES-301 ~4

-FIIVAL-

Lop Z Facility:

VOGTC£ 7,/BO{)o1 -ef'f&,o7

¥ Date of xam:

S enario Numbers:

/ S- / (. Operating Test No.:

I QUALITATIVE ATIRIBUTES Initials a

b*

c#

1.

The initial conditions are realistic, in that some equipment and/or instrumentation may be out

1. ~ ~ lJ:j.

of service, but it does not cue the operators into expected events.

J:t-

2.

The scenarios consist mostly of related events.

1. i <?-- I~

3.

Each event description consists of the point in the scenario when it is to be initiated the malfunction(s) that are entered to initiate the event (f/

e....

the symptoms/cues that will be visible to the crew J;i the expected operator actions (by shift position) f~"

the event termination point (if applicable)

{J/i fc,J

4.

No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario

.\\ (l..

without a credible preceding incident such as a seismic event.

{~

5.

The events are valid with regard to physics and thermodynamics.

~

<to ~J.

6.

Sequencing and timing of events is reasonable, and allows the examination team to obtain

~

(J....; Jj complete evaluation results commensurate with the scenario objectives.

7.

If time compression techniques are used, the scenario summary clearly so indicates.

W'

~)

Operators have sufficient time to carry out expected activities without undue time constraints.

~ I (l....

Cues are given.

~~\\

8.

The simulator modeling is not altered.

~{

1\\

(' (L I..tJ

9.

The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator

~;

~j performance deficiencies or deviations from the referenced plant have been evaluated t2...

to ensure that functional fidelity is maintained while running the planned scenarios.

1';.'

10.

Every operator will be evaluated using at least one new or significantly modified scenario.

W Cl-ltJ All other scenarios have been altered in accordance with Section 0.5 of ES-301.

11.

All individual operator competencies can be evaluated, as verified using Form ES-301-6

~

~ tJ (submit the form along with the simulator scenarios).

12.

Each applicant will be significantly involved in the minimum number of transients and events # ~ tJ specified on Form ES-301-5 (submit the form with the simulator scenarios).

13.

The level of difficulty is appropriate to support licensing decisions for each crew position.

~( (L.. );;:J Target Quantitative Attributes (Per Scenario; See Section D.5.d)

Actual Attributes

1.

Total malfunctions (5-8)

B / (, / 5 gf/

8 1-J;J.

2.

Malfunctions after EOP entry (1-2) 2

/ Z /

(

@r ~

~J

3.

Abnormal events (2--4) t.j / 'i / 'f ?JP tjt.-- 1:J

4.

Major transients (1-2) 2-

/ I /

( ~

~.t.J

5.

EOPs enteredlrequiring substantive actions (1-2) 2-

/ 'Z /

'Z '@' ~ l;,;J.

6.

EOP contingencies requiring substantive actions (0-2) 0

/ 0 /

(> @ 0...-11:.J.

7.

Critical tasks (2-3) 2.. /.3 /

2-, W- (2...- ~J ES-301, Page 25 of 27

-F)NAt.-

I of 2..

ES-301 Transient and Event Checklist Form ES-301-5 Facility:

VOGTc.k Date of Exam: 11'$0107 -8/IQ/<>? OperatinQ Test No.:

(

A E

Scenarios P

V P

E 1

2 3

4 T

M L

N 0

I I

T CREW CREW CREW CREW T

N C

A I

A T

POSITION POSITION POSITION POSITION L

M N

Y S

A B

S A

B S

A B

S A

B U

T P

R T

0 R

T 0

R T

0 R

T 0

M(*)

E 0

C P

0 C

P 0

C P

0 C

P R

I U

~

RX

[;

5"

{"

~

1 1 a SRO-I NOR

~

'2-1 1

1 D

IIC 3,'/. Go 1,2.,<-

~3,'I

2.,3 I.~I/I I,-t I 1,:1.,4 3,g 4

4 2

SRO-U MAJ

'1 1

c..

8 8

7 1

2 2

1 D

TS a

2 2

RO RX r;

5'

')

r S

5 S

S' 1

1 a D

NOR 5

5 1

1 1

~-I I/C I.:::: ~. 1,'f,(. I,Z,!' 1,'1.4

'>1 't'l,}

2., g

',]., 3, ti,:J, 1,Ir,7 1,:,3, 1,1.,~

~/S 4

4 2

Ir.. '7

~-U MAJ

'1

'1

(.,

(.

Co

~

8 7

7

'1 2

2 1

TS i/~,3,

',1./ <l 11'3

',':z..

a 2

2 RO RX 1

1 a D

NOR 1

1 1

SRO-I D

I/C 4

4 2

SRO-U MAJ 2

2 1

D TS a

2 2

RO RX 1

1 a D

NOR 1

1 1

SRO-I D

IIC 4

4 2

SRO-U MAJ 2

2 1

D TS a

2 2

Instructions:

1.

Check the applicant level and enter the operating test number and Form ES-D-1 event numbers for each event type; TS are not applicable for RO applicants. ROs must serve in both the "at-the-controls (A TC)"

and "balance-of-plant (BOP)" positions; Instant SROs must do one scenario, including at least two instrument or component (I/C) malfunctions and one major transient, in the ATC position.

2.

Reactivity manipulations may be conducted under normal or controlled abnormal conditions (refer to Section D.5.d) but must be significant per Section C.2.a of Appendix D. (*) Reactivity and normal evolutions may be replaced with additional instrument or component malfunctions on a 1-for-1 basis.

3.

Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to the applicant's competence count toward the minimum requirements specified for the applicant's license level in the right-hand columns.

ES-301, Page 26 of 27 I of 2..

ES-301 Transient and Event Checklist Form ES-301-5 Facility:

VOGTc.k Date of Exam: 11'$0107 -8/lo/<>? OperatinQ Test No.:

(

A E

Scenarios P

V P

E 1

2 3

4 T

M L

N 0

I I

T CREW CREW CREW CREW T

N C

A I

A T

POSITION POSITION POSITION POSITION L

M N

Y S

A B

S A

B S

A B

S A

B U

T P

R T

0 R

T 0

R T

0 R

T 0

M(*)

E 0

C P

0 C

P 0

C P

0 C

P R

I U

~

RX

[;

5"

{"

~

1 1 a SRO-I NOR

~

'2-1 1

1 D

IIC 3,'/. Go 1,2.,<-

~3,'I

2.,3 I.~I/I I,-t I 1,:1.,4 3,g 4

4 2

SRO-U MAJ

'1 1

c..

8 8

7 1

2 2

1 D

TS a

2 2

RO RX r;

5'

')

r S

5 S

S' 1

1 a D

NOR 5

5 1

1 1

~-I I/C I.:::: ~. 1,'f,(. I,Z,!' 1,'1.4

'>1 't'l,}

2., g

',]., 3, ti,:J, 1,Ir,7 1,:,3, 1,1.,~

~/S 4

4 2

Ir.. '7

~-U MAJ

'1

'1

(.,

(.

Co

~

8 7

7

'1 2

2 1

TS i/~,3,

',1./ <l 11'3

',':z..

a 2

2 RO RX 1

1 a D

NOR 1

1 1

SRO-I D

I/C 4

4 2

SRO-U MAJ 2

2 1

D TS a

2 2

RO RX 1

1 a D

NOR 1

1 1

SRO-I D

IIC 4

4 2

SRO-U MAJ 2

2 1

D TS a

2 2

Instructions:

1.

Check the applicant level and enter the operating test number and Form ES-D-1 event numbers for each event type; TS are not applicable for RO applicants. ROs must serve in both the "at-the-controls (A TC)"

and "balance-of-plant (BOP)" positions; Instant SROs must do one scenario, including at least two instrument or component (I/C) malfunctions and one major transient, in the ATC position.

2.

Reactivity manipulations may be conducted under normal or controlled abnormal conditions (refer to Section D.5.d) but must be significant per Section C.2.a of Appendix D. (*) Reactivity and normal evolutions may be replaced with additional instrument or component malfunctions on a 1-for-1 basis.

3.

Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to the applicant's competence count toward the minimum requirements specified for the applicant's license level in the right-hand columns.

ES-301, Page 26 of 27

~, ~ A--L-ES-301 Transient and Event Checklist Form ES-301-5 Facility:

VOf.,iu:....

Date of Exam:7/~ - P./lc.:>/o? Ooeratinq Test No.:

I A

E

~'1\\""1l()'

v,.".., \\0'1 Scenarios P

V

'11 P

E

~

/"

3 4

T M

L N

0 I

I T

CREW CREW CREW T

N C

CREW A

I A

T POSITION POSITION POSITION POSITION L

M N

Y S

A B

S A

B S

A B

S A

B U

T P

R T

0 R

T 0

R T

0 R

T 0

M(*)

E 0

C P

0 C

P 0

C P

0 C

P R

I U

~

RX

§ s

<f S

1 1

0 NOR I

I i

.j 1

1 1

SRO-I D

IIC

')..~3, 2:"~1 2,\\:1

J,!

'-1 4

4 2

SRO-U MAJ c,

7 7

2 2

1 D

TS 0

0 2

2 RO RX 5

5 s

If S

1 1

0 D

NOR

(

I I

3 1

1 1

~-I I

IIC

'2.,?; <of, :2.,] I "2.t/

1

'7-ll:~ 2/1-

~,.,;-

~8 4 4

2 W

U MAJ r."

"1 I

2 2

1 TS i,2/3 2,~i*

Ii 0 2

2 RO RX 1

1 0

D NOR SRO-I 1

1 1

D IIC 4

4 2

SRO-U MAJ 2

2 1

D TS 0

2 2

RO RX 1

1 0

D NOR SRO-I 1

1 1

D IIC 4

4 2

SRO-U MAJ D

2 2

1 TS 0

2 2

Instructions:

1.

Check the applicant level and enter the operating test number and Form ES-D-1 event numbers for each event type; TS are not applicable for RO applicants. ROs must serve in both the "at-the-controls (A TC)"

and "balance-of-plant (BOP)" positions; Instant SROs must do one scenario, including at least two instrument or component (IIC) malfunctions and one major transient, in the ATC position.

2.

Reactivity manipulations may be conducted under normal or controlled abnormal conditions (refer to Section D.S.d) but must be significant per Section C.2.a of Appendix D. (*) Reactivity and normal evolutions may be replaced with additional instrument or component malfunctions on a 1-for-1 basis.

3.

Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to the applicant's competence count toward the minimum requirements specified for the applicant's license level in the right-hand columns.

ES-301, Page 26 of 27 ES-301 Transient and Event Checklist Form ES-301-5 Facilitv:

VOl il.E,...

Date of Exam:"'/~ - A/Iv I"., 60eratina Test No.:

I A

E

~",,\\""1l()'

v,.".., \\..,....

Scenarios P

V

'11 P

E

~

/(,

3 4

T M

L N

0 I

I T

CREW CREW CREW T

N C

CREW A

I A

T POSITION POSITION POSITION POSITION L

M N

Y S

A B

S A

B S

A B

S A

B U

T P

R T

0 R

T 0

R T

0 R

T 0

M(*)

E 0

C P

0 C

P 0

C P

0 C

P R

I U

~

RX

§ s

<f S

1 1

0 SRO-I NOR I

I i

.j 1

1 1

D IIC

')..~3, 2:"~1 2,\\:1

J,!

'-1 4

4 2

SRO-U MAJ c,

7 7

2 2

1 D

TS 0

0 2

2 RO RX 5

5 s

If S

1 1

0 D

NOR

(

I I

3 1

1 1

~-I I

IIC

'2.,?; <of, :2.,; I "2.t/, '7-ll:~ 2/1-

~,.,;-

~8 4 4

2 W

U MAJ "1

I 2

2 1

TS i,2/3 2,~i*

Ii 0 2

2 RO RX 1

1 0

D NOR SRO-I 1

1 1

D IIC 4

4 2

SRO-U MAJ 2

2 1

D TS 0

2 2

RO RX 1

1 0

D NOR SRO-I 1

1 1

D IIC 4

4 2

SRO-U MAJ D

2 2

1 TS 0

2 2

Instructions:

1.

Check the applicant level and enter the operating test number and Form ES-D-1 event numbers for each event type; TS are not applicable for RO applicants. ROs must serve in both the "at-the-controls (A TC)"

and "balance-of-plant (BOP)" positions; Instant SROs must do one scenario, including at least two instrument or component (IIC) malfunctions and one major transient, in the ATC position.

2.

Reactivity manipulations may be conducted under normal or controlled abnormal conditions (refer to Section D.S.d) but must be significant per Section C.2.a of Appendix D. (*) Reactivity and normal evolutions may be replaced with additional instrument or component malfunctions on a 1-for-1 basis.

3.

Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to the applicant's competence count toward the minimum requirements specified for the applicant's license level in the right-hand columns.

ES-301, Page 26 of 27

Checklist FNL.

I

(

Checklist FNL.

I

(

}

FltJAL-fOF2.

ES-301 Competencies Checklist Form ES-301-6 Facility: V O~ T c.-E Date of Examination: 7/~o /07 -

Operating Test No.:,

8/1 0 /07 APPLICANTS RO 0"

RO 0

RO 0

RO 0

SRO-I 0 SRO-I c:1' SRO-I 0

SRO-I 0

SRO-U 0 SRO-U 0 SRO-U [Z(

SRO-U 0 Competencies SCENARIO SCENARIO SCENARIO SCENARIO 1

2 3

4 1

2 3

4 1

2 3

4 1

2 3

4

'/ '2.

I, <1-1,'3 1,1-I, z. I, 'l.-

1,:3 I, z.

1,'2-J, '2-1,'3 ',2-Interpret/Diagnose 3,'1',/"f 'f,{ "of },f >d If,"-

1, 'f 31'f 3.'f '1,1-J,f

';;c, Sf.

Sic, Events and Conditions

(",1.)/(,. I,B "/, e.

~,1 {,r., 7, '3 -l,s 1,,7 S', \\. 1, ~ 75

',2.-

',2 I, Z. f-1.,2. 1,2-

),1.-

} 7..

I Z i)'Z-1,3 1,2-Comply With and

~"

3,'t 3/1

,'1

3. 'f' 3,'1' 3, "

',,+ if 3,<< 'f,S' H Use Procedures (1)

,¥

.),\\. ),'" :5'/1-' 5i 1" £;c.. f,,,, 5",,,, ),1-fifo "'-,7 ~~

"7 B liS "1

1 8 7 'i.'

'7 8

12-

~z.

i/2., !J1--

1,3 I 2-1,2-1,2 Operate Control

1

,if 3.'f

,<of JfJ5 1,'f' 3, 'f '1>>

N/A NIp< nip< N/tA Boards (2)

?;~ >,'-' r,... S,"" ',-7 S',1. ),1

~,-}

1,~ 1,&

g I, '2.

'/2.:

I :I-I,Z, 1/1-I) 'l..

I 't I, 'l-I'L I,'Z.

1'1-

~~

Communicate 3,t ],'1 1.'1 VI' 'ltV 3, 'f 1~ 'f 3. <f 3; 'f 3,'( 1.'#' s,t,.

and Interact 5/

S,t,. S,t,. 5',1-

~~ 5,10:>

),>, ),<< ), \\> )I~ r,jO 1

1/ ff l,'i3.,

1 S 7,~

'1 Lis 'l/&

~2.

1,'1.-

I'l. Y"

,,2-

'/2-1'1-I'L Demonstrate N/Pr 1I!t. N/A-iliA-

,'I' 3,+

'1, 'f ',of 3.Q J,<j 3~'f 1','1 Supervisory Ability (3)

SlY ),'" S,s.

S,IP s-;" {?

SJ~ ~\\t 7 5 1'3.,

7'i3 7,S Comply With and l/l.. I, 'L i I 'L I,'z..

Use Tech. Specs. (3)

N)1t lilA 14/'" II/It-3 'f's I) ')

I, '/...

.3 tfl )' 1,3

~')..

Notes:

(1 )

Includes Technical Specification compliance for an RO.

(2)

Optional for an SRO-U.

(3)

Only applicable to SROs.

Instructions:

Check the applicants' license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.

ES-301, Page 27 of 27 FltJAL-fOF2.

ES-301 Competencies Checklist Form ES-301-6 Facility: V O~ T c.-E Date of Examination: 7/~o /07 -

Operating Test No.:,

8/1 0 /07 APPLICANTS RO 0"

RO 0

RO 0

RO 0

SRO-I 0 SRO-I c:1' SRO-I 0

SRO-I 0

SRO-U 0 SRO-U 0 SRO-U [Z(

SRO-U 0 Competencies SCENARIO SCENARIO SCENARIO SCENARIO 1

2 3

4 1

2 3

4 1

2 3

4 1

2

'/ '2.

I, <1-1,'3 1,1-I, z. I, 'l.-

1,:3 I, z.

1,'2-J, '2-1,'3 ',2-Interpret/Diagnose 3,'1',/"f 'f,{ "of },f >d If,"-

1, 'f 31'f 3.'f '1,1-J,f

';;c, Sf.

Sic, Events and Conditions

(",1.)/(,. I,B "/, e.

~,1 {,r., 7, '3 -l,s 1,,7 S', \\. 1, ~ 75

',2.-

',2 I, Z. f-1.,2. 1,2-

),1.-

} 7..

I Z i)'Z-1,3 1,2-Comply With and

~"

3,'t 3/1

,'1

3. 'f' 3,'1' 3, "

',,+ if 3,<< 'f,S' H Use Procedures (1)

,¥

.),\\. ),'" :5'/1-' 5i 1" £;c.. f,,,, 5",,,, ),1-fifo "'-,7 ~~

"7 B liS "1

1 8 7 'i.'

'7 8

12-

~z.

i/2., !J1--

1,3 I 2-1,2-1,2 Operate Control

1

,if 3.'f

,<of JfJ5 1,'f' 3, 'f '1>>

N/A NIp< nip< N/tA Boards (2)

?;~ >,'-' r,... S,"" ',-7 S',1. ),1

~,-}

1,~ 1,&

g I, '2.

'/2.:

I :I-I,Z, 1/1-I) 'l..

I 't I, 'l-I'L I,'Z.

1'1-

~~

Communicate 3,t ],'1 1.'1 VI' 'ltV 3, 'f 1~ 'f 3. <f 3; 'f 3,'( 1.'#' s,t,.

and Interact 5/

S,t,. S,t,. 5',1-

~~ 5,10:>

),>, ),<< ), \\> )I~ r,jO 1

1/ ff l,'i3.,

1 S 7,~

'1 Lis 'l/&

~2.

1,'1.-

I'l. Y"

,,2-

'/2-1'1-I'L Demonstrate N/Pr 1I!t. N/A-iliA-

,'I' 3,+

'1, 'f ',of 3.Q J,<j 3~'f 1','1 Supervisory Ability (3)

SlY ),'" S,s.

S,IP s-;" {?

SJ~ ~\\t 7 5 1'3.,

7'i3 7,S Comply With and l/l.. I, 'L i I 'L I,'z..

Use Tech. Specs. (3)

N)1t lilA 14/'" II/It-3 'f's I) ')

I, '/...

.3 tfl )' 1,3

~')..

Notes:

(1 )

Includes Technical Specification compliance for an RO.

(2)

Optional for an SRO-U.

(3)

Only applicable to SROs.

Instructions:

Check the applicants' license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.

ES-301, Page 27 of 27 3

4

F'N~-

2.oF 2..

ES-301 Competencies Checklist Form ES-301-6 Facility: VOG, 7 (,fE Date of Examination: ;j30~07 - Operating Test No.:

l c9 (u

Q")

APPLICANTS RO

~

RO 0

RO 0

RO 0

SRO-I 0 SRO-I @

SRO-I 0

SRO-I 0

SRO-U 0 SRO-U 0 SRO-U 0' SRO-U 0 i

Competencies SCENARIO SCENARIO SCENARIO SCENARIO

~1?--;\\O1

~ ~

~ ~

~

(.,.

A

')

A

') 1-4 1

2 3

4 InterpreUDiagnose I/L, 2../3 i/z... 2,1 1)'2. 2,$

'3,'f Sjl,o /IJ)A-tJllr :3,~ >,l.

tJ)A N/ll '3;1. 5/(,

~,,~ I-l/A Events and Conditions

(",1 1

7 7

7

"/

11-

" '2.,

1/ '2.

1,1-

"2.,3 I} 2-Comply With and

{,f 3."1' Nt;. I-l)A 3,-1 1,1 tJ)A NIA 'IS 3A IJ/A rJJf Use Procedures (1)

I'll-1';(.,

lS~ flY' I

I,.J 7 5,\\.<

,,'2.-

1/:1-1,2-r-Operate Control "f 3;'f' NJt>r N)fl 3/~

,'I Ii/flo N/Il tJ)A />I/A N/A ~)A Boards (2) 5'1,1-Sj,v S,lo S',v 1

I 1,'2-1,1.-

I, e. ',2-i,'L 1/'2.--

Communicate 3,'1 3.'/ Nip N)II 3.'1' 3, <( N)A tJ)A '3/'1 1. ~' N/,.

~/(.

!"/Y S}v

'7~

)11,.

~... 'S1'r and Interact I

"1

')

r I, z.

II'Z-Demonstrate tJ/ p. ilIA "'/A tJ/A

,'1'

,'i fJ)A NIA 1,4

'3,~ t</A ~/t4 Si" S,~

SIr

~/;;

Supervisory Ability (3) 7 1

Comply With and N)A tJ/A 4ft ~/4 i/2.

2,3 JJ)A NjA IJ '/.. :2,) /Jjlt tJ/A Use Tech. Specs. (3)

.3 S

3 S Notes:

(1 )

Includes Technical Specification compliance for an RO.

(2)

Optional for an SRO-U.

(3)

Only applicable to SROs, Instructions:

Check the applicants' license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.

ES-301, Page 27 of 27 F'N~-

2.oF 2..

ES-301 Competencies Checklist Form ES-301-6 Facility: VOG, 7 (,fE Date of Examination: 7;30)07 -

Operating Test No.:

c9 (u/o")

l APPLICANTS RO

~

RO 0

RO 0

RO 0

SRO-I 0 SRO-I @

SRO-I 0

SRO-I 0

SRO-U 0 SRO-U 0 SRO-U 0' SRO-U 0 Competencies SCENARIO SCENARIO SCENARIO SCENARIO

~1?--;\\O1

~ ~

~.~

1

(.,.

')

A

'2 A

') 1-4 1 2

e;..

InterpreUDiagnose I/l, 2../3 i/z... 2,1 1)'2. 2,$

'3,'f Sjl,o /IJ)A-tJllr :3,~ >,l.

tJ)A N/ll '3;1. 5/(,

~,,~ I-l/A Events and Conditions

(",1 1

7 7

7

"/

,1-

" 'l.,

l/'l.

1,1-

"2.,3 I} 2-Comply With and

{,f 3."1' Nt;. I-l)A 3,-1 1,1 tJ)A NIA 'IS 3A IJ/A rJ ItA I'll-1';(.,

lS~ flY' Use Procedures (1)

I I,.J 7 5,\\.<

,,'2.-

1/:1-1,2-r-Operate Control "f 3;'f' NJt>r N)fl 3/~

,'I Ii/flo N/Il tJ)A />I/A N/A N)A 5'1,1-Sj,v S,lo S',v Boards (2) 1 I

" 'l.

1,1.-

I, e. ',2-i,'L 1/'2.--

Communicate 3,'1 3.'/ Nip N)II 3.'1' 3, <( N)A tJ)A '3/'1 1. ~' N/,.

~/~

S"/Y S}v

,~~ )11,.

~... 'S1'r and Interact I

"1

')

r I, z.

II'Z-Demonstrate tJ/ p. ilIA "'/A tJ/A

,'1'

,'i fJ)A NIA 1,4 1,~ t</A III t4 Supervisory Ability (3)

Si" S,~

sIr

~/;;

7 1

Comply With and N)A tJ/A 4ft ~/4 i/2.

2,3 JJ)A NjA IJ '/.. :2,) /Jjlt tJ/A Use Tech. Specs. (3)

.3 S

3 S Notes:

(1 )

Includes Technical Specification compliance for an RO.

(2)

Optional for an SRO-U.

(3)

Only applicable to SROs, Instructions:

Check the applicants' license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.

ES-301, Page 27 of 27 3

4

Qual Checklist, FNL f

Qual Checklist, FNL f

ES-401 Written Examination Quality Checklist Facility:

v'O<:'TU Date of Exam:

wee;)(....~

8/Ii /01

1.

2.

3.

Questions and answers are tech to the

a.

b.

SRO NRC KlAs are referenced for all questions.

Facilitv learnina obiectives are referenced as "nnrnnri"t" in accordance with Section D.2.d of ES-401

4.

The sampling process was random and systematic (If more than 4 RO or 2 SRO questions the

5.

Question duplication from the license screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate:

_ the audit exam was systematically and randomly developed; or

6.

7.

_ the audit exam was completed before the license exam was started; or

_ the examinations were developed independently; or

/the licensee certifies that there is no duplication; or

_ other (explain)

Bank use meets limits (no more than 75 percent from the bank, at least 10 percent new, and the rest new or modified); enter the actual RO / SRO-only rii"trih"tinn{,,'

Bank Modified

/e.

" / 8 Memory CIA Between 50 and 60 percent of the questions on the RO exam are written at the comprehension/ analysis level; the SRO exam may exceed 60 percent if the randomly selected KlAs support the higher cognitive levels; enter the actual RO / SRO 31, / 10 1{1

/ /-(

8.

References/handouts provided do not give away answers or aid in the elimination of distractors.

9.

Question content conforms with specific KIA statements in the previously approved examination outline and is appropriate for the tier to which they are assigned;

10.

11.

a. Author

b. Facility Reviewer (*)

c. NRC Chief Examiner (#)

d. NRC Regional Supervisor Note:

ES-401, Page 29 of 33 Form ES-401-6 ES-401 Written Examination Quality Checklist Form ES-401-6 Facility:

Date of Exam:

wee;)(.... ~ Exam Level: RO [2I'SRO 8/Ii /01

1.

2.

3.

Questions and answers are accurate and ~~,~';,.~h'~ to the

a.

b.

SRO NRC KlAs are referenced for all questions.

referenced as "n,,,nnri,,,t,, in accordance with Section D.2.d of ES-401

4.

The sampling process was random and systematic (If more than 4 RO or 2 SRO questions the N

5.

Question duplication from the license screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate:

_ the audit exam was systematically and randomly developed; or

6.

7.

_ the audit exam was completed before the license exam was started; or

_ the examinations were developed independently; or

/the licensee certifies that there is no duplication; or

_ other (explain)

Bank use meets limits (no more than 75 percent from the bank, at least 10 percent new, and the rest new or modified); enter the actual RO / SRO-only Bank Modified

/e.

" / 8 Memory CIA Between 50 and 60 percent of the questions on the RO exam are written at the comprehension/ analysis level; the SRO exam may exceed 60 percent if the randomly selected KlAs support the higher cognitive levels; enter the RO/ SRO 31, / 10 1{1

/ /-(

8.

References/handouts provided do not give away answers or aid in the elimination of distractors.

9.

Question content conforms with specific KIA statements in the previously approved examination outline and is appropriate for the tier to which they are assigned;

10.

Question n"\\"rh,nrr,,,h*j,.

dix B.

11.

The exam contains the required number of one-point, multiple choice items; the total is correct and with the value on the cover sheet.

a. Author

b. Facility Reviewer (*)

c. NRC Chief Examiner (#)

d. NRC Regional Supervisor Note:

• The facility reviewer's initials/signature are not applicable for NRC-developed examinations.

NRC in Column "c'"

ief examiner I

ES-401, Page 29 of 33 Initial

ES-403, Rev. 9 Written Examination Grading Quality Checklist Form ES-403-1 Facility: VOQtle Date of Exam: 8/15/2007 Exam Level: RO/SRO Initials Item Description a

b c

1.

Clean answer sheets copied before grading

~ 1Ii4 L:.L I

2.

Answer key changes and question deletions justified and documented

~f-

3.

Applicants' scores checked for addition errors l,L (reviewers spot check> 25% of examinations)

4.

Grading for all borderline cases (80 +/-2% overall and 70 or 80, as applicable, +/-4% on the SRO-only) reviewed in detail

-eL

5.

All other failing examinations checked to ensure that grades

(

are justified

)

~L

6.

Performance on missed questions checked for training icJIK

)

deficiencies and wording problems; evaluate validity f ~

of questions missed by half or more of the applicants Printed Name/Signature Date C..1'".' 'i X"ol'it"Z- =;:¥=

YV&7

a. Grader

./

b. Facility Reviewer(*)

tV/A

c. NRC Chief Examiner (*) EdwIn L, 1:.;'1... *iIr./4/~~ ~,

~4q7 F

I P

/ '/

d. NRC Supervisor (*)

~

b",-:-l/~A/..\\G-J ~

._.~

~L7_;:>7

/'

J

(*)

The facility reviewer's signature is not applicable for examinations graded by the NR two independent NRC reviews are required.

ES-403, Rev. 9 Written Examination Grading Quality Checklist Form ES-403-1 Facility: Vogtle Date of Exam: 8/15/2007 Exam Level: RO/SRO Initials Item Description a

b c

1.

Clean answer sheets copied before grading

~ 1Ii4 l;L f

2.

Answer key changes and question deletions justified and documented

~f-

3.

Applicants' scores checked for addition errors l,L (reviewers spot check> 25% of examinations)

4.

Grading for all borderline cases (80 +/-2% overall and 70 or 80, as applicable, +/-4% on the SRO-only) reviewed in detail

-eL

5.

All other failing examinations checked to ensure that grades

(

are justified

~L

6.

Performance on missed questions checked for training icJIK

~

deficiencies and wording problems; evaluate validity

~

of questions missed by half or more of the aJ)plicants

~

Printed Name/Signature Date c..I'".' 'i X"ol'it"Z- ~¥=

YV&7

a. Grader

./

b. Facility Reviewer(*)

tV/A

c. NRC Chief Examiner (*) £du.JIn ~

1:.;1-. *iIr./4/~~ ~;:iJ.,

~4q7 F

I P

/ '/

d. NRC Supervisor (*)

/.> be.:- ~ I~ AI-\\~ j ~

Ji;v J

~L7_;:>7

/

J

(*)

The facility reviewer's signature is not applicable for examinations graded by the NR two independent NRC reviews are required.

ES-401, Rev. 9 Written Examination Review Worksheet SRO Form ES-401-9

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q# LOK (F/H)

LOD (1-5)

Stem Cues T /F Focus Credo Partial Job-Minutia

1. /

Back-Q=

SRO u/E/S Explanation Dist.

Link units ward KIA Only Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1.

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2.

Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3.

Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (Le., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4.

Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (Le., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (Le., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5.

Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable ).

6.

Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

7.

At a minimum, explain any "U" ratings e.g., how the Appendix B psychometric attributes are not being met).

1 H

x x

U Question written appears to be more systems related and RO level. In assuming procedure actions have been taken - identify the procedure.

What alarm would be in as a result of this event? How can distractor A be plausible? Why would one expect the reactor to trip because we are at a point to take/ record data. You stated that the question matched your objective - it does not. If the question was written to match the objective, then it would match the KIA and also be written to an SRO level. (EXPLAINED CONCERN FOR "A". EXCEPTED QUESTION)

(OK)

ES-401, Rev. 9 Written Examination Review Worksheet SRO Form ES-401-9

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q# LOK (F/H)

LOD (1-5)

Stem Cues T /F Focus Credo Partial Job-Minutia

1. /

Back-Q=

SRO u/E/S Explanation Dist.

Link units ward KIA Only Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1.

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2.

Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3.

Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (Le., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4.

Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (Le., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (Le., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5.

Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable ).

6.

Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

7.

At a minimum, explain any "U" ratings e.g., how the Appendix B psychometric attributes are not being met).

1 H

x x

U Question written appears to be more systems related and RO level. In assuming procedure actions have been taken - identify the procedure.

What alarm would be in as a result of this event? How can distractor A be plausible? Why would one expect the reactor to trip because we are at a point to take/ record data. You stated that the question matched your objective - it does not. If the question was written to match the objective, then it would match the KIA and also be written to an SRO level. (EXPLAINED CONCERN FOR "A". EXCEPTED QUESTION)

(OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q# LOK LOD (F/H)

(1-5)

Stem Cues TIF Credo Partial Job-Minutia

1. 1 Back-Q= SRO utE/S Explanation Focus Dist.

Link units ward KIA Only 2

H 2

X X

U Stem is weak. For the conditions given, why would anyone expect containment pressure to remain stable? If containment pressure rises over time, what time (s) are we looking at? Would it be increasing slowly? Based on the information provided can the applicant make this call? Could the applicant say stable and increasing slowly? This makes distractors A & C not plausible. Why would the operator be expected to immediately transfer as you indicated in two distractors. (CHANGED THE STEM) (OK) 3 H

1 X

X E

Based on the information given in the stem (1 NAAis capable of being energized) it appears that the others distractors are not plausible. Since this is an auto transfer, any operator should or would want to re-energize any bus available to them. (OK) 4 H

2 S

(OK) 5 H

3 S

(OK) 6 H

X U

Please explain how this KIA is a match. I do not see the modified question. (CHANGED INFORMATION IN STEM -

KIA MATCH))

7 F

1 X

?I Seems like an RO question. I do not see the bank question. (OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q# LOK LOD (F/H)

(1-5)

Stem Cues TIF Credo Partial Job-Minutia

1. 1 Back-Q= SRO utE/S Explanation Focus Dist.

Link units ward KIA Only 2

H 2

X X

U Stem is weak. For the conditions given, why would anyone expect containment pressure to remain stable? If containment pressure rises over time, what time (s) are we looking at? Would it be increasing slowly? Based on the information provided can the applicant make this call? Could the applicant say stable and increasing slowly? This makes distractors A & C not plausible. Why would the operator be expected to immediately transfer as you indicated in two distractors. (CHANGED THE STEM) (OK) 3 H

1 X

X E

Based on the information given in the stem (1 NAAis capable of being energized) it appears that the others distractors are not plausible. Since this is an auto transfer, any operator should or would want to re-energize any bus available to them. (OK) 4 H

2 S

(OK) 5 H

3 S

(OK) 6 H

X U

Please explain how this KIA is a match. I do not see the modified question. (CHANGED INFORMATION IN STEM -

KIA MATCH))

7 F

1 X

?I Seems like an RO question. I do not see the bank question. (OK)

I I

ES-401, Rev. 9 2

Form ES-401-9

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. /

Back-Q=

SRO utE/S Explanation Focus Dist.

Link units ward KIA Only

~

F 1

X X

U This appears to be a system/RO question. Questions talks about system but not about procedure requirements/operation/SRO decision making. Stem can be reworded to address the issues. (WROTE NEW QUESTION) (OK) 9 H

1 X

UtE D will always be correct for a scenario where there are procedure guidance and the crew/operator has not gotten there yet. As long as the operators are following procedures, with no transitions required D will all be correct. Should consider rewording stem/distractors. You told the operator in the stem that this action will be performed later......

(CHANGED STEM) (OK).

0 F

1 S

(OK) THEY WANTED TO CHANGE BUT I DID NOT AGREE 1

H X

E Missing word in stem. I would think other alarms would be in. Need to see more information on system operation and response to various conditions. (SAID VARIOUS ALARMS - CORRECTED WORD IN THE STEM) (OK) 2 F

X X

X U

Question not written at SRO level. Distractor A is not plausible - "close and lock the doors" Extra words in distractor D. Distractor D could also be correct. (REWORDED THE STEM AND DISTRACTORS.

3 F

S (OK) 4 F

S (OK) 5 H

2 S

(OK) 6 H

2 S

(OK) 7 H

1 X

U This is a true/false question (REWORDED STEM AND DISTRACTORS)

(OK) 8 F

1 (OK) 9 F

1 X

E Distractor D is not plausible. (OS) 0

?

This is a retake exam question from 2006 (QUESTION IS MODIFIED)

(OK)

ES-401, Rev. 9 2

Form ES-401-9

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. /

Back-Q=

SRO utE/S Explanation Focus Dist.

Link units ward KIA Only F

1 X

X U

This appears to be a system/RO question. Questions talks about system but not about procedure requirements/operation/SRO decision making. Stem can be reworded to address the issues. (WROTE NEW QUESTION) (OK)

H 1

X UtE D will always be correct for a scenario where there are procedure guidance and the crew/operator has not gotten there yet. As long as the operators are following procedures, with no transitions required D will all be correct. Should consider rewording stem/distractors. You told the operator in the stem that this action will be performed later......

(CHANGED STEM) (OK).

0 F

1 S

(OK) THEY WANTED TO CHANGE BUT I DID NOT AGREE 1

H X

E Missing word in stem. I would think other alarms would be in. Need to see more information on system operation and response to various conditions. (SAID VARIOUS ALARMS - CORRECTED WORD IN THE STEM) (OK) 2 F

X X

X U

Question not written at SRO level. Distractor A is not plausible - "close and lock the doors" Extra words in distractor D. Distractor D could also be correct. (REWORDED THE STEM AND DISTRACTORS.

3 F

S (OK) 4 F

S (OK) 5 H

2 S

(OK) 6 H

2 S

(OK) 7 H

1 X

U This is a true/false question (REWORDED STEM AND DISTRACTORS)

(OK) 8 F

1 (OK) 9 F

1 X

E Distractor D is not plausible. (OS)

~O

?

This is a retake exam question from 2006 (QUESTION IS MODIFIED)

(OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 8ack-Q=

SRO u/E/S Explanation Focus Dist.

Link units ward KIA Only 1:>1 2

S (OK) 2 H

1 X

X

??

This is a very basic question. Why did you organize the distractors as you did? It appears that you method may be cuing the applicant as which one to pick----- the first one out of order from the order given.

(CHANGED ORDER) (OK)

~3 H

1 X

X U

Each alarm is associated with the same system. Therefore, why would anyone select unexpected? As written, the applicant is only selecting from two answers. (REMOVED EXPECTED AND UNEXPECTED) (OK) 4 H

2 X

X E

Change stem such that it requires the applicant to make a decision on a set of given conditions.

Consider giving a set of conditions that would require the applicant to determine if cooldown and depressurization is required and then select the appropriate procedure. May need to change the distractors. (ADDED INFO TO THE STEM) (OK) 5 F

1 S

Memory level question (weak question) (CONVINCED ME THAT IT WAS NOT A MEMORY LEVEL QUESTION) (OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 8ack-Q=

SRO u/E/S Explanation Focus Dist.

Link units ward KIA Only P1 2

S (OK)

D2 H

1 X

X

??

This is a very basic question. Why did you organize the distractors as you did? It appears that you method may be cuing the applicant as which one to pick----- the first one out of order from the order given.

(CHANGED ORDER) (OK) 3 H

1 X

X U

Each alarm is associated with the same system. Therefore, why would anyone select unexpected? As written, the applicant is only selecting from two answers. (REMOVED EXPECTED AND UNEXPECTED) (OK)

~4 H

2 X

X E

Change stem such that it requires the applicant to make a decision on a set of given conditions.

Consider giving a set of conditions that would require the applicant to determine if cooldown and depressurization is required and then select the appropriate procedure. May need to change the distractors. (ADDED INFO TO THE STEM) (OK) 5 F

1 S

Memory level question (weak question) (CONVINCED ME THAT IT WAS NOT A MEMORY LEVEL QUESTION) (OK)

ES-401 Written Examination RO Form ES-401-9 Q#I

1.

2.

3. Psychometric Flaws LOK LOD (CIA) (1-5) Stem Cues T/F Credo Partial Focus Dist.

Job-Link

4. Job Content Flaws

5. Other

6.

Minutia

1. 1 8ack-Q=

SRO U/E/S units ward KIA Only Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.]

7.

Explanation

1.

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2.

Enter the level of difficulty (LOO) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3.

Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (Le., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

One or more distractors is not credible.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4.

Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (Le., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (Le., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5.

Check questions that are sampled for conformance with the approved KIA and those that are deSignated SRO-only (KIA and license level mismatches are unacceptable).

6.

Based on the reviewer's judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

7.

At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

1 H

2 X

2 F

2 X

3 H

2 X

X 4

H 2

X X

E U

E E

No discussion was made on when the ARVs would open.

Why are ARVs controllin~ tave plausible in distractor A and B? -

PROVIDED J STIFICATION INDICATING THAT A & B WERE PLAUSIBLE. MADE A CHANGE TO THE STEM. (OK)

The second sentence in A, B, C makes no sense. It looks like th author had 2 trains Ofthou9ht9,0ing. This Q is unsat because 3 choices are affecte. - CHANGED THE STEM AND ADDED WORDS TO THE DISTRACTORS (OK)

Define immediately in the notes or feedback section in the event this is challenged because several minutes later the opposite occurs. -

CHANGED THE STEM AND THE DISTRACTORS (OK)

Indent bullets Change Stem: Which one of the following is correct if NO operator action is taken?

Distractor B: an (s) missing on shift. - CHANGED THE STEM (OK)

ES-401 Written Examination RO Form ES-401-9 Q#I

1.

2.

3. Psychometric Flaws LOK LOD (CIA) (1-5) Stem Cues T/F Credo Partial Focus Dist.

Job-Link

4. Job Content Flaws

5. Other

6.

Minutia

1. 1 8ack-Q=

SRO U/E/S units ward KIA Only Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.]

7.

Explanation

1.

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2.

Enter the level of difficulty (LOO) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3.

Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (Le., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

One or more distractors is not credible.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4.

Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (Le., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (Le., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5.

Check questions that are sampled for conformance with the approved KIA and those that are deSignated SRO-only (KIA and license level mismatches are unacceptable).

6.

Based on the reviewer's judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

7.

At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

1 H

2 X

2 F

2 X

3 H

2 X

X 4

H 2

X X

E U

E E

No discussion was made on when the ARVs would open.

Why are ARVs controllin~ tave plausible in distractor A and B? -

PROVIDED J STIFICATION INDICATING THAT A & B WERE PLAUSIBLE. MADE A CHANGE TO THE STEM. (OK)

The second sentence in A, B, C makes no sense. It looks like th author had 2 trains Ofthou9ht9,0ing. This Q is unsat because 3 choices are affecte. - CHANGED THE STEM AND ADDED WORDS TO THE DISTRACTORS (OK)

Define immediately in the notes or feedback section in the event this is challenged because several minutes later the opposite occurs. -

CHANGED THE STEM AND THE DISTRACTORS (OK)

Indent bullets Change Stem: Which one of the following is correct if NO operator action is taken?

Distractor B: an (s) missing on shift. - CHANGED THE STEM (OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#I LOK LaD (CIA) (1-5) Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO U/E/S Explanation Focus Dist.

Link units ward KIA Only 5

F 2

X U

No HU rate is not plausible. Change distractors A and C to reflect a 50 F per hour max HU rate.

This Q is rated unsat due to 2 non plausible distractors.

~REWORDED THE STEM AND THE DISTRACTORS)

OK) 6 M?

2 X

X E

Indent bullets Stem: Change to "...Ioop suction valve power supplies" It is not clear what the correct answer is given the attached ref material. Fac please explain.

I think this is a "H" question. (IT WAS OK, NO NEED TO CHANGE THE WORDING JUST BECAUSE I WANTED TOO) 7 H

3 5

(OK) 8 F

2 X

U This Q does not revolve around a loss or malfunction of HPIILPI. It discusses minimum requirements onl).

Rewrite Q to meet KA. (REWROTE QUESTION (OK) 9 F

2 X

X E

To manF; unnecessary words in the stem. Restate: Which one of t e following is the correct description for the 8 hour9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> cooldown method of the PRT as stated in SOP-13004-1/2, PRT Operation? (REWORDED STEM AND DISTRACTORS) (OK) 10 F

2 X

E Distractor C is notfclausible. Replace "c" (CHANGED DISTRACTOR C) OK) 11 F

2 X

X E

State the current time in the stem to be 0130. 18022-C states "loss of ACCW greater than 10 minutes - stop RCPs.

Remove "no later than 0125" This is NOT needed anymore. (MADE CHANGES) (OK) 12 F

2 5

(Ok) 13 H

3 5

(OK) 14 H

2 X

X E

Reference a procedure in the stem... (OK) 15 H

2 X

E This is a memory question on basic power supplies.

Change "C" to "M. (LEAVE AS C)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#I LOK LaD (CIA) (1-5) Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO U/E/S Explanation Focus Dist.

Link units ward KIA Onlv 5

F 2

X U

No HU rate is not plausible. Change distractors A and C to reflect a 50 F per hour max HU rate.

This Q is rated unsat due to 2 non plausible distractors.

~REWORDED THE STEM AND THE DISTRACTORS)

OK) 6 M?

2 X

X E

Indent bullets Stem: Change to "...Ioop suction valve power supplies" It is not clear what the correct answer is given the attached ref material. Fac please explain.

I think this is a "H" question. (IT WAS OK, NO NEED TO CHANGE THE WORDING JUST BECAUSE I WANTED TOO) 7 H

3 5

(OK) 8 F

2 X

U This Q does not revolve around a loss or malfunction of HPIILPI. It discusses minimum requirements onl).

Rewrite Q to meet KA. (REWROTE QUESTION (OK) 9 F

2 X

X E

To manF; unnecessary words in the stem. Restate: Which one of t e following is the correct description for the 8 hour9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> cooldown method of the PRT as stated in SOP-13004-1/2, PRT Operation? (REWORDED STEM AND DISTRACTORS) (OK) 10 F

2 X

E Distractor C is notfclausible. Replace "c" (CHANGED DISTRACTOR C) OK) 11 F

2 X

X E

State the current time in the stem to be 0130. 18022-C states "loss of ACCW greater than 10 minutes - stop RCPs.

Remove "no later than 0125" This is NOT needed anymore. (MADE CHANGES) (OK) 12 F

2 5

(Ok) 13 H

3 5

(OK) 14 H

2 X

X E

Reference a procedure in the stem... (OK) 15 H

2 X

E This is a memory question on basic power supplies.

Change "C" to "M. (LEAVE AS C)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#I LOK LOD (CIA) (1-5) Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO utE/S Explanation Focus Dis!.

Link units ward KIA Only 16 H

2 X

X X

E Remove the first bullet (-)

Indent all other bullets.

Put a space before the "Which one" statement.

"A" Change "due to" to "since" The note on page 12 of 53,17006-1 states requires 2/3 channels but setpoint at the op say 1/3. Does this make "C" a potentiall~ correct answer? CAF (CHANGED WORDS TO C EW BRIEFING) (OK) 17 F

2 S

(BOLD WORDS IN THE STEM) (OK) 18 H

2 X

E If "B" was correct, "A" would be also. Need an "only" in "A." -

(MADE CHANGE) (OK) 19 H

3 X

U The statement "The ROD DEV annunciator does not alarm" is a non action statement and is moot. It further implies that neither "C" or "0" is the correct answer.

Delete this statement from "C" and "0" and add the word "only" at the be~inning. This Q is rated as unsat because of 2 nonplausib e distractors. (WILL ADD REFERENCE -

NO NEED TO MAKE ANY CHANGES) (OK) 20 F

2 X

E The first 4 bullets are not required to answer the question.

Furthermore, this question does not meet the KA which is RCP malfunction. (ADDED BULLETS TO HELP MEET THE KIA) (OK) 21 H

3 X

U Indent bullets The "NOT" words in the distractors make then nonplausible. In "B" and "C" use the word "only" and in "0" use the word "neither and nor." (MADE CHANGES TO BOTH THE STEM AND THE DISTRACTORS) (OK) MAY TAKE ANOTHER LOOK AT IT LATER) 22 H

2 X

E The first 2 bullets are not necessary to answer the question. Delete them. Format: Which one of the following is the correct indication representation if 5 core exit TC's are failed? (MADE CHANGES) (OK) 23 F

2 X

X E

Indent bullets Second part of KA not met. A procedure choice needs to be included. -

(SECOND PART OF THE KIA IS MET FIRST PART NOT DIRECTLY MET) (OK) 24 H

2 X

E Indent bullets - (CHANGES MADE) (OK) 25 H

2 X

E Look at the words in the stem: "Which ONE"...

Indent bullets..... - (REWORDED THE STEM) (OK)

(7/27/2007 ADDED INFORMATION TO STEM) (OK

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#I LOK LOD (CIA) (1-5) Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO utE/S Explanation Focus Dis!.

Link units ward KIA Only 16 H

2 X

X X

E Remove the first bullet (-)

Indent all other bullets.

Put a space before the "Which one" statement.

"A" Change "due to" to "since" The note on page 12 of 53,17006-1 states requires 2/3 channels but setpoint at the op say 1/3. Does this make "C" a potentiall~ correct answer? CAF (CHANGED WORDS TO C EW BRIEFING) (OK) 17 F

2 S

(BOLD WORDS IN THE STEM) (OK) 18 H

2 X

E If "B" was correct, "A" would be also. Need an "only" in "A." -

(MADE CHANGE) (OK) 19 H

3 X

U The statement "The ROD DEV annunciator does not alarm" is a non action statement and is moot. It further implies that neither "C" or "0" is the correct answer.

Delete this statement from "C" and "0" and add the word "only" at the be~inning. This Q is rated as unsat because of 2 nonplausib e distractors. (WILL ADD REFERENCE -

NO NEED TO MAKE ANY CHANGES) (OK) 20 F

2 X

E The first 4 bullets are not required to answer the question.

Furthermore, this question does not meet the KA which is RCP malfunction. (ADDED BULLETS TO HELP MEET THE KIA) (OK) 21 H

3 X

U Indent bullets The "NOT" words in the distractors make then nonplausible. In "B" and "C" use the word "only" and in "0" use the word "neither and nor." (MADE CHANGES TO BOTH THE STEM AND THE DISTRACTORS) (OK) MAY TAKE ANOTHER LOOK AT IT LATER) 22 H

2 X

E The first 2 bullets are not necessary to answer the question. Delete them. Format: Which one of the following is the correct indication representation if 5 core exit TC's are failed? (MADE CHANGES) (OK) 23 F

2 X

X E

Indent bullets Second part of KA not met. A procedure choice needs to be included. -

(SECOND PART OF THE KIA IS MET FIRST PART NOT DIRECTLY MET) (OK) 24 H

2 X

E Indent bullets - (CHANGES MADE) (OK) 25 H

2 X

E Look at the words in the stem: "Which ONE"...

Indent bullets..... - (REWORDED THE STEM) (OK)

(7/27/2007 ADDED INFORMATION TO STEM) (OK

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#I LOK LOD (CIA) (1-5) Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO utE IS Explanation Focus Dis!.

Link units ward KIA Only 26 F

2 S

(CHANGED DISTRACTOR "A") (OK) 27 H

2 X

E Indent bullets - (OK) 28 F

2 X

X E

Indent bullets Stem: Which one of the following actions is correct per 18001, Primary instrumentation malfunction. (MADE CHANGES) 29 H

2 X

X E

Indent bullets Typo in second bullet.

Consider moving everything except the bolded stuff to the stem and make it a "fill in the blank" - (INDENTED AND FIXED TYPO) (OK) 30 F

2 X

X?

E Indent bullets Second part of KA not met. A procedure choice needs to be included. - ~MADE CHANGES TO THE STEM AND DISTRACTOR ) (OK) 31 H

2 X

X E

Indent bullets Please explain BIT operation..... - (QUESTION IS OK) 32 H

2 X

X X

E Indent bullets Distractor 0 is not plausible.

Inadequate reference to support correct answer. Need to know wh6 trip occurs when -31 fuse blows. -(CHANGED DISTRA TOR "0")

33 F

2 X

X U

Indent bullets What would make the applicant think that there is a temperature control valve in this system if there NOT one and one never existed? Distractors "A" and "B" are not plausible. ~BASED ON KIA REQUIREMENT AND THE FACT OF OW THEIR SYSTEM IS DESIGNED I ACCEPTED THEIR DISTRACTORS - APPLICANT MUST KNOW HOW SYSTEM IS DESIGNED) (OK) 34 H

2 X

X U

Indent bullets Distractors "B" and "0" are not plausible. Q is ranked unsat due to 2 non plausible distractors. - (DECIDED TO ACCEPT ANSWER AFTER LOOKING AT CONTROL BOARD LAYOUT) (OK) 35 H

3 X

U Indent bullets. There is no Which ONE statement (stem)

- (ADDED THE STEM) (OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#I LOK LOD (CIA) (1-5) Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO utE IS Explanation Focus Dis!.

Link units ward KIA Only 26 F

2 S

(CHANGED DISTRACTOR "A") (OK) 27 H

2 X

E Indent bullets - (OK) 28 F

2 X

X E

Indent bullets Stem: Which one of the following actions is correct per 18001, Primary instrumentation malfunction. (MADE CHANGES) 29 H

2 X

X E

Indent bullets Typo in second bullet.

Consider moving everything except the bolded stuff to the stem and make it a "fill in the blank" - (INDENTED AND FIXED TYPO) (OK) 30 F

2 X

X?

E Indent bullets Second part of KA not met. A procedure choice needs to be included. - ~MADE CHANGES TO THE STEM AND DISTRACTOR ) (OK) 31 H

2 X

X E

Indent bullets Please explain BIT operation..... - (QUESTION IS OK) 32 H

2 X

X X

E Indent bullets Distractor 0 is not plausible.

Inadequate reference to support correct answer. Need to know wh6 trip occurs when -31 fuse blows. -(CHANGED DISTRA TOR "0")

33 F

2 X

X U

Indent bullets What would make the applicant think that there is a temperature control valve in this system if there NOT one and one never existed? Distractors "A" and "B" are not plausible. ~BASED ON KIA REQUIREMENT AND THE FACT OF OW THEIR SYSTEM IS DESIGNED I ACCEPTED THEIR DISTRACTORS - APPLICANT MUST KNOW HOW SYSTEM IS DESIGNED) (OK) 34 H

2 X

X U

Indent bullets Distractors "B" and "0" are not plausible. Q is ranked unsat due to 2 non plausible distractors. - (DECIDED TO ACCEPT ANSWER AFTER LOOKING AT CONTROL BOARD LAYOUT) (OK) 35 H

3 X

U Indent bullets. There is no Which ONE statement (stem)

- (ADDED THE STEM) (OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

0#1 LOK LOD (CIA) (1-5) Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-0=

SRO utEIS Explanation Focus Dist.

Link units ward KIA Only 36 F

1 X

U Indent bullets This Q has very little discriminatory value. Make it a multiple parts question to field a decent LOD. Q is unsat since LOD = 1. (REWRO E QUESTION CHANGED DISTRACTORS) (OK) 37 H

2 X

X?

E Indent bullets Second part of KA not met. (OK AS WRITTEN) 38 F

2 S

(CHANGED QUESTION) (OK) 39 F

1 X

E Indent bullets With Rx power @ 100%, there is no discriminato~ value.

Chan1e initial power to 77%. (CHANGED POWE LEVE ) (OK) 40 F

1 X

E If DC bus loads are consistent with design bases, distractors "8" and "D" are not plausible. Delete this statement and provide applicant with a discharSe rate and let him draw his own conclusions. - (THIS WA FROM THE 2006 TEST) (OK) 41 H

2 X

E Indent bullets (MADE CHANGES) (OK) 42 H

1 X

U Indent bullets If distractor "A", "8", or "C" was correct, "D" would also be correct. This Q has no discriminatory value.(REVIEWED PROCEDURE AND LOOKED AT DRAWING - AGREED THAT QUESTION IS OK) 43 F

2 X

E Stem is poorly written. (REWROTE STEM) (OK) 44 H

2 X

E Need a procedure reference (18016-C) in the stem to rule out distractor D. (ADDED INFORMATION) (OK) 45 F

2 X

E Indent 1 and 2 in the stem. (MADE CHANGE) (OK) 46 H

1 X

X U

Indent bullets. The distractors are not plausible.

(REMOVED WORDS FROM THE STEM) (OK) 47 H

2 X

E Indent bullets. (MADE CHANGES) (OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

0#1 LOK LOD (CIA) (1-5) Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-0=

SRO utEIS Explanation Focus Dist.

Link units ward KIA Only 36 F

1 X

U Indent bullets This Q has very little discriminatory value. Make it a multiple parts question to field a decent LOD. Q is unsat since LOD = 1. (REWRO E QUESTION CHANGED DISTRACTORS) (OK) 37 H

2 X

X?

E Indent bullets Second part of KA not met. (OK AS WRITTEN) 38 F

2 S

(CHANGED QUESTION) (OK) 39 F

1 X

E Indent bullets With Rx power @ 100%, there is no discriminato~ value.

Chan1e initial power to 77%. (CHANGED POWE LEVE ) (OK) 40 F

1 X

E If DC bus loads are consistent with design bases, distractors "8" and "D" are not plausible. Delete this statement and provide applicant with a discharSe rate and let him draw his own conclusions. - (THIS WA FROM THE 2006 TEST) (OK) 41 H

2 X

E Indent bullets (MADE CHANGES) (OK) 42 H

1 X

U Indent bullets If distractor "A", "8", or "C" was correct, "D" would also be correct. This Q has no discriminatory value.(REVIEWED PROCEDURE AND LOOKED AT DRAWING - AGREED THAT QUESTION IS OK) 43 F

2 X

E Stem is poorly written. (REWROTE STEM) (OK) 44 H

2 X

E Need a procedure reference (18016-C) in the stem to rule out distractor D. (ADDED INFORMATION) (OK) 45 F

2 X

E Indent 1 and 2 in the stem. (MADE CHANGE) (OK) 46 H

1 X

X U

Indent bullets. The distractors are not plausible.

(REMOVED WORDS FROM THE STEM) (OK) 47 H

2 X

E Indent bullets. (MADE CHANGES) (OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#I LOK LOD (CIA) (1-5) Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO UlE/S Explanation Focus Dis!.

Link units ward KIA On Iv 48 F

2 X

X?

E Indent bullets Note: Why does 17034-1, page 25 of 89 state that the DC distrobution panels do not have ground detection relus when panel AD1 does? Second part of KA not met. (

ILL REVISIT TOMORROW

• • • • • • • • • • • • • • • • • • • • • • • • • • • • • )

49 H

2 X

E Indent bullets (OK) 50 H

2 X

E There appears to be unnecessary information in the stem.

Eliminate the first 3 bullets. They are not needed to answer the Q (See caution on paae 20 of 26, 18028-C).

{~~*~RJ2Jgg*~*~t~~~1:~\\gl!:i***1t!1~*,91d~~!1£~)******

(LOCATED A REPLACEMENT QUESTION) (OK) 51 F

2 X

E Indent bullets (MADE CHANGE AND INFORMATION TO COVER BASES. THIS ENHANCED THE KIA MATCH) 52 H

1 X

U I disagree with distractor analysis plausibility statement. A diversion of potentiall6 radioactive water to a clean anythina is not~lausi Ie. Replace distractors "C" and "D."

kCHAN ED 01 TRACTOR "C" ADDED A WORD TO THE TEM) (OK) 53 F

2 5

(OK) 54 F

2 X

X X

E Indent bullets Why did you add "to limit the leakage" to the answer.

It is NOT in the procedure. In distractor "A": I think the word "at" should be "a" (MADE CHANGES) (OK) 55 H

2 X

X X

E Indent bullets As is, "B" is also a correct answer. Add the word "only" at the end. (MADE CHANGE) (OK) 56 H

2 X

E Indent bullets (OK) 57 F

2 X

E Indent bullets (CHANGED SET POINT & INDENTED)

(OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#I LOK LOD (CIA) (1-5) Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO UlE/S Explanation Focus Dis!.

Link units ward KIA On Iv 48 F

2 X

X?

E Indent bullets Note: Why does 17034-1, page 25 of 89 state that the DC distrobution panels do not have ground detection relus when panel AD1 does? Second part of KA not met. (

ILL REVISIT TOMORROW

• • • • • • • • • • • • • • • • • • • • • • • • • • • • • )

49 H

2 X

E Indent bullets (OK) 50 H

2 X

E There appears to be unnecessary information in the stem.

Eliminate the first 3 bullets. They are not needed to answer the Q (See caution on paae 20 of 26, 18028-C).

{~~*~RJ2Jgg*~*~t~~~1:~\\gl!:i***1t!1~*,91d~~!1£~)******

(LOCATED A REPLACEMENT QUESTION) (OK) 51 F

2 X

E Indent bullets (MADE CHANGE AND INFORMATION TO COVER BASES. THIS ENHANCED THE KIA MATCH) 52 H

1 X

U I disagree with distractor analysis plausibility statement. A diversion of potentiall6 radioactive water to a clean anythina is not~lausi Ie. Replace distractors "C" and "D."

kCHAN ED 01 TRACTOR "C" ADDED A WORD TO THE TEM) (OK) 53 F

2 5

(OK) 54 F

2 X

X X

E Indent bullets Why did you add "to limit the leakage" to the answer.

It is NOT in the procedure. In distractor "A": I think the word "at" should be "a" (MADE CHANGES) (OK) 55 H

2 X

X X

E Indent bullets As is, "B" is also a correct answer. Add the word "only" at the end. (MADE CHANGE) (OK) 56 H

2 X

E Indent bullets (OK) 57 F

2 X

E Indent bullets (CHANGED SET POINT & INDENTED)

(OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#I LOK LOD (CIA) (1-5) Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO U/E/S Explanation Focus Dis!.

Link units ward KIA Only 58 H

1 X

U Indent bullets Stem: The word "that" is missing after parameter.

This Q has no discriminatory value. Maybe make a Q that ask what is the correct order in which this are affected.

This Q has no discriminatory value. (REPLACED QUESTION) (OK) 59 F

2 X

E Indent bullets (MADE CHANGE) (OK) 60 F

2 X

E Indent bullets (MADE CHANGES; MODIFIED STEM AND DISTRACTORS) (OK) 61 F

1 5

(OK) 62 F

2 5

(OK) 63 H

1 X

X E

Unit 1 (not one..... ) There appears to be unnecessary information in the stem. Distractor A is not plausible. Why would one expect a significant increase in counts? Should there not be some alarm in at this time? (OK) 64 H

1 X

5 Bank question - Not modified. Only changed number from previous use. (CHANGED TO BANK) (OK) 65 H

1 X

?

Explain why you consider D plausible. What procedure Identifies the type of equipment that should be worn when there is known airborne radiation? ~CHANGED STEM AND DISTRAC OR) (OK) 66 H

2 5

(OK) 67 H

2 X

U Only the answer gives a reason/bases. The other three distractors say "if present due to."

Not a reason. Inadequate distractors.

(CHANGED STEM AND DISTRACTOR) (OK) 68 H

3 5

(OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#I LOK LOD (CIA) (1-5) Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO U/E/S Explanation Focus Dis!.

Link units ward KIA Only_

58 H

1 X

U Indent bullets Stem: The word "that" is missing after parameter.

This Q has no discriminatory value. Maybe make a Q that ask what is the correct order in which this are affected.

This Q has no discriminatory value. (REPLACED QUESTION) (OK) 59 F

2 X

E Indent bullets (MADE CHANGE) (OK) 60 F

2 X

E Indent bullets (MADE CHANGES; MODIFIED STEM AND DISTRACTORS) (OK) 61 F

1 5

(OK) 62 F

2 5

(OK) 63 H

1 X

X E

Unit 1 (not one..... ) There appears to be unnecessary information in the stem. Distractor A is not plausible. Why would one expect a significant increase in counts? Should there not be some alarm in at this time? (OK) 64 H

1 X

5 Bank question - Not modified. Only changed number from previous use. (CHANGED TO BANK) (OK) 65 H

1 X

?

Explain why you consider D plausible. What procedure Identifies the type of equipment that should be worn when there is known airborne radiation? ~CHANGED STEM AND DISTRAC OR) (OK) 66 H

2 5

(OK) 67 H

2 X

U Only the answer gives a reason/bases. The other three distractors say "if present due to."

Not a reason. Inadequate distractors.

(CHANGED STEM AND DISTRACTOR) (OK) 68 H

3 5

(OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#I LOK LOD (CIA) (1-5) Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO U/E/S Explanation Focus Dist.

Link units ward KIA Only 69 H

2 X

X X

U The stem states that the "Crew is at the step to use the tables to determine if...... can be stopped based on subcooling criteria"... Why would one not be expected to check a criteria, in this case subcooling, before action is taken to stop a pump? Stem talks about stoptng CCPs and SIPs (more than one). Why is "St00 a C P" needed in the stem? ~CHANGED STEM)~OKk 7/25/2007 ADDED CCP A WAS TOPPED TO STE ) 0 70 F

2 X

E Could distractor A not be correct if they are not able to open both PORVs? (REVISED STEM AND DISTRACTORS) (OK) 71

?

The stem ask for a method. Please explain why you think each of (c0ur distractors describe a method. (CHANGED STEM) OK) 72 H

2 S

(OK) 73 H

3 S

(REWORDED DISTRACTOR B) (OK) 74 H

2 X

U I do not understand why distractors C and Dare considered plausible. Also the second part of B is not plausible. I do not know of any situations where a procedure would require the crew to wait 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> if an undesirable condition exist. (CHANGED STEM AND DISTRACTOR) (OK) 75 H

3 X

X U

Change the wording in the stem form "may" to should be implemented...... I do not understand how distractors B and C are plausible based on the information provided in the stem. (MADE CHANGE TO STEM)

(DECIDED DISTRACTORS WERE OK) (OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#I LOK LOD (CIA) (1-5) Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO U/E/S Explanation Focus Dist.

Link units ward KIA Only 69 H

2 X

X X

U The stem states that the "Crew is at the step to use the tables to determine if...... can be stopped based on subcooling criteria"... Why would one not be expected to check a criteria, in this case subcooling, before action is taken to stop a pump? Stem talks about stoptng CCPs and SIPs (more than one). Why is "St00 a C P" needed in the stem? ~CHANGED STEM)~OKk 7/25/2007 ADDED CCP A WAS TOPPED TO STE ) 0 70 F

2 X

E Could distractor A not be correct if they are not able to open both PORVs? (REVISED STEM AND DISTRACTORS) (OK) 71

?

The stem ask for a method. Please explain why you think each of (c0ur distractors describe a method. (CHANGED STEM) OK) 72 H

2 S

(OK) 73 H

3 S

(REWORDED DISTRACTOR B) (OK) 74 H

2 X

U I do not understand why distractors C and Dare considered plausible. Also the second part of B is not plausible. I do not know of any situations where a procedure would require the crew to wait 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> if an undesirable condition exist. (CHANGED STEM AND DISTRACTOR) (OK) 75 H

3 X

X U

Change the wording in the stem form "may" to should be implemented...... I do not understand how distractors B and C are plausible based on the information provided in the stem. (MADE CHANGE TO STEM)

(DECIDED DISTRACTORS WERE OK) (OK)

ES-501, Rev. 9 Post-Examination Check Sheet Form ES-501-1

~--.... -.--.... -.---~

...... -.-.--~-----

Post-Examination Check Sheet Facility: _ Vogtle Date of Examination: 7/30 -8/15/2007 Task Description Date Complete

1.

Facility written exam comments or graded exams received and verified complete 811712007

2.

Facility written exam comments reviewed and incorporated and NRC grading completed, if necessary N/A

3.

Operating tests graded by NRC examiners 9/512007

4.

NRC chief examiner review of operating test and written exam grading completed 9/12/2007

5.

Responsible supervisor review completed 9/13/2007

6.

Management (licensing official) review completed 9/13/2007

7.

License and denial letters mailed 9/13/2007

8.

Facility notified of results 9/13/2007

9.

Examination report issued (refer to NRC MC 0612) 9/14/2007

10. Reference material returned after final resolution of any appeals N/A ES-501, Rev. 9 Post-Examination Check Sheet Form ES-501-1 Post-Examination Check Sheet Facility: _ Vogtle Date of Examination: 7/30 -8/15/2007 Task Description Date Complete

1.

Facility written exam comments or graded exams received and verified complete 811712007

2.

Facility written exam comments reviewed and incorporated and NRC grading completed, if necessary N/A

3.

Operating tests graded by NRC examiners 9/512007

4.

NRC chief examiner review of operating test and written exam grading completed 9/12/2007

5.

Responsible supervisor review completed 9/13/2007

6.

Management (licensing official) review completed 9/13/2007

7.

License and denial letters mailed 9/13/2007

8.

Facility notified of results 9/13/2007

9.

Examination report issued (refer to NRC MC 0612) 9/14/2007

10. Reference material returned after final resolution of any appeals N/A

Check List Check List