ML093280324

Initial Exam 2009-301 Final Administrative Documents

ML093280324
Person / Time
Site:	Vogtle
Issue date:	11/18/2009
From:	NRC/RGN-II
To:	Southern Nuclear Operating Co
References
50-424/09-301, 50-425/09-301, ES-201-1
Download: ML093280324 (30)

Text

ES-201 Examination Preparation Checklist Form ES-201-1 Facility:

Vogtle 2009-301 Date of Examination:

6/1-1212009 Examinations Developed by:

Facility Written / Operating Test Chief Target Task Description (Reference)

Examiner's Date

• Initials

-180

l. Examination administration date confirmed (C.1.a; C.2.a and b)

Ld

-120

2.

NRC examiners and facility contact assigned (C.l.d; C.2.e)

~~

-120

3.

Facility contact briefed on security and other requirements (C.2.c)

~~

-120

4.

Corporate notification letter sent (C.2.d)

LL

[-90]

[S. Reference material due (C.l.e; C.3.c; Attachment 2)]

LJ.-

{ -7S}

6.

Integrated examination outline(s) due, including Forms ES-201-2, ES-201-3, ES-301-1, ES-301-2, ES-301-S, ES-D-1's, ES-401-1I2, ESAOl-3, and ES-401-4, as.£cf.,

applicable (C.l.e and f; C.3.d)

{ -70}

{7. Examination outline(s) reviewed by NRC and feedback provided to facility

~;;i licensee (C.2.h; C.3.e)}

{ -4S}

8.

Proposed examinations (including written, walk-through JPMs, and scenarios, as applicable), supporting documentation (including Forms ES-301-3, ES-301-4, ES-301-S, ES-301-6, and ES-401-6), and reference materials due (C.l.e, f,g and 4~

h; C.3.d)

-30

9.

Preliminary license applications (NRC Form 398's) due (C.l.l; C.2.g; ES-202)

/tJ-

-14

10. Final license applications due and Form ES-201-4 prepared (C.l.l; C.2.i; ES-202) ~L

-14

11. Examination approved by NRC supervisor for facility licensee review L,/-

(C.2.h; C.3.+/-)

-14

12. Examinations reviewed with facility licensee (C.l.j; C.2.f and h; C.3.g)

~L

-7

13. Written examinations and operating tests approved by NRC supervisor

~c!-

(C.2.i; C.3.h)

-7

14. Final applications reviewed; 1 or 2 (if >10) applications audited to confirm qualifications / eligibility; and examination approval and waiver letters sent 4~

(C.2.i;Attachment 4; ES-202, C.2.e; ES-204)

-7 IS. Proctoring/written exam administration guidelines reviewed with facility licensee L,t.:£ (C.3.k)

-7

16. Approved scenarios, job performance measures, and questions distributed to 4'L NRC examiners (C.3.i)

Target dates are generally based on facility-prepared examinations and are keyed to the examination date identified in the corporate notification letter. They are for planning purposes and may be adjusted on a case-by-case basis in coordination with the facility licensee.

[Applies only] {Does not apply} to examinations prepared by the NRC.

ES-201 Examination Preparation Checklist FOTIn ES-201-1 Facility:

Vogtle 2009-301 Date of Examination:

6/1-12/2009 Examinations Developed by:

Facility Written 1 Operating Test Chief Target Task Description (Reference)

Examiner's Date

• Initials

-180

1.

Examination administration date confirmed (C.1.a; C.2.a and b)

£d

-120

2.

NRC examiners and facility contact assigned (C.1.d; C.2.e)

~~

-120

3.

Facility contact briefed on security and other requirements (C.2.c)

ecf

-120

4.

Corporate notification letter sent (C.2.d)

L£

[-90]

[S. Reference material due (C.1.e; C.3.c; Attachment 2)]

Lc:L

{ -7S}

6.

Integrated examination outline(s) due, including Forms ES-201-2, ES-201-3, ES-301-1, ES-301-2, ES-301-S, ES-D-l's, ES-401-1/2, ES-401-3, and ES-401-4, as ~~

applicable (C.1.e and f; C.3.d)

{ -70}

{7. Examination outline(s) reviewed by NRC and feedback provided to facility

~;;i licensee (C.2.h; C.3.e)}

{ -4S}

8.

Proposed examinations (including written, walk-through JPMs, and scenarios, as applicable), supporting documentation (including Forms ES-301-3, ES-301-4, ES-301-S, ES-301-6, and ES-401-6), and reference materials due (C.1.e, f, g and 4;[

h; C.3.d)

-30

9.

Preliminary license applications (NRC Form 398's) due (C.Ll; C.2.g; ES-202)

/t;L

-14

10. Final license applications due and Form ES-201-4 prepared (C.Ll; C.2.i; ES-202) 1"-

-14

11. Examination approved by NRC supervisor for facility licensee review

~;i (C.2.h; C.3.f)

-14

12. Examinations reviewed with facility licensee (C.1.j; C.2.f and h; C.3.g)

~L

-7

13. Written examinations and operating tests approved by NRC supervisor

~;i (C.2.i; C.3.h)

-7

14. Final applications reviewed; 1 or 2 (if >10) applications audited to confirm qualifications 1 eligibility; and examination approval and waiver letters sent

~J (C.2.i; Attachment 4; ES-202, C.2.e; ES-204)

-7 IS. Proctoring/written exam administration guidelines reviewed with facility licensee ~~

(C.3.k)

-7

16. Approved scenarios, job performance measures, and questions distributed to 4'L NRC examiners (C.3.i)

Target dates are generally based on facility-prepared examinations and are keyed to the examination date identified in the corporate notification letter. They are for planning purposes and may be adjusted on a case-by-case basis in coordination with the facility licensee.

[Applies only] {Does not apply} to examinations prepared by the NRC.

ES-201 Examination Outline Quality Checklist Form ES-201-2 6na.1 Facility: VOGTLE Date of Examination:

6-1-2009 Item

1.

W R

I T

T E

N

2.

S I

M U

L A

T o R

3.

W 1

T

4.

G E

N E

R A

L.

Task Description

a.

Verify that the outline(s) fit(s) the appropriate model, in accordance with ES-401.

b. Assess whether the outline was systematically and randomly prepared in accordance with Section D.1 of ES-401 and whether all KIA categories are appropriately sampled.

c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics.

d. Assess whether the justifications for deselected or rejected KIA statements are appropriate.

a.

Using Form ES-30i -5, verify that the proposed scenario sets cover the required number of normal evolutions, instrument and component failures, technical specifications, and major transients.

b. Assess whether there are enough scenario sets (and spares) to test the projected number and mix of applicants in accordance with the expected crew composition and rotation schedule without compromising exam integrity, and ensure that each applicant can be tested using at least one new or significantly modified scenario, that no scenarios are duplicated from the applicants' audit testes), and that scenarios will not be repeated on subsequent days.

c.

To the extent possible, assess whether the outline(s) conform(s) with the qualitative and quantitative criteria specified on Form ES-301-4 and described in Appendix D.

a. Verify that the systems walk-through outline meets the criteria specified on Form ES-30i-2:

(1) the outline(s) contain(s) the required number of control room and in-plant tasks distributed among the safety functions as specified on the form (2) task repetition from the last two NRC examinations is within the limits specified on the form (3) no tasks are duplicated from the applicants' audit testes)

(4) the number of new or modified tasks meets or exceeds the minimums specified on the form (5) the number of alternate path, low-power, emergency, and RCA tasks meet the criteria on the form.

b. Verify that the administrative outline meets the criteria specified on Form ES-30i-1:

(1) the tasks are distributed among the topics as specified on the form (2) at least one task is new or significantly modified (3) no more than one task is repeated from the last two NRC licensing examinations

c.

Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days.

a.

Assess whether plant-specific priorities (including PRA and IPE insights) are covered in the appropriate exam sections.

b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate.

c. Ensure that KIA importance ratings (except for plant-specific priorities) are at least 2.5.

d. Check for duplication and overlap among exam sections.

e. Check the entire exam for balance of coverage.

f.

Assess whether the exam fits the appropriate job level (RO or SRO).

a. Author

b. Facility Reviewer (*)

c. NRC Chief Examiner (#)

d. NRC Supervisor Note:

1. Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.

• Not applicable for NRC-prepared examination outlines Initials ES-201 Examination Outline Quality Checklist 6na.1 Facility: VOGTLE Date of Examination:

6-1-2009 Item

1.

W R

I T

T E

N

2.

S I

M U

L A

T o

R

3.

W 1

T

4.

G E

N E

R A

L.

Task Description

a.

Verify that the outline(s) fit(s) the appropriate model, in accordance with ES-401.

b.

Assess whether the outline was systematically and randomly prepared in accordance with Section D.1 of ES-401 and whether all KIA categories are appropriately sampled.

c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics.

d. Assess whether the justifications for deselected or rejected KIA statements are appropriate.

a.

Using Form ES-301-5, verify that the proposed scenario sets cover the required number of normal evolutions, instrument and component failures, technical specifications, and major transients.

b.

Assess whether there are enough scenario sets (and spares) to test the projected number and mix of applicants in accordance with the expected crew composition and rotation schedule without compromising exam integrity, and ensure that each applicant can be tested using at least one new or significantly modified scenario, that no scenarios are duplicated from the applicants' audit testes), and that scenarios will not be repeated on subsequent days.

c.

To the extent possible, assess whether the outline(s) conform(s) with the qualitative and quantitative criteria specified on Form ES-301-4 and described in Appendix D.

a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2:

(1) the outline(s) contain(s) the required number of control room and in-plant tasks distributed among the safety functions as specified on the form (2) task repetition from the last two NRC examinations is within the limits specified on the form (3) no tasks are duplicated from the applicants' audit testes)

(4) the number of new or modified tasks meets or exceeds the minimums specified on the form (5) the number of alternate path, low-power, emergency, and RCA tasks meet the criteria on the form.

b. Verify that the administrative outline meets the criteria specified on Form ES-301-1:

(1) the tasks are distributed among the topics as specified on the form (2) at least one task is new or significantly modified (3) no more than one task is repeated from the last two NRC licensing examinations

c.

Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days.

a. Assess whether plant-specific priorities (including PRA and IPE insights) are covered in the appropriate exam sections.

b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate.

c. Ensure that KIA importance ratings (except for plant-specific priorities) are at least 2.5.

d. Check for duplication and overlap among exam sections.

e. Check the entire exam for balance of coverage.

f.

Assess whether the exam fits the appropriate job level (RO or SRO).

a. Author

b. Facility Reviewer (*)

c. NRC Chief Examiner (#)

d. NRC Supervisor Form ES-201-2 Initials a

b*

c#

I I;~~ rAJ,1;1-

.Date

,I:; -e. iJ -0'1

&;-20-C>~

.$I.2@?

iifltk~1 Note:

1. Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.

• Not applicable for NRC-prepared examination outlines

ES*201 Examination Outline Quality Checklist Facility:

Vogtle Date of Examination:

06-26-2009 Item

1.

W R

I T

T E

N

2.

S I

M U

L A

T o R

3.

W 1

T

4.

G E

N E

R A

L Task Description

a.

Verify that the outline(s) fit(s) the appropriate model, in accordance with ES-401.

b.

Assess whether the outline was systematically and randomly prepared in accordance with Section D.1 of ES-401 and whether all KIA categories are appropriately sampled.

c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics.

d. Assess whether the justifications for deselected or rejected KJA statements are appropriate.

a.

Using Form ES-301-5, verify that the proposed scenario sets cover the required number of normal evolutions, instrument and component failures, technical specifications, and major transients.

b. Assess whether there are enough scenario sets (and spares) to test the projected number and mix of applicants in accordance with the expected crew composition and rotation schedule without compromising exam integrity, and ensure that each applicant can be tested using at least one new or significantly modified scenario, that no scenarios are duplicated from the applicants' audit test(s), and that scenarios will not be repeated on subsequent days.

c.

To the extent possible, assess whether the outline(s) conform(s) with the qualitative and quantitative criteria specified on Form ES-301-4 and described in Appendix D.

a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2:

(1) the outline(s) contain(s) the required number of control room and in-plant tasks distributed among the safety functions as specified on the form (2) task repetition from the last two NRC examinations is within the limits specified on the form (3) no tasks are duplicated from the applicants' audit test(s)

(4) the number of new or modified tasks meets or exceeds the minimums specified on the form (5) the number of alternate path, low-power, emergency, and RCA tasks meet the criteria on the form.

b. Verify that the administrative outline meets the criteria specified on Form ES-301-1:

(1) the tasks are distributed among the topics as specified on the form (2) at least one task is new or significantly modified (3) no more than one task is repeated from the last two NRC licensing examinations

c.

Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days.

a. Assess whether plant-specific priorities (including PRA and IPE insights) are covered in the appropriate exam sections.

b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate.

c. Ensure that KIA importance ratings (except for plant-specific priorities) are at least 2.5.

d. Check for duplication and overlap among exam sections.

e. Check the entire exam for balance of coverage.

f.

Assess whether the exam fits the appropriate job level (RO or SRO).

a. Author

-lrintAl4 rr:'IS":J:atureL T. N. Thompsonl ~ ~ A V

b. Facility Reviewer (*)

c. NRC Chief Examiner (#)

d. NRC Supervisor Form ES*201*2 Initials a

b*

c#

I W/A,,'"

Note:

1. Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.

• Not applicable for NRC-prepared examination outlines ES-201 Examination Outline Quality Checklist Form ES-201-2 Fill ;a. \\

Facility:

Vogtle Date of Examination:

06-26-2009 Initials Item Task Description a

b*

c#

1.

a.

Verify that the outline(s) fit(s) the appropriate model, in accordance with ES-401.

AlIA rJ/~ ~t w

R

b.

Assess whether the outline was systematically and randomly prepared in accordance with N/~ rJ/A li:J.

I Section 0.1 of ES-401 and whether all KIA categories are appropriately sampled.

T NIIt NiA l-lt T

c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics.

E N

d. Assess whether the justifications for deselected or rejected KIA statements are appropriate.

,~ A1 Lt.

v

2.

a.

Using Form ES-301-5, verify that the proposed scenario sets cover the required number

'" JA "/A of normal evolutions, instrument and component failures, technical specifications, NIA-S and major transients.

I M

b.

Assess whether there are enough scenario sets (and spares) to test the projected number U

and mix of applicants in accordance with the expected crew composition and rotation schedule AIM Ii/A L

without compromising exam integrity, and ensure that each applicant can be tested using A

at least one new or significantly modified scenario, that no scenarios are duplicated T

from the applicants' audit test(s), and that scenarios will not be repeated on subsequent days.

0 To the extent possible, assess whether the outline(s) conform(s) with the qualitative AI/It fll" R

c.

and quantitative criteria specified on Form ES-301-4 and described in Appendix D.

3.

a. Verify that the systems walk-through outline meets the criteria specified on Form ES-301-2:

(1 )

the outline(s) contain(s) the required number of control room and in-plant tasks

(

W distributed among the safety functions as specified on the form IJ/A I

(2) task repetition from the last two NRC examinations is within the limits specified on the form wl~

T (3) no tasks are duplicated from the applicants' audit test(s)

(4) the number of new or modified tasks meets or exceeds the minimums specified on the form (5) the number of altemate path, low-power, emergency, and RCA tasks meet the criteria on the form.

b. Verify that the administrative outline meets the criteria specified on Form ES-301-1:

IV/A #/"

(1 )

the tasks are distributed among the topics as specified on the form (2) at least one task is new or significantly modified (3) no more than one task is repeated from the last two NRC licensing examinations

c.

Determine if there are enough different outlines to test the projected number and mix I'IJA-IV/A of applicants and ensure that no items are duplicated on subsequent days.

'It

4.

a.

Assess whether plant-specific priorities (including PRA and IPE insights) are covered

<11 I1j ~.t in the appropriate exam sections.

G

<~' III.eX E

b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate.

N Ensure that KIA importance ratings (except for plant-specific priorities) are at least 2.5.

<~

~; :A.£::1' E

c.

R

d. Check for duplication and overlap among exam sections.

(~ ~

f{/ ~.f.

A AA 41,l;t.

L

e. Check the entire exam for balance of coverage.

,,<I(;!Y-

f.

Assess whether the exam fits the appropriate job level (RO or SRO).

W ~.

I~j.

v

~

'!T:'~~aturek

{p Date

a. Author T. N. Thompsonl

~...(Jyv "1-¥)

b. Facility Reviewer (*)

D. Scukanecl ~

J~~'L

{/

6~/1-

c. NRC Chief Examiner (#)

JidlAJ'" Le 2

Jr'. /~..1.>.. ~ /~ L~

,,-gS/olj

.uLI,-.~..... "'C. W(O~fAI /... ~ 1:

I 7

d. NRC Supervisor d"tflb'

'(!!I1.u,,'-'!TJ1J.Jy Note:

1. Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.

• Not applicable for NRC-prepared examination outlines

r ES-201 Examination Security Agreement Form ES-201-3

1.

Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of 6/1/2009 as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting ~s a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in.cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2.

Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of 6/1/2009. From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME JOB TITLE / RESPONSIBILITY

1. Thad N. Thompson Operations Instructor

-:J.-.~ '7

. r-._ f,.."._.. v,.......

..,..-.. -~-. --.

', "".-.~ ~

()..."..,/-[ ". ~.""" ~~ ~

0

_A n

j

, #~.

ES-201, Page 26 of 27 ES-201 Examination Security Agreement Form ES-201-3

1.

Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of 6/1/2009 as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations andlor an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2.

Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information conceming the NRC licensing examinations administered during the week(s) of 6/1/2009. From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME JOB TITLE 1 RESPONSIBILITY

1. Thad N. Thompson Operations Instructor ES-201, Page 26 of 27

ES*201 Examination Security Agreement Form ES*201*3

1.

Pre*Examination I acknowledge that I have acquired specialized knowledge' about the NRC licensing examinations scheduled for the week(s) of 6 It / to Qcr as of the date of my signature. I agree that I Will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback), Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2.

Post-Examination To the best of my know!~dge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of 6!1/J,o ~'f. From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME JOB TITLE / RESPONSIBILITY SIGNATURE (1)

DATE

$IGNATU2.E (2)

DATE NOTE

---

,.-------- ~-~----------------------------..,}~'.,.'---------

---

~---..::...---------

ES-201, P 27 of 28 Examination Security Agreement Form ES-201-3

1.

Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of 6/1'; :z~",o<l as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. J understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2.

Post-Examination To the best of my know!*;d~e,l,did not divulge to any unauthori~ed p~rsons a.ny information co~cerning the ~RC licensi~g ~xamina~i~ns a?minist.ered during the week(s) of 6/1/1. 0

~'f. From the date that I entered Into this secunty agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME JOB TITLE / RESPONSIBILITY SIGNATURE (1)

SIGNATU~E (2)

DATE NOTE

---

----------------------------------~+---------

ES-201, P 27 of 28

ES-201 Examination Security Agreement Form ES-201-3

1.

Pre-Examination I acknowledge that I have acquired specialized knowledgeabout the NRC licensing examinations scheduled for the week(s) of 6-/-0 T as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable ifthe individual does not select the training content or provide direct or indirect feedback). Furthermore, lam aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2.

Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of6"/- 0 1. From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME SIGNATURE (1)

DATE DATE NOTE

1. fP3L ~p7:

6-a~",_

2. 2~,i~~ -xiUty

~~

3'1::1L~~ J~f\\.{

,,-;1Q -Oq

~:%h* t/iiti£

~~~t;--g(

6. __

--=c.-___

I I,

7.

8. ________________________ - _____________

9.

10.

11.

12.

13.

14. __________________ _

15.

NOTES:

ES-201, P 27 of 28 ES-201 Examination Security Agreement Form ES-201-3

1.

Pre-Examination I acknowledge that I have acquired specialized knowledgeabout the NRC licensing examinations scheduled for the week(s) of 6-/-0 '1 as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if the individual does not select the training content or provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2.

Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of6'"1- 0 1. From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME SIGNATURE (1)

DATE DATE NOTE

~: ~~

~~~~~~~~~~

__ -ri~~~~~~~

__ ~~~~_6:;{gr

~e1.~

ri,~:g(

6. ___ ~~

_______________________ -'-_______ --------1--/----- ___

7. ____________________________________________________ _

8. _________________________________________ --_________ _

9. ____________________________________________________ _

10.. ____________________________________________________ _

11.. ____________________________________________________ _

12.. ____________________________________________________ _

13.. ____________________________________________________ _

14.. ____________________________________________________ _

15., _________________________________________________ _

NOTES:

ES-201, P 27 of 28

ES-301-3 Operating Test Quality Checklist P,,,, a.. I Facility: Vogtle Date of Examination: 6-1-2009 Operating Test Number: 2009-301

1. General Criteria

a.

The operating test conforms with the previously approved outline; changes are consistent with sampling requirements (e.g., 10 CFR 55.45, operational importance, safety function distribution).

b.

There is no day-to-day repetition between this and other operating tests to be administered during this examination.

c.

The operating test shall not duplicate items from the applicants' audit test(s). (see Section D.1.a.)

d.

Overlap with the written examination and between different parts of the operating test is within acceptable limits.

e.

It appears that the operating test will differentiate between competent and less-than-competent applicants at the designated license level.

2. Walk-Through Criteria

a.

Each JPM includes the following, as applicable:

initial conditions initiating cues references and tools, including associated procedures reasonable and validated time limits (average time allowed for completion) and specific designation if deemed to be time-critical by the facility licensee operationally important specific performance criteria that include:

detailed expected actions with exact criteria and nomenclature system response and other examiner cues statements describing important observations to be made by the applicant criteria for successful completion of the task identification of critical steps and their associated performance standards restrictions on the sequence of steps, if applicable

b.

Ensure that any changes from the previously approved systems and administrative walk-through outlines (Forms ES-301-1 and 2) have not caused the test to deviate from any of the acceptance criteria (e.g., item distribution, bank use, repetition from the last 2 NRC examinations) specified on those forms and Form ES-201-2.

3. Simulator Criteria The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES-301-4 and a co is attached.

a.

Author

b.

Facility Reviewer(*)

c.

NRC Chief Examiner (#)

d.

NRC Supervisor NOTE:

The facility signature is not applicable for NRC-developed tests.

Inde en dent NRC reviewer initial items in Column "c"; chief examiner concurrence re uired.

Initials Date

(

ES-301-3 Operating Test Quality Checklist P,4 s.. I Facility: Vogtle Date of Examination: 6-1-2009 Operating Test Number: 2009-301

1. General Criteria

a.

The operating test conforms with the previously approved outline; changes are consistent with sampling requirements (e.g., 10 CFR 55.45, operational importance, safety function distribution).

b.

There is no day-to-day repetition between this and other operating tests to be administered during this examination.

c.

The operating test shall not duplicate items from the applicants' audit test(s). (see Section D.1.a.)

d.

Overlap with the written examination and between different parts of the operating test is within acceptable limits.

e.

It appears that the operating test will differentiate between competent and less-than-competent applicants at the designated license level.

a.

b.

2. Walk-Through Criteria Each JPM includes the following, as applicable:

initial conditions initiating cues references and tools, including associated procedures reasonable and validated time limits (average time allowed for completion) and specific designation if deemed to be time-critical by the facility licensee operationally important specific performance criteria that include:

detailed expected actions with exact criteria and nomenclature system response and other examiner cues statements describing important observations to be made by the applicant criteria for successful com pletion of the task identification of critical steps and their associated performance standards restrictions on the sequence of steps, if applicable Ensure that any changes from the previously approved systems and administrative walk-through outlines (Forms ES-301-1 and 2) have not caused the test to deviate from any of the acceptance criteria (e.g., item distribution, bank use, repetition from the last 2 NRC examinations) specified on those forms and Form ES-201-2.

3. Simulator Criteria The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES-301-4 and a copy is attached.

a.

Author

b.

Facility Reviewer(*)

..!D~a![n~ie~I-,S~c<bul.!SkE.afrne~c;"'/L.c.~~~~~~.dJ!.~~~~::'::::::::=-___ _

c.

NRC Chief Examiner (#)

d.

NRC Supervisor NOTE:

The facility signature is not applicable for NRC-developed tests.

Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.

Initials Date 5~ "e,.()" (}j 5' -2<J-o tl 5iJ70,pJ)7 I ~7

~s1Z1-;;'7

ES-301-4 Simulator Scenario Quality Checklist r-J N "" LJ Facilty: Vogtle Date of Exam: 6-1-2009 Scenario Numbers: 1 /2/3 O(2erating Test No.: 2009-301 QUALITATIVE ATTRIBUTES Initials a

b*

c#

1.

The initial conditions are realistic, in that some equipment and/or instrumentation may be out

~

pf tJ of service, but it does not cue the operators into expected events.

2.

The scenarios consist mostly of related events.

~~ ~ III

3.

Each event description consists of the point in the scenario when it is to be initiated the malfunction(s) that are entered to initiate the event

(~ pi the symptoms/cues that will be visible to the crew Jt the expected operator actions (by shift position) the event termination point (if applicable) r~~ I~

4.

No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario

"\\ ~"

I jj without a credible preceding incident such as a seismic event.

5.

The events are valid with regard to physics and thermodynamics.

~~ ~

IJ;J--

6.

Sequencing and timing of events is reasonable, and allows the examination team to obtain

~~ pJ d complete evaluation results commensurate with the scenario objectives.

7.

If time compression techniques are used, the scenario summary clearly so indicates.

W~:

Operators have sufficient time to carry out expected activities without undue time constraints.

",tJ Cues are given.

<~r\\

'I

8.

The simulator modeling is not altered.

~~

I

.~ l,j

9.

The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator

\\ ' 4 performance deficiencies or deviations from the referenced plant have been evaluated

~~

);j.

to ensure that functional fidelity is maintained while running the planned scenarios.

~

M

10.

Every operator will be evaluated using at least one new or significantly modified scenario.

--:~

,tJ All other scenarios have been altered in accordance with Section 0.5 of ES-301.

11.

All individual operator competencies can be evaluated, as verified using Form ES-301-6

,#~ t;! 1i (submit the form along with the simulator scenarios).

12.

Each applicant will be significantly involved in the minimum number of transients and events

~~; '((f specified on Form ES-301-5 (submit the form with the simulator scenarios).

\\

\\

,\\

• jJ

13.

The level of difficulty is appropriate to support licensing decisions for each crew position.

~~

'At III Target Quantitative Attributes (Per Scenario; See Section D.S.d)

Actual Attributes

1.

Total malfunctions (5-8) 10/9/8

-ti

2.

Malfunctions after EOP entry (1-2) 4/5/3

.4:1

3.

Abnormal events (2-4) 5/3/3 Ll;J-

4.

Major transients (1-2) 1 /1 /2 I%J

5.

EOPs enteredlrequiring substantive actions (1-2) 2/3/2

~;l

6.

EOP contingencies requiring substantive actions (0-2) 0/2/0

~1.

7.

Critical tasks (2-3) 3/6/4

~

(

(

ES-301-4 Simulator Scenario Quality Checklist r-I IV "" L.!

Facilty: Vogtle Date of Exam: 6-1-2009 Scenario Numbers: 1 /2/3 ~T

"""" 301 QUALITATIVE ATTRIBUTES a

1.

The initial conditions are realistic, in that some equipment and/or instrumentation may be out

.~.

of service, but it does not cue the operators into expected events.

2.

The scenarios consist mostly of related events.

~ ~.

3.

Each event description consists of the point in the scenario when it is to be initiated the malfunction(s) that are entered to initiate the event the symptoms/cues that will be visible to the crew the expected operator actions (by shift position) the event termination point (if applicable)

4.

No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario

~ ~

without a credible preceding incident such as a seismic event.

5.

The events are valid with regard to physics and thermodynamics.

~

6.

Sequencing and timing of events is reasonable, and allows the examination team to obtain

~.~

complete evaluation results commensurate with the scenario objectives.

7.

If time compression techniques are used, the scenario summary clearly so indicates.

...\\'

Operators have sufficient time to carry out expected activities without undue time constraints.

~~\\

Cues are given.

8.

The simulator modeling is not altered.

tJ¥~

9.

The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator

\\,

performance deficiencies or deviations from the referenced plant have been evaluated

\\~~

to ensure that functional fidelity is maintained while running the planned scenarios.

10.

Every operator will be evaluated using at least one new or significantly modified scenario.

~~'

All other scenarios have been altered in accordance with Section 0.5 of ES-301.

11.

All individual operator competencies can be evaluated, as verified using Form ES-301-6

,~~

(submit the form along with the simulator scenarios).

12.

Each applicant will be significantly involved in the minimum number of transients and events r~K specified on Form ES-301-5 (submit the form with the simulator scenarios).

~

\\

\\

,\\

13.

The level of difficulty is appropriate to support licensing decisions for each crew position.

~~:

Target Quantitative Attributes (Per Scenario; See Section D.S.d)

Actual Attributes

1.

Total malfunctions (5-8) 10/9/8

2.

Malfunctions after EOP entry (1-2) 4/5/3

3.

Abnormal events (2-4) 5/3/3

4.

Major transients (1-2) 1 / 1 /2

5.

EOPs entered/requiring substantive actions (1-2) 2/3/2

6.

EOP contingencies requiring substantive actions (0-2) 0/2/0

7.

Critical tasks (2-3) 3/6/4 Initials b'

c#

~ tj

~ lJ d J/

~ jj

~

1,J--

pJ d d ~J A

r~ llt 4 );;f M It)

~

1.1

'?(f

• );j

'At III

-ti 11-2::£

£J

,t;f

~1.

A:J

ES-301-4 Simulator Scenario Quality Checklist

~/#"4.

Facilty: Vogtle Date of Exam: 6-1-2009 Scenario Numbers: 4 / /

O[!erating Test No.: 2009-301 QUALITATIVE ATTRIBUTES Initials a

b*

c#

1.

The initial conditions are realistic, in that some equipment and/or instrumentation may be out

~

i;;J l/qj of service, but it does not cue the operators into expected events.

"Ii~,-

2.

The scenarios consist mostly of related events.

~{ £fJ It

3.

Each event description consists of the point in the scenario when it is to be initiated the malfunction( s) that are entered to initiate the event the symptoms/cues that will be visible to the crew

~

'~

the expected operator actions (by shift position)

'\\

Jj the event termination point (if applicable)

4.

No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario

~:

~~ IJ) without a credible preceding incident such as a seismic event.

<,\\\\1'

5.

The events are valid with regard to physics and thermodynamics.

~01\\' Pf IJf

6.

Sequencing and timing of events is reasonable, and allows the examination team to obtain

~~ tl1 jj complete evaluation results commensurate with the scenario objectives.

~ \\~

t

7.

If time compression techniques are used, the scenario summary clearly so indicates.

~ q Operators have sufficient time to carry out expected activities without undue time constraints.

tJ Cues are given.

<,/,

8.

The simulator modeling is not altered.

~:@W~I I4r j;J

9.

The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator

~\\ t1I performance deficiencies or deviations from the referenced plant have been evaluated I\\~

1.j to ensure that functional fidelity is maintained while running the planned scenarios.

10.

Every operator will be evaluated using at least one new or significantly modified scenario.

~~.fJf

(\\'

);J All other scenarios have been altered in accordance with Section 0.5 of ES-301.

11.

All individual operator competencies can be evaluated, as verified using Form ES-301-6

~~~ 41 11; (submit the form along with the simulator scenarios).

W'~ QI

12.

Each applicant will be significantly involved in the minimum number of transients and events

,I' I~J specified on Form ES-301-5 (submit the form with the simulator scenarios).

~~~\\

13.

The level of difficulty is appropriate to support licensing decisions for each crew position.

<~

I~ >>

Target Quantitative Attributes (Per Scenario; See Section D.S.d)

Actual Attributes

i.

Total malfunctions (5-8) 9/ /

11

2.

Malfunctions after EOP entry (1-2) 5/ /

.II

3.

Abnormal events (2-4) 4/ /

~L

4.

Major transients (1-2) 2/ /

-l.:i

5.

EOPs entered/requiring substantive actions (1-2) 2/ /

,l;f-.

6.

EOP contingencies requiring substantive actions (0-2) 0/ /

.cJ

7.

Critical tasks (2-3) 2/ /

~;/

ES-301-4 Simulator Scenario Quality Checklist,::'/111""

II

~ L

.L....

1

/ /

0

.~. t No.: 2009-301 QUALITATIVE ATTRIBUTES Initials a

b*

c#

1.

The initial conditions are realistic, in that some equipment and/or instrumentation may be out

1. 'jJj /4J of service, but it does not cue the operators into expected events.

'{tl\\ (-

2.

The scenarios consist mostly of related events.

~~ I{f 1.1

3.

Each event description consists of the point in the scenario when it is to be initiated the malfunction(s) that are entered to initiate the event the symptoms/cues that will be visible to the crew it

'~

the expected operator actions (by shift position)

Jj the event termination point (if applicable)

4.

No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario

~:

~~ );j without a credible preceding incident such as a seismic event.

,\\\\1\\

5.

The events are valid with regard to physics and thermodynamics.

~~\\' pr j,J

6.

Sequencing and timing of events is reasonable, and allows the examination team to obtain w-~ a iJj complete evaluation results commensurate with the scenario objectives.

~ \\)1 I~

7.

If time compression techniques are used, the scenario summary clearly so indicates.

~

HI Operators have sufficient time to carry out expected activities without undue time constraints.

11)

Cues are given.

8.

The simulator modeling is not altered.

~~(

~

,.. Illi );J

9.

The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator ef' ¢(/

performance deficiencies or deviations from the referenced plant have been evaluated I\\~

1,,1 to ensure that functional fidelity is maintained while running the planned scenarios.

10.

Every operator will be evaluated using at least one new or significantly modified scenario.

~

It(!

~ "

('\\

j;J All other scenarios have been altered in accordance with Section D.5 of ES-301.

11.

All individual operator competencies can be evaluated, as verified using Form ES-301-6

<~~ 4! lJ (submit the form along with the simulator scenarios).

?f~ QJ

12.

Each applicant will be significantly involved in the minimum number of transients and events

/1' lJ specified on Form ES-301-5 (submit the form with the simulator scenarios).

~ '<~\\

13.

The level of difficulty is appropriate to support licensing decisions for each crew position.

~

~i I~ >>

Target Quantitative Attributes (Per Scenario; See Section D.S.d)

Actual Attributes

1.

Total malfunctions (5-8) 9/ /

.,It

2.

Malfunctions after EOP entry (1-2) 5/ /

~i

3.

Abnormal events (2-4) 4/ /

./,1

4.

Major transients (1-2) 2/ /

L:i

5.

EOPs entered/requiring substantive actions (1-2) 2/ /

Lbt

6.

EOP contingencies requiring substantive actions (0-2) 0/ /

l.ej

7.

Critical tasks (2-3) 2/ /

.1;1

/ (

05/28/2009 07:3l FAX 706 826 3953 VEGP TRAINING CENTER 141 003

.11

/'

ES-a01-4 Simulator Scenario Quality Checklist F / ~.4 L.J.

===----~===-===================------===========~--=======-=9

~~!!l;.!S:.;;V.;:.09!1,;tI;;;;e;,...,J!D~a~te1:i0~f:SE~x~am~;6-~1.;;;2,;OO:;;9;......3S~ce~na~rI~o~N~u~m!!!!be~rs~:==4~/~/=~O~pe~ra~ti~*n~g!=~~8!;1lSlt;;t~N~o*iio:~20~O::;9~-3~0~1r=======o====l1 QUALITATIVE ATTRIBUTES Initials a

b*

0#

~'--------------------------------------------------~~~-r~~1

1.

The initial conditions are realistic, in that some equipment and/or Instrumentation may be out

~~" r /JJ. ~~.t of service, but it does not OI.Ie the operators into expeeted events.

,,0:.

I YO r-------------------------~--------~~--------------------------~~~~~~~~I

~.

__ T_h_e_s_~

na_ri_o_s_eo_n_s_~_t_m_o_s~~y_~

~_I_a~

e_ve_n_ts_. __________________________________ 4~~~~-~4'~~~~~~~~.J

3.

Each event description consists of r--'

4.

~.

5. -.

6. -.

7.

8.

f----

9.

10.

11.

r-

12.

r-

13.

i---

1. -

2.

3.

~

4.

I--

5.

~.

6.

1--.

the point in the scenario when It Is to be Initiated the malfunetion(s) that are entered to initiate the event the symptoms/cues that will be visible to the crew Ifle expected operator actions (by shift position) i" the event termination paint (if appliclilble)

No mOre than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario without a credible preceding Incident such as a selsmie event The events are valid with regard to physics and thermodynamics.

Sequenclng and timing of events Is reasonable. and allows Ifle examination teem to obtain IJOITlplete evaluatioll resull$ oommensurate with the scenario objectives.

If time compression techniques are used, the scenario summary dearly so Indicates.

Operators have sufficient time to carry out expected activities without undue time constralnt:s.

Cues are given.

The simulator modeling Is not altered.

The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator performance deficiencies or deviations from the referenced plant have beell evaluated to ensure that functional fidelity is maintained while running the planned scenarios.

Every operator will be evaluated using at least one new or slgnlflcandy modified SOElnario.

All other scenarios have been altered in accordance with Section 0.6 of ES-.301.

All individual opera~or competencies can be evaluated, as verified using Fom'l 5S-301-6 (submit the folTT1 IiIlollg with the simuletor scenariOS).

Each applicant will be significantly involved in the minimum number of transients and events sp$CIfied on Fol'I'I'\\ ES-301-5 (submit the form with the simulator scenarios).

The level 01' difficulty is appropriate to support lioensing decisions for each crew position.

Target Quantitative Attributes (Per Scenario; See S8ctlon D.5.d)

Actual Attributes Total malfunctions (5-8) g/ /

Malfunctions after EOP entry (1-2) 6/1 Abnormal events (2-4) 41 I Major transients (1-2) 2/1 EOPs entered/requiring substantive actions (1-2) 2/ /

eop contingencies I'EIql,liring $ubstantive actions (0-2) 011 7,

Critical tasks (2~3) 2/ /

=.

~

I_/!Ii. l£tX

~~ l/{f ~;

~~ l,q ~~

4 ~~ Ig.b 1 i:trtt '/tF ~

~ ~~ 9

~;.

~

'(If/. Ib 05/28/2009 07:3L FAX 706 826 3953 VEGP TRAINING CENTER

""j..'

ES-:101-4 Simulator Scenario Quality Checklist.F /HALl I.E!£\\~ Vogtle Date of Exam: 6-1-2009 Scenario Numbers: 4 I I Operating Test No.: 2009-301

.:1 I'

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

1.

2.

3.

4.

5.

QUALITATIVE ATTRIBUTES The initial conditions are realistic, in that some equipment and/or Instrumentation may be out of seNice, bl,lt il does not we the operators into expected events.

Ti'le scena~ios consist mostly of related events.

Each event description consists of the point in the scenario when It Is to belnltlaled ti'le malfunction(s) that a~ entered to initiate the event the symptoms/cues that will be visible to the crew the expected operator actions (by Si'lift position) the event te~mlnatlon point <if applicOible)

No mOre than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario without a credible preceding Incident suci'l as a selsmie event The events are valid with regard to physics and thennodynamics.

Sequenclng and timing of events Is reasonable. and allows the examination teOim to obtain complete evaluation results commensurate with the scenario objectives.

If time compression techniques are used, the scenario summary clearly so Indicates.

Operators have sufficient time to carry out expected activities without undue time constraints.

Cues are given.

The simulator modeling Is not altered.

The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator performance deficiencies or devialions ft'om !;he referenced planl i'lave I;Jeen evaluated to ensure that functional fidelity is maintained while running the planned scenarios.

Every operator wlll be evaluated using at least one new or significantly modified scenario.

All other scenarios have been altered in accordance with Section D.5 of ES-301.

All individual operator competencies can be evaluated, as verified using ~on'1'\\ eS-301-6 (submit the funn 0I10n9 with ti'le simulOitor scenariOS).

Each applicant will be significantly involved in the minimum number of transients and events spacltied on Fonn ES-301-5 (submit !;he form with the simulator scenarios).

The level of diffioulty is appropriate to support lioensing decisions for each crew position.

Target Quantitative Attributes (Per SCflnarlo; See Section D.5.d)

Actual Attributes Total malfunctions (5-8) 91 /

Malfunctions afief EOP entry (1-2) 51 J Abnonnal events (2-4) 41 /

Major transients (1-2) 2/ I EOPs enteredJrequiring substantive aotions (1-2) 2 J I EOP contingencies requiring substantive actions (0-2) o I I Critical tasks (2-3) 21 /

a

~

~

(~

4~~

, ~~

.oC ~~

~

141003 Initials b*

c#

II¥. I~;t:

1A¥ ~/

1#1' ~..t Iq h

'AY..H

,q

~,;t.

q. g.

05/28/2009 07:3l FAX 706 826 3953 VEGP TRAINING CENTER III 002

~

Simulator Scenario Quality Checklist p" N4 4

~==========================----~==========--------~

~

ill VogUe Data of Exam: 6-1-2009 Scenario Numbers: 1 12/3 Operating Test No.: 2009-301 QUALITATIVE ATTRIBUTES Initiais a

b" 0#-

---

~------------------------------~r_~~~~~~I

1.

The Inllial c:ondltlons are realistic:, in that some equiprnent end/or instrumentation may be out

~ ~.i:-t of service, but it does not cue the operators into expected events.

~

~------~------------~------~------------------------~~~~~~I

2.

The seenariQ$ consl$t mostly of related events.

~~. IJf.J1

~------------------~~--------------------------------------------~~_4~~+~~,~,_~1

3.

Each event description consists of

4.

r---

5..

r---

6. -,

7.

B. -,

9.

r----

10.

r----

11.

12.

~-

13.

=,

the point in the scenario when it is to be initiated 6

the malfunction(l;) that <Ire entered to initililte the event the symptoms/cues that will be visible to the crew the expected operator actions (by shift posl1:lon) the event termination point (if applicable)

No more than one non*mechanlstlc failure (e.g.* pipe break) Is incorporated InlO the sc:enar'lo without iii credible preceding incident such as iii 5eismic event.

The events are valid with regard to physics and thermodynamics.

Sequencing and timing of events i5 reasonable, and allows the examination team to obtain complete evaluation results ccmmensurate wIth the scenario objectives.

If tim e compression techniques are used, the scenario summary clearly so Indicates.

Operators have sufficient time to carry out expected activities without undue time ccnstraints.

Cues are given.

The simulator modeling is not altered.

The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator performance deficiencies !;Ir deviations from the referenced plant have been evah.Jated to ensure that functional fidelity Is maintained while running the planned scenarios.

Every operator will be evaluated I,lslng at least one new or significantly modified scenario.

All other scenarios have been altered in accordance with Section D.5 of eS*301.

All individual operator c:ompetencles can be evaluated. as verified using Form ES*301*6 (submit the form along with.the simulator scenarios).

Each applicant will be significantly Involved In the minimum number of transients and events specified on Form ES*301-5 (submit the form with the 5irnl,llOltor 5Ceflarl05).

The level of dlfflClJlty Is appropriate to support licensing decisions for each crew position.

Target Quantitative Attribl,lte$ (Per Scenario: See Saetion D.S.d)

---

~--~------------~--------~----------~----+_------------_4~~~~~--~1 Actual Attributes

~

~

AI' 11/

1.

Total malfunctions (5-8) 10/9/8 4/5/3

~ra f)/ 0 5/3/3

~ ~tii Id ~~

1/1/2

~

,pp M V"........

2/3/2

,~ tJ/ ~

0/2/0

~

~. "Ifj 3/6/4

~

It.~ d.

t'i1i!Jl(

2.

Malfunctions after eop entry (1-2)

~----------------~~~------------------~--+-------------~~~~~~~I

~~A~bn~o~rm~al~e~ve~n~ts~(~2_-4~)~ ________________________ --____ -+ ________ ~

____ ~~~~~~~

4.

Major transients (1-2)

~

5.

eops entered/requiring substantive actions (1-2)

~----------~~~----------~~--~--------~+-~~--~----~~~S6~~~

6.

EOP contingencies requiring substantive actions (0-2)

---

~--~~~~----------~--~------------+-------------~~~~~~~~I

7.

CritiC<lI task& (2-3)

=

05/28/2009 07:3L FAX 706 826 3953 VEGP TRAINING CENTER 141002 ES-::01-4 Simulator Scenario Quality Checklist Fnv44 r==="- '

. ~

ffi Vogtle Date of Exam: 6" 1 "2009 Scenario Number!:!: 1 12/3 O~ratlng Test No.: 2Q09-301 QUAl.ITATIVE AlTRIElUTES Initials a

b" C#

~

IJf

1.

The In!tlal condilions are realistic. in that some equipment andlor in5trumentation may be out

~-t of service. but it dOe:! not cue the operators into expected events.

r--

2.

The sceMrios consist mostly of related events,

~~. If ~:l r--

fi7

3.

Each event description consists of the point in the scenario when it is to be initiated the malfunction(s) that Olre entered to initiate the event d

the symptoms/cues that wI!! be visible to the crew the expected operator actlon$ (by $hlft posl~on) tJ-.

the event termination point (if applicable) r.~ IP

4.

No more than one non-mechanistic failure (e.g.* pipe break) Is incorporated Into the scenado

~

~.:f without a credible preceding incident such as a 5eisrnic event.

r--

A 5..

The events are valid with regard to physics and thermodynamics.

~~

.~ L:f-I---

6.

Sequencing and timing of events is reasonable, and allows the examination team to obtain

~~,oJ It.J complete evaluation results commensurate with the scenario objectives.

-7.

If time compression techniques are used, the scenario summary clearly so Indicates.

,\\.' at Operators have sufficient time to carry out expected activities without undue time constraints.

~~ It

~

Cues are given.

A B.

The simulator modeling i5 not altered,

~~

.~ L.i L

9.

Thl'! scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator

,@* Ii performance deficiencies or deviations from the referaneed plant ha.ve been evaluated

~J-to enSure that functional fidelity Is maintained while running the planned scenarios.

~~ fJI'

10.

Every operator will be evaluated using at least one new or significantly modified scenario,

~;t All other scenarios have been altered in accordance with Section 0.5 of eS*301.

~

rII,

11.

All individual operator competencies can b@ evaluated. as verified using Form ES"301*6

~,;

~cf (submit the form along with.the simulator scenarios).

r----'

w: 'klf

12.

Each applicant will be significantly involved In the minimum number of transients and events

~

lt~

specified on Form ES*301-5 (submit the form with the simulOltor 5cenarios).

r----

~~

'A'.! k,),,1

13.

The level of difficulty Is appropriate to support licensing decisions for each crew position.

~.

Target Quantitative Attribl,ltes (Pet Seen;'lfio; See Seotion D.5.d)

Actual Attributes

1.

Total malfunctions (5-8) 10/9/8

~

R lu

~

/lI' lQ

2.

Malfunctions after eop entry (1-2) 4/513

~,

-

; ~ l.d

3.

Abnormal events (2-4) 5/3/3 pI.J

/ -,

,~1 I2fI M

4.

Major transients (1-2) 1 11/2 r----

~

d §

5.

eops entered/requiring substantive actions (1-2) 2/3/2 r--

~

~

,.1#

6.

EOP contlngencies requiring substantive actions (0-2) 0/2/0 r----

A..l ~~ d

7.

Critic",1 tasks (2-3) 3/6/4

~fJl!)J"

=

(

ES-301-5

Vogtle Nuclear Plant A

P P

L I

C A

N T

SRO-I x

SRO-U x

RO x

SRO-I x

RO x

SRO-I x

Instructions:

E V

E N

T T

Y P

E RX NOR I/C MAJ TS RX NOR IIC MAJ TS RX NOR I/C MAJ TS 1

CREW POSITION S

R o A

T C

B o P

Transient and Event Checklist Date of Exam: June 1-5,2009 Operating Test No.: 2009-301 2

CREW POSITION S

R o A

T C

B o P

Scenarios 3

CREW POSITION S

R o A

T C

B o P

4 CREW POSITION S

R o

A T

C B o P

T o

T A

L 4

2 M

I N

I M

U M(*)

R U

o 23 4

4 2

6 2

2 11 0

2 2

4 o

13 4

4 2

6 2

2 o

o 2

2 4

o 11 4

4 2

6 2

2 o

o 2

2

1.

Check the applicant level and enter the operating test number and Form ES-D-1 event numbers for each event type; TS are not applicable for RO applicants. ROs must serve in both the "at-the-controls (ATC)" and "balance-of-plant (BOP)"

positions; Instant SROs must serve in both the SRO and the ATC positions, including at least two instrument or component (I/C) malfunctions and one major transient, in the ATC position. If an Instant SRO additionally serves in the BOP position, one I/C malfunction can be credited toward the two IIC malfunctions required for the ATC position.

2.

Reactivity manipulations may be conducted undernormal or controlled abnormal conditions (refer to Section D.5.d) but must be significant per Section C.2.a of Appendix D. (*) Reactivity and normal evolutions may be replaced with additional instrument or component malfunctions on a 1-for-1 basis.

3.

Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to the applicant's competence count toward the minimum requirements specified for the applicant's license level in the right-hand columns.

ES-301-5 Transient and Event Checklist

=================================================================9

Vogtle Nuclear Plant A

P P

L I

C A

N T

SRO-I x

SRO-U x

E V

E N

T T

Y P

E RX NOR I/C MAJ TS RX RO NOR x

SRO-I x

RO x

SRO-I x

Instructions:

IIC MAJ TS RX NOR I/C MAJ TS 1

CREW POSITION S

R o

A T

C B

o P

Date of Exam: June 1-5,2009 Operating Test No.: 2009-301 2

CREW POSITION S

R o

A T

C B

o P

Scenarios 3

CREW POSITION S

R o

A T

C B

o P

4 CREW POSITION S

R o

A T

C B

o P

T o

T A

L 4

2 M

I N

I M

U M(*)

R U

o 23 4

4 2

6 2

2 11 0

2 2

4 o

13 4

4 2

6 2

2 o

o 2

2 4

o 11 4

4 2

6 2

2 o

o 2

2

1.

Check the applicant level and enter the operating test number and Form ES-D-1 event numbers for each event type; TS are not applicable for RO applicants. ROs must serve in both the "at-the-controls (ATC)" and "balance-of-plant (BOP)"

positions; Instant SROs must serve in both the SRO and the ATC positions, including at least two instrument or component (I/C) malfunctions and one major transient, in the ATC position. If an Instant SRO additionally serves in the BOP position, one IIC malfunction can be credited toward the two IIC malfunctions required for the ATC position.

2.

Reactivity manipulations may be conducted under normal or controlled abnormal conditions (refer to Section D.5.d) but must be significant per Section C.2.a of Appendix D. (*) Reactivity and normal evolutions may be replaced with additional instrument or component malfunctions on a 1-for-1 basis.

3.

Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to the applicant's competence count toward the minimum requirements specified for the applicant's license level in the right-hand columns.

ES-301-6 Competencies Checklist r01/A-L..

Facility: Vogtle Date of Examination: 6-1-2009 Operating Test No.: 2009-301 APPLICANTS Competencies RO X

SRO-I X

SRO-U X SCENARIO SCENARIO SCENARIO 1

2 3

4 1

2 3

4 1

2 3

4 Interpret/Diagnose 12 12 1 2 1 2 1 2 1 2 1 2 12 123 12 12 12 Events and Conditions 34 34 34 34 34 34 34 34 456 34 34 34 56 56 56 56 56 56 56 56 789 56 56 56 78 78 7 78 78 78 7

78 78 7

78 9

9 9

9 9

Comply With and 12 12 1 2 12 1 2 1 2 1 2 12 123 12 12 12 Use Procedures (1) 34 34 34 34 34 34 34 34 456 34 34 34 56 56 56 56 56 56 56 56 789 56 56 56 78 78 7 78 78 78 7

78 78 7

78 9

9 9

9 9

Operate Control 12 1 2 12 12 Boards (2) 34 34 34 34 56 56 56 56 78 78 7

78 9

9 Communicate 12 1 2 12 12 1 2 1 2 1 2 1 2 123 12 12 12 and Interact 34 34 34 34 34 34 34 34 456 34 34 34 56 56 56 56 56 56 56 56 789 56 56 56 78 78 7 78 78 78 7

78 78 7

78 9

9 9

9 9

Demonstrate 12 12 12 12 123 1 2 1 2 12 Supervisory Ability (3) 34 34 34 34 456 34 34 34 56 56 56 56 789 56 56 56 78 78 7

78 78 7

78 9

9 9

Comply With and 1 2 1 2 12 12 123 1 2 12 12 Use Tech. Specs. (3) 34 34 34 34 456 34 34 34 56 56 56 56 789 56 56 56 78 78 7

78 78 7

78 9

9 9

Notes:

(1 )

Includes Technical Specification compliance for an RO.

(2)

Optional for an SRO-U.

(3)

Only applicable to SROs.

Instructions:

Check the applicants' license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.

ES-301-6 Competencies Checklist r/#A-£...

Facility: Vogtle Date of Examination: 6-1-2009 Operating Test No.: 2009-301 APPLICANTS Competencies RO X

SRO-I X

SRO-U X SCENARIO SCENARIO SCENARIO 1

2 3

4 1

2 3

4 1

2 3

4 I nterpretlDiagnose 12 12 1 2 12 1 2 1 2 12 1 2 123 1 2 1 2 12 Events and Conditions 34 34 34 34 34 34 34 34 456 34 34 34 56 56 56 56 56 56 56 56 789 56 56 56 78 78 7

78 78 78 7

78 78 7

78 9

9 9

9 9

Comply With and 12 12 1 2 12 1 2 1 2 1 2 12 123 12 1 2 1 2 Use Procedures (1) 34 34 34 34 34 34 34 34 456 34 34 34 56 56 56 56 56 56 56 56 789 56 56 56 78 78 7

78 78 78 7

78 78 7

78 9

9 9

9 9

Operate Control 12 1 2 12 1 2 Boards (2) 34 34 34 34

(

56 56 56 56

\\

78 78 7

78 9

9 Communicate 12 1 2 12 12 1 2 1 2 1 2 1 2 123 12 12 1 2 and Interact 34 34 34 34 34 34 34 34 456 34 34 34 56 56 56 56 56 56 56 56 789 56 56 56 78 78 7

78 78 78 7

78 78 7

78 9

9 9

9 9

Demonstrate 1 2 1 2 12 12 123 1 2 1 2 12 Supervisory Ability (3) 34 34 34 34 456 34 34 34 56 56 56 56 789 56 56 56 78 78 7

78 78 7

78 9

9 9

Comply With and 1 2 1 2 1 2 1 2 123 1 2 1 2 12 Use Tech. Specs. (3) 34 34 34 34 456 34 34 34 56 56 56 56 789 56 56 56 78 78 7

78 78 7

78 9

9 9

Notes:

(1 )

Includes Technical Specification compliance for an RO.

(2)

Optional for an SRO-U.

(3)

Only applicable to SROs.

Instructions:

Check the applicants' license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.

ES-401 Record of Rejected KI As Form ES-401-4 Tier / Group Randomly Reason for Rejection Selected KIA SRO T2/G1 061A2.02 Vogtle design does not have air operated steam supply valves for the AFW system. 061A2.03 was randomly selected as a replacement.

SRO T2/G2 071A2.03 Vogtle design does not use rupture disks for the Waste Gas Disposal System. 071A2.09 was randomly selected as a replacement.

ROT3 G2.4.41 Emergency classification thresholds are not RO duties at Vogtle.

G2.4.43 was randomly selected as a replacement.

ES-401, Page 27 of 33 ES-401 Record of Rejected KlAs Form ES-401-4 Tier / Group Randomly Reason for Rejection Selected KIA SRO T2/Gl 061A2.02 Vogtle design does not have air operated steam supply valves for the AFW system. 061A2.03 was randomly selected as a replacement.

SRO T2/G2 071A2.03 Vogtle design does not use rupture disks for the Waste Gas Disposal System. 071A2.09 was randomly selected as a replacement.

ROT3 G2.4.41 Emergency classification thresholds are not RO duties at Vogtle.

G2.4.43 was randomly selected as a replacement.

(

ES-401, Page 27 of 33

ES-401-6 Written Examination Quality Checklist fi N t:I /.J Vogtle Date of Exam: 6-26-2009 Exam Level: RO X SRO X

1.

2.

a.

b.

SRO Item on ns and answers are technicall accurate and a icable to the facili NRC K/As are referenced for all questions.

ectives are referenced as available.

in accordance with Section D.2.d of ES-401

4.

The sampling process was random and systematic (If more than 4 RO or 2 SRO questions were from the last 2 NRC licensi consult the NRR OL ram office

5.

Question duplication from the license screening/audit exam was controlled

6.

as indicated below (check the item that applies) and appears appropriate:

_ the audit exam was systematically and randomly developed; or

_ the audit exam was completed before the license exam was started; or

_ the examinations were developed independently; or XX the licensee certifies that there is no duplication; or

_ other (explain)

Bank Modified Bank use meets limits (no more than 75 percent from the bank, at least 10 percent new, and the rest new or modified); enter the actual RO / SRO-only 11 /2 10/ 1

7.

estion distributi ht.

Between 50 and 60 percent of the questions on the RO exam are written at the comprehension/ analysis level; the SRO exam may exceed 60 percent if the randomly selected K/As support the higher cognitive levels; enter the actual RO / SRO uestion d

8.

References/handouts provided do not give away answers or aid in the elimination of distractors.

Memory 33/2

9.

Question content conforms with specific KIA statements in the previously approved examination outline and is appropriate for the tier to which they are assigned; deviations are

10.

Question n<"",nnmetric and format meet the ix B.

11.

The exam contains the required number of one-point, multiple choice items; the total is correct and with the value on the cover sheet.

a. Author

b. Facility Reviewer (*)

c. NRC Chief Examiner (#)

d. NRC Regional Supervisor New 54/22 CIA 42/23 Note:

• The facility reviewer's initials/signature are not applicable for NRC-developed examinations.

NRC reviewer initial items in Column "c"* chief examiner concurrence uired.

Initial ES-401-6 Written Examination Quality Checklist F! N ~ L.

Facility:

Vogtle Date of Exam: 6-26-2009 Exam Level: RO X SROX Initial Item Description a

b*

c#

1.

Questions and answers are technically accurate and applicable to the facility.

~;:

"'1~~

.; I(( l--t.t

2.

a.

NRC K/As are referenced for all questions.

~i d 1,j

b.

Facility learning objectives are referenced as available.

3.

SRO questions are appropriate in accordance with Section D.2.d of ES-401

~

_(2i ltt 1-10;" :

4.

The sampling process was random and systematic (If more than 4 RO or 2 SRO questions

~l, t were repeated from the last 2 NRC licensing exams, consult the NRR OL program office).

5.

Question duplication from ihe license screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate:

W

_ the audit exam was systematically and randomly developed; or d

_ the audit exam was completed before the license exam was started; or

_ the examinations were developed independently; or

);1 XX the licensee certifies that there is no duplication; or

.~

_ other (explain)

6.

Bank use meets limits (no more than 75 percent Bank Modified New

~

PI from the bank, at least 10 percent new, and the rest t'

);1 new or modified); enter the actual RO / SRO-only 11 /2 10/ 1 54/22 I,r question distribution(s) at right.

(

7.

Between 50 and 60 percent of the questions on the RO Memory CIA

-4JI exam are written at the comprehension/ analysis level; q}

the SRO exam may exceed 60 percent if the randomly 33/2 42/23

~ I:

~1 selected K/As support the higher cognitive levels; enter t..~"

the actual RO / SRO question distribution(s) at right.

8.

References/handouts provided do not give away answers or aid in the elimination of distractors.

{)/ 4' J;.1

9.

Question content conforms with specific KIA statements in the previously approved

J~ W examination outline and is appropriate for the tier to which they are assigned; it deviations are justified.

10.

Question psychometric quality and format meet the guidelines in ES Appendix B.

@. ?if v1;J

11.

The exam contains the required number of one-point, multiple choice items;

~: ¢if )i the total is correct and agrees with the value on the cover sheet.

Printed ~nat~~l~

Date I

'-19 "()1

a. Author Thad N. Thompson /

I rI;,'J

'2'L,,-r.;~

b. Facility Reviewer (*)

Daniel Scukanec /

Po -e::k",11'" ;<_ {/

6-;1;.)~<'.}1

c. NRC Chief Examiner (#)

.edu..*.n he.a -dV' /~

~'><..-::J 2'~ th~

t$Bt;

d. NRC Regional Supervisor LUJ,~ -r: WiTJAUIiJ,J / ~

~

/

fJ',

/ ~'qlMff ~

Note:

• The facility reviewer's initials/signature are not applicable for NRC-developed examinations.

1. Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.

ES-401 Written Examination Review Worksheet -- Vogtle SRO Form ES-401-9

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. /

Back-Q=

SRO B/M/N u/EiS Explanation Focus Dist.

Link units ward KIA Only Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1.

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2.

. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3.

Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (Le., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4.

Check the appropriate box if a job content error is identified:

The question is not.linked to the job requirements (Le., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (Le., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5.

Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).

6.

Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7.

Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8.

At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stein, I Cues I T/F I c~ed'l Partial JOb-I Minutia I #/, I Back-Q= ISRO B/M/N U/EiS Explanation Focus Dlst.

Link units ward KIA Only ES-401 Written Examination Review Worksheet -- Vogtle SRO Form ES-401-9

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. /

Back-Q=

SRO B/M/N U/E/S Explanation Focus Dis!.

Link units ward KIA Only Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1.

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2.

. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3.

Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4.

Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5.

Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).

6.

Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7.

Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8.

At a minimum, explain any "u" ratings (e.g., how the Appendix B psychometric attributes are not being met).

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. /

Back-Q=

SRO B/M/N U/E/S Explanation Focus Dis!.

Link units ward KIA Only

1. I 2.

8.

Q# I LOK LOD (F/H)

(1-5)

Explanation H

X I X

N U

2 I H

I 1

I I

X I

I N

I U I Based on the information provided, this question

.§.

appears to be a collection of true false statements. The questions can be answered with general knowledge. Based on note associated with TS and procedures concerning PORV operation. It is also general knowledge, that when a peace of TS equipment is not operable, proceeding to a higher mode is not allowable.

Very low level of difficult. SRO only. ---

CONVENCED US THAT THIS WAS AN SRO QUESTION AND WAS ACCEPTABLE. NOT A U. (OK AS WRITTEN) 3 I I X I N I u I

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO B/M/N U/E/S Explanation Focus Dist.

Link units ward KIA Only 1

H X

X X

N U

Supporting documentation provided in package is incomplete. Underlining and bolding in stem provides cue. Charging line flow fluctuating between 30 and 130 gpm will not result in charging line HIILO FLOW alarm. Based on the information given, explain what would cause the VCT HI/LOW LEVEL alarm? Based on the information provided, explain why one would conclude there was a loss of letdown flow. If that conclusion can not be reached based on information given, then distractors C and Dare not plausible.. It also appears that this question can be answered based on RO knowledge alone.

Not SRO. REVISED -- MODIFED STEM AND DISTRACTORS (OK) 2 H

1 X

N U

Based on the information provided, this question

§ appears to be a collection of true false statements. The questions can be answered with general knowledge. Based on note associated with TS and procedures concerning PORV operation. It is also general knowledge, that when a peace of TS equipment is not operable, proceeding to a higher mode is not allowable.

Very low level of difficult. SRO only. ---

CONVENCED US THAT THIS WAS AN SRO QUESTION AND WAS ACCEPTABLE. NOT A U. (OK AS WRITTEN) 3 x

N u

The question can be answered with RO only knowledge. Determining that an ATWS exist is passed on system knowledge and the LCO entered is based on the applicants understanding of what LCO must be entered when an A TWS occurs. This is RO knowledge. CHANGED STEM AND DISTRACTORS (OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO B/M/N U/E/S Explanation Focus Dist.

Link units ward KIA Only I

4 N

S MADE A CHANGE TO THE STEM AND DISTRACTORS (OK) 5 H

N S

MADE CHANGES TO THE DISTRACTORS (OK) 6 M

S DID REWORD STEM (Ok) 7 X

N U

8 H

2 X

E E

May be two correct answers. Both B & D.

REWORDED DISTRACTORS TOOK OUT RETURN TO LEVEL. UNDERSTANDING LCO REQUIREMENT ASSURES KIA MATCH (OK) 9 F

X N

10 H

2 X

N 11 H

3 N

S ADDED WORDS TO THE DISTRACTOR (OK) 12 H

3 N

s REWORDED DISTRACTORS AFTER REVIEWING QUESTION. (OK) 13 2

N S

(OK) 14 H

2 X

N U

15 H

3 N

S REWORDED THE STEM (OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO B/M/N U/E/S Explanation Focus Dist.

Link units ward KIA Only 4

N S

MADE A CHANGE TO THE STEM AND DISTRACTORS (OK) 5 H

N S

MADE CHANGES TO THE DISTRACTORS (OK) 6 M

S DID REWORD STEM (Ok) 7 X

N U

Not SRO only. Question can be answered with system only knowledge REWORDED THE STEM AND DISTRACTORS (OK) 8 H

2 X

E E

May be two correct answers. Both B & D.

REWORDED DISTRACTORS TOOK OUT RETURN TO LEVEL. UNDERSTANDING LCO REQUIREMENT ASSURES KIA MATCH (OK) 9 F

1 X

N u

LOD. Simple Memory. Which one describes the....

Not SRO only WROTE A NEW QUESTiON (OK) 10 H

2 X

N U

Question can be answered using RO knowledge only WROTE NEW QUESTiONS (OK) 11 H

3 N

s ADDED WORDS TO THE DISTRACTOR (OK) 12 H

3 N

s REWORDED DISTRACTORS AFTER REVIEWING QUESTION. (OK) 13 2

N s

(OK) 14 H

2 X

N U

Not SRO only. Question can be answered with RO/systems knowledge. For the given conditions wilen should feed and bleed be terminated? I would think that the RO would know that SG level must be restored before terminating. NEW QUESTION (OK) 15 H

3 N

s REWORDED THE STEM (OK)

1. I 2.

, 5. Other,

Q# I LOK LOD (F/H)

(1-5) 16,

H, 2 17 I F?

I I X,

18,

H I I x I 19 I F

2

6., 7.,

8.

Explanation N, u 'Not SRO only. What steps are marked N/A?

Could not locate in the procedure where recirculation of ~ 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> is required prior to sampling. Is this a general knowledge question?

I do not see where procedures were used to correct/control...... CORRECTED INFORMATION IN THE STEM. QUESTION SHOULD NOT HAVE BEEN UNSAT. IT IS THE SRO RESPONSIBILITY TO REVIEW/APPROVE I

I RELEASE PERMITS. (OK}

N U

N

, 8? I Need to make sure there are not two correct (B &

D) answers. Reference procedure was not provided. REMOVED PART OF THE DISTRACTORS. CRAIG WAINTED TOO....

HAD TO CHANGE QUESTION BACK TO INCLUDE WHAT CRAIG SUGGESTED DID NOT WORK...... QUESTION IS OK

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO B/M/N U/E/S Explanation Focus Dist.

Link units ward KIA Only 16 H

2 X

N U

Question can be answered with system knowledge only. What actions should be taken if a loss of CV is occurring? Reduce power. How is this achieved? What should be observed while reducing power and what actions are required to keep ROD BANK LO-LO LIMIT alarm clear?

What was the initial power level? WROTE NEW QUESTION (OK) JOHN DOES NOT LIKE THE QUESTION, THINKS IT IS BEYOND SRO KNOWLEDGE.... I DISSAGREE. WE DISCUSSED THIS AGAIN AND CAME TO AN AGREEMENT.

17 F?

X N

U Not SRO only. What steps are marked N/A?

Could not locate in the procedure where recirculation of - 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> is required prior to sampling. Is this a general knowledge question?

I do not see where procedures were used to correct/control...... CORRECTED INFORMATION IN THE STEM. QUESTION SHOULD NOT HAVE BEEN UNSAT. IT IS THE SRO RESPONSIBILITY TO REVIEW/APPROVE RELEASE PERMITS. {OK}

18 H

X N

U Not SRO only. Can be answered with system knowledge and knowledge of expected actions after a component/equipment problem has been identified. WROTE NEW QUESTiON (OK) 19 F

2 N

S?

Need to make sure there are not two correct (B &

D) answers. Reference procedure was not provided. REMOVED PART OF THE DISTRACTORS. CRAIG WAINTED TOO....

HAD TO CHANGE QUESTION BACK TO INCLUDE WHAT CRAIG SUGGESTED DID NOT WORK...... QUESTION IS OK

10 20 30 Psychometric Flaws 40 Job Content Flaws 50 Other 60 70 80 Q#

LOK LOD j

(F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q= SRO B/M/N U/EIS Explanation Focus Dist.

Link units ward KIA Only 20 H

2 X

N s

(OK) 21 H

2 N

s (OK) 22 F

2 X

N E

Consider changing distractors such that a higher value is give for one of the other limits identified in the emergency exposure guideline ADDED INFORMATION TO THE STEM AND ORGNIZED DISTRACTOR (OK) 23 F

N s

ADDED WORDS TO THE STEM (OK) 24 X

X u

The stem of the question provides part of the note. The answer provides the rest Memory question. Are RO not responsible for the notes/cautions? REWORDED QUESTIION (OK) 25 H

2 X

X N

U Not SRO only. As written the question only requires RO knowledge to answer. Although the question was used on a Wolf Creek exam, the way it was modified made it RO only. As written the question appears to be a T/F' type question.

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD til (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia Back-Q=

SRO B/M/N U/E/S Explanation Focus Dis!.

Link units ward KIA Only 20 H

2 X

N s

(OK) 21 H

2 N

s (OK) 22 F

2 X

N E

Consider changing distractors such that a higher value is give for one of the other limits identified in the emergency exposure guideline ADDED INFORMATION TO THE STEM AND ORGNIZED DISTRACTOR (OK) 23 F

N s

ADDED WORDS TO THE STEM (OK) 24 X

X u

The stem of the question provides part of the note. The answer provides the rest. Memory question. Are RO not responsible for the notes/cautions? REWORDED QUESTIION (OK) 25 H

2 X

X N

U Not SRO only. As written the question only requires RO knowledge to answer. Although the question was used on a Wolf Creek exam, the way it was modified made it RO only. As written the question appears to be a T/F type question.

ES-401 Written Examination Review Worksheet -- Vogtle RO Form ES-401-9

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem JcuesJ T/F lcredjpartial JOb-j Minutil ~Ck-Q= jSRO B/M/N U/E/S Explanation Focus Dist.

Link units ward KIA Only Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.)

1.

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2.

Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3.

Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (Le., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4.

Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (Le., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (Le., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5.

Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).

6.

Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7.

Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8.

At a minimum, explain any "u" ratings (e.g., how the Appendix B psychometric attributes are not being met).

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

.6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. /

Back-Q=

SRO B/M/N U/E/S Explanation Focus Dist.

Link units ward KIA Only 1

F 2

S (OK) 2 H

4 S?

Is this an RO question? LICENSEE SAID IT WAS SRO. WE REMOVED #s FROM DISTRACTORS (OK)

ES-401 Written Examination Review Worksheet -- Vogtle RO Form ES-401-9

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. /

Back-Q= SRO B/M/N U/E/S Explanation Focus Dis!.

Link units ward KIA Only Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1.

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2.

Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3.

Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4.

Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5.

Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).

6.

Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7.

Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8.

At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. /

Back-Q= SRO B/M/N U/E/S Explanation Focus Dis!.

Link units ward KIA Only 1

F 2

S (OK) 2 H

4 S?

Is this an RO question? LICENSEE SAID IT WAS SRO. WE REMOVED #s FROM DISTRACTORS (OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO B/M/N U/E/S Explanation Focus Dist.

Link units ward KIA Only 3

H 2

X u

Possible two correct answers (B &C).

Identify the failures that a multiplexing failure could result in (failures within the rod control system) EXPLAINED WHY THERE WERE NOT TWO CORRECT ANSWERS (QUESTION IS OK) 4 F

2 MlB S

MADE MINOR CHANGES (OK) 5 H

3 N

S CHANGED 32 TO 35 MINOR CHANGEES (OK) 6 3

N S

(OK) 7 F

2 M

(OK) 8 H

3 N

S (OK) 9 H

1 N

S? Added clarity to stem and identified testing on A RT Breakers. Changed distracter wording.

SATLOD 10 2

B S

(OK) 11 N

S (OK) 12 H

3 X

X M/B U

Based on the information given, I do not consider A & B plausible. You also give a reason in two distractors (cue). What is "indicating properly?" If the applicant concludes indication properly based on conditions, there could be two correct answers.

Question replaced with Farley 2003 question.

SAT 13 H

1 S

(OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO B/M/N U/E/S Explanation Focus Dist.

Link units ward KIA Only 3

H 2

X U

Possible two correct answers (B &C).

Identify the failures that a multiplexing failure could result in (failures within the rod control system) EXPLAINED WHY THERE WERE NOT TWO CORRECT ANSWERS (QUESTION IS OK) 4 F

2 M/B S

MADE MINOR CHANGES (OK) 5 H

3 N

S CHANGED 32 TO 35 MINOR CHANGEES (OK) 6 3

N S

(OK) 7 F

2 M

(OK) 8 H

3 N

S (OK) 9 H

1 N

S?

Added clarity to stem and identified testing on A RT Breakers. Changed distracter wording.

SATLOD 10 2

B S

(OK) 11 N

S (OK) 12 H

3 X

X M/B U

Based on the information given, I do not consider A & B plausible. You also give a reason in two distractors (cue). What is "indicating properly?" If the applicant concludes indication properly based on conditions, there could be two correct answers.

Question replaced with Farley 2003 question.

SAT 13 H

1 S

(OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. /

Back-Q=

SRO B/M/N U/E/S Explanation Focus Dis!.

Link units ward KIA Only 14 H

S (OK) 15 F

2 s

(OK) 16 H

2 s

(OK) 17 H

X s

Distractor C does not appear to be plausible.

Both CCPs have a common suction. Not possible to have trains alignment for suctions sources.

Evaluated drawings, procedures and determined distractors are minimally acceptable. SAT 18 F

s (OK) 19 2

S (OK) 20 2

N s

(OK) 21 H

2/3 s

(OK) 22 H

2 N

s (OK) 23 F

N s

(OK) 24 H

X E

Rearrange information in the stem (order of events - unit shut down..... )

Stem rearranged SAT 25 F

X SI? Please provide documentation where you discuss simi-auto swapover of RHR References provided - SAT

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

B.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q= SRO B/M/N U/E/S Explanation Focus Dist.

Link units ward KIA Only 14 H

S (OK) 15 F

2 s

(OK) 16 H

2 s

(OK) 17 H

X s

Distractor C does not appear to be plausible.

Both CCPs have a common suction. Not possible to have trains alignment for suctions sources.

Evaluated drawings, procedures and determined distractors are minimally acceptable. SAT 1B F

s (OK) 19 2

S (OK) 20 2

N s

(OK) 21 H

2/3 s

(OK) 22 H

2 N

s (OK) 23 F

N s

(OK) 24 H

X E

Rearrange information in the stem (order of events - unit shut down..... )

Stem rearranged SAT 25 F

X Sf? Please provide documentation where you discuss simi-auto swapover of RHR References provided - SAT

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q= SRO B/M/N U/E/S Explanation Focus Dist.

Link units ward KIA Only 26 F

1 X

u KIA asked for a reason. Asking for a reason might increase LOD Question replaced and edited - LOD 2 SAT 27 F

X X

M U

KIA failed to address operational implications.

Stem gives the reason why.

Discussed operational implications - reason why is in stem but is not required to be tested by KA.

28 H

2 s

(OK) 29 H

2 X

X N

u If CNTM is in service for respirable air quality control, would one expect any dampers to be I

closed? How couid a system operate properly with one set of dampers closed?

Added containment pressure to stem to increase plausibility of distractors. System normally operated with only exhaust open.

SAT 30 F

x u

Given the fact that you have a power supply failure for a detector, why would you not enter a procedure to repair/correct the problem?

Distractors B & D are not plausible Question altered to include core alterations. SAT

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q= SRO B/M/N U/E/S Explanation Focus Dist.

Link units ward KIA Only 26 F

1 X

U K/ A asked for a reason. Asking for a reason might increase LOD Question replaced and edited LOD 2 SAT 27 F

X X

M u

KIA failed to address operational implications.

Stem gives the reason why.

Discussed operational implications - reason why is in stem but is not required to be tested by KA.

28 H

2 s

(OK) 29 H

2 X

X N

U If CNTM is in service for respirable air quality control, would one expect any dampers to be closed? How could a system operate properly with one set of dampers closed?

Added containment pressure to stem to increase plausibility of distractors. System normally operated with only exhaust open.

SAT 30 F

X u

Given the fact that you have a power supply failure for a detector, why would you not enter a procedure to repair/correct the problem?

Distractors B & 0 are not plausible Question altered to include core alterations. SAT

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO B/M/N UlE/S Explanation Focus Dist.

Link units ward KIA Only 31 F

1 X

UlE LOD. May be able to reword stem/distractors.

Discussed makeup water requirements (normal vs requirement to use borated water).

32 H

3 5

(OK) 33 F

N un I Please Axpla C&

10.

to in I

Imai

Repl, 34 F

1 X

U ILl not see A&

It I Iwi out

)n lof I en lei !:jO! iCY L0ratilr 35 F

2 51?

Need to review additional information on excessive hydrogen production.

Plausable due to hydrogen production during charging but not a concern during discharging.

36 H

2 5

(OK) 37 H

3 5

(OK) 38 H

3 5

CHANGED WORDING IN STEM AND DISTRACTORS (OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO B/M/N UlE/S Explanation Focus Dis!.

Link units ward KIA Only 31 F

1 X

u/E LOD. May be able to reword stem/distractors.

Discussed makeup water requirements (normal vs requirement to use borated water).

32 H

3 S

(OK) 33 F

N un Please explain the plausibility of distractors C &

D. Unable to locate information in reference material Replaced distracotrs C&D with basis for ARV's.

34 F

1 X

u LOD. I do not see the plausibility of A & C. Are there any scenarios which would allow rod withdrawal would be done with out the permission of the S8?

Replaced second half of distractor with emergency boration termination criteria.

35 F

2 SI? Need to review additional information on excessive hydrogen production.

Plausable due to hydrogen production during charging but not a concern during discharging.

36 H

2 S

(OK) 37 H

3 S

(OK) 38 H

3 S

CHANGED WORDING IN STEM AND DISTRACTORS (OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO B/M/N U/E/S Explanation Focus Dist.

Link units ward KIA Only 39 H

N s

(OK) 40 H

M/B Sat - Higher order - level systems knowledge 41 H

X X

E Why do we need the alarm information in the stem. Consider stating "all systems functioned as designed following the trip... "We would not expect the RO to identify TS Alarm indicates where in the UOP the shift is and is required to correctly answer question.

TS has been removed 42 H

X N

E Explain the plausibility of A & B. I do not know/nor could I locate an cases where power is required to be produced by 100 MW increments.

MW reference removed - distractors modified to enhance clarity 43

?

1 X

X X

u LOD. KIA ask for knowledge of reason. This question failed to address the KIA. Possible two answers as written. Do not see in the procedure where it is required to verify NSCW discharge valves are closed.

Valve closure is a required interlock.

Reason for valve closure added to distractor.

44 F

2 B

s (OK) 45 H

2 N

s (OK) 46 H

1 N

s?

LOD MODIFITED QUESTION. (OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Sack-Q=

SRO S/M/N U/E/S Explanation Focus Dis!.

Link units ward KIA Only 39 H

N 8

(OK) 40 H

MIS Sat - Higher order - level systems knowledge 41 H

X X

E Why do we need the alarm information in the stem. Consider stating "all systems functioned as designed following the trip... "We would not expect the RO to identify TS Alarm indicates where in the UOP the shift is and is required to correctly answer question.

TS has been removed 42 H

X N

E Explain the plausibility of A & B. I do not know/nor could I locate an cases where power is required to be produced by 100 MW increments.

MW reference removed - distractors modified to enhance clarity 43

?

1 X

X X

u LOD. KIA ask for knowledge of reason. This question failed to address the KIA. Possible two answers as written. Do not see in the procedure where it is required to verify NSCW discharge valves are closed.

Valve closure is a required interlock.

Reason for valve closure added to distractor.

44 F

2 S

8 (OK) 45 H

2 N

8 (OK) 46 H

1 N

8?

LOD MODIFITED QUESTION. (OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD I

I (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO B/M/N U/E/S Explanation Focus Dist.

Link units ward KIA Only 47 H

2 N

S (OK) 48 H

2 N

S?

What are Hagen controllers? REMOVED CONTROLLER NAME - ADDED AUTO MANUAL CONTROLLER (OK) 49 F

X N

WE IThere two answers.

at l~iE in (OK) 50 F

X OKiS ???? need additional information on the detector.

Need more information to determine the plausibility of distractors B & D. DECIDED TO USE ORIGINAL QUESTION. PROVED THAT DISTRACTORS WERE OK BASED ON DESIGN OF MONITORS. (OK) 51 H

3 X

WE IF!

condi"uul j:::;

one t:Xj.Jt:l;t to I perla; a

If is A

&C not

lou, (OK) 52 H

3 M

S (OK) 53 F

2 X

N u IRad are I il

)1;Hl~ible

'QvlUI;::'"

10UI (OK) 54 F

N S

(OK) 55 H

2 s

(OK) 56 H

S (OK) 57 F

S (OK) 58 H

2 S

(OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q= SRO B/M/N UlE/S Explanation Focus Dis!.

Link units ward KIA Only 47 H

2 N

S (OK) 48 H

2 N

S?

What are Hagen controllers? REMOVED CONTROLLER NAME - ADDED AUTO MANUAL CONTROLLER (OK) 49 F

X N

WE There could be two correct answers. Look at requirement identified in ODCM. CHANGED STEM AND DISTRACTORS. (OK) 50 F

X OKiS ???? need additional information on the detector.

Need more information to determine the plausibility of distractors B & D. DECIDED TO USE ORIGINAL QUESTION. PROVED THAT DISTRACTORS WERE OK BASED ON DESIGN OF MONITORS. (OK) 51 H

3 X

WE For the conditions given why would one expect to perform a rapid power reduction? If this is true, A

& C would not be plausib!e. WROTE NEW QUESTION. (OK) 52 H

3 M

S (OK) 53 F

2 X

N U

Rad Chern Lab monitors are disable. Two implausible distractors.. WROTE NEW QUESTION. (OK) 54 F

N S

(OK) 55 H

2 S

(OK) 56 H

S (OK) 57 F

S (OK) 58 H

2 S

(OK)

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q= SRO B/M/N U/E/S Explanation Focus Dist.

Link units ward KIA Only 59 F

Y, X

X N

E Consider rewording the stem/distractors to have the applicant identify the window. It appears that as written the applicant only need to choose from two correct answers (A & 8) WROTR NEW QUESTION (OK) 60 F

1 s

LOD (OK) 61 F

1 s

LOD (OK) 62 F

1 X

E LOD Distractor A is not plausible. Review def of local control. Appears acceptable for in-plant JPM.. ACCEPTED QUESTION AS WRITTEN.

{OK} NOT A U MINOR CHANGES 63 F

1 X

B u

~OD A

64 H

2 s

(OK) 65 H

3 s

(OK) 66 F

X X

u Where are the preaccess filter units. Distractors do not appear to be plausible. CHANGED STEM.AND DISTRACTORS (OK} QUESTION SHOULD NOT HAVE BEEN A U 67 H

2 s

(OK) 68 H

1 X

u As written the question has a very LOD. Two distractors are implausible. Inadequate supporting documentation and wording in distractors. PROVIDED JUSTIFICATION AS THE WHY THE QUESTION IS ACCEPTABLE (OK} NOT AU

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO B/M/N U/E/S Explanation Focus Dis!.

Link units ward KIA Only 59 F

y" X

X N

E Consider rewording the stem/distractors to have the applicant identify the window. It appears that as written the applicant only need to choose from two correct answers (A & B) WROTR NEW QUESTION (OK) 60 F

1 s

LOD (OK) 61 F

1 s

LOD (OK) 62 F

1 X

E LOD Distractor A is not plausible. Review def of local control. Appears acceptable for in-plant JPM.. ACCEPTED QUESTION AS WRITTEN.

(OK} NOT A U MINOR CHANGES 63 F

1 X

B u

LOD. implausible distractors A & B. WROTE NEW QUESTION 64 H

2 s

(OK) 65 H

3 s

(OK) 66 F

X X

u Where are the preaccess filter units. Distractors do not appear to be plausible. CHANGED STEM.AND DlSTRACTORS (OK} QUESTION SHOULD NOT HAVE BEEN A U 67 H

2 s

(OK) 68 H

1 X

u As written the question has a very LOD. Two distractors are implausible. Inadequate supporting documentation and wording in distractors. PROVIDED JUSTIFICATION AS THE WHY THE QUESTION IS ACCEPTABLE

{OK} NOT A U

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q=

SRO B/M/N u/E/S Explanation Focus Dist.

Link units ward KIA Only 69 H

2

??

Is this an RO question? AGREED THAT IT WAS NOT AN RO QUESTION. SELECT A DIFFERENT KIA AND WRITE A NEW QUESTION 70 H

1 S

(OK) 71 F

2 S

(OK) 72 H

X X

B/M E/S Based on the information/lack of information there may be two correct answers. The level in the other SGs are not provided. Take a look at the procedure and answer questions concerning SGs levels and work through procedure. Prove that there are not two correct answers. REWROTE QUESTION (OK)..

73 F

2 X

E Based on the question asked, distractor D is not plausible. What dose higher priority of ORANGE vs RED have to do with the question asked?

Check to see if this is similar to an SRO question.

WROTE NEW QUESTION.

74 H

2 S

(OK) 75 F

1 X

X E

LOD. More information could be provided in the stem. Ask the applicant to identify the procedure to be used and the bases for using the procedure.

ACCEPTED QUESTION AS IS (OK)

Red - Should not have been identified as Unsat.

Question that had to be replaced that were Identified as E or had a ?

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

8.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Credo Partial Job-Minutia

1. 1 Back-Q= SRO B/M/N UlE/S Explanation Focus Dist.

Link units ward KIA Only 69 H

2

??

Is this an RO question? AGREED THAT IT WAS NOT AN RO QUESTION. SELECT A DIFFERENT KIA AND WRITE A NEW QUESTION 70 H

1 S

(OK) 71 F

2 S

(OK) 72 H

X X

B/M ElS Based on the information/lack of information there may be two correct answers. The level in the other SGs are not provided. Take a look at the procedure and answer questions concerning SGs levels and work through procedure. Prove that there are not two correct answers. REWROTE QUESTION (OK)..

73 F

2 X

E Based on the question asked, distractor 0 is not plausible. What dose higher priority of ORANGE vs RED have to do with the question asked?

Check to see if this is similar to an SRO question.

WROTE NEW QUESTION.

74 H

2 S

(OK) 75 F

1 X

X E

LOD. More information could be provided in the stem. Ask the applicant to identify the procedure to be used and the bases for using the procedure.

ACCEPTED QUESTION AS IS (OK)

Green Unsat.

Red - Should not have been identified as Unsat.

Question that had to be replaced that were Identified as E or had a ?

11/13/2009 14:21 FAX 706 826 3953 VEGP TRAINING CENTER l4J 001

.\\......./.

ES-40:1 Written Examination Grading Quality Checklist FOrm ES-403-1 I Facil

Vogtle Date of Exam: 06/26/09 Exam Level: RO[ZJ SRO[Z]

Itsm Descri tion

1.

2.

Answer key changes and question deletions justified and documented

3.

Applicants' scores checked for addition errors reviewers S ot check> 25% of examinations

4.

Grading for all borderline cases (80 +/-21% overall and 70 or 80, as a licable, +/-4% on the SRO-onl reviewed in detail 5,

All other failing examinations checked to ensure that grades are 'ustified

6.

Performance on missed questions checked for training deficiencies and wording problems; evaluate validity of uestions missed b half or more of the a plicants Printed Name/Signature

a. Grader

b. F ~cility Reviewer(")

c. N ~c Chief Examiner (*)

d. NRC Supervisor (*)

Initials Date

(.. "-z...l-, 0; 6-;?6-0?

"1-15..- 0 :\\

o//¢l:Jt:I'j

(*)

The facility reviewer's signature is not applicable for examinations graded by the NRC; two independent NRC reviews are re uired.

ES-403, Page 5 of 5 11/13/2009 14:21 FAX 706 826 3953 VEGP TRAINING CENTER Written Examination Grading Quality Checklist 141001 FOrm ES-403-1 I Facil

Vogtle Date of Exam: 06/26/09 Exam Level: RO[ZJ SRO[Z]

Itsm Descri tion

1.

2.

Answer key changes and question deletions justified and documented 3-Applicants' scores checked for addition errors reviewers s ot check> 25% of examinations

4.

Grading for all borderline cases (80 +/-21% overall and 70 or 80, as a licable, +/-4% on the SRO-onl reviewed in detail 5,

All other failing examinations checked to ensure that grades are 'ustified

6.

Performance on missed questions checked for training deficiencies and wording problems; evaluate validity of uestions missed b half or more of the a plicants Printed Name/Signature

a. Grader

b. F ~cility Reviewer(*)

c. N ~c Chief Examiner (*)

d. NRC Supervisor (*)

Initials Date

(.. "-z...~, 0; C-;?6-0?

"1-12-- 0:1 o//¢?L:J4?'j

(*)

The facility reviewers signature is not applicable for examinations graded by the NRC; two independent NRC reviews are re uired_

ES-403, Page 5 of 5