ML052160115

Final Qa/ Related Forms (Folder 1)

ML052160115
Person / Time
Site:	Nine Mile Point
Issue date:	07/01/2005
From:	Barr S Operations Branch I
To:	Conte R Operations Branch I
Conte R
References
Download: ML052160115 (5)

Text

ES-201 Examination Preparation Checklist Form ES-201-1

/

NRC (circleone)

Target Date*

Task Description (Reference)

-180

-120

2.

NRC examiners and facility contact assigned (C.1.d; C.2.e)

-120

-120

4.

Corporate notification letter sent (C.2.d)

[-go]

[5.

Reference material due (C.1.e; C.3.c; Attachment 2)]

-75

6.

Integrated examination outline(s) due, including Forms ES-201-2. ES-201-3,

1.

Examination administration date confirmed (C.l

.a; C.2.a and b)

3.

Facility contact briefed on security and other requirements (C.2.c)

ES-301-1, ES-301-2, ES-301-5, ES-D-l's, ES-401-112, ES401-3. and ES401-4. as applicable (C.1.e and f; C.3.d)

Examination outline(s) reviewed by NRC and feedback provided to facility licensee (C.2.h; C.3.e) scenarios, as applicable), supporting documentation (including Forms ES-301-3. ES-3014, ES-301-5, ES-301-6. and ES-401-6), and reference materials due (C.l.e, f. g and h; C.3.d)

7.

-70

-45

8.

Proposed examinations (including written, walk-through JPMs, and

-30

9. Preliminary license applications (NRC Form 398's) due (C.1.I; C.2.g;

10. Final license applications due and Form ES-2014 prepared (C.1.l; C.2.i; 1 I. Examination approved by NRC supervisor for facility licensee review ES-202)

-14 ES-202)

-14 (C.2.h; C.3.f)

-14

-7

13. Written examinations

12. Examinations reviewed with facility licensee (C.1.j; C.2.f and h; C.3.g)

• approved by NRC supervisor (C.2.i; C.3.h)

(C.2.i; Attachment 4; ES-204) with facility licensee (C.3.k) distributed to NRC examiners (C.3.i)

-7

14. Final applications reviewed; examination approval and waiver letters sent

15. Proctoringlwntten exam administration guidelines reviewed

16. Approved scenarios, job performance measures, and questions

-7

-7 Chief Examiner's Initials y13 y6 773 76

'in y3 33 j1!3

%J id Pf *

"J Y

rJ /A Target dates are keyed to the examination date identified in the corporate notification letter.

They are for planning purposes and may be adjusted on a case-by-case basis in coordination with the facility licensee.

Applies only to examinations prepared by the NRC.

[ ]

I

.1.

.I ES-201, Page 24 of 27

ES-201 Examination Outline Quality Checklist Form ES-20 1-2 Facility:

Item

1.

Written

2.

Sim

3.

WIT

4.

General NINE MILE POINT 1 Date of Examination: June f7.2005 Task Description I

Initials la

a. Verify that the outline(s) fit(s) the appropriate model per ES-401.

1M

b. Assess whether the outline was systematically and randomly prepared in accordance with Section D.l of ES-401 and whether all WA categories are appropriately sampled

c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics.

~

d. Assess whether the justifications for deselected or rejected WA statements are appropriate.

a. Using Form ES-301-5, verify that the proposed scenario sets cover the required number of normal evolutions, instrument and component failures, and major transients.

b. Assess whether there are enough scenario sets (and spares) to test the projected number and mix of applicants in accordance with the expected crew composition and rotation schedule without compromising exam integrity; ensure each applicant can be tested using at least one new or significantly modified scenario, that no scenarios are duplicated from the applicants audit test@)*. and scenarios will not be repeated on subsequent days.

c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative and quantitative criteria specified on Form ES-301-4 and described in Appendix D.

a. Verify that:

(1) the outline(s) contain(s) the required number of control room and in-plant tasks, (2) no more than 30% of the test material is repeated from the last NRC examination, (3)' no tasks are duplicated from the applicants audit test(s). and (4) no more than 80% of any operatino test is taken directlyfrom the licensees exam banks.

PP

b. Verify that:

(1) the tasks are distributed among the safety function groupings as specified in ES-301, (2) one task is conducted in a low-power or shutdown condition, (3) 4 - 6 (2 - 3 for SRO-U) of the tasks require the applicant to implement an alternate path procedure, (4) one in-plant task tests the applicants response to an emergency or abnormal condition, and (5) the in-plant walk-through requires the applicant to enter the RCA.

c. Verify that the required administrative topics are covered.

• P

d. Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days.

a. Assess whether plant-specific priorities (including PRA and IPE insights) are covered in the appropriate exam section.

b. Assess whether the 10 CFR 55.41143 and 55.45 sampling is appropriate.

c. Ensure that WA importance ratings (except for plant-specific priorities) are at least 2.5.

d. Check for duplication and overlap among exam sections.

lib

~

~

e. Check the entire exam for balance of coverage.

f. Assess whether the exam fits the appropriate job level (RO or SRO).

a.

b.

C.

d.

Author Facility Reviewer (')

NRC Chief Examiner (#)

NRC Supervisor Note:

• Not applicable for NRC-developed examinations.

1. IndeDendent NRC reviewer initial items in Column "c:" chief examiner concurrence reauired I

24 of 25 NUREG-1021 Draft Revision 9

I ES-401 WRITTEN EXAMINATION QU FACILITY:

NINE MILE POINT 1 DATE OF EXAM JUNE 17,2005 EXAM LEVEL RO Item Description a

1 2

a. NRC WAS referenced for all questions.

Question and answers are technically accurate.

b. Facility learning objectives referenced as available.

3 SRO questions are appropriate in accordance with Section D.2.d of ES-401.

NA 4

If more than four RO and two SRO questions are repeated from the last two NRC licensing exams, the facility licensees sampling process was random and systematic.

Question duplication from the license screeninglaudit exam was controlled as indicated below (check the item that applies) and appears appropriate:

X the audit exam was systematically and randomly developed the audit exam was completed before the license exam was started the examinations were developed independently the licensee certifies that there is no duplication other (explain) 5 6

Bank use meets limits (no more than 75 Bank Modified New 13175 4/75 58/75 17.3%

5.4%

77.3%

percent from the bank, at least 10 percent new, and the rest modified); enter the actual question distribution(s) at right Between 50 and 60 percent of the questions on the RO exam (including 10 new questions)

Memory CIA are written at the comprehensionlanalysis level; the SRO exam may exceed 60 percent if the randomly selected WAS support the higher question distribution(s) at right.

Referencedhandouts provided do not give away answers or aid in the elimination of distractors.

Question content conforms with specific WA statements in the previously approved examination outline and is appropriate for the tier to which they are assigned; deviations are justified.

7 32/75 43/75 cognitive levels; enter the actual ROISRO 42.6%

57.4%

8 9

10 Question psychometric quality and format meet the guidelines in ES Appendix B.

11 The exam contains the required number of one-point, multiple choice items; the total is correct and agrees with value on cover sheet.

nitials ZE I

I @

a. Author Date

b. Facility Reviewer (*)

Date 573b5-

c. NRC Chief Examiner(#) 57-7 MIL Date L/?/ar

d. NRC Regional Supervisor (lKHA3 J CdrJTt Date c*/tbr Note:

U The facility reviewers initialslsignature are not applicable for NRC developed examinations.

1. Independent NRC reviewer initials Column c ; chief examiner concurrence required.

NUREG 1021 Rev 9 ES-401 Final Exam Submittal

ES-403 Written Examination Grading Form ES-403-1 Quality Checklist

5.

All other failing examinations checked to ensure that grades

a. Grader

b. Facility Reviewer(*)

b Printed NameEignature Date

c. NRC Chief Examiner (*)

sr@KN "i C?A(z(l k fflw 7// Z/b r

d. NRC Supervisor (*)

4 \\ ( M 3 WAI7-f 7,h Z,/b S-

• )

The facility reviewer's signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.

/

/I ES-403, Page 5 of 5

ATTACHMENT 6: NRC EXAMINATION SECURITY AGREEMENT I

U N \\ r l ~JAC N R I ~ N wEKAY I

rob(31+

form ES-201-3 m-201 Examinalion Securlty Agreemenl

1.

Pre-Fxeminallon I

1 acknowledge that I'have acquired spedallzed knowledge about,the NRC tiienslng examfnalfons schedu!ed for the week(s) of klR/b % as of Ihe date of my signature. I aQree thal I will nol knowingly dhrulge any Informallon about lheee examinaliona to any parsons whqhave not been a u l h o ~ d by he NRC chlef curamlner. I ccnderalmd lhrt I am not to Instruct, OVdUa?43, or piavide pmrformancs foodback lo !home rppllcants rchsdukd to bs admlnlrtclnd thema Ilconrlng mxamlnaflonr from thia date until ccrmplrtlon ol oxamlnrtlon rdmlnlstratlon, rxespt PI ap&ciflc@lty noted b b l w and authorlzsd by the NRC. Furthermare. I em aware 01 Ihe phyicet security measurss and repulremenls (as documented In Ihs facllity~lbsnsee'o pm&dor+)

and understand tha! vlolelion of (he conditions of thls agreement may result In cancfjltetlon ol !he exarnlnellone adotun enforcement actlon against me or fhe facility Ilcensse. I Will lmrnedialsly report to Iacllily management 01 the NRC chlel eweminer any indbllons M SuggeetbM thp1 examinalion securily may have been cornpromled.

2.

Post-Examinetion

~

I c

To the best of my kno ed e, 1 did nol dtvulge to my unauthon'zed per~ons any information concetnlng the NRC ItaenolnQ ewamlnnliwr6 adrnlnlstered.

dudng the week(s) of la i3 O f. From Ihe date that I entered into this security agreement until Ihe completion of examina8llan adminhstretlon, 1 dM not instrucl, evaluate, or provide performance feedback to (hose applicanb who were admfnistered these llcenslng examinations, except a6 SpecikdY '

noted below and aulhorized by (he N KC.

. PAINTEONAME JOB TITLE / RESPONSIB!LTr/

Page 36 TAP-TQS-OB Rev 11 r