ML043210261

Various Checklists for the Fermi Initial Examination - September 2004

ML043210261
Person / Time
Site:	Fermi
Issue date:	11/05/2004
From:	Pederson C Division of Reactor Safety III
To:	O'Connor W Detroit Edison
Shared Package
ML043000346	List: ML043210251 ML043210261
References
50-341/04-301
Download: ML043210261 (25)

Text

VARIOUS CHECKLISTS FOR THE FERMI INITIAL EXAMINATION SEPTEMBER 2004

ES-201 Examination Preparation Checklist F o r m ES-201-1 Target Chief Date* Task Description / Reference Exami ner's Initials

-180 1, Examination administration date confirmed (C.1.a; C.2.a & b) &

-120 2. NRC examiners and facility contact assigned (C.1 .d; C.2.e) i9

-120 3. Facility contact briefed on security & other requirements (C.2.c) fl

-120 4. Corporate notification letter sent (C.2.d)

- [-go] [5. Reference material due (C.1.e; C.3.c)l /e&

-75 6. Integrated examination outline(s) due (C.l .e & f; C.3.d)

-70 7. Examination outline(s) reviewed by NRC and feedback provided to facility licensee (C.2.h; C.3.e) &

I -45 I 8. Proposed examinations, supporting documentation, and reference materials due (C.l.e, f, g & h; C.3.d)

I -30 9. Preliminary license applications due (C.l .I; C.2.g; ES-202) $E@@

I( -I4 1 IO. Final license applications due and assignment sheet prepared (C.1.l; C.2.g; ES-202)

-14 11. Examination approved by NRC supervisor for facility licensee review (C.2.h; C.3.9

-14 12. Examinations reviewed with facility licensee (C.l .j; C.2.f & h; C.3.g)

-7 13. Written examinations and operating tests approved by NRC supervisor (C.2.i; C.3.h) fl

-7 14. Final applications reviewed; assignment sheet updated; waiver letters sent (C.2.g, ES-204) v!+@

15. Proctoring/written exam administration guidelines reviewed with

-7 facility licensee and authorization granted to give written exams (if applicable) (C.3.k)

-7 16. Approved scenarios, job performance measures, and questions distributed to NRC examiners (C.3.i)

William T. OConnor, J e

,::; ...; , -- Yice Iresitl PIX, NuCltiir Gencratio 11 Fermi 2 6400 North Divie H~vy..,Kewport, blichigan 4SlGG Tel: 734-586-5201 Fax: 734-586-4172 DTE Energy May 12,2004 NRC-04-0034 Mr. Mike Bielby U. S. Nuclear Regulatory Commission Region III 2443 Warrenville Road, Suite 210 Lisle, Illinois 60532-4352

Reference:

NRC Correspondence from Roger D. Lanksbury to William OConnor, Jr., dated May 5,2004

Subject:

Administration of Licensing Examinations The attached Examination Outline is provided per Reference in support of the licensing examinations at the Fermi 2 Nuclear Power Plant the week of September 13 and September 20,2004.

The Examination Outline was prepared using the sampling process described in ES-401 Attachment 1 of Draft Revision 9 of NUREG-1021.

Additionally, find attached Form ES-20 1-1, Examination Outline Quality Assurance Checklist, with the facility portion completed.

To maintain examination security and integrity, the examination outlines shall be withheld from public disclosure until after the examinations are complete.

Should you have any questions regarding any aspect of the licensing examinations, please contact Steve Bollinger at (734) 586-4105, or Mike Cadden at (734) 586-4721. 7y

ES-201 Examination Outline Form ES-201-2 Quality Checklist FaciliC Date of Examinatior Initials Item Task Description

1. a. Verify bat the outline(s) fit@)the appropriate model per ES-401.

w R b. Assess whether the outline was systematically and randomly prepared in accordance with I Section D.l of ES-401 and whether all WA categories are appropriately sampled.

T T c. Assess whether the outline over-emphasizes any systems, evolutions, or generic topics.

E

- N d. Assess whether the justifications for deselected or rejected WA statements are appropriate.

2. a. Using Form ES-301-5, verify that the proposed scenario sets cover the required number of normal evolutions, instrument and component failures, and major transients.

7.

M

b. Assess whether there are enough scenario sets (and spares) to test the projected number and mix of applicants in accordance with the expected crew composition and rotation schedule without compromising exam integrity; ensure each applicant can be tested using at least one new or significantly modified scenario, that no scenarios are duplicated from the applicants audit test(sr, and scenarios will not be repeated on subsequent days.

c. T o the extent possible, assess whether the outline(s) conform(s) with the qualitative and

- quantitative criteria specified on Form ES-301-4 and described in Appendix D.

3. a. Verify hat:

(1) the outline(s) contain(s) the required number of control room and in-plant tasks, .

w (2) no more than 30% of the test material is repeated from the last NRC examination,J il I (3)* no tasks are duplicated from the applicants audit test(s), and L/

T (4) no more than 80% of any operating test is taken directly from the licensees exam banks:/

b. Verify hat:

(1) the tasks are distributed among the safety function groupings as specified in ES-3Ol.d (2) one task is conducted in a low-power or shutdown condition. J (3) 4 - 6 (2-3 for SRO-U) of the tasksrequire the applicant to implement an alternate path/

procedure, (4) one in-plant task tests the applicants response to an emergency or abnormal condition, and ,

( 5 ) the in-plant walk-through requires the applicant to enter the RCA. J

c. Verify that the required administrative topics are covered.

d. Determine if there are enough different outlines to test the projected number and mix of applicants and ensure that no items are duplicated on subsequent days.

4. a. Assess whether plant-specific priorities (including PRA and IPE insights) are covered in the appropriate exam section.

G E b. Assess whether the 10 CFR 55.41/43 and 55.45 sampling is appropriate.

N E c. Ensure that W A importance ratings (except for plant-specific priorities) are at least 2.5.

R A d. Check for duplication and overlap among exam sections.

L

e. Check the entire exam for balance of coverage.

- f. Assess whether the exam fits the appropriate job level (RO or SRO). n Printed Na P. Author

3. Facility Reviewer r)

. NRC Chief Examiner (#)

1. NRC Supervisor

\late: Not applicable for NRC-developedexaminations.

1. Independent NRC reviewer initial items in Column c; chief examiner concurrence required.

NUREG 1021,Draft Revision 9 24 of 25

William T.OConnor, Jr.

Vice President, Nuclear Generation Fermi 2 6400 North Dixie Hwy., Newport, Michigan 48166 Tel: 734.686.6201 Fax: 734.686.4172 Detroit Edison May 27,2004 NRC-04-0039 Mr. Mike Bielby U. S . Nuclear Regulatory Commission Region Dl 2443 Warrenville Road, Suite 210 Lisle, Illinois 60532-4352

Reference:

NRC Correspondence from Roger D. Lanksbury to William OConnor, Jr., dated May 5,2004

Subject:

Administration of Licensing Examinations The attached revised Examination Outline is provided per Reference in support of the licensing examinations at the Fermi 2 Nuclear Power Plant the week of September 13 and September 20,2004.

The Examination Outline was prepared using the sampling process described in ES-401 Attachment 1 of Draft Revision 9 of NUREG-1021.

Additionally, find attached Form ES-201-1, Examination Outline Quality Assurance Checklist, with the facility portion completed.

To maintain examination security and integrity, the examination outlines shall be withheld from public disclosure until after the examinations are complete.

Should you have any questions regarding any aspect of the licensing examinations, please contact Steve Bollinger at (734) 586-4 105, or Mike Cadden at (734) 586-4721.

Sincerely,

E S-20 1 Examination Outline Quality Checklist Form ES-201-2 (1) the tasks are distributed among the safety function groupings as specified in ES-301, (2) one task is conducted in a low-power or shutdown condition, (3) 4 - 6 (2 - 3 for SRO-U) of he tasks require the applicant to implement an alternate path procedure, (4) one in-plant task tests the applicants response to an emergency or abnormal condition, and (5) the in-plant walk-through requires the applicant to enter the RCA.

a. Author

b. Facility Reviewer [)

Note: Not applicable for NRC-developed examinations.

1. Independent NRC reviewer initial items in Column c; chief examiner concurrence required.

A NUREG-1021, Draft Revision 9 24 of 25

Fermi 2 6400 North Dixie Hiiy., Newport, h4ichigan 4816G Tel: 731-686-5201 Fax: 734-586-4172 DTE Energy October 11,2004 NRC-04-0105 Mr. H. Peterson U. S. Nuclear Regulatory Commission Region I11 2443 Warrenville Road, Suite 210 Lisle, Illinois 60532-4352

Dear Mr. Peterson:

Enclosed are the original Forms ES-201-3, Examination Security Agreement, with pre- and post-examination signatures by every individual who had detailed knowledge of the exams. You should have already received the remainder of the information described in ES-501 Initial Post-Examination Activities section C. 1.a.

If you have any further questions, please contact Mr. Allen Mann at (734) 586-4067.

Sincerely, Enclosures

A FERMI Examination Security Agreement c-Form ES-201-3

1. Pre-Exam ination date of my signature. I agree that I will not knowingly divulge any information about r an enforcement examinat ion security may have been corn promised.

2. Post-Examination ion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applic ing examinations, except as specifically noted below and authorized by the NRC .

PRINTED NAME

1. D-G s e u NOTES:

25 of 25 NUREG-I 021, Draft Revision 9

'I I L FERMI ES-201 Examination Security Agreement Form ES-201-3

Fermi 2 6400 North Dixie Hwy., Newport, MI 48166 Detroit Edison July 29,2004 NANT-04-0101 0801.26 Mr. H. Peterson Region III U.S. Nuclear Regulatory Commission 2443 Warrenville Road Suite 210 Lisle, IT., 60532-4352

Dear Mr. Peterson:

Enclosed please find the following examination materials submitted to the NRC in preparation for the upcoming Fermi 2 initial license examination scheduled to occur during the week of September 20, 2004:

Modified examination outlines (modified during the examination development process) 75 written examination items for the Reactor Operator examination

• 25 written examination items for the Senior Reactor Operator examination Ten plant system related Job Performance Measures Administrative topic evaluation items Two (2) sirnulato1 scenarios.

Appropriate quality verification checklists from Draft Revision 9, Supplement 1 of NUREG 1021 The examination materials were developed using the appropriate guidance contained in Draft Revision 9, NUREG 1021. These materials shall be withheld from public disclosure until after the examinations are complete.

We look forward to working with you and your examination team during the examination development and administration process. If you have any questions or comments regarding the contents of the items listed above please contact Mr. A1 Mann at 734-586-4067.

Sincerely, William T. OConnor w

Vice President, Nuclear Ge eration Fermi 2 Enclosure

ES-301 Operating Test Quality Checklist Form ES-30 1-3 Facility: 6?tMi Date of Examination: 9/20/04 Operating Test Numbe 2004401 lnitia

1. GENERAL CRITERIA

a. The operating test conformswith the previously approved outline; changes are consistent with sampling requirements (e.g., 10 CFR 55.45, operational importance, safety function distribution).

b. There is no day-to-day repetition between this andother operating tests to beadministered during this examination.

C. The operating test shall not duplicate items from the applicants' audit test(S(see Section D.1.a).

d. Overlap with the written examination and between operating testcategories is within acceptable

e. It appears that the operating test will differentiate between competent and less-than-competent applicants at the designated license level.

1 t

2. WALK-THROUGH CRITERIA

a. Each JPM ncludes the following, as applicable:

initial conditions J

' initiating cues J

' references and tools, including associated procedures J

' reasonable and validated time limits (average time allowed for completion) and specific/

designation if deemed to be time critical by the facility licensee J

' specific performance criteria that include:

- detailed expected actions with exact criteria and nomenclatureJ

- system response and other examiner cues/

- statements describing important observations to be made by the applicant/

- criteria for successful completion of the task J

- identification of critical steps and their associated performance standards

- restrictions on the sequence of steps, if applicable /

b. Repetition from operating tests used during the previous licensing examination is within J' acceptable limits (30% for the walk-through) and do not compromise test integrity.

C. At least 20 percent of the JPMs on each test are new or significantly modified. 4

3. SIMULATOR CRITERIA - I -

a. The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES-301-4 and a copy is attached.

Date

a. Author

b. Facility Reviewer(*)

c. NRC Chief Exami NOTE:

• The facility signature is not applicable for NRC-developed tests.

1. Independent NRC reviewer initial items in Column "c;" chief examiner concurrence required.

ES-30 1 Simulator Scenario Quality Checklist Form ES-301-4 Facility: Fermi 2 Date of Exam: 9/20/04 Scenario Numbers: 1 I 2 I Operating Test No.: 2004401 I

QUALITATIVEATTRIBUTES p t -

S -

- b' C#

1. The initial conditions are realistic, in that some equipment andlor instrumentation may be out of service, but it does not cue the operators into expected events. dP

2. The scenarios consist mostly of related events.

1"

3. Each event description consists of the point in the scenario when it is to be initiated the malfunction(s) that are entered to initiate the event the symptomslcues that will be visible to the crew the expected operator actions (by shift positim) the event termination point (if applicable)

4. No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario without a credible preceding incident such as a seismic event. R

5. The events are valid with regard to physics and thermodynamics.

6. Sequencing and timing of events is reasonable, and allows the examination team to obtain complete evaluation results commensuratewith the scenario objectives. 4

.' . If time compression techniques are used, the scenario summary clearly so indicates. Operators have sufficient time to carry out expected activities without undue time constraints. Cues are -/

8. The simulator modeling is not altered. -R

9. The scenarios have been valdated. Any open simulator performancedeficiencies have been evaluated to ensure that functional fidelity is maintained while running the planned scenarios.

IO. Every operator will be evaluated using at least one new or significantly modified senario. All other scenarios have been altered in accordance with Section D.5 of ES-301.

11. All individual operator competencies can be evaluated, as verified using Form ES-301-6 (submit the form along with the simulator scenarios).

12. Each applicant will be significantly involved in the minimum number of transients and events specified on Form ES-301-5 (submit the form with the simulator scenarios). R

13. The level of difficulty is appropriate to support licensing decisions for ech crew position.

TARGET QUANTITATWE ATTRIBUTES (PER SCENARIO: SEE SECTON D.4.D) Actual Attributes -

1. Total malfunctions (5-8) 8 1 7 1

2. Malfunctions after EOP entry (1-2) 3 1 3 1 A

3. Abnormal events (2-4) 2 1 3 1 x

4. Major transients (1-2) 2 I 2 1 d

5. EOPs enteredlrequiring substantive actions (1-2) 2 1 2 1

5. EOP contingencies requiring substantive actions (0-2) 1 1 0 1

7. Critical tasks (2-3) 3 I 2 I 25 of 27 NUREG-1021, Draft Revision 9

ES-301 Transient and Event Checklist Form ES-301-5 OPERATING TEST NO.: Fermi 2 for 9/20/04 2004-401 Reactivity 1 6 1 Normal 0 As RO Instrument / 2 4-53 2,4,8 Component Major 1 799 56 Reactivity 0 1 Normal 1 1.6 I AsSRO Instrument / 2 2.3.4.

5 2.3.4 Component Major 1 7,9 5.6 Reactivity 0 1 Normal 1 1,6 SRO-U Instrument / 2 224, 23.4 Component 5 Major 1 7,9 56 NUREG-1021, Draft Revision 9 26 of 27

ES-301 Competencies Checklist Form ES-301-6 Notes:

(1) Includes Technical Specification compliance for an RO.

(2) Optional for an SRO-U.

(3) Only applicable to SdOs.

Instructions:

Circle the applicant's license type and enter one or more event numbers that will allow the examiners to evaluate ev ble competency for every applicant.

Author:

NRC Reviewer:

27 of 27 NUREG-I 021, Draft Revision 9

ES-401 Written Examination Form ES-401-6 Quality Checklist Facility: Fermi 2 Date of Exam: 09120104 E am Level: RO/SRO Initial I

Item Description

1. Questions and answers technically accurate and applicable to facility

2. a. NRC WAS referenced for all questions

b. Facility learning objectives referenced as available

3. SRO questions are appropriate per Section D.2.d of ES-401

4. Question selection and duplication from the last two NRC licensing exams appears consistent with a systematic sampling process

5. Question duplication from the license screening/auditexam was controlled as indicated below (checkthe item that applies) and appears appropriate:

-the audit exam was systematically and randomly developed; or the audit exam was completed before the license exam was started; or g t h e examinationswere developed independently;or

- the licensee certifies that there is no duplication; or other (explain)

I I I

6. Bank use meets limits (no more than 75 percent from the bank at least 10 percent new, Bank [ Modified I New and the rest modified); enter the actual question distribution at right P

(71%)

1 (1%)

- 1 21 (28%) A

7. Between 50 and 60 percent of the questions on Memory CIA the exam (including 10 new questions) are ,.- 1

' I 3U written at the comprehension/analysis'level; .

(49%) (51%)

enter the actual question distribution at right

3. References/handouts provided do not give away answers

3. Question content conforms with specific WA statements in the previously approved examinationoutline and is appropriate for the Tier to which they are assigned; deviations are justified

10. Question psychometricquality and format meet ES, Appendix B,guidelines

11. The exam contains the required number, one-point, multiple choice items; the total is correct and agrees with value on cover sheet Printed Name / Signature Date

3. Author

1. Facility Reviewer (*)

. NRC Chief Examiner (#)

1. NRC Regional Supervisor Q!!K rlote:

• The facility reviewer's initials/signatureare not applicable for NRCdeveloped examinations.

1. Independent NRC reviewer initial items in Column "c:" chief examiner concurrence reauired.

ES-40 1 Written Examination Form ES-401-6 Quality Checklist Facility: Fermi 2 Date of Exam: 09/20/04 Exam Level: RO/SRO

- Initial

?k Item Description a

1. Questions and answers technically accurate and applicable to facility Irh T

5. Question duplication from the license screeninglaudit exam was controlled as indicated below (check the item that applies) and appears appropriate:

- the audit exam was systematicallyand randomly developed; or

-the audit exam was completed before the license exam was started: or

& the examinations were developed independently:or x

the licensee certifies that there is no duplication; or

7. Between 50 and 60 percent of the questions on the RO exam are written at the

a. Author

b. Facility Reviewer (*)

Date 7-30..

7 F q u +

c. NRC Chief Examiner (#)

d. NRC Regional Supervisor w!L Note:

• The facility reviewer's initialskignature are not applicable for NRCdeveloped examinations.

1. Independent NRC reviewer initial items in Column "c;" chief examiner concurrence required.

+ Nd! Jdd& d s -9a/-T.

NUREG-1021, Draft Revision 9 30 of 34

Fermi 2 6400 Iforth Dixie Hwyi Newport, Zllichigan 4SlGG Tel: 734-586-5201Fax: 734-586-4178 DTE Energy September 8,2004 NRC-04-0067 Mr. H. Peterson Region III U. S, Nuclear Regulatory Commission 2443 Warrenville Road Suite 210 Lisle, Illinois 60532-4352

Dear Mr. Peterson:

Enclosed please find the following examination materials submitted to the NRC in preparation for the upcoming Fermi 2 initial license examination scheduled to occur during the week of September 20,2004:

0 Modified examination outlines (modified during the examination development process) 0 75 written examination items for the Reactor Operator examination 25 written examination items for the Senior Reactor Operator examination 0 Ten plant system related Job Performance Measures 0 Administrative topic evaluation items 0 Two (2) simulator scenarios.

0 Appropriate quality verification checklists from Draft Revision 9, Supplement 1 of NUREG 1021 The examination materials were developed using the appropriate guidance contained in Draft Revision 9, NUREG 1021. These materials shall be withheld from public disclosure until after the examinations are complete.

We look forward to working with you and your examination team during the examination development and administration process. If you have any questions or comments regarding the contents of the items listed above please contact Mr. A1 Mann at 734-586-4067.

Enclosure

ES-301 Operating Test Quality Checklist F o ES-301-3

~

Facility: Fermi 2 Date of Examination:9/20/04 Operating Test Number: 2004401 l 1. GENERAL CRITERIA E-between competent a I 2. WALK-THROUGHCRITERIA Each JPM includesthe following, as applicable:

. initial conditions I

initiating cues

. references and tools, including associated procedures

. reasonable and validated time limits (average time allowed for completion) and specific designation if deemed to be time critical by the facility licensee

. specific performancecriteria that include:

- detailed expected actions with exact criteria and nomenclature

- system response and other examiner cues

- statements describing important observations to be made by the applicant

- criteria for successful completion of the task

- identification of critical steps and their associated performancestandards

- restrictionson the sequence of steps, if applicable Repetitionfrom operating tests used during the previous licensingexamination is within acceptable limits (30% for the walk-through) and do not compromise test integrity.

I c' At least 20 percent of the JPMs on each test are new or significantly modified.

3. SIMULATOR CRITERIA

a. The associated simulator operating tests (scenario sets) have been reviewed in accordancewith Form ES-301-4 and a copy is attached.

Printed Name / Signature Date

a. Author

b. Facility Reviewer(

c. NRC Chief Examiner (#) I

d. NRCSupeMsor q(IS l0.C NOTE:

• The facility signature is not applicable for NRCdevelopedtests.

1. Independent NRC reviewer initial items in Column "c;" chief examiner mncurence required.

I

. r NUREG-1021, Draft Revision 9 24 of 27

ES-301 Simulator Scenario Quality Checklist Form ES-301-4 Facility: Fermi 2 Date of Exam: 9/20/04 ScenarioNumbers: I/ 2 / OperatingTest No.: 20M401

~~

QUALITATIVE ATTRIBUTES Initials I I

1. The initial conditions are realistic, in that some equipment and/or instrumentationmay be out of service, but it does not cue the operators into expected events. .ab &&/

2. The scenarios consist mostly of related events.

3. Each event description consists of the point in the scenario when it is to be initiated the malfunction(s) that are entered to initiate the event the symptomslcues that will be visible to the crew the expected operator actions (by shift position) the event termination point (if applicable)

4. No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario without a credible preceding incident such as a seismic event.

5. The events are valid with regard to physics and thermodynamics.

6. Sequencing and timing of events is reasonable, and allows the examinationteam to obtain complete evaluation results commensurate with the scenario objectives.

7. If time compression techniques are used, the scenario summary clearly so indicates. Operators have sufficient time to carry out expected activities without undue time constraints. Cues are given.

8. The simulator modeling is not altered.

9. The scenarios have been validated. Any open simulator performancedeficiencies have been evaluated to ensure that functional fidelity is maintained while runningthe planned scenarios.

10. Every operator will be evaluated using at least one new or significantly modified scenario. All other scenarios have been altered in accordance with Section D.5 of ES-301.

11. All individual operator competencies can be evaluated, as verified using Form ES301-6 (submit the form along with the simulator scenarios).

12. Each applicant will be significantly involved in the minimum number of transients and events specified on Form ES-3014 (submit the form with the simulator scenarios).

13. The level of difficulty is appropriate to support licensing decisions for each crew position.

TARGET QUANTITATIVE ATTRlBUTES (PER SCENARIO; SEE SECTION b.4.D) Aptual Attributes - - -

1. Total malfunctions (58) 25 of 27 N U R E G - 1 0 2 1 , Draft Revision 9

ES-301 Transient and Event Checklist Form ES-301-5 OPERATING TEST NO.: Fermi 2 for 9/20/04 - 2004-401 As RO SRO-I As SRO Instructions: (I) Enter the operating test number and Form ES-D-1 event numbers for each evolution type.

(2) Reactivity manipulations may be conducted under normal or controlled abnormal conditions (refer to Section D.5.d) but must be significant per Section C.2.a of Appendix D.

• Reactivity and normal evolutions may be replaced with additional instrument or component malfunctions on a one-for-one basis (3) Whenever practical, both instrument and component malfunctions should be included; only those that require verifiable actions that provide insight to ts competence count toward the minimum requirement.

NRC Reviewer:

NUREG-1021, Draft Revision 9 26 of 27

ES-301 Competencies Checklist Form ES-301-6 SRO RO I BOP Competencies SCENARIO 1 2 3 4 Interpret / Diagnose Events 3,4, 3,4,5, and Conditions 56, 63 7

Comply With and ~ 4 , 4-5, Use Procedures (1) 58, 6,7 Q

Y Operate Control Boards (2)

Communicate and 34, 13, Interact 53, 43, 9 6,7 Demonstrate SuDervisotv 4,6, 13, 8.9 7 Ability (3)

Comply With and 2,4 33 UseTech. Specs. (3) I l l I Notes:

(1) Includes Technical Specification compliance for an RO.

(2) Optional for an SRO-U.

(3) Only applicable to SROs.

Instructions:

Circle the applicant's license type and enter one or more event numbers that will allow the examiners to evaluate ev able competency for every applicant.

Author:

NRC Reviewer:

27 of 27 NUREG-1021, Draft Revision 9

ES-401 Written Examination Form ES-401-6 Quality Checklist Item Description I a I b '

1. Questions and answers technically accurate and applicable to facility

2. a. NRC WAS referenced for all questions

b. Facility learning objectives referenced as available

3. SRO questions are appropriate per Sedion D.2.d of E M 0 1

4. Question selection and duplication from the last two NRC licensing exams appears consistent with a systematic sampling process

5. Question duplication from the license screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate:

- the audit exam was systematically and randomly developed; or the audit exam was completed before the license exam was started; or the examinations were developed independently; or

- the licensee certifies that there is no duplication; or other (explain)

I I I

6. Bank use meets limits (no more than 75 Bank Modified New percent from the bank at least 10 percent new, and the rest modified); enter the actual question distribution at right 48 (64%)

0 (0%)

27

(=%I @ A

7. Between 50 and 60 percent of the questions on Memory CIA the exam (induding 10 new questions) are 35 written at the comprehensiodanalysislevel; enter the actual question distribution at right (47%) %I: ( /&I b-

8. Referenceslhandouts provided do not give away answers fi9-

9. Question content conforms with specific WA statements in the previously approved examination outline and is appropriate for the Tier to which they are assigned; deviations are justified I 10. Question psychometric quality and format meet ES, Appendix B, guidelines Printed Name / Signature Date

a. Author

b. Faciltty Reviewer (*)

c. NRC Chief Examiner (#)

d. NRC Regional Supervisor I Note: The facility reviewer's initials/signature are not applicable for NRC-developed examinations.

1. Independent NRC reviewer initial items in Column 'c;" chief examiner concurrence required.

ES-401 Written Examination Form ES-401-6 Quality Checklist Item Description

1. Questions and answers technically accurate and applicableto facility

2. a. NRC WASreferenced for all questions

b. Facility learning objectives referenced as available

3. SRO questions are appropriate per Section 0.2.d of E W 1

4. Question selection and duplication from the last two NRC licensing exams appears consistent with a systematic sampling process

5. Question duplication from the license screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate:

- the audit exam was systematically and randomly developed; or

- the audit exam was completed before the license exam was started; or X the examinationswere developed independently; or

- the licensee certifies that there is no duplication; or 6.

other (explain)

Bank use meets limits (no more than 75 percent from the bank at least 10 percent new, and the rest modified); enter the actual RO/SRO auestion distributionls\ at rioht

-1 4

I I Modified (2  %.

I

(@?

I 1 I

7. Between 50 and 60 percent of the questions on Memory the RO exam are written at the comprehensiorVanalysislevel; the SRO exam may exceed 60 percent if the randomly selected WAS support the higher cognitiie 9 (=%I levels; enter the actual RO/SRO question distnbution(s) at right

a. Author

b. Facility Reviewer (*)

c. NRC Chief Examiner (#)

d. NRC Regional Supenisor

• The facility reviewer's initialdsignature are not applicable for NRCdeveloped examinations.

1. Independent MRC reviewer initial items in Column "c;" chief examiner concurrence required.

NUREG-1021, Draft Revision 9 30 Of 34

ES-403 Written Examination Grading Form ES-403-1 Quality Checklist Facilitv: Date of Exam: Exam Level: RO/SRO

6. Performance on missed que Date NUREG-1021, Draft Revision 9

ES-501 Post-Examination Check Sheet Form ES-501-1 Post-Examination Check Sheet

-~

Task Description Date Complete

1. Facility written exam comments or graded exams received 1 October 1,2004 and verified complete

2. Facility written exam comments reviewed and incorporated October 6, 2004 and NRC grading completed, if necessary

3. Operating tests graded by NRC examiners I October 6, 2004

4. NRC chief examiner review of operating test and written exam October 7, 2004 grading completed jlfMd4 3 / 8 y

5. Responsible supervisor review completed

6. Management (licensing official) review completed

/D f lliy

7. Licens- mailed P&k/$Y

8. Facility notified of results

/ U

9. Examination report issued (refer to NRC MC 0612)

10. Reference material returned after final resolution of any appeals