ML032480901

Various Checklists for the Retake Examination - July 7, 2003

ML032480901
Person / Time
Site:	Fermi
Issue date:	07/07/2003
From:	Lanksbury R NRC/RGN-III/DRS/OLB
To:	Detroit Edison
References
50-341/03-302
Download: ML032480901 (15)

Text

VARIOUS CHECKLISTS FOR THE FERMI RETAKE EXAMINATION - JULY 7,2003

ES-201 Examination Preparation Checklist Form ES-201-1 Target Date*

Mpid~q &+cj ke K Y Q ~

Task Description / Reference

-1 20

-1 20

[-go]

~~

~

~

~

3. Facility contact briefed on security & other requirements (C.2.c)

4. Corporate notification letter sent (C.2.d)

[5. Reference material due (C.1.e; C.3.c)l Date of Examination: 7-7- 03 Chief Examiner's Initials

1)

-1 80 I 1. Examination administration date confirmed (C.l.a; C.2.a & b) 11

-1 20 I 2. NRC examiners and facility contact assigned (C.1.d; C.2.e) 5e 6P 11

-75 I 6. Integrated examination outline(s) due (C.1.e & f; C.3.d)

-70

7. Examination outline@) reviewed by NRC and feedback provided

8. Proposed examinations, supporting documentation, and

9. Preliminary license applications due (C.1.I; C.2.g; ES-202)

10. Final license applications due and assignment sheet prepared to facility licensee (C.2.h; C.3.e) reference materials due ((3.1.e, f, g & h; C.3.d)

(C.l.I; C.2.g; ES-202)

-45

-30 P P

-1 4 64 1 1. Examination approved by NRC supervisor for facility licensee review (C.2.h; C.3.9

12. Examinations reviewed with facility licensee (C.l.j; C.2.f & h; C.3.g)

II

-7

13. Written examinations +

,c//t NRC supervisor (C.2.i; C.3.h) o f il

-7

14. Final applications reviewed; assignment sheet updated; waiver letters sent (C.2.g, ES-204)

15. Proctoring/written exam administration guidelines reviewed with facility licensee and authorization granted to give written exams (if applicable) (C.3.k)

OP

-7

16. Approved scenarios, job performance measures, and questions

-7 I distributed to NRC examiners (C.3.i)

Target dates are keyed to the examination date identified in the corporate notification letter.

They are for planning purposes and may be adjusted on a case-by-case basis in coordination with the facility licensee.

Applies only to examinations prepared by the NRC.

[ ]

NUREG-1021, Revision 8, Supplement 1 22 of 24

Detroit Edison July 17,2003 NANT-03 -0 1 10 080 1.26 Mr. B. Palagi Region III U. S. Nuclear Regulatory Commission 80 1 Warrenville Road Lisle, IL 60532-4351

Dear Mr. Palagi:

Fermi 2 administered an initial written retake license exam on July 7, 2003. Enclosed please find the remaining material required in accordance with NUREG-102 1, ES-50 1.

Original Form ES-201-3, "Examination Security Agreement," with a pre and post-examination signature for each individual who had detailed knowledge of the written examination 0

If you have any questions or comments regarding the contents of these materials please contact Mr. AI Mann at 734-586-4067.

Sincerely, Larry'D. d&ders Manager, Nuclear Training Fermi 2 Enclosure

x Examination Security Agreement Form ES-201-3 ES-201

1.

Pre-Examination I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of 717 /AS as of the date of my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the NRC. Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have been compromised.

2.

Post-Examination To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered during the week(s) of 7/7/03. From the date that I entered into this security agreement until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted below and authorized by the NRC.

PRINTED NAME JOB TiTLE I RESPONSIBILITY SIGNATURE (I)

DATE SIGNATURE (2) DATE NOTE NOTES:

25 of 25 NUREG-I 021, Draft Revision 9

SECURITY AGREEMENT CLARIFICATION SIMULATOR DYNAMIC EXAMINATION PRINT NAME PHE-EXAMINATION CERTIFICATION DATE~TIME SUPERVISOR POST-EXAMINATION CERTIFICATION DATE~TIME SIGNATURE INITIALS SIGNATURE JPM EXAMINATION PRINT NAME PRE-EXAMINATION CERTIFICATION DATE~TIME SuPERvrson POST-EXAMINATION CERTIFICATION DATE~TIME SIGNATURE INITIALS SIGNATURE I

I I

I I

DTC: TPMMNT DSN: MNT04008 Rev. 1 P2/2 File: 1703.22

\\

c SECURITY AGREEMENT CLARIFICATION c

WKITTEN EXAMINATION PRINT NAME 1

PRE-EXAMINATION CERTIFICATION I DATERIME 1

SUPERVISOR I POST-EXAMINATION CERTIFICATION I DATERIME I DTC. "DMMNT DSN: MNT04008 Rev. 1 P2/2 File: 1703.22

6' SECURITY ACREEhENT CLARIFICATION 1

WRITTEN EXAMINATION DATE/TIME PRINT NAME PRE-EXAMINATION CERTIFICATION DATE/TIME SUPERVISOR POST-EXAMINATION CERTIFICATION SIGNATURE INITIALS SIGNATURE KE O b k

/2W&

on\\o 7f-l 0.3 1 025

b. zc-CS /

J?AA SIMULATOR DYNAMIC EXAMINATION DATE/TIME PRINT NAME PRE-EXAMINATION CERTIFICATION DATE/TIME SUPERVISOR POST-EXAMINATION CERTIFICATION SIGNATURE INITIALS SIGNATURE I

A IN 6~I1103~;rdkb 7/8/03 flsr I

I PUE-EXAMINATION CERTIFICATION DATE/TIME SIGNATURE 1

I I

1 SUPERVISOR POST-EXAMINATION CERTIFICATION DATE~IME INITIAIS SIGNATURE DTC: TPMMNT DSN: MNT04008 Rev. 1 P2/2 File: 1703.22

W W

SECURITY AGREEMENT ROSTER EXAMINATION PERIOD --

7/7/05 to 7h3 '

DTC: TPMMNT DSN: MNT04009 Rev. 1 P1/1 File: 1703.22Date: 101195 DTC: VSNTDP DSN:

File: 0210.06.04 IP: I

ES-401

6.

Bank use meets limits (no more than 75 percent from the bank at least 10 percent new, SRO-only question distribution at right and the rest modified); enter the actual RO /

Written Examination Qualitv Checklist Bank Modified New 35 9

3/

Form ES-40 1-6

7.

Between 50 and 60 percent of the questions on the RO exam are written at the Facility: Fermi 2 Date of Exam: 07107l03 Exam Level: RO/SRO Memoty CIA I

comprehension/analysis level; the SRO Exam may exceed 60 percent if the randomly selected WAS support the higher cognitive levels; enter the actual question distribution at right 33

+L

( 44%)

( 56 %)

Printed Name I Sianature Initial

• P

a. Author

b. Facility Reviewer (*)

c. NRC Chief Examiner (#)

d. NRC Regional Supervisor Date Note:

• The facility reviewers initials/signature are not applicable for NRC-developed examinations.

1. Independent NRC reviewer initial items in Column c; chief examiner concurrence required.

ES-403 Written Examination Grading Form ES-403-1 Quality Checklist Item Description Facility: Fermi 2 Date of Exam: 07/07/03 Exam Level: 80 Initials a

b c

2.

3.

Answer key changes and question deletions justified and Applicants' scores checked for addition errors documented AloAlJ& o f (reviewers spot check 25% of examinations)

1.

Clean answer sheets copied before grading I

p.

3.

Grading for all borderline cases (80% +/- 2% overall and 70 +/- 4% on the SRO-only) reviewed in detail 92%@ i$4

$v oe

4.

5.

All other failing examinations checked to ensure that grades are justified A I O M G ~ ~ ~

Performance on missed questions checked for training deficiencies and wording problems; evaluate validity of questions missed by half or more of the applicants

a. Grader >

R I 7-7-03

b. Facility Reviewer(*)

c. NRC Chief Examiner (*)

(./A 6P w P oe

(*)

The facility reviewer's signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.

5 of 5 NUREG-1021, Draft Revision 9

f

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other s1#

LOK LOD (F/H)

(1-5) Stem Cues T/F Cred. Partial Job-Minutia

1. /

Back-Q= SRO Focus Dist.

Link units ward WA Only 1

H 3

X 2

F 2

3 F

3 4

F 2

5 H

3 X

6 F

2 7

F 2

8 H

4 X

9 H

4 10 H

3 X

CY

6.

7.

U/E/S Explanation E/S

1. Distractor D. not credible RESOLUTION: Distractor changed s

s E

S U

S s

S U

S s

S Question is Fundamental not Higher cognitive level. Also editorial comment on stem. RESOLUTION: stem changed Does not ask question for WA called for in outline RESOLUTION: In stem trip point exceeded with no scram = KA Can we expect the candidate to know the limit for Stator cooling water out let temperature? RESOLUTION: Stem changed ES-401 Written Examination Form ES-401-9 (R8, Si)

Review Worksheet

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1.

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2.

Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3.

Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or dist,ractors contain cues (!.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

More than one distractor is not credible.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4.

Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (Le., the question has a valid WA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (Le., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5.

Check questions that are sampled for conformance with the approved WA and those that are desiqnated SRO-only (WA and license level mismatches are unacceptable).

6.

Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

7.

At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

• Question reviewed for 30 question WA match

1. Question reviewed for 30 question psychometric quality review

I' ES-401 2

Form ES-401-9 (R8, S1)

1. I 2.

I

3. Psychometric Flaws I

4. Job Content Flaws I 5. Other I

6.

7.

Explanation

• 1. A. IS NOT CREDIBLE, D. ALSO A CORRECT ANSWER RESOLUTION: Distractors A and D changed

• 1. C&D NOT CREDIBLE RESOLUTION: Distractors C and D changed Edit to add procedure names. RESOLUTION: done

• Question reviewed for 30 question WA match

1. Question reviewed for 30 question psychometric quality review

ES-401 2

Form ES-401-9 (R8, S1)

1.

2

3. Psychometric Flaws 4 Job Content Flaws Q#

LOK LOD (F/H)

(1-5) Stem Cues T/F Cred. Partial Job-Minutia

1. /

Back-Focus Dist Link units ward 31 F

2 32 H

2 33 F

2 X

34 H

2 35 H

2 36 H

2 X

37 H

3 38 F

2 39 F

2 X

40 F

3 41 F

4 X

X 42 H

3 43 H

3 44 H

3 45 F

3 46 H

4 47 H

2 48 H

3 49 H

3 50 H

4 5 Other

6.

7 Q= SRO U/E/S Explanation WA Only s

S U

S changed S

S U/S E/s S

• 1. C IS THE ONLY CREDIBLE ANSWER RESOLUTION: Distractors

• # A & C NOT CREDIBLE RESOLUTION: Distractors changed Edit stem for clarity RESOLUTION: done E/S

• 1. A. NOT CREDIBLE RESOLUTION: Distractor changed S

U S

correct. RESOLUTION: Distractors changed S

S s

S s

S S

S S

C & B require candidate to remember alarm set points A also

• Question reviewed for 30 question WA match

1. Question reviewed for 30 question psychometric quality review

I ES-401 2

Form ES-401-9 (R8, Sly

• Question reviewed for 30 question WA match

1. Question reviewed for 30 question psychometric quality review

ES-401 2

Form ES-401-9 (R8, SI)

• Question reviewed for 30 question WA match

1. Question reviewed for 30 question psychometric quality review