ML071340135: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
 
(Created page by program invented by StriderTol)
Line 5: Line 5:
| author name =  
| author name =  
| author affiliation = Entergy Nuclear Operations, Inc
| author affiliation = Entergy Nuclear Operations, Inc
| addressee name = Johnson G A
| addressee name = Johnson G
| addressee affiliation = NRC/RGN-I/DRS/OB
| addressee affiliation = NRC/RGN-I/DRS/OB
| docket = 05000293
| docket = 05000293
Line 22: Line 22:
3 1 L G fc- 1 M Developed by: Written - Facility Date of Examination:
3 1 L G fc- 1 M Developed by: Written - Facility Date of Examination:
3/1 /UY NRC 'L3 // Operating - Facility NRC r? Target Date' -180 -1 20 -120 ~ 3. Facility contact briefed on security and other requirements (C.2.c) -120 4. Corporate notification letter sent (C.2.d)
3/1 /UY NRC 'L3 // Operating - Facility NRC r? Target Date' -180 -1 20 -120 ~ 3. Facility contact briefed on security and other requirements (C.2.c) -120 4. Corporate notification letter sent (C.2.d)
I mk 1-90] 15. Reference material due (C.1.e; C.3.c; Attachment 2)]  
I mk 1-90] 15. Reference material due (C.1.e; C.3.c; Attachment 2)]
: 6. Integrated examination outline(s) due, including Forms ES-201-2, ES-201-3, ES-301-1, ES-301-2. ES-301-5, ES-D-l's, ES-401-1/2, ES-401-3, and ES-401-4. as applicable (C.1.e and f; C.3.d) Examination outline(s) reviewed by NRC and feedback provided to facility licensee (C.2.h: C.3.e)} Proposed examinations (including written, walk-through JPMs, and scenarios, as applicable), supporting documentation (including Forms ES-301-3. ES-301-4, ES-301-5, ES-301-6, and ES-401-6, and any Form ES-201-3 updates), and reference materials due (C.l .e, f. g and h; C.3.d) b# {7. 8 (-75) I-45) -30 9. Preliminary license applications (NRC Form 398's) due (C.l .I; C.2.g; ES-202) -14 10. Final license applications due and Form ES-201-4 prepared (C.1.l; C.2.i; ES-202) -14 11. Examination approved by NRC supervisor for facility licensee review (C.2.h; C.3.f)  
: 6. Integrated examination outline(s) due, including Forms ES-201-2, ES-201-3, ES-301-1, ES-301-2. ES-301-5, ES-D-l's, ES-401-1/2, ES-401-3, and ES-401-4. as applicable (C.1.e and f; C.3.d) Examination outline(s) reviewed by NRC and feedback provided to facility licensee (C.2.h: C.3.e)} Proposed examinations (including written, walk-through JPMs, and scenarios, as applicable), supporting documentation (including Forms ES-301-3. ES-301-4, ES-301-5, ES-301-6, and ES-401-6, and any Form ES-201-3 updates), and reference materials due (C.l .e, f. g and h; C.3.d) b# {7. 8 (-75) I-45) -30 9. Preliminary license applications (NRC Form 398's) due (C.l .I; C.2.g; ES-202) -14 10. Final license applications due and Form ES-201-4 prepared (C.1.l; C.2.i; ES-202) -14 11. Examination approved by NRC supervisor for facility licensee review (C.2.h; C.3.f)
: 12. Examinations reviewed with facility licensee (C.1 .j; C.2.f and h; C.3.g) I kak, 7 13. Written examinations and operating tests approved by NRC supervisor (C.2.i; C.3.h)  
: 12. Examinations reviewed with facility licensee (C.1 .j; C.2.f and h; C.3.g) I kak, 7 13. Written examinations and operating tests approved by NRC supervisor (C.2.i; C.3.h)  
-7 14. Final applications reviewed; 1 or 2 (if >IO) applications audited to confirm qualifications I eligibility; and examination approval and waiver letters sent (C.2.i; Attachment 4; ES-202, C.2.e; ES-204) -7 15. Proctoring/written exam administration guidelines reviewed with facility licensee (C.3.k)
-7 14. Final applications reviewed; 1 or 2 (if >IO) applications audited to confirm qualifications I eligibility; and examination approval and waiver letters sent (C.2.i; Attachment 4; ES-202, C.2.e; ES-204) -7 15. Proctoring/written exam administration guidelines reviewed with facility licensee (C.3.k)
I b.3 -7 16. Approved scenarios, job performance measures, and questions distributed to NRC examiners (C.3.i) Target dates are generally based on facility-prepared examinations and are keyed to the examination da Y e identified in the corporate notification letter.
I b.3 -7 16. Approved scenarios, job performance measures, and questions distributed to NRC examiners (C.3.i) Target dates are generally based on facility-prepared examinations and are keyed to the examination da Y e identified in the corporate notification letter.
They are for planning purposes and may be adjusted on a case-by- case basis in coordination with the facility licensee.  
They are for planning purposes and may be adjusted on a case-by- case basis in coordination with the facility licensee.
[Applies only] {Does not apply) to examinations prepared by the NRC. ES-201, Page 24 of 27 ES-201 Examination Outline Quality Checklist FOI~ ES-201-2 Date of Examination:
[Applies only] {Does not apply) to examinations prepared by the NRC. ES-201, Page 24 of 27 ES-201 Examination Outline Quality Checklist FOI~ ES-201-2 Date of Examination:
Initials a I b'I c# Task Description Item 15-0 !5+\. a. Verify that the outline(s) m(s) the appropriate model, in accordance with ES-401. so 7% 59 ;";c b. Assess whether the outline was systematically and randomly prepared in accordance with Section D.l of EM01 and whether all KIA categories are appropriately sampled. c. Assess whether the outline overemphasizes any systems, evolutions, or generic topics. d. Assess whether the justifications for deselected or rejected WA statements are appropriate.  
Initials a I b'I c# Task Description Item 15-0 !5+\. a. Verify that the outline(s) m(s) the appropriate model, in accordance with ES-401. so 7% 59 ;";c b. Assess whether the outline was systematically and randomly prepared in accordance with Section D.l of EM01 and whether all KIA categories are appropriately sampled. c. Assess whether the outline overemphasizes any systems, evolutions, or generic topics. d. Assess whether the justifications for deselected or rejected WA statements are appropriate.
: 2. S and major transients.
: 2. S and major transients.
I M U L A T 0 R a. Using Form ES-301-5, veri9 that the proposed scenario sets cover the required number of normal evolutions, instrument and component failures, technical specifications.  
I M U L A T 0 R a. Using Form ES-301-5, veri9 that the proposed scenario sets cover the required number of normal evolutions, instrument and component failures, technical specifications.
: b. Assess whether there are enough scenario sets (and spares) to test the projected number and mix of applicants in accordance with the expected crew composition and rotation schedule without compromising exam integrity, and ensure that each applicant can be tested using at least one new or significantly modified scenario, that no scenarios are duplicated from the applicants' audit test(s), and that scenarios will not be repeated on subsequent days. - 3. W I T c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative and quantitative criteria specified on Form ES301-4 and described in Appendix D. - 59 a. Venfy that the systems walk-through outline meets the criteria specified on Form ES-301-2:  
: b. Assess whether there are enough scenario sets (and spares) to test the projected number and mix of applicants in accordance with the expected crew composition and rotation schedule without compromising exam integrity, and ensure that each applicant can be tested using at least one new or significantly modified scenario, that no scenarios are duplicated from the applicants' audit test(s), and that scenarios will not be repeated on subsequent days. - 3. W I T c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative and quantitative criteria specified on Form ES301-4 and described in Appendix D. - 59 a. Venfy that the systems walk-through outline meets the criteria specified on Form ES-301-2:
(1) the outline(s) contain@) the required number of control room and in-plant tasks distributed among the safety functions as specilied on the form J (2) task repetition from the last two NRC examinations is within the limits specified on the form J (3) no tasks are duplicated from the applicants' audit test@)  
(1) the outline(s) contain@) the required number of control room and in-plant tasks distributed among the safety functions as specilied on the form J (2) task repetition from the last two NRC examinations is within the limits specified on the form J (3) no tasks are duplicated from the applicants' audit test@)  
/ (4) the number of new or modified tasks meets or exceeds the minimums specitid on the form/ (5) the number of alternate path, low-power. emergency, and RCA tasks meet the criteria  
/ (4) the number of new or modified tasks meets or exceeds the minimums specitid on the form/ (5) the number of alternate path, low-power. emergency, and RCA tasks meet the criteria
: f. Assess whether the exam tits the appropriate job level (RO or SRO). Date a. Author ;7-/6- c? b. Facillty Reviewer (*) c. NRC Chief Examiner (#) d. NRC Supervisor Note: # Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.
: f. Assess whether the exam tits the appropriate job level (RO or SRO). Date a. Author ;7-/6- c? b. Facillty Reviewer (*) c. NRC Chief Examiner (#) d. NRC Supervisor Note: # Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.
ES-201, Page 25 of 27   
ES-201, Page 25 of 27   
, *', 11. 12. 14 15. 3%
, *', 11. 12. 14 15. 3%
ES-30 1 Operating Test Quality Checklist Form ES-301-3  
ES-30 1 Operating Test Quality Checklist Form ES-301-3
: a. The operating test conforms to the previously approved outline; changes are consistent with sampling requirements (e.g. 10 CFR 55.45, operational importance, safety function distribution).  
: a. The operating test conforms to the previously approved outline; changes are consistent with sampling requirements (e.g. 10 CFR 55.45, operational importance, safety function distribution).  
' b. c. There is no day-to-day repetition between this and other operating tests to be administered during this examination. The operating test shall not duplicate items from the applicants' audit test@) (see Section D.l .a). Overlap with the written examination and between different parts of the operating test is 1 d. within acceDtable limits. 2. WALK-TH ROUGH CRITERIA a. Each JPM includes the following, as applicable:
' b. c. There is no day-to-day repetition between this and other operating tests to be administered during this examination. The operating test shall not duplicate items from the applicants' audit test@) (see Section D.l .a). Overlap with the written examination and between different parts of the operating test is 1 d. within acceDtable limits. 2. WALK-TH ROUGH CRITERIA a. Each JPM includes the following, as applicable:
Line 44: Line 44:
* references and tools, including associated procedures J reasonable and validated time limits (average time allowed for completion) and specific designation if deemed to be time critical by the facility licensee
* references and tools, including associated procedures J reasonable and validated time limits (average time allowed for completion) and specific designation if deemed to be time critical by the facility licensee
* operationally important specific performance criteria that include: - detailed expected actins with exact criteria and nomendature 4 - system response and other examiner cues J - statements describing important observations to be made by the applicant J - criteria for successful completion of the task - identification of critical steps and their associated performance standards J - restrictions on the sequence of steps, if applicable 4 test to deviate from any of tion from the last 2 NRC NRC 70 4 F/3 ;D 25.L Date The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES-3014 and a copy is attached.
* operationally important specific performance criteria that include: - detailed expected actins with exact criteria and nomendature 4 - system response and other examiner cues J - statements describing important observations to be made by the applicant J - criteria for successful completion of the task - identification of critical steps and their associated performance standards J - restrictions on the sequence of steps, if applicable 4 test to deviate from any of tion from the last 2 NRC NRC 70 4 F/3 ;D 25.L Date The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES-3014 and a copy is attached.
Printed Name / Signature 2- $67 zn.-L37 a. Author b. Facility Reviewer (") c. NRC Chief Examiner  
Printed Name / Signature 2- $67 zn.-L37 a. Author b. Facility Reviewer (") c. NRC Chief Examiner
(#) ~ l d. NRC Supervisor iitials NOTE: # The facility signature is not applicable for NRC-developed tests. Independent NRC reviewer initial items in Column "c", chief examiner concurrence required.
(#) ~ l d. NRC Supervisor iitials NOTE: # The facility signature is not applicable for NRC-developed tests. Independent NRC reviewer initial items in Column "c", chief examiner concurrence required.
NUREG-1021, Revision 9 ES-301 Simulator Scenario Quality Checklist Form ES-301-4 I Facilitv: Pilarim Date of Exam: 2/26/07 Scenario Numbers: I / 2 I3 / 4 Operat QUALITATIVE AlTRlBUTES The initial conditions are realistic, in that some equipment andlor instrumentation may be out of service, but it does not cue the operators into expected events.
NUREG-1021, Revision 9 ES-301 Simulator Scenario Quality Checklist Form ES-301-4 I Facilitv: Pilarim Date of Exam: 2/26/07 Scenario Numbers: I / 2 I3 / 4 Operat QUALITATIVE AlTRlBUTES The initial conditions are realistic, in that some equipment andlor instrumentation may be out of service, but it does not cue the operators into expected events.
Line 53: Line 53:
If time compression techniques are used, the scenario summary clearly so indicates. Operators have sufficient time to carry out expected activities without undue time constraints.
If time compression techniques are used, the scenario summary clearly so indicates. Operators have sufficient time to carry out expected activities without undue time constraints.
II 8. The simulator modeling is not altered.
II 8. The simulator modeling is not altered.
evaluated to ensure 12. Each applicant will be significantly involved in the minimum number of transients and events specified on Form ES-301-5 (submit the form with the simulator scenarios).  
evaluated to ensure 12. Each applicant will be significantly involved in the minimum number of transients and events specified on Form ES-301-5 (submit the form with the simulator scenarios).
: 1. Total malfunctions (5-8) 2. Malfunctions after EOP entry (1-2)  
: 1. Total malfunctions (5-8) 2. Malfunctions after EOP entry (1-2)
: 3. Abnormal events (2-4) 4. Major transients (1-2) 5. 6. EOPs enteredlrequiring substantive actions (1-2)
: 3. Abnormal events (2-4) 4. Major transients (1-2) 5. 6. EOPs enteredlrequiring substantive actions (1-2)
EOP contingencies requiring substantive actions (0-2) 7. Critical tasks (2-3)  
EOP contingencies requiring substantive actions (0-2) 7. Critical tasks (2-3)  
Line 62: Line 62:
2/26/07 Operating Test No. NRC 14 Competencies InterpretIDiag-nose Events and Conditions Comply With and Use Procedures (1) Operate Control Boards (2) Communicate and Interact Demonstrate Supervisory Ability (3) Comply With and Use Tech. Specs. (3) Notes: (1) Includes Technical Specification compliance for an RO. (2) Optional for an SRO-U. (3) Only applicable to SROs. - 1 NIA NIA - SCENARIO , Instructions:
2/26/07 Operating Test No. NRC 14 Competencies InterpretIDiag-nose Events and Conditions Comply With and Use Procedures (1) Operate Control Boards (2) Communicate and Interact Demonstrate Supervisory Ability (3) Comply With and Use Tech. Specs. (3) Notes: (1) Includes Technical Specification compliance for an RO. (2) Optional for an SRO-U. (3) Only applicable to SROs. - 1 NIA NIA - SCENARIO , Instructions:
Circle the applicants' license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.
Circle the applicants' license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.
NUREG-I 021, Revision 9 ES-401 Written Examination Quality Checklist Form ES-401-6 Facility: Pilgrim Date of Exam: 2/26/07 Item Description  
NUREG-I 021, Revision 9 ES-401 Written Examination Quality Checklist Form ES-401-6 Facility: Pilgrim Date of Exam: 2/26/07 Item Description
: 1. 2. a. NRC WAS referenced for all questions.  
: 1. 2. a. NRC WAS referenced for all questions.
: 3. 4. Questions and answers technically accurate and applicable to facility.  
: 3. 4. Questions and answers technically accurate and applicable to facility.
: b. SRO questions are appropriate in accordance with Section D.2.d of ES401 The sampling process was random and systematic. (If more than 4 RO and 2 SRO questions were repeated from the last 2 NRC licensing exams, consult the NRR OL Program Office) Question duplication from the license screeninglaudit exam was controlled as indicated below (check the item that applies) and appears appropriate: Facility learning objectives referenced as available.  
: b. SRO questions are appropriate in accordance with Section D.2.d of ES401 The sampling process was random and systematic. (If more than 4 RO and 2 SRO questions were repeated from the last 2 NRC licensing exams, consult the NRR OL Program Office) Question duplication from the license screeninglaudit exam was controlled as indicated below (check the item that applies) and appears appropriate: Facility learning objectives referenced as available.
: 5. the audit exam was systematically and randomly developed the audit exam was completed before the license exam was started the examinations were developed independently the licensee certifies that there is no duplication other (explain)
: 5. the audit exam was systematically and randomly developed the audit exam was completed before the license exam was started the examinations were developed independently the licensee certifies that there is no duplication other (explain)
X 6. Bank use meets limits (no more than 75 percent Bank Modified New from the bank at least IO percent new, and the rest new or modified); enter the actual RO I SRO-only 13 I 8 1 1 2 61 1 15 question distribution(s) at right.
X 6. Bank use meets limits (no more than 75 percent Bank Modified New from the bank at least IO percent new, and the rest new or modified); enter the actual RO I SRO-only 13 I 8 1 1 2 61 1 15 question distribution(s) at right.
Line 76: Line 76:
\ I (9 b I ES-403 Written Examination Grading Form ES-403 Quality Checklist Facility:
\ I (9 b I ES-403 Written Examination Grading Form ES-403 Quality Checklist Facility:
K~G, ( , Date of Exam:
K~G, ( , Date of Exam:
Exam Level: RO SRO 0 Initials Item Description a b C 1. Clean answer sheets copied before grading MQ ,9- *Y " I 2. Answer key changes and question deletions justified and documented  
Exam Level: RO SRO 0 Initials Item Description a b C 1. Clean answer sheets copied before grading MQ ,9- *Y " I 2. Answer key changes and question deletions justified and documented
: 3. Applicants' scores checked for addition errors (reviewers spot check  
: 3. Applicants' scores checked for addition errors (reviewers spot check  
> 25% of examinations) fl9P /r/ 4. Grading for all borderline cases (80 22% overall and170 or 80, As applicable, &4% on the SRO-only) reviewed in detail I 5. All other failing examinations checked to ensure that grades are justified  
> 25% of examinations) fl9P /r/ 4. Grading for all borderline cases (80 22% overall and170 or 80, As applicable, &4% on the SRO-only) reviewed in detail I 5. All other failing examinations checked to ensure that grades are justified
: 6. Performance on missed questions checked for training I I I, deficiencies and wording problems; evaluate validity fig IT(-- I//i of questions missed by half or more of the applicants Printed NameEignature Date a. Grader b. Facility Reviewer(*)  
: 6. Performance on missed questions checked for training I I I, deficiencies and wording problems; evaluate validity fig IT(-- I//i of questions missed by half or more of the applicants Printed NameEignature Date a. Grader b. Facility Reviewer(*)
: c. NRC Chief Examiner  
: c. NRC Chief Examiner
(*) d. NRC Supervisor  
(*) d. NRC Supervisor
(*) (*) The facility reviewer's signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.
(*) (*) The facility reviewer's signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.
ES-403, Page 5 of 5   
ES-403, Page 5 of 5   
\ ES-403 Written Examination Grading Form ES-403-1 Quality Checklist ~~ ~ ~~ ___ ~ ~ ~ Faci\ity:FR (6 ,.A Date of Exam: Exam Level: RO 0 SRO d Initials Item Description a b C 1. Clean answer sheets copied before grading A4 G5c/7 and documented I E3/19- $4 dtfl 2. 3. Answer key changes and question deletions justified Applicants' scores checked for addition errors (reviewers spot check  
\ ES-403 Written Examination Grading Form ES-403-1 Quality Checklist ~~ ~ ~~ ___ ~ ~ ~ Faci\ity:FR (6 ,.A Date of Exam: Exam Level: RO 0 SRO d Initials Item Description a b C 1. Clean answer sheets copied before grading A4 G5c/7 and documented I E3/19- $4 dtfl 2. 3. Answer key changes and question deletions justified Applicants' scores checked for addition errors (reviewers spot check  
> 25% of examinations)  
> 25% of examinations)
: 4. Grading for all borderline cases (80 &2% overall andl70 or 80, As applicable, &4% on the SRO-only) reviewed in detail  
: 4. Grading for all borderline cases (80 &2% overall andl70 or 80, As applicable, &4% on the SRO-only) reviewed in detail
: 5. All other failing examinations checked to ensure that grades are justified  
: 5. All other failing examinations checked to ensure that grades are justified
: 6. Performance on missed questions checked for training deficiencies and wording problems; evaluate validity of questions missed by half or more of the applicants Printed Name/Signature Date a. Grader b. Facility Reviewer(*)  
: 6. Performance on missed questions checked for training deficiencies and wording problems; evaluate validity of questions missed by half or more of the applicants Printed Name/Signature Date a. Grader b. Facility Reviewer(*)
: c. NRC Chief Examiner (*) d. NRC Supervisor  
: c. NRC Chief Examiner (*) d. NRC Supervisor
(*) (*) The facility reviewer's signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.
(*) (*) The facility reviewer's signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.
ES-403, Page 5 of 5}}
ES-403, Page 5 of 5}}

Revision as of 01:22, 13 July 2019

Final Qa/Related Forms (Folder 1)
ML071340135
Person / Time
Site: Pilgrim
Issue date: 02/20/2007
From:
Entergy Nuclear Operations
To: Johnson G
Operations Branch I
Sykes, Marvin D.
Shared Package
ML060800104 List:
References
50-293/07-301, ES-201, ES-201-1 50-293/07-301
Download: ML071340135 (10)
v • d • e


Text

Task Description (Reference)

Chief Examiner's initials 1. 2. Examination administration date confirmed (C.1 .a; C.2.a and b) NRC examiners and facility contact assigned (C.l .d; C.2.e) /sax- Form ES-201-1 T ES-201 Examination Preparation Checklist Facility:

3 1 L G fc- 1 M Developed by: Written - Facility Date of Examination:

3/1 /UY NRC 'L3 // Operating - Facility NRC r? Target Date' -180 -1 20 -120 ~ 3. Facility contact briefed on security and other requirements (C.2.c) -120 4. Corporate notification letter sent (C.2.d)

I mk 1-90] 15. Reference material due (C.1.e; C.3.c; Attachment 2)]

6. Integrated examination outline(s) due, including Forms ES-201-2, ES-201-3, ES-301-1, ES-301-2. ES-301-5, ES-D-l's, ES-401-1/2, ES-401-3, and ES-401-4. as applicable (C.1.e and f; C.3.d) Examination outline(s) reviewed by NRC and feedback provided to facility licensee (C.2.h: C.3.e)} Proposed examinations (including written, walk-through JPMs, and scenarios, as applicable), supporting documentation (including Forms ES-301-3. ES-301-4, ES-301-5, ES-301-6, and ES-401-6, and any Form ES-201-3 updates), and reference materials due (C.l .e, f. g and h; C.3.d) b# {7. 8 (-75) I-45) -30 9. Preliminary license applications (NRC Form 398's) due (C.l .I; C.2.g; ES-202) -14 10. Final license applications due and Form ES-201-4 prepared (C.1.l; C.2.i; ES-202) -14 11. Examination approved by NRC supervisor for facility licensee review (C.2.h; C.3.f)
12. Examinations reviewed with facility licensee (C.1 .j; C.2.f and h; C.3.g) I kak, 7 13. Written examinations and operating tests approved by NRC supervisor (C.2.i; C.3.h)

-7 14. Final applications reviewed; 1 or 2 (if >IO) applications audited to confirm qualifications I eligibility; and examination approval and waiver letters sent (C.2.i; Attachment 4; ES-202, C.2.e; ES-204) -7 15. Proctoring/written exam administration guidelines reviewed with facility licensee (C.3.k)

I b.3 -7 16. Approved scenarios, job performance measures, and questions distributed to NRC examiners (C.3.i) Target dates are generally based on facility-prepared examinations and are keyed to the examination da Y e identified in the corporate notification letter.

They are for planning purposes and may be adjusted on a case-by- case basis in coordination with the facility licensee.

[Applies only] {Does not apply) to examinations prepared by the NRC. ES-201, Page 24 of 27 ES-201 Examination Outline Quality Checklist FOI~ ES-201-2 Date of Examination:

Initials a I b'I c# Task Description Item 15-0 !5+\. a. Verify that the outline(s) m(s) the appropriate model, in accordance with ES-401. so 7% 59 ;";c b. Assess whether the outline was systematically and randomly prepared in accordance with Section D.l of EM01 and whether all KIA categories are appropriately sampled. c. Assess whether the outline overemphasizes any systems, evolutions, or generic topics. d. Assess whether the justifications for deselected or rejected WA statements are appropriate.

2. S and major transients.

I M U L A T 0 R a. Using Form ES-301-5, veri9 that the proposed scenario sets cover the required number of normal evolutions, instrument and component failures, technical specifications.

b. Assess whether there are enough scenario sets (and spares) to test the projected number and mix of applicants in accordance with the expected crew composition and rotation schedule without compromising exam integrity, and ensure that each applicant can be tested using at least one new or significantly modified scenario, that no scenarios are duplicated from the applicants' audit test(s), and that scenarios will not be repeated on subsequent days. - 3. W I T c. To the extent possible, assess whether the outline(s) conform(s) with the qualitative and quantitative criteria specified on Form ES301-4 and described in Appendix D. - 59 a. Venfy that the systems walk-through outline meets the criteria specified on Form ES-301-2:

(1) the outline(s) contain@) the required number of control room and in-plant tasks distributed among the safety functions as specilied on the form J (2) task repetition from the last two NRC examinations is within the limits specified on the form J (3) no tasks are duplicated from the applicants' audit test@)

/ (4) the number of new or modified tasks meets or exceeds the minimums specitid on the form/ (5) the number of alternate path, low-power. emergency, and RCA tasks meet the criteria

f. Assess whether the exam tits the appropriate job level (RO or SRO). Date a. Author ;7-/6- c? b. Facillty Reviewer (*) c. NRC Chief Examiner (#) d. NRC Supervisor Note: # Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.

ES-201, Page 25 of 27

, *', 11. 12. 14 15. 3%

ES-30 1 Operating Test Quality Checklist Form ES-301-3

a. The operating test conforms to the previously approved outline; changes are consistent with sampling requirements (e.g. 10 CFR 55.45, operational importance, safety function distribution).

' b. c. There is no day-to-day repetition between this and other operating tests to be administered during this examination. The operating test shall not duplicate items from the applicants' audit test@) (see Section D.l .a). Overlap with the written examination and between different parts of the operating test is 1 d. within acceDtable limits. 2. WALK-TH ROUGH CRITERIA a. Each JPM includes the following, as applicable:

  • initial conditions J initiating cues
  • references and tools, including associated procedures J reasonable and validated time limits (average time allowed for completion) and specific designation if deemed to be time critical by the facility licensee
  • operationally important specific performance criteria that include: - detailed expected actins with exact criteria and nomendature 4 - system response and other examiner cues J - statements describing important observations to be made by the applicant J - criteria for successful completion of the task - identification of critical steps and their associated performance standards J - restrictions on the sequence of steps, if applicable 4 test to deviate from any of tion from the last 2 NRC NRC 70 4 F/3 ;D 25.L Date The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES-3014 and a copy is attached.

Printed Name / Signature 2- $67 zn.-L37 a. Author b. Facility Reviewer (") c. NRC Chief Examiner

(#) ~ l d. NRC Supervisor iitials NOTE: # The facility signature is not applicable for NRC-developed tests. Independent NRC reviewer initial items in Column "c", chief examiner concurrence required.

NUREG-1021, Revision 9 ES-301 Simulator Scenario Quality Checklist Form ES-301-4 I Facilitv: Pilarim Date of Exam: 2/26/07 Scenario Numbers: I / 2 I3 / 4 Operat QUALITATIVE AlTRlBUTES The initial conditions are realistic, in that some equipment andlor instrumentation may be out of service, but it does not cue the operators into expected events.

I 2. The scenarios consist mostly of related events. 3. Each event description consists of 0 the point in the scenario when it is to be initiated J the symptoms/cues that will be visible to the crew J the expected operator actions (by shift position)

J the event termination point (if applicable)

/ 0 the malfunction(s) that are entered to initiate the event J 0 0 0 No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the scenario without a credible preceding incident such as a seismic event.

II 5. The events are valid with regard to physics and thermodynamics. Sequencing and timing of events is reasonable, and allows the examination team to obtain complete evaluation results commensurate with the scenario objectives.

If time compression techniques are used, the scenario summary clearly so indicates. Operators have sufficient time to carry out expected activities without undue time constraints.

II 8. The simulator modeling is not altered.

evaluated to ensure 12. Each applicant will be significantly involved in the minimum number of transients and events specified on Form ES-301-5 (submit the form with the simulator scenarios).

1. Total malfunctions (5-8) 2. Malfunctions after EOP entry (1-2)
3. Abnormal events (2-4) 4. Major transients (1-2) 5. 6. EOPs enteredlrequiring substantive actions (1-2)

EOP contingencies requiring substantive actions (0-2) 7. Critical tasks (2-3)

~ 4342 ~1 Test No.: NRC Initials NUREG-1021, Revision 9 ES-301 Transient and Event Checklist Form ES-301-5 Facility:

Pilgrim Date of Exam: 2/26/2007 Operating Test No.: NRC E V E N T T Y P E - RX NOR - Scenarios I"" f' : CREW POSITION CREW POSITION 3 4 M I/C SROl1 MAJ TS RX NOR IIC SROl2 MAJ TS RX - - NOR I/C SROl3 MAJ NOR I/C 30 1 MAJ RX - NOR I/C 30 2 NUREG 1021 Revision 9

ES-301 Competencies Checklist Form ES-301-6 SRO SCENARIO 1 2 34 2,3A 3-10 6,7,8, 3-8 9 3'4'5' 63 ALL ALL ALL ALL NIA NIA NIA NIA ALL ALL ALL ALL ALL ALL 1-8 1-6,9 43 2,4 4,6,7 3,4,5 RO (ATC) 1 NIA NIA - T ----t-- NIA I NIA I NIA I NIA 4 NIA NIA - BOP Facility: Pilgrim Date of Examination:

2/26/07 Operating Test No. NRC 14 Competencies InterpretIDiag-nose Events and Conditions Comply With and Use Procedures (1) Operate Control Boards (2) Communicate and Interact Demonstrate Supervisory Ability (3) Comply With and Use Tech. Specs. (3) Notes: (1) Includes Technical Specification compliance for an RO. (2) Optional for an SRO-U. (3) Only applicable to SROs. - 1 NIA NIA - SCENARIO , Instructions:

Circle the applicants' license type and enter one or more event numbers that will allow the examiners to evaluate every applicable competency for every applicant.

NUREG-I 021, Revision 9 ES-401 Written Examination Quality Checklist Form ES-401-6 Facility: Pilgrim Date of Exam: 2/26/07 Item Description

1. 2. a. NRC WAS referenced for all questions.
3. 4. Questions and answers technically accurate and applicable to facility.
b. SRO questions are appropriate in accordance with Section D.2.d of ES401 The sampling process was random and systematic. (If more than 4 RO and 2 SRO questions were repeated from the last 2 NRC licensing exams, consult the NRR OL Program Office) Question duplication from the license screeninglaudit exam was controlled as indicated below (check the item that applies) and appears appropriate: Facility learning objectives referenced as available.
5. the audit exam was systematically and randomly developed the audit exam was completed before the license exam was started the examinations were developed independently the licensee certifies that there is no duplication other (explain)

X 6. Bank use meets limits (no more than 75 percent Bank Modified New from the bank at least IO percent new, and the rest new or modified); enter the actual RO I SRO-only 13 I 8 1 1 2 61 1 15 question distribution(s) at right.

Between 50 and 60 percent of the question on the RO exam are written at the comprehension/

analysis level; the SRO exam may exceed 60 percent if the randomly selected WAS support the higher cognitive levels; enter the actual RO I SRO question distribution(s) at right Referenceslhandouts provided do not give away answers or aid in the elimination of distractors. Question content conforms with specific WA statements in the previously approved examination outline and is appropriate for the tier to which they are assigned; deviations are justified. Question psychometric quality and format meet the guidelines in ES Appendix B. CIA 7. Memo 33 I 10 42 115 8. 9. IO. 11. The exam contains the required number of one-point, multiple choice items; the total is correct and agrees with value on cover sheet.

P 'nted Name I Signature Pr n Exam Level: ROlSRO a so 59 59 50 5'9 sn 50 SD so SD Initial b* ac R- E Date 211 6107 a. Author b. Facility Reviewer (9 c. NRC Chief Examiner (#) d. NRC Regional Supervisor Note:

  • The facility reviewer's initialslsignature are not applicable for NRC-developed examinations.
  1. Independent NRC reviewer initial items in Column "c"; chief examiner concurrence required.

NUREG-1021, Revision 9

\ I (9 b I ES-403 Written Examination Grading Form ES-403 Quality Checklist Facility:

K~G, ( , Date of Exam:

Exam Level: RO SRO 0 Initials Item Description a b C 1. Clean answer sheets copied before grading MQ ,9- *Y " I 2. Answer key changes and question deletions justified and documented

3. Applicants' scores checked for addition errors (reviewers spot check

> 25% of examinations) fl9P /r/ 4. Grading for all borderline cases (80 22% overall and170 or 80, As applicable, &4% on the SRO-only) reviewed in detail I 5. All other failing examinations checked to ensure that grades are justified

6. Performance on missed questions checked for training I I I, deficiencies and wording problems; evaluate validity fig IT(-- I//i of questions missed by half or more of the applicants Printed NameEignature Date a. Grader b. Facility Reviewer(*)
c. NRC Chief Examiner

(*) d. NRC Supervisor

(*) (*) The facility reviewer's signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.

ES-403, Page 5 of 5

\ ES-403 Written Examination Grading Form ES-403-1 Quality Checklist ~~ ~ ~~ ___ ~ ~ ~ Faci\ity:FR (6 ,.A Date of Exam: Exam Level: RO 0 SRO d Initials Item Description a b C 1. Clean answer sheets copied before grading A4 G5c/7 and documented I E3/19- $4 dtfl 2. 3. Answer key changes and question deletions justified Applicants' scores checked for addition errors (reviewers spot check

> 25% of examinations)

4. Grading for all borderline cases (80 &2% overall andl70 or 80, As applicable, &4% on the SRO-only) reviewed in detail
5. All other failing examinations checked to ensure that grades are justified
6. Performance on missed questions checked for training deficiencies and wording problems; evaluate validity of questions missed by half or more of the applicants Printed Name/Signature Date a. Grader b. Facility Reviewer(*)
c. NRC Chief Examiner (*) d. NRC Supervisor

(*) (*) The facility reviewer's signature is not applicable for examinations graded by the NRC; two independent NRC reviews are required.

ES-403, Page 5 of 5

Retrieved from "https://kanterella.com/w/index.php?title=ML071340135&oldid=1825935"