ML052580617

IR 50456457-05-008, Braidwood Requalification Program Inspection Plan (IP 71111.11B)

ML052580617
Person / Time
Site:	Braidwood
Issue date:	09/12/2005
From:	Mcneil D, Hironori Peterson, Richard Skokowski NRC/RGN-III/DRP/RPB3, Division of Reactor Safety III
To:
References
IR-05-008
Download: ML052580617 (12)

Text

LICENSED OPERATOR REQUALIFICATION PROGRAM INSPECTION PLAN Facility: Braidwood Nuclear Station Dates: (Prep) August 29 - September 2, 2005; (On-Site Inspection) September 12 - 16, 2005 Inspectors: Dell McNeil, (Lead Inspector); Chuck Zoia (Inspector)

Inspection Report No: 50-456/457-2005-008 (Input to SRI Integrated Report); IP 71111.11B (Biennial)

DRP Notified: Yes, Branch Chief and Resident Staff via face-to-face or telephone Licensee Notified: Yes, Inspection Letter and Verbally; Carl B. Dunn, Trg Director. Ph# 815-417-4000 Inspection Type: U Baseline Supplemental Event Response Other PPR/PIM Weaknesses: None SRA INPUT: None Inspection Focus IP No./(Section #)

Bold Priority 1 Completion Facility Operating History: Assess operator performance since the last requalification program evaluation to determine if performance deficiencies have been addressed through requal training 71111.11 (02.03) program.

Licensee Requalification Examinations: Assess the adequacy of the facility licensees written examinations and operating tests for Requalification. 71111.11 (02.04)

Licensee Administration of Requalification Examinations: Observe examinations and tests in progress and interview personnel to assess the facility licensee's effectiveness in conducting written examinations and operating tests to ensure operator mastery of the requalification training program 71111.11 (02.05) content.

Licensee Training Feedback System: Assess the effectiveness of the facility licensees training feedback process for revising and maintaining its licensed operator continuing training program up to 71111.11 (02.06) date, including the use of feedback from plant events and industry experience information.

Licensee Remedial Training Program: Assess the adequacy and verify the effectiveness of the remedial training conducted since the last requalification examination and the training planned for the current examination cycle to ensure that it addresses weaknesses in licensed operator or crew 71111.11 (02.07) performance identified during training and plant operations.

Conformance With Operator License Conditions: Review the licensees program for maintaining active operator licenses and ensuring the medical fitness os its licensed operator. Assess the facility 71111.11 (02.08) and operator licensees compliance with 10 CFR 55.53.

Written Examination and Operating Test Results: For each Requal cycle, review the number of applicants and pass/fail rates of written examinations, individual operating tests, and simulator 71111.11 (02.09) operator tests.

Conformance With Simulator Requirements Specified in 10 CFR 55.46: Assess the adequacy of the facility licensees simulation facility for use in operator licensing examinations and for satisfying experience requirements as prescribed in 10 CFR 55.46. Assess the effectiveness of the facility 71111.11 (02.11) licensees process for continued assurance of simulator fidelity with regard to identifying, reporting, correcting, and resolving simulator discrepancies via a corrective action program.

Other items as discussed with DRP and Resident Staff, as necessary.

Plan Prepared By: Date:

Dell R. McNeil, Reactor Inspector, Operators Branch Plan Approved By: Date:

Hironori Peterson, Chief, Operators Branch Plan Reviewed By: Date:

R. Skokowski, Chief, Reactor Projects Branch 3

INSPECTION PLAN BRAIDWOOD NUCLEAR PLANT REQUALIFICATION TRAINING PROGRAM I. Inspection Objectives (Module IP 71111 Attachment 11):

01.01 To verify that the facility licensee's requalification program for licensed reactor operators (ROs) and senior reactor operators (SROs) ensures safe power plant operation by adequately evaluating how well the individual operators and crews have mastered the training objectives, including training on high-risk operator actions with senior reactor analysts (SRAs) input.

01.02 To assess the facility licensee's effectiveness in evaluating and revising the requalification program for licensed operators based on their operational performance, including requalification examinations.

01.03 To assess the facility licensee's effectiveness in ensuring that the individuals who are licensed to operate the facility satisfy the conditions of their licenses as specified in 10 CFR 55.53.

01.04 To supply regional management with the information necessary to assess the performance of the facility licensee's licensed operator requalification program and determine the need for additional inspections or NRC-conducted examinations.

L In addition, to assess the adequacy of facility licensees simulation facility for use in operator licensing examinations and for satisfying experience requirements as prescribed in simulator regulation 10 CFR 55.46.

II. Inspection Schedule:

A. Partial in-office inspection preparation and review will be conducted, as necessary, during the week of August 30, 2005. Overall documents to be reviewed include past operating tests and written examinations, and the requalification training program governing documents.

B. On site inspection activities will be conducted during September 12 - 16, 2005, to support the licensees requalification schedule.

1. Inspector badging - Complete.

2. Inspection requirements per IP 71111 Attachment 11 with assignment as licensee's examination schedule permits (see Attached schedule).

3. Debriefing meeting with Training Staff at end of each day. Exit meeting tentatively scheduled for 2 pm on September 16, 2005.

C. Working Hours: In order to observe the examination process, it may be necessary to adjust individual normal working hours. Overtime hours will be worked as needed. Overtime is not anticipated.

III. Team Members:

Dell McNeil, Reactor Engineer (Lead Inspector)

C. Zoia, Reactor Inspector IV Inspection Assignments: In Accordance With IP 71111.11B Inspectors should prioritize their activities to ensure that inspection requirements 02.03, 02.04, 02.05, 02.09, and 02.11 are completed first. Inspection requirements 02.06, 02.07, and 02.08 are to be considered and performed to the extent necessary to conclude that the objectives of the inspection procedure have been met. (Note:

Section 02.10 is for Quarterly Resident Staff Observations)

V. Inspection Documentation:

During the inspection process maintain a record of all documents reviewed including the revision number. All potential findings will be documented with all the specifics (who, what, where, when, how, etc.) and provided to the lead inspector prior to the end of each day.

Each inspector will be responsible for writing detailed observations of their findings which provides all the specifics regarding their inspection activities assigned. Brief summaries of the findings shall be provided to the lead inspector prior to the exit meeting. The overall results will be incorporated into the Resident Integrated Inspection Report # 2005-08.

VI. Starfire Information:

Inspection Report No.: 05000456/2005008; 05000457/2005008 (SRI Integrated Report)

IPE code: BI (Baseline Inspection); BIP (Prep); BID (Documentation)

AT (Inspection Related Travel)

Inspection Procedure: 2515/7111111B

BRAIDWOOD REQUALIFICATION INSPECTION PROPOSED SCHEDULE WEEK OF SEPTEMBER 12, 2005 WEEK PRIOR TO INSPECTION WEEK:

In office review of licensees examination material. Final logistics and preparation.

INSPECTION WEEK:

MONDAY 9/12:

Morning Travel to Site. Inspect requalification written and operating examinations.

Entrance Mtg.

Afternoon Training Program Review Review remainder of exam material and results of last years exam.

Begin Simulator verification inspection Observe simulator training for effectiveness/overlap with the annual op test.

TUESDAY 9/13:

Morning Review Plants operating history (PIM, Inspection Reports, Exam Reports, Requal Inspection Reports, LERs, In-house Audits, etc...)

Complete simulator verification inspection.

Afternoon Training Program Review (Medical, License Maintenance)

WEDNESDAY 9/14:

All Day Observe annual operating test (dynamic simulator scenarios), interviews.

THURSDAY 9/15:

All Day Observe annual operating test (dynamic simulator scenarios), interviews.

FRIDAY 9/16:

Morning Observe annual operating test (JPMs).

Review findings with Senior Resident Inspector.

2:00 PM Exit Meeting with Licensee Management.

Afternoon Return Travel.

NOTE: Specifics of the inspection schedule may vary, as schedule and time permits.

REQUAL INSPECTION ASSIGNMENT SHEET & CHECKLIST FACILITY: BRAIDWOOD DATE: August 25-30, 2003 Inspectors: D. McNeil (DM); C. Zoia (CZ)

INSPECTOR ACTIVITY STATUS*

CZ Review licensees operating history (SALP, RI reports and observations / input)

DM Review last QA audit on the requal program (feedback system)

DM/CZ Review licensees requal exam administration/evaluation practices (IP 71111.11B, section 03.05) [Check that operators are being evaluated in their appropriate licensed positions; check critical task assignments -how, procedure]

DM/CZ Review requal exams - Written parts A and B (use Appendix A)

DM/CZ Review requal exams - JPMs (use Appendix A)

DM/CZ Review requal exams - Scenarios (use Appendix A)

DM/CZ Review remedial training program DM Review program for maintaining active licenses (how time is accounted for, compliance with 10 CFR 55.53(e) and (f))

DM/CZ Review program for maintaining medical requirements (compliance with 10 CFR 55.53(i) [Focus on ANSI/ANS - 3.4, 1983 or 1996]

DM/CZ Interview Operations and Training staff (Appendix B as guidance)

DM/CZ Review Simulator Requirements Specified in 10 CFR 55.46 (use Appendix C and NRR OLB Guidance Memo)

CZ Review Exam Security program (both initial and requal) 10 CFR 55.49. (How security is maintained on simulator scenarios and JPM simulator setups - if saved on simulator computer what are the security protocol (procedures)?)

CZ Review program for testing/maintaining UFSAR EOP time requirements DM/CZ Review any concern items from SRI or from last Requal Inspection Report DM/CZ Control Room Observations w/ RI (as schedule permits)

• STATUS: Done, Defered, N/A, mark as needed

APPENDIX A Checklist for Evaluating Facility Testing Material Yes/No

____ 1. Does each question have a documented link to important licensee tasks, K/As, and/or facility learning objectives?

____ 2. Is each question operationally oriented (i.e., is there a correlation between job demands and test demands)?

____ 3. Is each question written at the appropriate level of knowledge (fundamental knowledge, comprehension, or application/analysis)? Refer to Appendix B, Written Examination Guidelines, of NUREG-1021, Operator Licensing Examinations for Power Reactors, for guidance.

____ 4. Is the context of the questions realistic and free of window dressing and backwards logic?

____ 5. Does the question possess a high K/A importance factor (3 or greater) for the job position?

____ 6. Does the question appear to have the ability to discriminate between an operator who possesses a satisfactory level of safety significant knowledge and an operator who does not?

____ 7. Is each question appropriate for the written examination and the selected written examination format (e.g., short answer; multiple choice)?

____ 8. Does any question have the potential of being a "double-jeopardy" question?

____ 9. Is each question clear, precise, and easy to read and understand?

____ 10. Is there only one correct answer to each question?

____ 11. Does each question pose situations and problems that differ from those presented during training?

ADDITIONALLY, FOR OPEN-REFERENCE QUESTIONS:

____ 1. Does the question require an appropriate use of references (i.e., use of analysis skills or synthesis of information either to discern what procedures were applicable or to consult the procedures to obtain the answer)?

____ 2. Is the question a "direct look-up" question (i.e., one that immediately directs an operator to a particular reference where the answer is readily available?

____ 3. Are there questions given in a static scenario setup that takes advantage of the simulator control room setting?

Job Performance Measure (JPM) Quality Checklist Yes/No

____ 1. Is each task supported by facility's job task analysis?

____ 2. Is each task operationally important (i.e., meets threshold criterion of K/A at 3 or above or as determined by the facility)?

____ 3. Is each task designed as either SRO only, RO/SRO or AO/RO/SRO?

4. Does each JPM include: (Refer to Appendix C, Job Performance Measures Guidelines, of NUREG-1021, Operator Licensing Examinations for Power Reactors, for guidance.)

____ a. Initial conditions

____ b. Initiating cues

____ c. References, including associated procedures

____ d. Performance standards which are specific in that exact control and indication nomenclature and criteria (switch position, meter reading) are specified, even if these criteria are not specified in the procedural step

____ e. System response cues in the performance standards that are complete and correct so that the examiner can properly cue the operator, if asked

____ f. Statements describing important actions or observations that should be made by the operator

____ g. Criteria for successful completion

____ h. Identification of the critical steps and their associated performance standards

____ i. Validated time limits (average time allowed for completion)

____ j. JPMs identified as time critical or not time critical

____ k. Restrictions on the sequence of steps

Simulator Scenario Review Checklist Qualitative Attributes Yes/No

____ 1. Is each scenario of the appropriate scope, depth, and complexity with clearly stated objectives? (Refer to Appendix D, Simulator Testing Guidelines, of NUREG-1021 for guidance.

____ 2. Are the initial conditions realistic? (i.e., in that some equipment and/or instrumentation may be out of service, but it does not cue crew into expected events?)

____ 3. Does the scenario consist mostly of related events?

4. Does each scenario event description include:

____ a. The point in the scenario when it is to be initiated?

____ b. The malfunction(s) that are entered to initiate the event?

____ c. The symptoms/cues that will be visible to the crew?

____ d. The expected operator actions (by shift position)?

____ e. The event termination point?

____ 5. Is no more than one non-mechanistic failure (e.g., pipe break) incorporated into the scenario without a credible preceding incident such as a seismic event?

____ 6. Is each event valid with regard to physics and thermodynamics?

____ 7. Is the sequencing/timing of events reasonable, and does it allow for the examination team to obtain complete evaluation results commensurate with the scenario objectives?

____ 8. Has the simulator modeling been altered?

____ 9. Can each rating factor in each crew competency be evaluated?

____ 10. Has the scenario been validated?

____ 11. If the sampling plan indicates that the scenario was used for training during the requalification cycle, has the facility determined whether it should be modified or not used?

Simulator Scenario Review Checklist (continued)

Note: The following criteria list scenario traits that are numerical in nature. A second set of numbers indicates a range to be met for a set of two scenarios. Therefore, to complete this part of the review, the set of scenarios must be available.

Quantitative Attributes Yes/No

____ 1. Total malfunctions inserted: 4 to 8 / 10 to 14

____ 2. Malfunctions that occur after EOP entry: 1 to 4 / 3 to 6

____ 3. Abnormal Events: 1 to 2 / 2 to 3

____ 4. Major Transients: 1 to 2 / 2 to 3

____ 5. EOPs used beyond primary scram response EOP: 1 to 3 / 3 to 5

____ 6. EOP Contingency Procedures used: 0 to 3 / 1 to 3

____ 7. Approximate scenario run time: 45 to 60 minutes (one scenario may approach 90 minutes)

____ 8. Crew Critical Tasks: 2 to 5 / 5 to 8

____ 9. Are Technical Specifications exercised during the test?

COMMENTS:

END

APPENDIX B Activity Suggested Interview Topics/Questions General Former positions at the facility: How long? Licensed?

Current position and duties: How long? Licensed? Requalification program responsibilities?

03.02 & 03.03 Examinations: How developed? Sampling plan? Appropriate coverage?

- License level? Practiced/covered in training? Duplicate quizzes? Too Easy/Hard? Too long/short? Were references necessary? How compare Exams, with NRC exams?

performance standards, Performance standards: How are they formulated? Operations versus simulator, and Training? Are they endorsed by management? Are they objective? How security are they communicated to evaluators? Do the operators know what is expected of them? Are they applied consistently?

Performance feedback: Is it timely? Is it objective? What happens if you fail? How could feedback be improved?

Administration: Operating/training crew = test crew? What if you miss an exam? Measures to mitigate undue stress? Less stressful than NRC?

Simulator: Does it respond correctly? Is hardware current? Any negative training?

Security: Are exams common? How is security ensured? Are there formal procedures? Who is responsible? Do you feel comfortable with process? Do security measures cause undue stress? Are you aware of any incidents? What would you change if you could?

03.04 - Feedback collection: How is it done? Who collects comments? Who is solicited? Does the QA/QC department oversee the program?

Feedback system Comment resolution: Who does it? Are they timely? Safety basis for changes? How is management involved? How are changes promulgated? Were they to your satisfaction? Feedback to originator?

Recent examples?

Overall, how effective is your training program? The examination program? The feedback system? How would you improve it?

03.05 - Program development: How are remedial training needs identified?

Individual/crew exam results? On-the-job performance/events? Generic Remedial weaknesses? Who develops remedial training programs? How is training Operations involved?

program Implementation: Is it appropriate? Is it effective? How is remediation verified? What would you change if you could?

APPENDIX C (IP 7111111B)

SIMULATOR FIDELITY GUIDANCE IP 71111.11 Simulator Fidelity Inspection Guidance for Sections 2.11, 3.11, and Appendix C

This is not intended to replace IP 71111.11 but to provide guidance to inspectors who are responsible for conducting this part of the inspection for the first time.

Meet w/ Simulator Software Lead

1. Re-emphasize the scope of the simulator inspection. (Scope statement is in Section 02.11.)

2. Inquire if the licensee has read and understood the revised IP. (If not ensure they have a copy and walk them through the new section 02.11, 03.11, and Appendix C)

3. Ask if they have any questions regarding the new sections.

4. Inquire as to what ANSI 3.5 Standard the simulator is operating under.

5. Inquire if the plant-referenced simulator has been used for reactivity manipulations for initial license applicants to meet eligibility requirements.

Recommended Simulator Inspection Protocol

1. Otain hard copy of organization chart, in particular, location of people on the simulator staff.

2. Request/obtain hard copy of Simulator Management and Configuration Procedure(s) -

i.e., the administrative procedure, including the procedure that governs simulator discrepancies (prioritization scheme, corrective action process and resolution schedule),

and performance testing.

3. Request list of all open simulator discrepancies (DR-s). (Review for importance relative to impact on 10 CFR 55.45 & 59 operator actions as well as nuclear & thermal hydraulic operating characteristics.)

4. Request list of closed DRs for the last 12 months.(Review for same type of issues as open DR-s and timeliness for resolution.)

5. Request list of simulator performance tests:(Test results should meet acceptance criteria

- confirm that significant DR-s are processed in the licensee's simulator corrective action program)

Transient Tests - Review at least 2 (suggest Rx trip at 100% power and all RCP pumps trip)

Malfunction Tests - Review at least 5 (2 or 3 malfunctions can be from the requal operating test scenarios.) 3-4 of the malfunction tests should relate to nuclear and thermal-hydraulic operating characteristics and 1 -2 of the tests should relate to extensive logic/interlocks performance (Suggestions include loss of a feed-water pump, main turbine-trip at rated power, loss of all off-site power, or main steam line isolation.)

Core Performance Tests - Examples of items to review include plant heat balance, determination of shutdown margin and measurement of reactivity coefficients and control rod worth using permanently installed instrumentation. Nuclear and thermal-hydraulic operating characteristics such as xenon and samarium effects, 1/M plots, and other low power physics test can be used to assess the simulated core performance.

Normal Plant Evolutions Tests - Confirm that the simulator has been operated from cold shutdown conditions to rated full power and back. Generally, if these test are performed

during use of the simulator for operator training, credit may be taken for having performed the required verification, provided that the evolutions are performed in accordance with reference unit procedures, and are evaluated and documented. Operator conducted surveillance tests - on safety related equipment or systems. (Same general comment applies as for normal evolutions)

Note: For any performance test that has plots/trends, ensure that parameters are not divergent from expected plant results.