IR 05000263/2003301

Er 05000263/2003301(DRS); 09/23/2003-09/26/2003; Monticello Nuclear Generating Plant; Initial Operator License Examination Report

ML033240032
Person / Time
Site:	Monticello
Issue date:	11/19/2003
From:	Lanksbury R Division of Reactor Safety III
To:	Thomas J. Palmisano Nuclear Management Co
References
50-263/03-301 50-263/03-301
Download: ML033240032 (20)

Text

ber 19, 2003

SUBJECT:

MONTICELLO NUCLEAR GENERATING PLANT NRC INITIAL LICENSE EXAMINATION REPORT 05000263/2003301(DRS);

Dear Mr. Palmisano:

On September 26, 2003, the U.S. Nuclear Regulatory Commission (NRC) completed administration of initial operator licensing examinations at your Monticello Nuclear Generating Plant. The NRC finalized the results of the examination on November 4, 2003, following the review of the post-examination comments submitted by your staff on October 6, 2003. The enclosed report documents the results of the examinations.

NRC examiners administered initial operator licensing examinations to four applicants. The examiners administered the operating test during the week of September 22, 2003. Members of the Monticello Nuclear Generating Plant training staff administered the written examination on September 26, 2003. One Senior Reactor Operator (SRO) and two Reactor Operator (RO)

applicants were administered written examinations and operating tests. One SRO retake applicant who failed the 2002 initial operator license written examination was administered a written retake examination. Of the three applicants who took both the written examination and the operating test, one SRO applicant failed the operating test and two RO applicants failed the written examination. These three applicants will not be issued a license. The two SRO applicants passed the written examination; however, they both scored 81 percent or less on the written examination; and, in accordance with the guidelines of NUREG 1021, Operator Licensing Examination Standards for Power Reactors, ES-501.D.3.c, the SRO retake applicants license will be withheld until any appeal rights of the other proposed license applicant failures, which may impact the outcome of the examination, are exhausted. Three of four applicants failing the examination was an abnormally high failure rate. Your staff would be expected to evaluate these failures to determine whether deficiencies exist in your initial licensed operator training program.

In accordance with 10 CFR Part 2.790 of the NRC's Rules of Practice, a copy of this letter and its enclosure will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRC's document system (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room). We will gladly discuss any questions you have concerning this examination.

Sincerely,

/RA by M. Bielby Acting for/

Roger D. Lanksbury, Chief Operations Branch Division of Reactor Safety Docket No. 50-263 License No. DPR-22

Enclosures:

1. Operator Licensing Examination Report 05000263/2003301(DRS)

2. Post Examination Comments and Resolution 3. Simulation Facility Report 4. Written Examinations and Answer Keys (RO & SRO)

REGION III==

Docket No: 50-263 License No: DPR-22 Report No: 05000263/2003301(DRS)

Licensee: Nuclear Management Company, LLC Facility: Monticello Nuclear Generating Plant Location: 2807 West Highway 75 Monticello, MN 55362 Dates: September 23 through 26, 2003 Examiners: H. Peterson, RIII NRC Chief Examiner P. Young, RIII NRC Examiner G. Johnson, RI NRC Examiner (Validation Week Only)

Observers: J. Drake, RIV NRC Examiner Approved by: Roger Lanksbury, Chief Operations Branch Division of Reactor Safety Enclosure 1

SUMMARY OF FINDINGS

ER 05000263/2003301(DRS); 09/23/2003-09/26/2003; Monticello Nuclear Generating Plant;

Initial Operator License Examination Report.

The announced operator licensing initial examination was conducted by Region III examiners in accordance with the guidance of NUREG-1021, Operator Licensing Examination Standards for Power Reactors, Revision 8, Supplement 1.

Examination Summary:

• Four examinations were administered. One Senior Reactor Operator (SRO) and two Reactor Operator (RO) applicants were administered initial operator license examination operating tests and written examinations. One SRO retake applicant who failed the 2002 operator license written examination was administered a written retake examination.

• Of the three applicants who took both the operating test and written examination, one SRO applicant failed the operating test and two RO applicants failed the written examination. These three applicants will not be issued a license. (Section 4OA5.1)

• Two SRO applicants passed the written examination. These two applicants each scored 81 percent or less on the written examination, and in accordance with the guidelines of NUREG 1021, "Operator Licensing Examination Standards for Power Reactors," ES-501.D.3.c, the SRO retake applicants license will be withheld until any appeal rights of the other proposed license applicant failures, which may impact the outcome of the examination, are exhausted. (Section 4OA5.1)

REPORT DETAILS

4OA5 Other

.1 Initial Licensing Examinations

a. Examination Scope

The NRC examiners conducted an announced operator licensing initial examination during the week of September 22, 2003. The facilitys training staff used the guidance established in NUREG-1021, Operator Licensing Examination Standards for Power Reactors, Revision 8, Supplement 1, to prepare the examination outline and to develop the written examination and operating test. The NRC examiners administered the operating test during the week of September 22, 2003. Members of the Monticello training department administered the written examination on September 26, 2003. Two Reactor Operator (RO) and two Senior Reactor Operator (SRO) applicants were examined.

b. Findings

Written Examination The licensee developed the written examination. The NRC examiners determined that the written examination, as originally submitted by the licensee, was within the NUREG-1021, Operator Licensing Examination Standards for Power Reactors, Revision 8, Supplement 1, ES-201.C.2.h and ES-401.E.3.b, acceptable quality range expected by the NRC. However, the margin to the threshold was at a minimum. This determination was based on the fact that 23 out of 128 written questions required replacement or significant modification and an additional 38 questions required enhancements. The problems identified with the written examination included, but were not limited to, questions submitted with multiple correct answers, questions submitted containing non-plausible distractors, questions where one or more distractors is (are)partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the question stem), overlap with operating test categories, and double distractor set questions written with a low discriminatory value (e.g., use of a common knowledge component that reduced the question to 50/50 selection verses selection of a correct answer from four plausible choices). Examination changes, agreed upon during the examination validation week of August 11, 2003, between the NRC and the licensee, were made according to the guidance contained in NUREG-1021.

The licensee graded the examination on September 26, 2003, and conducted a review of each question to determine accuracy and validity of the examination questions. The licensee submitted one post-examination question change on October 6, 2003. The examiners reviewed the recommended change with facility personnel and requested additional references to clarify the proposed change. The examiners accepted the recommended change. The change resulting from the examiners post-examination review are documented in Enclosure 2, Post Examination Comments and Resolution.

Operating Test The NRC examiners determined that the operating test, as originally submitted by the licensee, was within the NUREG-1021, Operator Licensing Examination Standards for Power Reactors, Revision 8, Supplement 1, ES-201.C.2.h and ES-301.E.2 &

.3 ,

acceptable quality range expected by the NRC. However, the margin to the threshold was at a minimum. This determination was based on the problems identified with the operating test including, but not limited to, duplication of items from the audit test(s),overlap with the written examination, flaws discovered during validation that resulted in significant revisions of two administrative Job Performance Measures (JPMs), flaws discovered during validation of a scenario event involving the failure of the Rod Worth Minimizer that resulted in significant revision, and simulator scenario events that did not require the applicant to perform sufficient verifiable actions to provide insight to the applicant's competence.

Examination changes, agreed upon during examination validation the week of August 11, 2003, between the NRC and the licensee, were made according to the guidance contained in NUREG-1021 with one exception. The examiners requested a change to the content of the emergency notification follow-up information that would be provided to the applicant. This information contained additional wording that removed the applicants decision making process from the evaluation of the radiation release data provided. This was discovered during a review of the material just one day prior to administration and was corrected prior to administration.

Examination Results One SRO and two RO applicants were administered initial operator license examination operating tests and a written examination. One SRO applicant who failed the 2002 initial operator license written examination was administered a written retake examination. Of the three applicants who took both the written examination and the operating test, one SRO applicant failed the operating test and two RO applicants failed the written examination. These three applicants will not be issued a license.

The two SRO applicants passed the written examination; however, they both scored 81 percent or less on the written examination; and, in accordance with the guidelines of NUREG 1021, "Operator Licensing Examination Standards for Power Reactors,"

ES-501.D.3.c, the SRO retake applicants license will be withheld until any appeal rights of the other proposed license applicant failures, which may impact the outcome of the examination, are exhausted. Should the RO applicants who failed the written examination appeal, a subsequent review of the written examination may result in question deletions or changes which may affect the licensing decision of the SRO applicants with a score of 81 percent or less.

Three of four applicants failing the initial operator license examination was an abnormally high failure rate. The licensee wrote condition report (CR) 03009914, Initial License Class Results Do Not Meet Site Expectations, and initiated a root cause investigation to address the submitted examination quality and the overall examination results. The licensee would be expected to incorporate any lessons learned from this effort into future examination submittals.

.2 Examination Security

a. Inspection Scope

The NRC examiners briefed the facility contact on the NRCs requirements and guidelines related to examination physical security (e.g., access restrictions and simulator considerations) and integrity (e.g., predictability and bias). The examiners observed the implementation of examination security and integrity measures (e.g., security agreements, sampling criteria, bank use, and test item repetition)throughout the examination process. The examiners also reviewed the facilitys examination security procedure, MTCP-03.35, Initial and Requalification Examination Security, Revision 4.

b. Findings

The licensees implementation of examination security requirements during examination preparation and administration were acceptable and met the guidelines provided in NUREG 1021, Operator Licensing Examination Standards for Power Reactors. No violations of 10 CFR 55.49, Integrity of Examinations and Tests, occurred during the examination preparation and administration.

4OA6 Meetings

Examination Exit Meeting The chief examiner and Mr. P. T. Young, RIII NRC Examiner (certification examination for chief examiner), presented the examination teams preliminary observations and findings on September 26, 2003, to Mr. J. Purkis and other members of the licensee management. In addition, following the receipt of the post-examination comment, the chief examiner conducted a subsequent exit meeting via telephone conference call on October 7, 2003, with Mr. J. Grubb and other members of the licensee management.

The licensee acknowledged the observations and findings presented. No proprietary information was identified.

ATTACHMENT:

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee

D. Wilson, Site Vice President

J. Purkis, Plant Manager

W. Cheever, Training Manager

J. Grubb, Operations Manager

G. Lashinski, General Supervisor Operations Training

J. Earl, Supervisor Operations Training

K. Markling, Control Room Supervisor

D. Neve, Regulatory Affairs Manager

G. Bregg, Nuclear Oversight Manager

G. Allex, Lead Examination Developer

D. Foster, Examination Developer

J. Shriver, Principle Engineering Analyst - Simulator

R. Baumer, Regulatory Affairs

J. Ruth, Operations Senior Instructor

B. MacKissock, Shift Operations Manager

K. Haugen, Shift Operations Manager

NRC

S. Burton, Senior Resident Inspector

R. Orlikowski, Resident Inspector

LIST OF ITEMS

Opened

None

Closed

None

Discussed

None Attachment