ML23156A019

PR-055 - 62FR42426 - Initial Licensed Operator Examination Requirements

ML23156A019
Person / Time
Issue date:	08/07/1997
From:	Hoyle J NRC/SECY
To:
References
62FR42426, PR-055
Download: ML23156A019 (1)

Text

ADAMS Template: SECY-067 DOCUMENT DATE: 08/07/1997 TITLE: PR-055 - 62FR42426 - INITIAL LICENSED OPERATOR EXAMINATION REQUIREMENTS CASE

REFERENCE:

PR-055 62FR42426 KEYWORD: RULEMAKING COMMENTS Document Sensitivity: Non-sensitive - SUNSI Review Complete

DOCKET NO. PR-055 (62FR42426)

In the Matter of INITIAL LICENSED OPERATOR EXAMINATION REQUIREMENTS DATE DATE OF TITLE OR DOCKETED DOCUMENT DESCRIPTION OF DOCUMENT 08/05/97 07/31/97 FEDERAL REGISTER NOTICE - PROPOSED RULE

• 08/21/97 08/14/97 COMMENT OF KEN ERICKSON ( 1) 09/08/97 07/30/97 COMMENT OF PAUL M. STEINER ( 2) 10/10/97 10/01/97 COMMENT OF STATE OF ILLINOIS (THOMAS W. ORTCIGER, DIRECTOR) ( 3) 10/17 /97 10/14/97 COMMENT OF ENTERGY OPERATIONS, INCL (JERROLD G. DEWEASE, VICE PRESIDENT) ( 4) 10/20/97 10/20/97 COMMENT OF NUCLEAR ENERGY INSTITUTE (RALPH E. BEEDLE, SR. V. P.) ( 5) 10/22/97 10/21/97 COMMENT OF RICHARDS. BALDWIN ( 6)

.10/24/97 10/21/97 COMMENT OF FLORIDA POWER AND LIGHT COMPANY (H. N. PADUANO) ( 7) 10/27/97 10/17 /97 COMMENT OF ARIZONA PUBLIC SERVICE COMPANY (JAMES M. LEVINE, SR. V.P.) ( 8) 10/28/97 10/22/97 COMMENT OF JOHN MUNRO ( 9) 10/29/97 10/21/97 COMMENT OF CAROLINA POWER & LIGHT COMPANY (D. B. ALEXANDER) ( 10) 10/30/97 10/24/97 COMMENT OF S. CAROLINA ELECTRIC & GAS COMPANY (GARY J. TAYLOR, VICE PRESIDENT) ( 11) 12/03/97 12/02/97 COMMENT OF WALTER W. HUNT ( 12) 01/12/98 10/23/97 COMMENT OF MASSACHUSETTS INSTITUTE OF TECHNOLOGY (JOHN A. BERNARD, PH.D., DIRECTOR) ( 13) 04/21/99 04/19/99 FEDERAL REGISTER NOTICE - FINAL RULE

DOCKET NUMBER PROPOSED RULE PB 5S-OOCK~TFO

( IP:2 /:"Rq :J. 'I;it,) IC ,....,, .

[75 o::b1-PJ NUCLEAR REGULATORY COMMISSION !:l APR 21 A1G :QQ 10 CFR Part 55 Ot I ,.

RlN 3150-AF62 r~d :

ADJUL' Initial Licensed Operator Examination Requirements AGENCY: Nuclear Regulatory Commission.

ACTION: Final rule.

SUMMARY

The Nuclear Regulatory Commission (NRC) is amending its regulations to allow nuclear power facility licensees to prepare, proctor, and grade the required written examinations and to prepare the required operating tests that the NRC uses to evaluate the competence of individuals appl,4ng for operator licenses at those plants. The amendment requires facility licensees that elect to prepare the examinations to prepare the examinations in accordance with NRC operator licensing examination standards for power reactors; establish, implement, and

• maintain procedures to control examination security and integrity; submit, upon approval by an authorized representative of the facility licensee, each examination and test to the NRC for review and approval; and proctor and grade the written examinations upon NRC approval. In making this final rule change, the NRC will continue to administer (i.e., manage and oversee) the initial operator licensing examination process by: (1) developing the generic fundamentals examinations (which are also proctored by facility licensees); (2) reviewing and approving the facility-developed, site-specific written examinations and operating tests; and (3) independently conducting and grading both the dynamic simulator and walk-through portions of the operating test, which is considered the most performance-based aspect of the licensing process and

2 permits the NRC to evaluate the operator and senior operator applicants' competence under normal and abnormal plant conditions. The amendment preserves the NRC's authority to prepare the examinations and tests in lieu of licensees and to exercise its discretion and reject a power reactor facility licensee's determination to prepare , proctor, and grade the written examinations and prepare the operating tests . The Commission is concerned with examination integrity; therefore, the amendment will also revise the regulations to ensure that applicants, licensees, and facil ity licensees understand the scope of the regulation .

~ ,:J 0 I 999 EFFECTIVE DATE : This final rule is effective on (iAscFt the ele:te 180 ele:-y~ f, u111 date of i,ublioa~ion iA H=te FemH-al Rcgis~er).

FOR FURTHER INFORMATION CONTACT: Siegfried Guenther, Office of Nuclear Raactor Regulation , U.S, Nuclear Regulatory Commission, Washington, DC 20555-0001, telephone :

301-415-1056; e-mail : sxg@nrc.gov.

SUPPLEMENTARY INFORMATION:

Background

Section 107 of the Atomic Energy Act (AEA) of 1954, as amended, requires the NRC to determine the qualifications of individuals applying for an operator's license, to prescribe uniform conditions for licensing these individuals, and to issue licenses as appropriate.

Pursuant to the AEA, 10 CFR Part 55 requires an applicant for an operator license to pass an examination that satisfies the basic content requirements specified in the regulation. The licensing examination consists of the following parts: (1) a written generic fundamentals

3 examination (covering reactor theory, thermodynamics, and components) that license applicants have to pass as a prerequisite for taking the site-specific examination; (2) a site-specific written examination covering plant systems, emergency and abnormal plant procedures, and plant-wide generic knowledge and abilities; and (3) a site-specific operating test consisting of three categories, including a crew-based, dynamic simulator performance demonstration, an individual, task-based walk-through covering control room and in-plant systems, and various plant administrative requirements . Although neither the AEA nor Part 55 specifies who must prepare, proctor, or grade these examinations, the NRC has traditionally performed those tasks itself or through its contract examiners. The NRC and its contract examiners have used the guidance in NUREG-1021, "Operator Licensing Examination Standards for Power Reactors," once titled "Operator Licensing Examiner Standards," to prepare the initial operator licensing examinations. This document has been revised as experience has been acquired in preparing the examinations. The current version is designated Revision 8.1 In accordance with 10 CFR 170.12 (i), the NRC's staff and contractual costs are recovered from facility licensees that receive examination services. In Fiscal Year (FY) 1995, the NRC spent approximately $3 million on contractor support for the preparation and administration of the initial operator licensing examinations and for support of requalification program inspections. On March 24, 1995, in SECY-95-075, "Proposed Changes to the NRC Operator Licensing Program," the staff advised the Commission of its intent to eliminate the use of contractors by allowing facility licensees to prepare the examinations. The NRC staff's proposal was motivated by the general improvement in the performance level of power reactor

1. Copies are available for inspection or copying for a fee from the NRC Public Document Room (PDR) at 2120 L Street NW, Washington, DC 20555-0001; the PDR's mailing address is Mail Stop LL-6; telephone is 202-634-3273; fax is 202-634-3343. Revision 8 of NUREG-1021 is also available for downloading from the internet at http://www.nrc.gov.

4 facility licensees' training programs, the NRC's continuing efforts to streamline the functions of the Federal government, and the need to accommodate anticipated resource reductions.

On April 18. 1995 , the Commission approved the NRC staff's proposal to initiate a transition process to revise the operator licensing program and directed the NRC staff to consider carefully the experience from pilot examinations before fully implementing the changes. On August 15, 1995, the NRC issued Generic Letter (GL) 95-06, "Changes in the Operator Licensing Program," 1 outlining the revised examination development process and soliciting volunteers to participate in pilot examinations to evaluate and refine the methodology.

Between October 1, 1995, and April 5, 1996, the NRC reviewed and approved 22 operator licensing examinations, including both the written examinations and the operating tests, prepared by facility licensees as part of a pilot progmm. These examinations were prepared using the guidance in Revision 7 (Supplement 1) of NUREG-1021 1 and the additional guidance in GL 95-06.

The results of the pilot examinations were discussed in SECY-96-123, "Proposed Changes to the NRC Operator Licensing Program," dated June 10, 1996. Based on the results of the pilot program, the NRC staff recommended that the Commission approve the implementation of the new examination process on a voluntary basis until rulemaking could be completed to require all power reactor facility licensees to prepare the entire initial operator licensing examination and to proctor and grade the written portion of the examination. On July 23, 1996, the Commission authorized the staff to continue the pilot examination process on a voluntary basis and directed the staff to develop a rulemaking plan to justify the changes that would be necessary to 10 CFR Part 55. The Commission also directed the staff to address a number of additional items (e.g., pros, cons, and vulnerabilities) regarding the revised examination process to facilitate a Commission decision on whether to implement the revised process on an industrywide basis.

5 With Commission approval, the NRC staff resumed conducting pilot-style examinations on August 19, 1996, and by the end of June 1998 had reviewed, approved, and administered 80 additional examinations that were developed by facility licensees. This raised the total number of examinations completed using the pilot process to 102.

On September 25, 1996, the NRC staff forwarded the rulemaking plan and a response to the additional items to the Commission in SECY-96-206, "Rulemaking Plan for Amendments to 10 CFR Part 55 to Change Licensed Operator Examination Requirements." SECY-96-206 identified a number of areas (i.e., quality and consistency, independence and public perception, examination security, NRC resources, program stability, and examiner proficiency) in which the NRC could be more vulnerable under the revised examination process and described the measures that the NRC has taken to manage the vulnerabilities. On December 17, 1996, the Commission directed the staff to proceed with the proposed rulemaking. The NRC staff forwarded the prnposed rule (SECY-97-079, "Proposed Rule - Initial Licensed Operator Examination Requirements") to the Commission on April 8, 1997, and on June 26, 1997, the Commission approved publication of the proposed rule for a 75-day comment period. The proposed rule was published in the Federal Register (62 FR 42426) on August 7, 1997.

After the public comment period expired on October 21, 1997, 11 comment letters were received. Two additional comment letters arrived after the expiration date but were also considered in the development of the final rule.

As written, the proposed rule would have required all power reactor facility licensees to prepare their operator licensing examinations and to proctor and grade the written portion of those examinations. Although the proposed rule would have imposed new requirements on facility licensees, the NRC took the position that the backfit rule, 10 CFR 50.109, did not apply because the shift in responsibility for preparing the examinations would not: (1) constitute a "modification of the procedures required to operate a facility" within the scope of the backfit

6 rule; (2) affect the basic procedures for qualifying licensed operators ; or (3) require facility licensees to alter their organizational structures . However, based upon further review after issuing the proposed rule. the NRC has concluded tha t there is insufficient basis to support the original position. Therefore , the NRC has decided to revise the final rule so power reactor facility licensees may elect to prepare their written examinations and operating tests (and proctor and grade the written examinations) in accordance with NUREG-1021 , or to have the NRC prepare the examinations, thereby making a backfit analysis unnecessary.

Discussion The pilot examinations demonstrated that the revised process, under which facility licensees prepare the written examinations and operating tests , is generally effecti *e and efficient. From tbe time the pilot program began in October 1995 through the end of June 1998, the NRC staff reviewed, approved, and administered a total of 102 examinations that were voluntarily developed by facility licensees under the pilot examination and transition program.

Facility licensees prepared the written examinations and the operating tests, proctored the written examinations, and graded the written examinations using the guidance provided by the NRC in GL 95-06 during the early stages of the pilot program, and subsequently in interim Revision 8 of NUREG-1021, "Operator Licensing Examination Standards for Power Reactors."

NRC examiners thoroughly reviewed the examinations and tests to determine if they were consistent with NRC standards, directed facility licensees to make whatever changes were necessary to achieve NRC standards if the submitted examinations and tests were deficient, and approved the examinations and tests before they were administered. NRC examiners independently administered all of the operating tests, reviewed the written examination grading,

7 and made the final licensing recommendations for approval by NRC management.

Comments from the NRC chief examiners who evaluated the pilot examinations indicate that the quality and level of difficulty of the licensee-prepared examinations (when modified as directed by the NRC) were generally comparable to the examinations prepared by the NRG (i.e., by the staff or NRC contractors) . The passing rate on the 102 pilot-style examinations administered through the end of June 1998 was only slightly lower than the passing rate on the power reactor licensing examinations administered during FY 1995, the last year in which all

• examinations were prepared by the NRC. However, considering the historical fluctuation in the average examination passing rates and the other factors (e.g., training program quality and screening of applicants by facility licensees) that could be responsible for some or all of the observed difference, the Commission has concluded that the observed change in the passing rates is not significant. The average grades on the facility-prepared, NRG-approved written examinations were also comparable if slightly lower than the grades on examinations prepared by the NRC during FY 1995. These data support the conclusion that the facility-prepa_red examinations are discriminating at a conservative and acceptable level and that the revised

• examination process is effective. Therefore, the fact that some facility licensees will be preparing the examinations with NRC review and approval, should have no negative effect on the safe operation of the plants.

Although the NRG-approved examinations were comparable to NRG-prepared examinations, essentially all of the examinations prepared by facility licensees required some changes subsequent to NRC review, and many of the examinations required significant rework.

The NRC had originally believed that, with training and experience, the industry would quickly gain proficiency in preparing the examinations, but the overall quality of the examinations submitted to the NRC during the pilot program did not improve as expected over time. Although approximately half of the 17 facility licensees that had prepared more than one examination by

8 the end of FY 1997 did maintain or improve the quality of their second or third examination submittals, the quality of the other facility licensees' second or third examinations was lower.

Consequently, the NRC has asked the industry to address the issue of examination quality and determine the need for additional training on examination development. The NRC will continue to: (1) direct facility licensees that prepare their examinations to revise the examinations as necessary to achieve an acceptable level of quality and discrimination; (2) withhold approval of those examinations that do not meet NRC standards; (3) oversee the regional implementation of the operator licensing process to ensure consistency; (4) address significant deficiencies in the submitted examinations as licensee performance issues in the examination reports, as appropriate; (5) conduct or participate in workshops, as necessary, to ensure that facility licensees understand the NRC's examination criteria; 2nd (6) prepare the licensing examinations for those facility licensees that elect not to prepare their own examinations.

With regard to the efficiency of the revised examination process, the experience to date supports the conclusion that the average industry cost will not differ significantly from the cost of NRG-prepared examinations. Comments from the industry reflect that the cost for some facility licensees to prepare the examinatior. was higher than it would have been for an NRC-prepared examination; howev*er, other licensees prepared good quality examinations at lower cost than the NRC. The industry generally attributed the higher cost to the revised examination and administrative criteria under the pilot examination process. Although the NRC acknowledges that the revised criteria contribute somewhat to the elevated cost, many of the variables that affect the quality and, consequently, the cost of the examination will be under the facility licensees' control and can present an opportunity for cost savings. For example, facility licensees that elect to prepare the examinations will be able to manage the size and quality of their examination banks and the training and experience of the personnel they select to write their licensing examinations. The revised examination process allows facility licensees to

9 control the development of the examinations and holds them responsible for their quality. If a facility licensee submits an acceptable quality examination , it is likely to save resources despite the additional administrative criteria ; however, if the facility licensee submits an examination that requires many changes, it will likely cost more than if the NRC had prepared the examination.

Comments from the NRC chief examiners who worked on the pilot examinations indicate that the average amount of time spent reviewing and revising the facility-prepared examinations was generally consistent with the estimates developed before starting the pilot program. Although a number (approximately 20 percent in FY 1997) of the examinations required significantly more NRC effort than originally anticipated to bring them up to the NRC's standards, the resource burden was generally offset by other examinations that required less effort to review and revise. The increased efficiency of the revised examination process has enabled the NR8 to eliminate the use of contractors in the operator licensing program and conduct the initial operator licensing and requalification inspection programs with the existing NRC staff. Before initiating the pilot examination and transition process at the beginning of FY

• 1996, the NRC spent approximately $3 million per year on contractor assistance for initial examinations and requalification inspections. In FY 1997, when facility licensees prepared approximately 75 percent of the examinations, the NRC's spending on contractor assistance for the licensing examinations and requalification inspections decreased to approximately $0.5 million. The FY 1998 and FY 1999 budgets reflect the complete elimination of contractor support for the operator licensing program (with the exception of the generic fundamentals examination). Future resource requirements for the operator licensing program will, in large part, be driven by changes in the level of facility participation in the voluntary examination development process.

In order to maintain the integrity of the operator licensing written examinations required

10 by 10 CFR 55.41 and 55.43 and the operating tests required by 10 CFR 55.45, the Commission has amended the final rule by adding a requirement for those power reactor facility licensees that elect to prepare. proctor, and grade the written examinations and prepare the operating tests , to establish , implement, and maintain procedures that control the security and integrity of those examinations and tests. The Commission's regulations in 10 CFR 55.49 already prohibit applicants, licensees (operators), and facility licensees from engaging in any activity that compromises the integrity of any examination or test required by 10 CFR 55.

However, based on the number of examination security incidents that have occurred since the pilot examination program began, the Commission has concluded that applicants, licensees, and facility licensees may not be aware that the requirements of 10 CFR 55.49 cover more than just those activities directly involving the physical administration of an examination or test. In that regard, the Commission considers the integrity of an examination or test to be compromised if.any activity occurs that could affect the equitable and consistent administration of the examination or test, regardless of whether the activity takes place before, during, or after the administration of the examination or test. Therefore, in addition to requiring certain facility licensees to establish, implement, and maintain procedures that control the security and integrity of the examinations and tests, the Commission is also amending 10 CFR 55.49 to clarify the scope of that regulation.

Revision 8 of NUREG-1021 identifies a number of examination security and integrity guidelines (e.g., physical security precautions, including the use of simulators and the mailing of examination materials) that the affected facility licensees (i.e., those that elect to prepare their own written examinations and operating tests) should consider when establishing their procedures. Although the security and integrity guidelines in NUREG-1021 are not regulatory requirements, once a facility licensee has established its required procedures, the Commission intends to monitor this area to ensure that the procedures are implemented and maintained.

11 Consistent with the examination security and integrity guidelines in NUREG-1021, facility employees with specific knowledge of any NRC exam ination before it is given should not communicate the examination contents to unauthorized individuals and should not participate in any further instruction of the students scheduled to take the examination. Before they are given access to the examination, facility employees are expected to sign a statement acknowledging their understanding of the restrictions . When the examinations are complete, the same employees are expected to sign a post-examination statement certifying that they have not

• knowingly compromised the examination .

NRC examiners are expected to be attentive to the facility licensee's examination security measures, to review the security expectations with the facility licensee at the time the examination arrangements are confirmed, and to report any security concerns to NRC management. If the NRC determines during its preparation that an examination may have been compromised, it:- will not administer the examination until the scope of the potential compromise is determined and measures can be taken to address the integrity and validity of the examination . Pursuant to 10 CFR 55.51, the NRC must make a determination before issuing a license that the test or examination is valid, meeting the requirements of the AEA and the Commission's regulations. If the compromise is discovered after the examination has been administered, the NRC will not complete the licensing action for the affected applicants until the NRC staff can make a determination regarding the validity of the examination. If the compromise is not discovereci until after the licensing action is complete, the NRC will reevaluate the licensing decision. If the NRC determines that the original licensing decision was based on an invalid examination, it will take appropriate action pursuant to 10 CFR 55.61 (b)(2).

As a separate action, the Commission is modifying its "General Statement of Policy and

12 Procedures for NRC Enforcement Actions" (Enforcement Policy) to provide examples of violations that may be used as guidance in determining the appropriate severity level for violations involving the compromise of an examination or test. The NRC staff will evaluate all potential compromises of an examination or test required by 10 CFR 55 to determine whether a violation of 10 CFR 55.49 has occurred. A compromise that is not detected before a license is issued would be considered a significant regulatory concern and categorized at least at Severity Level Ill. However, depending on the circumstances as explained in the Enforcement Policy, the severity level may be increased or decreased . The NRC intends to utilize its enforcement authority including , as warranted, civil penalties and orders against individuals and facility licensees who: (1) compromise the integrity of an examination in violation of 10 CFR 55.49; (2) commit deliberate misconduct in violation of 10 CFR 5G.5; or (3) provide incomplete or inaccurate information to the NRC in violation of 10 CFR 50.9. In addition, cases involving willful violations r-nay be referred to the Department of Justice for criminal prosecution.

The Commission has reviewed the vulnerabilities and costs associated with the revised examination process and considered the measures that the NRC staff has taken to mitigate the vulnerabilities. With regard to examination quality and level of difficulty, the Commission acknowledges that the effectiveness of the revised examination process is contingent on the NRC staff's review of the facility-proposed examinations to ensure that NRC standards are achieved. The Commission has concluded, based on the results of the pilot examination program, that the controls implemented by the NRC staff will provide reasonable assurance that the examinations that are administered to the license applicants will provide a valid and consistent basis upon which to make the licensing decisions regardless of whether the examinations were prepared by the facility licensee or the NRC. The Commission also realizes that the frequency of examination security incidents and the risk of undetected compromises may increase for those examinations that are prepared by facility licensees. However, the

13 Commission is confident that the measures discussed above will sufficiently control the vulnerability in this area .

The Commission is aware that the original expectation that facility licensees would eventually realize cost savings under the revised process as they gain proficiency in preparing the examinations has not yet been realized. However, the Commission has concluded that neither the increased vulnerabilities nor the absence of clear industry cost benefit provides sufficient basis for discontinuing the revised examination process. The Commission also finds that the revised examination process is more consistent with the NRC's other oversight programs because it requires NRC examiners to review materials prepared by facility licensees.

The revised process enables NRC examiners to focus more on the psychometric quality of examinations (e.g., the cognitive level at which the questions are written and the plausibility of the distractors or wrong answer choices) prepared by the facility licensees than on the technical accuracy of the ~xaminations, which was their primary focus when the examinations were prepared by NRC contractors. This shift in the NRC examiners' focus, coupled with the facility licensees' technical expertise, has the potential to improve the overall quality of the facility-prepared licensing examinations.

In the proposed rule, the NRC took the position that the backfit rule (1 0 CFR 50.109) did not apply to this rulemaking. However, in its review of the final rule, the Committee To Review Generic Requirements (CRGR) opined that it was inclined to view the rule as a backfit and recommended that the provisions of the proposed rule be implemented on a voluntary basis, which would not constitute a backfit. Although the NRC had considered and dismissed that alternative during the proposed rulemaking because of concerns regarding resource planning, it has since concluded that the benefits of the revised examination process (e.g., improved regulatory efficiency and greater licensee control over the examination costs) remain substantial even if every facility licensee is not required to prepare its own examinations.

14 Rather than terminate the pilot program and resume the NRC-prepared examination process on an industrywide basis, the NRC has decided to amend the final rule to give facility licensees the option to prepare their own examinations or to have them prepared by the NRC.

Summary of Public Comments The 75-day public comment period began when the notice of proposed rulemaking was published in the Federal Register (62 FR 42426) on August 7, 1997, and closed on October 21, 1997. The notice (FRN) requested public comment on the proposed rule, on the implementation guidance in interim Revision 8 of NUREG-1021, and on the following two questions:

1. Are there portions of the operator exams that are common to all licenseas, and would, therefore, be more efficiently developed by the NRC?

2. Is the conclusion in the regulatory analysis correct that it would be less costly for each licensee to prepare its own initial operator examinations to be reviewed, revised, and administered by the NRC, than to have one NRC contractor prepare these exams for all licensed operators with the costs to be reimbursed by licensee fees?

The NRC received 13 comment letters on the proposed rule; two of the letters arrived-after the comment period closed, but they were considered nonetheless. The respondents included three NRC examiners, one contract examiner, five nuclear utilities and one utility employee, one nonpower reactor facility licensee, the State of Illinois, and the Nuclear Energy Institute (NEI), which submitted its comments on behalf of the nuclear power industry. Copies of the public comments are available in the NRC Public Document Room, 2120 L Street, NW (Lower Level), Washington, DC, and on the internet ct "http://ruleforum.llnl.gov/cgi-bin/

rulemake?source=OE_ PRULE".

15 Seven of the respondents (three NRC examiners, one contract examiner, one utility employee, one nonpower facility licensee, and the State of Illinois) recommended that the rule change be disapproved . Five of the industry respondents (NEI and four utilities) supported the rule change; however, one utility endorsed NEl's comments but stated that it did not agree with the proposed rule in its present form. NEI and two of the utilities stated that they would rather continue with a voluntary program because it would allow greater flexibility for those facility licensees with small training staffs. However, they would support mandatory participation with the rule change rather than return to the previous process under which NRC contractors wrote most of the examinations.

Those comments related to the two specific questions raised in the proposed rule and those that have a direct bearing on the rule are discussed below. The comments are categorized as they relate to reactor safety and the vulnerabilities discussed in SECY-96-206 (i.e., quality and consistency, independence and public perception, security, NRC resources, and examiner proficiency) . The NRC received no comments related to program stability.

One NRG examiner, NEI, four of the utilities, and the utility employee also provided specific comments and recommendations regarding the implementation guidance in interim Revision 8 of NUREG-1021. Those comments are addressed in Attachment 1 of the Commission (SECY) paper associated with this rulemaking. A copy of the SECY is available in the NRC Public Document Room, on the internet at http://www.nrc.gov, or from Siegfried Guenther, Office of Nuclear Reactor Regulation, U.S. Nuclear Regulatory Commission, at 301-415-1056 or e-mail at sxg@nrc.gov.

Comment: With regard to the first specific question included in the proposed rulemaking, 2 of the 13 respondents (NEI and one utility) stated that all of the common material is already included in the generic fundamentals examination (GFE) and that the remaining

16 elements are best covered as part of the site-specific examination.

Response: It appears that the current allocation of topics between the GFE and site-specific written examinations is generally perceived to be an efficient method of covering the topics required by 10 CFR 55.41 and 55.43 . Therefore, the Commission finds no basis for changing the process to have the NRC separately develop portions of the initial examination that would be common to all facilities.

Comment: Seven of the 13 respondents (NEI , two utilities, a utility employee, and three examiners) directly or indirectly addressed the second specific question in their letters. NEI and one utility stated that the revised examination criteria in interim Revision 8 of NUREG-1021 have increased the level of effort and will result in high13r licensing fees regardless of who prepares the examinations . However, NEI and another utility agreed that comparing the cost of facility-prepared examinations to those prepared by the NRC is difficult, but they concluded that it should be less costly for facility licensees to prepare the examinations than to have the NRC prepare them under the same crite ria.

NEI also stated that the relative cost of the two examination processes should not be the only factor in deciding whether to proceed with the rulemaking that would have required all power reactor facility licensees to prepare their licensing examinations. NEI indicated that preparing higher cognitive level questions requires detailed plant knowledge, better provided by facility licensees, and that the revised process (which has eliminated the use of NRC contractors to administer the operating tests) will allow NRC staff to evaluate each applicant without relying on third-party observers.

Two NRC examiners, one contract examiner, and a utility employee asserted that the facility licensees' cost has increased under the revised examination process. They cited various reasons for the increased cost, including training personnel to write the examinations

17 and then restricting them from training the applicants, and upgrading equipment to maintain examination security. The NRC examiners based their comments on feedback from facility training personnel: one examiner indicated that it took facility licensees an average of 700 hours0.0081 days <br />0.194 hours <br />0.00116 weeks <br />2.6635e-4 months <br /> to prepare each examination. The utility employee stated that the rule change will simply transfer the cost of contractors from the NRC to the utilities.

Response: The NRC acknowledges that the revised administrative criteria in particular (e.g., the restrictions on which facility training personnel would be allowed to write the pilot examinations and the need to document the source of the test items) have probably caused the cost of preparing the examinations to be somewhat higher than it would have been if facility licensees had been allowed to prepare the examinations using the same criteria that applied to the NRC and its contractors before starting the pilot program. However, when the NRC first developed the revised examination process, with its additional administrative criteria, the NRC still believed that-the cost for facility licensees to prepare the examinations would be offset by the reduction in the licensing fees and that a cost savings could be realized as facility licensees gained experienee with the process. Many of the facility licensees that participated in the pilot .

program demonstrated that it is possible to prepare an acceptable quality examination at the same or lower cost than the NRC or its contractors could prepare a comparable examination.

The fact that a number of facility licensees did not prepare acceptable examinations may be as much an indication of the licensees' inefficiency and inexperience as it is a symptom of deficiencies in the examination criteria. Those facility licensees that did not initially submit acceptable examinations, eventually paid more in fees because of the additional effort required for the NRC to review, and the licensees' staffs to rewrite, the examinations. Finally, it is possible that the magnitude of the increase in effort and cost may be perceived to be higher than it actually is because the industry had originally expected to save money if the NRC would have allowed facility licensees to prepare the examinations using the version of NUREG-1021

18 that was in effect before beginning the pilot program .

With regard to the additional security costs cited by the examiners, the Commission has stressed the importance of maintaining examination security, but the NRC has not required facility licensees to invest in additional physical security systems. However, the frequency of security incidents since beginning the pilot examination program has prompted the NRC to: (1) clarify the intent of 10 CFR 55.49 in the final rule; (2) amend the final examination rule to require facility licensees that elect to prepare their examinations to establish, implement, and maintain procedures to control examination security and integrity; and (3) include additional security guidance in the final version of Revision 8 of NUREG-1021 . These actions will help ensure, among other things, that facility licensees understand their responsibility for maintaining control over the examination process.

The pilot examinations demonstrated that some of the people assigned by facility licensees to develop the examinations did not have sufficient expertise required to prepare good quality examination materials consistent with NRC standards. As noted earlier, the NRC has asked the industry to address the issue of examination quality and the need for additional training on examination development. The NRC acknowledges that the restrictions on the use of instructors to prepare the licensing examinations may be partially responsible for limiting the availability of qualified examination preparers. Moreover, the NRC has concluded that the restrictions have placed an unnecessary burden on facility licensees with minimal benefit and, therefore, has revised the personnel restrictions in the final version of Revision 8 of NUREG-1021 to allow facility instructors to prepare the licensing examinations (including the written and operating test outlines, the written examination questions, and the operating test details) without regard to the amount of time they spent training the license applicants. However, the instructors will still be precluded from instructing the applicants once they begin working on the licensing examination. This change is consistent with NRC policy regarding instructor

19 participation in requalification examinations and should provide licensees that elect to prepare their examinations with increased flexibility in managing their resources and possibly reduce their costs.

The NRC has revised the regulatory analysis in response to the public comments and lessons learned from the pilot program. The NRC has also reevaluated the additional administrative criteria in interim Revision 8 of NUREG-1021 and considers them reasonable and essential to mitigate the vulnerabilities (e.g., quality, security, and conflict of interest) of the new examination process and to facilitate the NRC staff's review of the proposed examinations .

These criteria are retained in the final version of Revision 8 of NUREG-1021.

The issue of cost has lost much of its importance because the NRC has decided to continue the revised examination process on a voluntary basis rather than require each power reactor facility licensee to prepare the examinations. It will be up to each facility licensee to compare the cost of preparing its own examinations in accordance with the criteria in the effective revision of NUREG-1021 with the cost of having the NRC staff prepare the examinations and then make a decision based on its available resources (and other considerations) .

Comment: Two NRC examiners with pilot-examination experience asserted that the quality of the simulator and walk-through tests has decreased significantly and that, in most cases, the quality and difficult~, of the submitted examinations have been below NRC standards.

All four examiners who submitted comments cited various reasons why the quality and difficulty of the facility-prepared examinations might be lower than examinations prepared by the NRC or its contract examiners, including: (1) the facility licensees' tendency to narrow the scope of the operating test to those procedures that the facility believes are important (and emphasized in the training program); and (2) the belief that most facility training personnel do not have the

20 expertise to develop valid test items. Two NRC examiners asserted that the quality of the examinations has not improved during the pilot program and is not likely to improve because there is nothing to prevent licensees from using different people to develop successive examinations. A utility employee asserted that the utilities' limited contact with the process by preparing an examination once every 18 to 24 months will not foster consistency or develop skilled examination writers.

Two NRC examiners asserted that the elimination of NRC contract examiners who participated in examinations across the four NRC regions will be detrimental to examination consistency. One NRG examiner asserted that the guidance in interim Revision 8 of NUREG-1021 is not sufficiently prescriptive to ensure nationwide consistency in the level of knowledge tested and the level of difficulty of the exarrinations and that several specific changes should be included in NUREG-1021 to address his concerns.

The State of Illinois asserted that the quality and consistency of the written examination questions can be maintained because the NRC can change and approve the questions before they are used . However, the State also recommended that the NRC should compile the examination questions and proctor the examinations (refer to the conflict-of-interest discussion below).

According to NEI, the recent facility-prepared examinations were of higher quality than the examinations prepared by the NRG before the pilot program started. Many of the NRC-prepared examinations had to be revised in response to the facility licensees' technical reviews.

Response: Essentially all of the facility-prepared examinations required some changes and many required significant changes to make them conform to the NRG's standards for quality and level of difficulty. According to the questionnaires completed by the NRC chief examiners responsible for the pilot examinations, the average facility-prepared written examination required approximately 1O to 20 changes, which is consistent with the number of

21 changes often required on examinations prepared by NRG contract examiners. Most NRG chief examiners judged the final examinations (with the NRG's changes incorporated) to be comparable to recent NRG-prepared examinations in terms of quality and level of difficulty.

Moreover, the fact that the passing rate on the facility-prepared examinations is generally consistent with the historical passing rate on examinations prepared by the NRG suggests that the NRG-approved examinations have discriminated at an acceptable level and that they have provided an adequate basis for licensing the applicants at those facilities.

Although the NRG expected that the proposed examination quality would improve as facility licensees gained experience and familiarity with the NRG's requirements and expectations, the overall quality of examinations submitted to the NRG during the transition process did not improve appreciably over time . Although approximately half of the 17 facility licensees that had prepared more than one examination by the end of FY 1997 did maintain or improve the quality of their second or third examination submittals, the quality of the other facility licensees' second or third examinations was lower. Although it is unclear to what extent the problems wtth proposed examination quality and difficulty have been caused by a lack of

• sufficient expertise on the part of the examination writers, the NRG has asked the industry to address this issue. Furthermore, the NRG staff has conducted and participated in a number of public meetings and workshops in an effort to communicate its expectations to the facility employees who will be preparing the examinations. Additional NRC and industry workshops will be conducted to address examination quality and solicit industry feedback.

In SECY-96-206, the NRC staff discussed the issues of examination quality and consistency and how they might be affected when a large number of facility employees assume the role that had been filled by a smaller number of experienced NRC and contract examiners.

The.NRG staff's comprehensive examination reviews versus the examination criteria in NUREG-1021 , in combination with supervisory reviews and the examination oversight activities

22 conducted by the Office of Nuclear Reactor Regulation , should mitigate the vulnerability in this area. Moreover, the industry and staff initiatives to improve the expertise of the examination writers should eventually enhance the quality and consistency of the facility-prepared examinations.

Comment: All four examiners who submitted comments, a nonpower reactor facility licensee, and the State of Illinois asserted that allowing the facility licensees to prepare the operator licensing examinations decreases the level of independence and creates a conflict of interest for facility personnel having responsibility for training and licensing the operators. Their letters maintained that the new process makes it possible for the utilities to "teach the examination," to test applicants only on what was taught, or to avoid testing in areas with known difficulties. One NRC examiner noted that the new process places training managers in a no-win situation bec.ause if applicants fail the examination, the managers look like poor trainers, and if the examination is too easy, the NRC gives them a bad report. He and another NRC examiner asserted, based on their experience during the pilot examinations, that some facility personnel openly admitted that they would develop the easiest possible examination to ensure that all their applicants would pass.

One NRC examiner noted that the NRC review and approval process cannot adequately compensate for the conflict-of-interest problems inherent in the revised examination process and recommended a change to interim Revision 8 of NUREG-1021 that would limit the licensees' latitude in selecting topics for the examination outline. The State of Illinois suggested that the NRC should compile the questions and proctor the examination to maintain more of the checks and balances that existed under the old process.

The nonpower reactor facility licensee noted that most professional licensing examinations are developed by independent agencies, and that this fosters a sense of

23 professionalism in the license applicants .

Response : The NRC agrees that the revised examination process decreases the level of independence in the licensing process and may create a potential conflict of interest for facility personnel involved in preparing the examination . However, the Commission has concluded that restricting the training activities of those individuals when they become involved in preparing the licensing examination, in combination with the NRC's enforcement authority, will adequately mitigate the vulnerability in this area . Although the NRC has amended the final

• version of Revision 8 of NUREG-1021 to allow instructors to participate in the examination development regardless of their involvement in training the license applicants (as discussed above in response to comments concerning the industry burden under the revised examination process), the NRC has also amended NUREG-1021 to include an expectation that facility licensees will use an objective, systematic process for preparing the written examination outlines. This process enhancement should limit the potential for bias in the selection of topics to be evaluated on the written examination.

The NRC will continue to monitor the facility licensees' examination development

• programs and implement additional restrictions, as necessary, if actual bias problems are identified. Moreover, if the NRC determines that a facility licensee has intentionally biased the scope, content, or level of difficulty of an examination (i.e ., compromised its integrity contrary to 10 CFR 55.49) to enhance the chances that its applicants would pass the examination, the NRC will utilize its enforcement authority including, as warranted, civil penalties, orders against the individuals involved, and charging the individuals Jnvolved with deliberate misconduct pursuant to 10 CFR 50.5.

Concerns regarding the potential for conflict of interest and the frequency of security incidents since beginning the pilot examination program have prompted the NRC to review the clarity of 10 CFR 55.49. The regulation encompasses not only activities like cheating and

24 lapses in security but also activities that compromise the integrity or validity of the examination itself (e .g.~ noncompliance with the criteria designed to limit the potential for bias in the selection of topics to be evaluated on the written examination) . Therefore. the NRC has concluded that it would be beneficial to amend 10 CFR 55.49 to clarify its intent and to amend the examination rule to require power reactor facility licensees that elect to prepare their licensing examinations to establish procedures to control examination security and integrity.

Comment: Three NRC examiners and the State of Illinois asserted that the revised examination process increases the threat to examination security. One examiner noted that the examination is onsite for a longer period of time, thereby proportionally increasing the risk of being compromised. Another examiner cited the fact that a number of examination reports have documented problems with security.

Response: As discussed in SECY-96-206 and SECY-97-079, the Commission is aware of the vulnerability in this area because several security incidents have occurred since beginning the pi<<)t examination program. Therefore, based on the comments received and the experience with security incidents, the NRC has: (1) clarified 10 CFR 55.49 in the final rule to ensure that applicants, licensees, and facility licensees understand the scope and intent of the regulation; (2) amended the final examination rule to require facility licensees that elect to prepare their licensing examinations to establish, implement, and maintain procedures to control examination security and integrity; (3) strengthened the discussion of examination security in the final version of Revision 8 of NUREG-1021; and (4) modified NUREG-1600, "General Statement of Policy and Procedures for NRC Enforcement Actions," to address enforcement action against parties subject to the requirements in 10 CFR 55.49. NRC examiners are expected to review the NRC's physical security guidelines and the facility licensee's specific plans for ensuring examination security at the time the examination

25 arrangements are confirmed with the designated facility contact. Furthermore, the NRC has issued an Information Notice to advise power reactor facility licensees of the NRC's perspective and expectations regarding the integrity of examinations developed by the facility licensees' employees and representatives, and it has asked NEI to take the initiative in developing a model for securing examinations.

As a separate action, the NRC will not administer any examination that may have been compromised until the scope of the potential compromise is determined and measures can be

• taken to address the integrity and validity of the examination. If the compromise is discovered after the examination has been administered, the NRC will not complete the licensing action for the affected applicants until the staff can make a determination regarding the impact that the compromise has had on the examination process. If the compromise is not discovered until after the licensing action is complete, the NRC will reevaluate the licensing decision pursuant to 10 CFR 55.61 (b}(2) if it determines that the original licensing decision was based on an invalid examination .

• Comment: One NRC examiner disagreed with the conclusion in the proposed rulemaking that the facility-prepared examination process is an efficient use of NRG resources when compared to the NRG-prepared or contractor-prepared examinations. He noted that, in most cases, the quality and difficulty of the proposed examinations have been below NRC standards (as discussed above) and that it has taken a significant effort on the part of the NRC chief examiner to achieve an acceptable product.

An NRC contract examiner asserted that NRC cost-saving is a poor reason for changing the rule, since the utilities pay for the examinations anyway. He noted that the pilot examination process has led to a loss of certified examiners and contends that those NRC examiners who are left will become more dissatisfied with their jobs and will leave because they will be required

26 to travel more to compensate for the loss of contractors .

Response : The NRC acknowledges that many of the facility-prepared examinations (about 20 percent in FY 1997) required significantly more NRC exam iner time than desired or planned in order to achieve NRC quality standards. However, questionnaires filled out by NRC chief examiners for the pilot examinations indicate that the average amount of time spent on reviewing and upgrading the examinations is generally consistent with the estimates developed before starting the pilot program (i.e. , approximately 170 examiner-hours). As noted in SECY-97-079, the NRC has issued a memorandum to its regional administrators emphasizing the importance of: (1) assigning adequate resources to carry out the operator licensing task; (2) completing a review of every facility-prepared examination; and (3) not administering any examination that fails to meet NRC standards for quality and level of difficulty. Furthermore, all the time that NRC examiners spend reviewing an examination and modifying it so that it meets NRC standards is ultimately billed to the facility licensee.

The Commission acknowledges that facility licensees bear the cost of preparing the licensing examinations whether or not the NRC performs this function . However, this rule will give facility licensees more control over the cost of licensing operators at their facility, and the pilot examination program has demonstrated that some facility licensees will save resources if they elect to prepare their own licensing examinations.

The NRC's budget cuts have necessitated agencywide downsizing, which can be expected to increase the burden of travel for many NRG employees, not just the operator licensing examiners. The number of NRC full-time equivalent (FTE) license examiners has remained essentially constant throughout the pilot program and, aside from normal attrition and staff turnover, the loss of certified examiners has been limited to NRC contractors.

Comment: Two NRC examiners expressed concern that examiner proficiency will

27 decrease as a result of implementing the revised examination process. One of the examiners stated that examination reviewers will not maintain the same base of knowledge as examination writers maintained and that they will lose their familiarity with plant operating procedures.

Response: The Commission has concluded that the revised examination process affords sufficient NRC staff involvement that NRC examiners will maintain an acceptable level of proficiency. An NRC examiner will review and approve every facility-prepared examination before it is administered to ensure that it conforms to the criteria specified in NUREG-1021 for content, format, quality, and level of knowledge and difficulty. NRC examiners will also continue to independently administer and grade both the dynamic simulator and the plant walk-through portions of the operating tests. Because NRC examiners will be administering all of the operating tests, the Commission believes that the revised process will enable the examiners to accrue more experience in a shorter period of time and to maintain their proficiency. New NRC license examiner-s will still be required to complete a standardized training program, including the development of a written examination and operating test, as part of their qualification process. Moreover, the NRC will ensure that the in-house capability to prepare the examinations is maintained by: (1) requiring a regional supervisor to review and approve every examination and the Office of Nuclear Reactor Regulation to conduct periodic examination reviews; (2) conducting examiner refresher training; and (3) convening an operator licensing examiners' training conference at intervals not to exceed 24 months. Although experience during the voluntary pilot prog~am and informal feedback from the industry suggests that facility licensees are likely to request the NRC to prepare a suffici_ent number of examinations to maintain the proficiency of its examiners, each region will be required to write at least one initial operator licensing examination per calendar year.

Comment: A utility employee asserted that the revised examination process will not

28 enhance the competency of the operators or reactor safety because the facilities' training resources will be diverted from their primary purpose (i.e. , training the applicants) as much as six months before the examination date. Three NRC examiners also took issue with the conclusion in the proposed rulemaking that the NRC staff's focus on operator performance and its core of experience will improve under the pilot examination process because contractors will no longer be used to administer the operating tests. Two of the examiners asserted that the reduction in the amount of procedural research by examiners will result in the identification and correction of fewer procedural problems. Two of the examiners also stated that the contract examiners help maintain examination consistency across the NRC regions and that their contribution to the operator licensing program goes beyond simple task completion.

Response: The Commission expects that thos *a training departments that cannot readily and safely absorb the examination development work will use the funds that they were previously payin9 to the NRC through the fee recovery program to secure the additional personnel to do the extra work or request the NRC to prepare the examinations. If a facility licensee decides to prepare the examination and, as a result, places insufficient resources on either training or testing, the quality of its proposed licensing examinations or the passing rate on those examinations would most likely suffer. Although many of the facility-prepared examinations have required significant changes to achieve NRC quality standards, the examination results, to date, are generally consistent with the results on previous NRG-prepared examinations, suggesting that the quality of the facility licensees' training programs has not been affected. Therefore, the fact that facility licensees will have the option of preparing the examinations is not expected to have a negative effect on reactor safety.

The NRC acknowledges that the contract examiners identified procedural and training problems in addition to their primary responsibility for preparing and administering the licensing examinations, and that they helped maintain examination consistency by working on

29 examinations in each of the NRC's regions. As noted in connection with the discussion of examination quality, the Commission realizes that the revised examination process increases the possibility of inconsistency. but it believes that the examination criteria in the final version of Revision 8 of NUREG-1021, in combination with the NRC's examination oversight programs, will minimize these inconsistencies so that they remain within acceptable limits.

When the NRC initiated the pilot program, its goal was to eliminate the need for NRC contract examiners without compromising the existing levels of reactor safety. Because NRC examiners will be administering all of the operating tests, the revised process will enable the NRC examiners to accrue more experience in a shorter period of time and may improve the consistency of the operating test evaluations and the licensing decisions. Although the total number of procedures reviewed in the process of developing examinations may be fewer under the revised method, NRC examiners will still be expected to review and identify discrepancies in the procedures tt,at will be exercised during the walk-through portion of the operating test and during the simulator scenarios.

Other Comments Since beginning the pilot examination program, the NRC has sought to obtain up-to-date insights regarding the effectiveness of the revised examination process based on the staff's growing body of experience in reviewing the facility-prepared examinations. Many of the staff comments received have paralleled the public comments and require no further attention in this notice. However, one recommendation to amend the wording of the proposed regulation is considered worthy of discussion and incorporation. Specifically, it was recommended that the rule should indicate that a key manager would be responsible for submitting the examination because that individual would be in a position to ensure that the facility licensee's operations

30 and training departments apply sufficient resources to prepare a quality examination . The NRC finds that the recommendation is consistent with normal NRC practice and the analogous regulatory requirement in §55.31 (a)(3), which requires **... an authorized representative of the facility licensee by which the applicant will be employed ..." to submit a written request that examinations be administered to the applicant. Therefore, the wording of the final examination rule has been amended to require an authorized representative of the facility licensee to approve the written examinations and operating tests before they are submitted to the NRC for review and approval.

Availability of Guidance Document for Preparing Operator Licensing Examinations As a consequence of preparing and administering the initial operator licensing examinations over a number of years, the NRC has developed a substantial body of guidance to aid its examiners. That guidance has been published in various versions of NUREG-1021, the latest version of which (final Revision 8) incorporates lessons learned since interim Revision 8 was published in February 1997, as well as refinements prompted by the comments submitted in response to the FAN of August 7, 1997 (62 FR 42426), which solicited public comments in conjunction with the proposed rulemaking. A copy of the final version of Revision 8 of NUREG-1021 will be mailed to each facility licensee; in accordance with NRC practice, revision$ of NUREG-1021 are announced in the Federal Register when they are issued and become effective six months after the date of issuance. Copies may be inspected and/or copied for a fee at the NRC's Public Document Room, 2120 L Street NW (Lower Level),

Washington, DC. Final Revision 8 of NUREG-1021 is also electronically available for downloading from the internet at "http://www.nrc.gov "

The NRC will prepare, administer, and grade initial operator licensing examinations

31 when requested by facility licensees and at least four times a year to maintain the proficiency of its examiners. NRC examiners will use the criteria in the effective version of NUREG-1021 to evaluate whether an applicant meets the Commission*s regulations. In this regard, NUREG-1021 is comparable to the Standard Review Plan (SRP), which establishes the criteria that the NRC uses to evaluate Part 50 license applications. Licensees that elect to prepare their own licensing examinations will also be required to use the guidance in the effective version of NUREG-1021 . As provided in NUREG-1021 , licensees may identify differences from the NUREG-1021 examination criteria and evaluate how the proposed alternatives provide an acceptable method of complying with the Commission 's regulations. The NRC staff will review any proposed alternatives and make a decision regarding their acceptability. The NRC will not approve any alternative that would compromise its statutory responsibility of prescribing uniform conditions for the operator licensing examinations .

Final Rule This regulation adds a new section, §55.40, "Implementation," to Subpart E of 10 CFR Part 55. Paragraph (a) of §55.40 states the NRC's intent to use the criteria in the version of NUREG-1021, "Operator Licensing Examination Standards for Power Reactors," in effect six months before the examination date when preparing and evaluating the written examinations required by §§55.41 and 55.43, and the operating tests required by §55.45. The NRC uses the criteria in NUREG-1021 to evaluate whether an applicant meets the Commission's regulations. In this regard, NUREG-1021 is comparable to the Standard Review Plan, which establishes the criteria that the NRC uses to evaluate Part 50 license applications.

Pursuant to Section 107 of the AEA of 1954, as amended, the NRC must prescribe uniform conditions for licensing individuals applying for operator licenses.

32 Based on the success of the pilot examination program, paragraph (b) of §55.40 allows power reactor facility licensees to prepare, proctor, and grade the written examinations required by §§55.41 and 55.43 and to prepare the operating tests required by §55.45, subject to the following conditions:

(1) To ensure uniformity pursuant to the AEA, the facility licensee shall prepare the examinations and tests in accordance with NUREG-1021; (2) To minimize the possibility that the required written examinations and operating tests might be compromised, the facility licensee shall establish, implement, and maintain procedures to control the security and integrity of the examinations and tests; (3) To ensure that the facility licensee's operations and training departments apply sufficient resources to prepare a quality examination, 8.n authorized representative of the facility licensee shall approve the examinations before they are submitted to the NRC for review and approval; and .

(4) To ensure that NRC standards for quality are maintained, the facility licensee must receive Commission approval of its proposed written examinations and operating tests before they are given.

These requirements are contained in §§55.40(b)(1 ), (2), (3), and (4) respectively.

As provided in NUREG-1021, licensees may identify differences from the NUREG-1021 examination criteria and evaluate how the proposed alternatives provide an acceptable method of compliance with NRC regulations. The NRC staff wiil review any proposed alternatives and make a decision regarding their acceptability. However, the NRC will not approve any alternative that would compromise its statutory responsibility of prescribing uniform conditions for the operator licensing examinations. The NRC staff will review the facility-prepared written examinations and operating tests against the criteria in NUREG-1021 and direct whatever changes are necessary to ensure that adequate levels of quality, difficulty, and consistency are

33 maintained. After the NRC staff reviews and approves a written examination, the facility licensee will proctor and grade the examination consistent with the guidance in NUREG-1021.

The NRC staff will continue to independently administer and grade the operating tests, review and approve the written examination results . and make the final licensing decisions. The facility licensee will not conduct parallel operator evaluations during the dynamic simulator or the walk-through tests.

Pursuant to the requirements in §55.40(c) , the NRC staff will prepare the licensing examinations and tests upon written request by a power reactor facility licensee in accordance with §55.31 (a)(3). In addition, the NRC may exercise its drscretion to reject a power reactor facility licensee's determination to prepare the required written examinations and operating tests, and to proctor and grade the written examinations. The NRC will then prepare, proctor, and grade the written examinations and prepare the operating tests for the facility licensee.

This provision of-the regulation allows the NRC to maintain its proficiency and to perform these activities if the NRC questions a licensee's ability to prepare an acceptable examination.

Paragraph (d) of §55.40 reasserts that the NRC will continue to prepare and administer the written examinations and operating tests for non-power reactor facility licensees. The NRC has taken this position because the non-power reactor community does not have an accreditation process for training and qualification or the resources to prepare the examinations.

This regulation also amends §55.49 because the NRC has determined, since the proposed rule was published, that applicants, licensees, and facility licensees may be interpreting §55.49 too narrowly by limiting it to actual cases of cheating. The amendment clarifies that the regulation pertains to all activities that could affect the equitable and consistent administration of the examination, including activities before, during, and after the examination is administered.

34 Environmental Impact: Categorical Exclusion The NRC has determined that this rule is the type of action described as a categorical exclusion in 1o CFR 51 .22(c)(1 ). Therefore. neither an environmental impact statement nor an environmental assessment has been prepared for this regulation.

Paperwork Reduction Act Statement This final rule amends information collection requirements that are subject to the Paperwork Reduction Act of 1995 (44 U.S.C. 3501 et seq). These were approved by the Office of Management and Budget (0MB), approval number 3150-0101. The additional public reporting burden for this collection of information is estimated to average 500 hours0.00579 days <br />0.139 hours <br />8.267196e-4 weeks <br />1.9025e-4 months <br /> per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information (i.e., preparing the examinations). The additional, one-time burden for power reactor facility licensees that elect to prepare their licensing examinations to establish procedures to prevent the examinations from being compromised is not expected to exceed 100 hours0.00116 days <br />0.0278 hours <br />1.653439e-4 weeks <br />3.805e-5 months <br /> per facility; and the burden of maintaining those procedures is estimated at approximately 1O hours per facility per year. Send comments on any aspect of this collection of information, including suggestions for reducing the burden, to the Information and Records Management Branch (T-6F33), U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001, or by internet electronic mail to bjs1@nrc.gov, and to the Desk Officer, Office of Information and Regulatory Affairs, NEOB-10202, (3150-0101 ), Office of Management and Budget, Washington, DC 20503.

35 Public Protection Notification The NRC may not conduct or sponsor, and a person is not required to respond to , a collection of information unless it displays a currently valid 0MB control number.

Regulatory Analysis

• The Commission has prepared a regulatory analysis on this regulation. The analysis examines the costs and benefits of the alternatives considered by the Commission. The regulatory analysis is available for inspection in the NRC Public Document Room, 2120 L Street NW (Lower Level), Washington, DC. Single copies of the analysis may be obtained from Siegfried Guenther, Office of Nuclear Reactor Regulation, U.S. Nuclear Regulatory Commission, at 301-415-1056 or by e-mail at sxg@nrc.gov.

Regulatory Flexibility Certification In accordance with the Regulatory Flexibility Act of 1980, (5 U.S.C. 605(b)), the Commission certifies that this rule does not have a significant economic impact on a substantial number of small entities. Thi~ rule affects only the licensing and operation of nuclear power plants. The companies that own these plants do not fall within the scope of the definition of "small entities" described in the Regulatory Flexibility Act or the Small Business Size Standards stated in regulations issued by the Small Business Administration at 13 CFR Part 121.

36 Small Business Regulatory Enforcement Fairness Act In accordance with the Small Business Regulatory Enforcement Fairness Act of 1996, the NRC has determined that this action is not a major rule and has verified this determination with the Office of Information and Regulatory Affairs of 0MB.

Backfit Analysis In the proposed rule , the NRC took the position that the backfit rule (10 CFR 50.109) did not apply because the proposed shift in responsibility for preparing the examinations: (1) would not constitute a "modification of the procedures requirr;d to operate a facility" within the scope of the backfit rule; (2) would not have affected the basic procedures for qualifying licensed operators; and (3) would not have required facility licensees to alter their organizational structures. However, upon further review, the NRC has concluded that there is insufficient basis to support the original position. Therefore , the NRC has decided to revise the final rule so that power reactor facility licensees may elect to prepare their written examinations and operating tests (and proctor and grade the written examinations) in accordance with NUREG-1021 or to have the NRC prepare the examinations. Eliminating the requirement for all facility licensees to prepare their examinations and tests obviates the need for a backfit analysis.

Enforcement Policy In conjunction with this final rule, the Commission is separately publishing modifications to NUREG-1600, "General Statement of Policy and Procedure for NRC Enforcement Actions,"

to address enforcement action against parties subject to the requirements in 10 CFR 55.49

37 (i.e., Part 55 license applicants/licensees and Part 50 licensees) .

List of Subjects in 10 CFR Part 55 Criminal penalties, Manpower training programs , Nuclear power plants and reactors, Reporting and recordkeeping requirements.

For the reasons given in the preamble and under the authority of the Atomic Energy Act of 1954, as amended; the Energy Reorganization Act of 1974, as amended; and 5 U.S.C. 552 and 553; the NRC adopts the following amendments to 1o CFR Part 55.

PART 55 - OPERATORS' LICENSES

1. The authority citation for Part 55 continues to read as follows:

AUTHORITY: Secs. 107, 161, 182, 68 Stat. 939, 948, 953 , as amended, sec. 234, 83

• Stat. 444, as amended (42 U.S.C. 2137, 2201, 2232, 2282); secs. 201, as amended, 202, 88 Stat. 1242, as amended, 1244 (42 U.S.C. 5841, 5842).

Sections 55.41, 55.43, 55.45, and 55.59 also issued under sec. 306, Pub. L.97-425, 96 Stat. 2262 (42 U.S.C. 10226). Section 55.61 also issued under secs. 186, 187, 68 Stat.. 955 (42 U.S.C. 2236, 2237) .

38

2. In §55.8, paragraph (c)(4) is revised to read as follows:

§55 .8 Information collection requirements: 0MB approval.

(4) In §§55.40. 55.41 , 55.43, 55.45, and 55.59, clearance is approved under control number 3150-0101.

3. A new §55.40 is added to read as follows:

§55.40 Implementation.

(a) The Commission shall use the criteria in NUREG-1021, "Operator Licensing Examination Standards for Power Reactors, 1 in effect six months before the examination date to prepare the written examinations required by §§55.41 and 55.43 and the operating tests required by §55.45. The Commission shall also use the criteria in NUREG-1021 to evaluate the written examinations and operating tests prepared by power reactor facility licensees pursuant to paragraph (b) of this section.

(b) Power reactor facility licensees may prepare, proctor, and grade the written examinations required by §§55.41 and 55.43 and may prepare the operating tests required by

§55.45, subject to the following conditions:

(1) Power reactor facility licensees shall prepare the required examinations and tests in accordance with the criteria in NUREG-1021 as described in paragraph (a) of this section; (2) Pursuant to §55.49, power reactor facility licensees shall establish, implement, and maintain procedures to control examination security and integrity; (3) An authorized representative of the power reactor facility licensee shall approve the required examinations and tests before they are sutmitted to the Commission for review and approval; and

39 (4) Power reactor facility licensees must receive Comm ission approval of their proposed written exam inations and operating tests .

(c) In lieu of paragraph (b) of this section and upon written request from a power reactor facility licensee pursuant to §55.31 (a)(3) , the Commission shall, for that facility licensee, prepare , proctor, and grade, the written examinations required by §§55.41 and 55.43 and the operating tests required by §55.45. In addition , the Commission may exercise its discretion and reject a power reactor facility licensee's determination to elect paragraph (b) of this section , in which case the Commission shall prepare, proctor, and grade the required written examinations and operating tests for that facility licensee.

(d) The Commission shall prepare, proctor, and grade the written examinations required by §§55.41 and 55.43 and the operating tests required by §55.45 for non-power reactor facility licensees.

1 Copies of NUREGs may be purchased from the Superintendent of Documents, U.S.

Government Printing Office, P.O. Box 38082, Washington, DC 20402-9328. Copies are also available from the National Technical Information Service, 5285 Port Royal Road, Springfield, VA 22161. A copy is available for inspection and/or copying in the NRC Public Document Room, 2120 L Street, NW (Lower Level), Washington, D.C.

4. Section 55.49 is revised to read as follows:

§55.49 Integrity of examinations and tests.

Applicants, licensees, and facility licensees shall not engage in any activity that compromises the integrity of any application, test, or examination required by this part. The integrity of a test or examination is considered compromised if any activity, regardless of intent, affected, or, but for detection, would have affected the equitable and consistent administration

40 of the test or examination . This includes activities related to the preparation and certification of license applications and all activities related to the preparation, administration, and grading of the tests and examinations required by this part.

Dated at Rockville, Maryland, this i '1 f'.h day of April, 1999.

For the Nuclear Regulatory Commission .

~

, ,c:/} ~ ., . ~

- y l -

Annette Vietti-Cook,

/

Secretary of the Commission.

1j DOCKETED NUCLEAR REACTOR LABORATOR{,J SNRC AN INTERDEPARTMENTAL CENTF:R OF MASSACHUSETTS INSTITUTE OF TEC~LOj~ 12 p5 :34 JOHN A BERNARD Activation Analysis Director Coolant Chemistry Director at Reactor 8~.>?'.:1'. ,:r:: Nuclear Medicine Principal Researcr E'"',::::,rt-t:"' Reactor Eng1neer1ng October 23, 1997 DOCKET NUMBER PROPOSED RULE PR 55 (tp~Fl<<</~'121-)

U.S. Nuclear Regulatory Commission Washington, D.C. 20555 Attn: Document Control Desk

Subject:

RIN 3150-AF62, "Initial Licensed Operator Examination Requirements" Gentlemen:

The Massachusetts Institute of Technology is opposed to the adoption of the proposed amendment that would require licensees to prepare, proctor, and grade the written examinations and prepare the operating tests that the NRC currently uses to evaluate the competence of individuals applying for operator licenses. While MIT recognizes that the proposed rule would only affect power reactors and hence have no direct impact on the research reactor that is operated by MIT, MIT nevertheless feels that the proposed change is detrimental. Our reasons are as follows:

I. The administration of exams to determine competency for the receipt of an initial license in most professions is normally performed by an independent agency. Examples include the medical, legal, professional engineering, and health physics professions. There is good reason for this practice and the proposed rule would represent a very substantial deviation from it. .) j

2. The proposed rule, if adopted, would undermine public confidence in the regulation of \\,

nuclear activities. This loss of confidence would extend to all reactor operations, not 3.

merely power plants.

The preparation and administration of initial licensing exams by an independent agency serves as a quality assurance check on the entire training process at each licensee.

\1 Adoption of the proposed rule would undermine that independent check and over time quite possibly result in the ingrowth of undetected weaknesses in training programs.

4. The administration of initial licensing exams to operator candidates by an independent agency fosters a sense of professionalism in the candidates. It makes the candidates realize that they have both a responsibility to their employer and also a larger responsibility to the nuclear community as a whole. If individual employers prepare and administer the exams, then the candidates' sense of responsibility to the commui1it:> as a whole will be greatly lessened.

Acknowledged by cam __!!! -2 199 9710290004 971023 PDR ADOCK 05000020 P PDR 1111111111111111111 1 I t 7 Hiil

~ A 1111 6lfil 1111 1

U.S. NUCLEAR REGULATORY COMMIS RULEMAKINGS &ADJUDICATIONS STf.

OFFICE OF THE SECRETARY OF THE COMMISSION Docll1\8l'1t Statistics Postmark Date M -01>t.ttmu,T Co"Tro/ 'Pes/c on 1/rz,fq&-

Copies Receiv8d l *--==-

Add'I Reproduced H :_

S~I DistrlbUtion I()y me .ssi.atJ1

_j)-~ II-j h< r; ?I> I? , "R. l n.s

U.S. Nuclear Regulatory Commission Page 2 for .the above reasons, we would urge that the proposed amendment be rejected.

Sincerely.

£:n~B~nar~

Director MIT Nuclear Reactor Laboratory JAB/CRM cc: USNRC - Region l - Project Scientist Radiation Safety Branch USNRC - Regi<w I - Senior Project Manager Nuclear Power Reactor and Decommissioning Projects Directorate

DOCKETED December 2, 1997 USNRC ET Walter W. Hunt PfaOSED -...... 55 "97 DEC -3 All :oa 2408 Westwind Drive Soddy Daisy, TN 37379

( t, ~ F ~ 4-a. l/:J. ~)

OFFIC:: OF SECHc ! *B (

RUL Efv! . nr* .::i ,,/ D Secretary, U.S. Nuclear Regulatory Commission AOJUDIC," i Iv, S -~TAFF Washington, DC 20555 The purpose of this letter is to provide comments on the proposed rule change to 10 CFR Part 55. The proposed rule change would require all nuclear power facility licensees to prepare, proctor, and grade the written examinations, and the operating tests used to evaluate individuals applying for operator licenses. The proposed rule change should NOT be approved.

- The catalyst for this proposed rule change was a reduction in the NRC budget.

Subsequently, a utility proposed that the utilities prepare the examination, similar to the methods used for preparation of the requalification examination. The proposed rule making in no way resembles the original utility suggestion. Once life was breathed into this ill conceived proposed rule change, it took on a life of it's own and pundits within the NRC, NEI, and the industry have taken positions trying to explain the proposed rule making's logic. Having been involved in operator licensing for over 20 years, and having had the very unpleasant experience of participating in the "pilot" process I am convinced that the proposed rule making has no logical basis, is a model of poor change management, is destructive for the industry and in fact, will increase greatly the cost of operator licensing.

A rational analysis of the motives of those involved in the process of licensing operators for nuclear facilities would conclude that we all; NRC examiners, utility managers and utility training instructors want to assure that only fully competent individuals receive a license to operate a nuclear facility. In order to achieve this goal, each of us should focus very specifically on the tasks necessary to prepare and examine a candidate. The proposed rule making diverts the attention of all involved in the process and dilutes their focus . The NRC examiners, now assumed by the Commission to have excessive free time, are assigned to other inspection duties and devote only the time required to "review" the examination materials. The NRC examiners are so burdened with additional assignments that trying to get an examiner's attention to review exam material is all but impossible, except during dedicated time periods. Many examiners have publicly documented their opposition to the proposed rule making. Making a comparison of the published comments of examiners to recent NRC Exam Reports, the conclusion could be drawn that examiners' opinions of the proposed rule making have influenced the results of examinations given under the pilot process.

OEC 199L Acknowledged by

U.S. Mn.EAR REGUlATORYCOMM1Ss10N RUl.EMAKJNGs &ADJUDICATIONS 8TAfT OFFICE OF THE SECRETARY OF THE COMMISSION DocllnentStaUsb Postma11c Date Coples Received

>>-em,* ff~I 6V ~,,-"/ r;~(/"'_j/,.(1,,

/ ~

t),') 12./2/t:;7 ('t"efrit't"hl-Fro,,., ,~-kr4cf;~~ ruk l,,,,e.iln.5 iv,6.11'ft!)

Add'I ~ Reproduoocf If

. . 1(-~ - - - -

- -~ .;.J.. /,~ J----':.....k!.1~.L.l:.L~ -

The facility employees, instead of totally focusing on assuring the candidates are trained to the greatest level, are diverted from their primary purpose, as much as six months before the exam, placed under a security agreement and can no longer provide input to the candidates training. The only way the facilities can fully devote their attention to the training of their candidates is to hire contractors to write the exam. As a result, many facilities will choose to utilize contractors to write their exams. Therefore, this proposed rule making will simply transfer the cost of contractors from the NRC to the utilities. At most, a utility will prepare one examination every year and a half to two years. This minimal contact with examination preparation does not foster consistency or develop highly skilled examiners.

The proposed rule making will NOT enhance the competency of operators. The proposed rule making will NOT enhance the safe operation of nuclear facilities. The proposed rule making will NOT enhance the consistency of examinations given to candidates.

The proposed rule making WILL increase the facilities' cost of licensing individuals. The proposed rule making WILL divert training resources from ensuring the competency of operators.

Attached are specific comments on the proposed NUREG 1021.

Cordially, Walter W. Hunt

10 CFR 55 Rule Change If it is NRC' s intent to require licencee' s to follow NUREG 1021 verbatim for the development of written examinations and operating tests per 10 CFR 55.40(a), then 10 CFR 55.40(a) should reference the NUREG or state " ... per NRC guidance."

NUREG 1021,interim revision 8 ES201 Item C.3.j - The crew composition used during the simulator evaluations should be the same used by the licensee at the plant. If the plant uses a Shift Manager and an STA, then a shift manager and a STA should be used during the simulator evaluations. This practice will eliminate any negative training conducted just to prepare for the NRC licensing examination. NOTE: ES-301, item D.1.e requires SRO applicant to be examined for the highest on-shift position for which the SRO license is required. For SQN, that position is the Shift Manager.

Item D.2.a - The restrictions imposed for the individuals who develop the test outlines are overly restrictive with no apparent benefit. The rules applied to those developing written questions would also be appropriate to those developing test outlines. The adoption of less restrictive use of test outline developers would allow the licensee more flexibility in the use of their training resources with no significant impact on test integrity.

Attachment 2 - This attachment should be rewritten to clearly state that licensees who develop their written examinations and operating tests only need to provide the NRC the material necessary to review and validate the examination. The list of submittal items in the attachment is necessary only if the NRC develops the written examinations and operating tests.

ES301 Item B.2.3 - Same comments as ES201, Item C.3.j.

Item D.l.c - If the licensee has an approved SAT based training program, it would be inappropriate to examine a candidate based on NRC's KIA catalog vice the site-specific task list. The site-specific task list should reflect the jobs the individual would perform at the facility.

Items D.2.f, D.3.b, and D.4.d - The development and submittal of operating test outlines is an unnecessary administrative burden for licensees if they are developing the operating test. Licensees should only be required to submit the operating test materials. To facilitate NRC review, it would be appropriate for the licensee to submit the operating tests 60 to 90 days before the test date.

NUREG 1021,interim revision 8 ES401 Items C.l.a and D.1 -The development and submittal of a written examination outline is an unnecessary administrative burden for licensees if they are developing the examination. Licensees should only be required to submit the proposed examination. To facilitate NRC review, it would be appropriate for the licensee to submit the examination 60 to 90 days before the test date.

Item D .1. b - The requirement to select at least one topic from each category in every tier of the outline can not be met unless examination questions can include more than one test outline topic. Both the SRO and the RO outline contain more than 100 topics.

Item D.l.b - If the licensee has an approved SAT based training program, it would be inappropriate to examine a candidate based on NRC's KIA catalog vice the site-specific task list. The site-specific task list should reflect the jobs the individual would perform at the facility.

Item D.2.f - The requirements listed to assure examination integrity and security are overly complicated and require an inordinate amount of licensee resources with little or no increase in examination integrity or security. Sufficient controls would exist if:

• Repeat of no more than 25% of examination questions used during the candidates' training program.

• No repeat of examination questions used during the licensee's final audit examination or similar testing vehicle given at or near the end of the candidates' training program.

• No repeat of examination questions used during the last two NRC licensing examinations at the facility.

• Provide controls for preventing the review and study of licensee question banks by license applicants.

Following the above guidelines, ensures that 75% of the examination would be examination questions that the candidate had not seen.

NUREG 1021,interim revision 8 Avvendix E Item Part E - Five minutes to familiarize yourself with plant conditions is not appropriate for all scenarios. Consideration should be given for scenarios with abnormal initial conditions that are not normally encountered at the plant.

December 2. 1997 NOTE TO: Emile Julian Chief. Docketing and Services Branch FROM: Carol Ga 11 agher RES. ORA

SUBJECT:

DOCKETING OF COMMENT ON PROPOSED RULEMAKING Attached for docketing is a comment letter related to the Proposed Rulemaking on Initial Licensed Operator Examination Requirements. Amendments to 10 CFR 55. This letter was received on our interactive rulemaking website on December 2. 1997 . The commenter's name and address are Walter W. Hunt.

2408 Westwind Drive. Soddy Daisy. TN 37379. Please send a copy of the docketed comment to Harry Tovmassian (mail stop T9-F-31) for his records.

Attachment:

As stated cc w/o attachment :

H. Tovmassian

South Carolina Electric & Gas Company Gary J. Taylor A

SCE&G P.O. Box88 Jenkinsville, SC 29065 (803) 345-4344 DOC KETED USNRC Vice President Nuclear Operations ASC.-.=:Coo;,any "97 OCT 30 P4 :19 October 24, 1997 OFFIC : :*: ;: 2r:,::.Rcr,9 V~fl 219 RUt f::./ ,;.., * ":-* ;'1 * -,

Secretary ADJU ICAT.!i_::. -~ :~ .. *-F U.S. Nuclear Regulatory Commission Washington, DC 20555 Attn: Rulemakings and Adjudications Staff

Dear Sir,

P'"~ '~

I ,- *, I \ 11 I ER na

, o PULE..!.r~5~ '---

Subject:

vIRGIL c. SUMMER NUCLEAR STATION {__6J_ F fl '-I 1.'-f'J.O)

DOCKET NO. 50/395 OPERATING LICENSE NO. NPF-12 PROPOSED RULE ON INITIAL OPERATOR EXAM REQUIREMENTS COMMENT South Carolina Electric and Gas (SCE&G) offers the following comments to the proposed changes to 10CFR55.

SCE&G does not agree with the proposed rule in its present form. SCE&G supports the comments as provided by NEI in response to the Federal Register Notice concerning proposed rulemaking on initial licensed operations examination requirements (62 Federal Register 42426 of August 7, 1997). In addition to the comments made by NEI, SCE&G believes that critical tasks should not be used during Integrated Plant Operation evaluations, as required by ES-301, D.4.d. Critical tasks, defined in Appendix D, place an unnecessary development burden on utilities. The competencies outlined in ES-303 are sufficient to conduct an accurate evaluation of a candidate's ability to safely operate the plant's systems under dynamic, integrated conditions.

The administrative requirements that have been added during and after the pilot program have increased the examination preparation effort significantly. Not only is this a large burden on the training staffs, the requirements of the process very much limits the versatility of the training staffs. An example of this is the requirement that the personnel involved in developing the exam can have limited previous involvement with the class, and once a member of the training staff acquires knowledge of the exam, he can no longer have any training involvement with the class. This may prevent instructors that are subject matter experts from working with students due to their potential involvement in the exam a year later. The complexities associated with NU REG 1021 , by definition, limits the utility staff expertise to a few individuals, thereby minimizing their full utilization in the operator training programs. This situation is also a problem for 0 2 6 1997 Acknowledged by

- - - - - - - -- y SGE&G Comments on Initial Operator Exam Proposed Rule Page 2 of 2 training supervision and management who are responsible for accomplishing both tasks of training the candidates and preparing the exams.

With the revised criteria contained in Revision 8 to NU REG 1021, future contractor prepared examinations will involve more effort, resulting in higher fees and does not necessarily ensure a better exam. Exams developed by the NRG are not subject to the same restrictions that utilities are. This may lead to the development of an exam that meets the requirements of NUREG 1021 but is not a representative sampling of the candidates real knowledge. For example, a utility can not have more that 25% of the questions on the exam that were used in the training program that was built and evaluated using a systems approach to training, where the NRC is not limited to this requirement because they have no knowledge of what was taught.

SGE&G suggests that a better "partnering" approach between the NRG and the utilities should be developed to improve the examination process.

We appreciate the opportunity to comment on this proposed rule. If you have any questions, please call Mr. Terry Matlosz of my staff at (803) 931- 5100.

cac c: J. L. Skolds W. F. Conway R.R. Mahan R. J. White A. R. Johnson NSRC RTS (PR 970004)

File (811.02, Folder 55.006)

OMS

DOCKETED CP&L US C Carolina Power & Light Company VI OCT 29 P4 :14 PO Box 1551 411 Fayetteville Street Mall

.IC; ,...,L: ,- - - ., _ i ,. o* CP&L Letter: PE&RAS-97-089 Raleigh NC 27602 OFF c. 1..,, ,, 1

• l~-* "'yo:. ._ b 2 1997 RUl F:M~f."'*'..\.'D ctoer 1, AD "JUDI(.,;, -';\. l_ I d

• • *-*1*AFF

~.)

~~ciP'61~~u~B~; PR 5 (62 FYl LJ1s-J 26)

Secretary U.S. Nuclear Regulatory Commission Washington, DC 20555 Attn: Rulemakings and Adjucations Staff

Subject:

Comments on NRC Proposed Rule on Initial Licensed Operator Examination Process (62 FR 42426)

Dear Sir or Madam:

Attached are the comments of Carolina Power & Light Company (CP&L) on the NRC Proposed Rule on Initial Licensed Operator Examination Process. In general, CP&L supports this change

- as a cost-effective improvement in the examination process.

Please contact me at (919) 546-6901 if you have questions.

Sincerely, D.B. Alexander, Manager Performance Evaluation & Regulatory Affairs HAS Attachment f<<W 2 6 1997 Acknowledged by cam ...... . . . . . . . . ._,_ __

Page 2 CP&L Letter PE&RAS-97-089 October 21, 1997 Comments on NRC Proposed Rule on Initial Licensed Operator Examination Process (62 FR 42426) cc: Mr. L.J. Callan, Executive Director for Operations Mr. S.J. Collins, Director, USNRC Office of Nuclear Reactor Regulation Mr. L.A. Reyes, Regional Administrator, Region II Mr. J.B. Brady, USNRC Resident Inspector - HNP, Unit 1 Mr. B.B. Desai, USNRC Resident Inspector - HBRSEP, Unit 2 Mr. V.L. Rooney, USNRC Project Manager - HNP, Unit 1 Ms. B.L. Mozafari, USNRC Project Manager - HBRSEP, Unit 2 Mr. C.A. Patterson, USNRC Resident Inspector - BSEP, Units 1 and 2 Mr. D.C. Trimble, USNRC Project Manager - BSEP, Units 1 and 2 Chairman J.A. Sanford - North Carolina Utilities Commission USNRC Document Control Desk

. Page 3 CP&L Letter PE&RAS-97-089 October 21, 1997 Attachment Comments on NRC Proposed Rule on Initial Licensed Operator Examination Process (62 FR 42426)

1. With regard to the specific NRC questions, CP&L has the following comments:

a) Question 1: Are there portions of the operator exams that are common to all licensees, and would therefore be more efficiently developed by the NRC?

CP&L considers that common elements have already been included in the Generic Fundamentals Examination Section (GFES).

b) Question 2: Is the conclusion in the regulatory analysis correct that it would be less costly for each licensee to prepare their own initial operator examinations to be reviewed, revised, and administered by the NRC, than to have one NRC contractor prepare these exams for all licensed operators with the costs to be reimbursed by licensee fees?

CP&L considers that the cost of future licensee-prepared examination packages, while difficult to predict accurately, is anticipated to be less expensive than NRC-contractor prepared examinations.

2. With regard to Interim Revision 8 of NUREG-1021, CP&L has the following specific comments:

a) ES-201 D.2.a: CP&L recommends that the restrictions on facility employees during the initial licensed operator examination process be the same as those in the requalification process.

b) ES-301 D.4: The instructions for Category C require a specific number of Component and Instrument Failures for each candidate. But, in the past, there has been some confusion about whether the component/instrument failure may be placed in the scenario after EOP entry. CP&L recommends that ES-301 be revised to provide stronger guidance on the NRC's intent to restrict component/instrument failures after EOP entry.

c) ES-201 C.3.j and ES-302 D.1.j: For some critical evolutions, station procedures may require the allocation to the control room of additional resources, beyond those required by the Technical Specifications. In order to be consistent with the training program and representative of how the plant will be operated, CP&L recommends that NUREG-1021 be revised to permit the use of additional surrogates during critical evolutions when the station procedures require the allocation of additional resources, as well as when required by the Technical Specifications.

d) ES-501 and Form ES-501-2: On Form ES-501-2, Power Plant Licensing Examination Results Summary Sheet, Marginal Pass is denoted with a "#" sign.

However, the precise definition of what constitutes "Marginal Pass" is not included.

CP&L recommends that ES-501 or Form ES-501-2 be revised to clarify the definition of "marginal pass."

, . Page 4 CP&L Letter PE&RAS-97-089 October 21, 1997 Attachment Comments on NRC Proposed Rule on Initial Licensed Operator Examination Process (62 FR 42426) e) ES-202 D.1.b.5: The general improved reliability of nuclear power plants over the past five years presents more limited opportunity for applicants (and licensed operators) to manipulate the controls of the reactor during a diversity of significant reactivity and power changes. Also, the simulator reactivity and power models have generally improved and become more realistic. To reduce the possibility of a plant perturbation solely to fulfill a training requirement, CP&L recommends that the NUREG be revised to acknowledge the manipulation of the controls on the simulator as a fully acceptable substitute for all but one manipulation of the controls on the reactor. If deemed necessary by the NRC, CP&L suggests that the NUREG be revised to increase the number, significance or diversity of the simulator reactivity and power changes to compensate for the reduction in the required number of reactor reactivity and power changes.

f) ES-301 D.4.f: The requirement for critical tasks was included in neither the pilot program nor the draft revision 8 of the NUREG. The generation of scenarios in accordance with the guidance provided in interim revision 8 of the NUREG will ensure that each candidate is evaluated on all required failures and that the scenarios are designed to evaluate each candidate in all listed competency areas. The inclusion of the requirement for critical tasks will create an area of subjectivity for the chief examiner and will require additional work, for both the utility and the chief examiner to develop and validate these individual critical tasks, which add low value to the examination process. For these reasons, CP&L recommends that the requirement for critical tasks be eliminated from the NUREG.

g) ES-301-4 1: There appears to be no purpose or value in having clearly stated objectives in the scenario summaries, since the scenario objectives and qualities are driven by the development process as stated in the NUREG. CP&L recommends deleting the statement about " ... having clearly stated objectives in the scenario summaries" from the NUREG.

h) ES-301-4 11: The statements that "Each operator will be evaluated using at least one new scenario ... All other scenarios have been modified in accordance with Section D.4." appear to conflict with ES-301 D.4 which states "In order to maintain test integrity, every applicant shall be tested on at least one new or significantly modified scenario ... " CP&L recommends the statements in ES-301-4 11 be modified to agree with ES-301 D.4.

i) ES-501 E-1: ES-501 E.l states, "If any of the applicants failed the written examination, the region shall delay issuing licenses to those applicants that passed the written examination with a grade of 82 percent or below until all written examination appeals have been reviewed for impact on the licensing decisions." A relatively quick evaluation would permit the issuance of those licenses for which the potential impact of the appeal process could not result in failure. CP&L recommends that ES-501 be clarified to indicate that the specified review is such an evaluation and to permit the issuance of those licenses for which the appeal process cannot result in failure.

DOCKET NU vi BER PR r- C-PR *? 0 ED RULE...::...:.::.....;:;;J;....;;..

DOCKETED october 22 1f 992. F- (<, J:./ 2. 1-/ 1.. 6 )

US C '

John Munro "97 OCT 28 P3 :15 20855 Blossom Landing Way Sterling, Va. 20165 The purpose of this letter is to provide comments on the proposed rule change to 10 CFR Part 55 that would require all nuclear power facility licensees to prepare, proctor, and grade the written examinations, and prepare the operating tests for NRC use in the evaluation of individuals applying for operator licenses at those plants. Although the effort expended by the NRC staff and a number of facilities to develop the examinations has been commendable, the proposed rulemaking and associated revisions to the Examiner Standards (Interim Revision 8), as presently configured, are not sufficiently prescriptive to ensure continued nationwide consistency with respect to level of knowledge tested and examination difficulty. NUREG-1021 does not provide objective criteria that ensure consistent and objective measurement of examination levels of knowledge and difficulty. Furthermore, the NRC review and approval process cannot adequately compensate for the clear conflict of interest problems that are inherent to this approach. (This last concern is best illustrated by a comment made by a facility licensee training manager to the NRC examination team during preparation of a recent examination. This individual apparently indicated that his intent was to make the examination as easy as possible to ensure all the candidates passed (Memorandum from P. Steiner to L. Reyes dated July 30, 1997).)

Therefore, for these reasons and the additional concerns discussed below regarding certain rationale provided in support of the rule change in the discussion sections of 62 FR 42426 and SECY-97-079, I respectfully recommend the proposed rulemaking be disapproved and that other options be considered for changing the examination process to effect the necessary resource

- savmgs.

- The expectation that with training and experience the industry will gain proficiency in preparing the examinations does not appear to be supported by recent experience. Two years into the process, many of the facility developed examinations continue to require significant rework. In some of these cases, parts of the facility licensee_submitted examinations have been determined to be inadequate. (62 FR 42426)

- The assertion that this approach will improve the NRC's staff focus on operator performance may be faulty. None of the commission papers associated with the rule change have assessed or shown this expectation to be correct. While NRC contact time with the applicants will clearly increase, it does not necessarily follow that this increased focus will result in the identification of additional operational weaknesses. In fact, some examiners would likely argue the contrary view as more correct. They will now become so focused on the examination tasks at hand that their ability to identify generic Page 1 of Acknowledged by card .........................~

"'S

operational concerns and weaknesses will be affected. Finally, the characterization of the contractor examiners' efforts as of little benefit to the NRC beyond the examination task is not correct. The contractor examiners were fully functioning members of the examination team and were often instrumental in the identification of numerous operational and procedural weaknesses. (62 FR 42426)

- SECY-97-079 states that NUREG-1021 has been modified to establish a standard that at least half of the examination questions be written at the comprehension or analysis level.

However, since the NUREG-1021 guidelines place no constraints on the quality or size of the facility bank before its use for examination development, the inevitable result will be the improper categorization of simple memory level questions as higher level comprehension/analysis level questions. The examination will then fail to meet the standard that at least half the examination questions be written at cognitive levels greater than the memory, i.e., up to 90% of the questions on the examination could then be written at the memory level.

- SECY 079 also states that NUREG-1021 contains criteria that should help establish an appropriate level of difficulty for the examinations but then indicates the personal experience and judgement of the author and reviewer remain the most important factors.

While this is the methodology that has been traditionally utilized, the addition of an indeterminate number of facility employees who may not have been trained in examination development principles will significantly increase the variation in the level of examination difficulty unless more objective and consistent measurements for adequate test item level of difficulty are developed.

Also enclosed are several comments regarding the proposed Revision 8 to NUREG-1021, Operator Licensing Examination Standards for Power Reactors.

Respectfully, John Munro Page 2 of

ENCLOSURE COMMENTS - INTERIM REVISION 8 TO NUREG-1021 Comment #1: ES-201D.2.c requires that all personnel who will receive detailed knowledge of any portion of the NRC licensing examination acknowledge their responsibilities by reading and signing Form ES-201-3, Examination Security Agreement. This agreement precludes the signer from participation in any instruction, evaluation, or other training-related activity. Although ES-401 allows up to five questions to be duplicated from the applicants' audit examination or other similar testing vehicle given near the end of the licensee training class, the Examiner Standards are silent concerning parallel development by the same authors of both the audit examination, including the operating test audit examination, and the NRC examination. As a result, the same individual(s) may have developed both the applicants' audit written examinations and operating tests, and the NRC licensing examinations. This practice represents a conflict of interest that NUREG-1021 should prohibit. An NRC test drafted by the same individual(s) who wrote the applicants' audit examination would likely be unconsciously biased toward the specific material and level of difficulty tested by the applicants' audit examination, albeit with different questions.

Furthermore, development of the audit examinations for the applicants is clearly a "training-related activity" precluded by the security agreement required to be signed by "all personnel who will receive detailed knowledge of any portion of the NRC licensing examination." Finally, the allowance provided by ES-401 D.2.f for up to five questions to be duplicated from the applicants' audit examination is not consistent with item No. 6 of Form ES-401-6, Written Examination Quality Assurance Checklist, which allows "no question duplication from the licensee screening/audit exam."

Recommendation - The Examiner Standards should clearly prohibit the same personnel from developing both the applicants' audit examination and the NRC examination.

Comment #2: ES-201, Attachment 1, Examination Security and Integrity Considerations, states the facility licensee will exercise the same physical security precautions with the initial examinations as it does with its requalification examinations. However, administration of the pilot-type examinations has shown that not all facility licensees have procedures in place governing the development and administration of requalification examinations. As the NRC examination is now more susceptible to compromise and considering the increased number of examination security breaches that have occurred in the last year, the NRC should specify, in detail, a proceduralized listing of the minimum physical security precautions they will require the facility licensee to set up before the facility licensee can begin development of the NRC licensing examination. ES-201 currently specifies that they will positively control examination-specific materials but provides almost no detail for accomplishing this objective. Furthermore, ES-201 is silent regarding the responsibility for the required physical security precautions should the facility licensee chose to have the examination developed off-site by a contractor.

Page 3 of

Recommendation - For nationwide consistency, the NRC should revise the ES-201 guidelines and include a detailed listing of the minimum physical security precautions the facility will be required to set up. The NRC should require the facility licensees to submit a

listing of the specific steps taken to implement these precautions for NRC review and approval before their commencing any examination development work. The NRC should also require the facility licensee's examination security procedures to specify the steps that they will set up should the test be assigned to an outside contractor organization for development.

Comment #3: ES-301 D.1.a allows for up to three JPMs to be repeated on successive examination days to different applicants. This policy fails to ensure an adequate level of examination security, in that, it does not preclude or address the possible interaction among the applicants between successive examination days. Compliance with 10 CFR 55.49 will be in jeopardy should information on the administered JPMs be passed between applicants thus allowing for an unfair advantage in the examination process for certain applicants. This policy is also inconsistent with that in place for the dynamic simulator examination which precludes the repeat of any scenarios on successive days.

Recommendation - Change ES-301 D.1.a to preclude overlap or duplication between operating tests administered on successive days.

Comment #4: ES-401 D.2.c states that to maintain examination quality and consistency, at least 50% of the questions on the examination will be written at the comprehension/analysis level.

However, the guideline then, contrary to the guidance discussed in Appendix A of NUREG-1021, states the cognitive level of any question drawn directly from a bank will be counted at its face value and makes no provision for the number oftimes the applicants may have been exposed to them. This policy - allowing bank questions to be counted at their initial level of knowledge in perpetuity - was not in draft Revision 8 ofNUREG-1021 but was added to Interim Revision 8 in response to an industry comment that was provided with no supporting basis or justification. The policy is inconsistent with, and contrary to, the goal of maintaining an adequate level discrimination validity for the examination. Appendix A ofNUREG-1021 states that discrimination could decrease to the simple recognition level if the item bank is small and available for the examinees to study. Questions and answers drawn from previously published and reviewed question banks are subject to memorization, and consequently, will fail to properly discriminate between safe and unsafe operators. Since the NUREG-1021 guidelines place no constraints on the quality or size of the facility bank before its use for examination development, the inevitable result will be the improper categorization of simple memory level questions as higher level comprehension/analysis level questions. The examination will then fail to meet the standard that at least half the examination questions be written at cognitive levels greater than the memory, i.e., up to 90% of the questions on the examination could then be written at the memory level.

Page 4 of

Recommendation - ES-401 D.2.c should be changed as follows to reflect the fundamental tenet of discriminant validity as discussed in Appendix A of NUREG-1021:

"The cognitive level of any question drawn directly from a bank will be considered at the memory."

Comment #5 (ES-600 series): Information Notice 95-24 noted that some facilities did not sufficiently control how many test items they repeated between practice and comprehensive examinations or among successive examinations, and revisions made by some facilities to the dynamic simulator scenarios between administrations were so superficial that the required operator actions and mitigation strategies were essentially unchanged. The present NUREG-1021 requalification examination guidelines have no restrictions regarding the reuse of test items and therefore permit the opportunity for excessive test item duplication from examination to examination and from crew to crew. Excessive test item duplication may adversely affect examination validity and jeopardize examination integrity (10 CFR 55.49). Limitations on the use of written exam bank questions and duplication from test to test were considered for Interim Revision 8 but not incorporated because of industry comment.

Recommendation - For consistency, the guidance provided to the regional staff on this issue in task Interface Agreement, 95TIA009, and Report on Interaction, No. 96-34, should be incorporated in the ES-600 series guidelines.

Comment #6: ES-401 D.l provides general instructions for development of the written examination outline and emphasizes the identification of plant-specific items for testing on the written examination. However, the guidance does not specify that the sample plan should be developed consistent with the principles discussed in Appendix A of NUREG-1021. This oversight, and the guideline's apparent emphasis on selection of plant specific items, can lead to a biased examination that does not ensure the inferences regarding the examinee's competency for the thousands of required knowledges and abilities not sampled by the test are proper.

Examination report Nos. 50-369/97-300 and 50-370/97-300 indicated that facility test developers had eliminated from consideration for testing any knowledge and ability from NUREG-1122, Knowledge and Abilities Catalog for Nuclear Power Plant Operators: Pressurized Water Reactors, that did not possess a facility licensee learning objective.

Recommendation - The guidance provided in ES-401 for development of the written examination outline should be revised consistent with the principles detailed in Appendix A ofNUREG-1021 and require the sample plan to be systematically developed. Further, the guidance should indicate that while plant-specific concerns may be used to override some items selected for testing by sample plan, these deviations must be discussed with and approved by the NRC chief examiner prior to their implementation.

Page 5 of

October 27 . 1997 NOTE TO: Emile Julian Chief. Docketing and Services Branch FROM: Carol Gallagher RES . ORA ~~

SUBJECT:

DOCKETING OF COMMENT ON PROPOSED RULEMAKING Attached for docketing is a comment letter related to the Proposed Rulemaking on Initial Licensed Operator Examination Requirements. Amendments to 10 CFR 55 . This letter was received on our interactive rulemaking website on October 24 . 1997. The commenter's address is 20855 Blossom Landing Way, Sterling, VA 20165. Please send a copy of the docketed comment to Harry Tovmassian (mail stop T9-F31) for his records.

Attachment:

As stated cc w/o attachment :

H. Tovmassian

Commitment. Innovation. Energy.

DOCKETED USHRC James M. Levine TEL (602)393-5300 Mail Station 7602 Palo Verde Nuclear Generating Station Senior Vice Presid~

Nuclear ocr 2Ffx Aff)~a6o77 P.O. Box 52034 Phoenix, AZ 85072-2034 OFF!:;:-: .,f  :);:-:r 1,1~~ ,94031 - JMUAKK/KR RULEf'v-1_: r 1 ... '.~ 1Q~ ber 17, 1997

• * *

• ADJUDI,r* ;--.,* I I( - ,.-- ,..1AFF J .:: I \.)

Rulemakmgs and AdJudrcatrons Staff U. S. Nuclear Regulatory Commission Washington, D.C. 20555

Subject:

Palo Verde Nuclear Generating Station (PVNGS)

Units 1, 2, and 3 Docket Nos. STN 50-528/529/530 Comments for Proposed Rule for NUREG-1021, Interim Revision 8, "Operator Licensing Examination Standards for Power Reactors" These comments are submitted by Arizona Public Service Company (APS) on behalf of PVNGS Units 1, 2, and 3 in response to the subject draft NUREG-1021, Interim Revision 8. We appreciate the opportunity to have participated in the voluntary pilot examination program and to comment on the NRC's proposed rule.

APS particularly wishes to emphasize comment #10 in the enclosure, regarding significant reactivity manipulations. The use of simulators for these manipulations should be allowed. The present requirement could place an economic impact on utilities where full power operation could be limited in order for trainees to manipulate the controls. In addition, maneuvering the plant for training purposes can result in unnecessary plant transients and could impact plant safety. The industry has invested heavily in fidelity simulation facilities. These facilities accurately reflect the plant conditions and provide the controlled training environment necessary for effective training. A large variety of complex control manipulations can be conducted on these simulators that are not allowable on the actual plant.

If you have any questions regarding these comments, please contact F. W. Riedel, Operations and Engineering Training Department Leader at (602) 393-6580.

Sincerelf l_

µ~

JMUAKK/KR/mah ct Ackn ,,,1 e ged y card .............................._.

Enclosure

1:! ~'-; :

r*r,*

1"-.J'.

Rulemaking and Adjudications Staff Comments for Proposed Rule for NUREG-1021, Interim Revision 8, "Operator Licensing Examination Standards for Power Reactors" Page2 cc: K. E. Perkins (all with enclosures)

J. L. Pellet S. L. McCrory J. H. Moorman

ENCLOSURE NUREG-1021 OPERATOR LICENSING EXAMINATION STANDARDS FOR POWER REACTORS INTERIM REVISION 8

NUREG-1021, Operator Licensing Examination Standards for Power Reactor, Interim Revision 8 The following comments are in regards to the proposed rule change to 10 CFR 55.40 and the resultant changes to NUREG-1021.

The proposed rule change is as follows:

(a) Power reactor facility licensees shall prepare the required site-specific written examinations and operating tests.

(b) Power reactor facility licensees shall submit the written examinations and operating tests to the Commission for review and approval.

(c) Power reactor facility licensees shall proctor the site-specific written examinations.

(d) In lieu of requiring a power reactor facility licensee to prepare the examination and tests, and to proctor the site-specific written examinations, the Commission may elect to perform those tasks.

(e) The Commission will prepare and administer the written examinations and operating tests at non-power reactor facilities.

PVNGS' comments regarding NUREG 1021 Interim Revision 8 are as follows:

1. ES-201 B second paragraph: Changed from "shall participate" to "that choose to participate."

COMMENT: Rulemaking will require the utilities to participate. This paragraph will need to change to reflect rulemaking .

- 2. ES-201 C.1.e second paragraph: Allows for "Supervisors who approved the course materials but did not perform instructional activities for the license applicants may review and approve the examination materials."

COMMENT: Clarification is needed as to what constitutes "instructional activities." Is the supervisor allowed to have contact with the class in accordance with ES-201, D.

Personnel Restrictions?

3. ES-201 C 2.h second paragraph: States "If any of the facility-developed examination materials (written, walk-through, or simulator) require substantive changes and cannot be made to conform with the examination standards at least five working days before the scheduled examination date, regional management shall consult OLB and make a decision whether to proceed with the facility-developed examinations or develop the examinations in-house." It goes on to further state that "If examination problems are identified at the last minute, the examinations should be postponed: substantive examination changes will not be made during the examination week."

COMMENT: Substantive changes should not be made during the examination week.

Some guidance as to what constitutes "substantive" would be helpful.

4. ES-201 C 3.f third sentence: Adds requirement that the Chief Examiner exercise their judgement to ensure that the exams are prepared to the appropriate level of difficulty.

COMMENT: This requirement is entirely subjective and results in the "moving target" acceptance criteria. This is currently being interpreted differently in the different NRC regions. Clear guidance needs to be established so that the facility writers can hit the mark the first time and not require substantive changes to the exam.

5. ES-201 D 2.a: Significantly changed the personnel restrictions. The guidance now states that "Facility employees who had any direct involvement in training the license applicants shall not prepare the outlines for the written examinations or the operating tests." Also, "no one who provided 15 percent or more of the scheduled classroom instruction or 20 percent or more of the total scheduled classroom and simulator instruction may participate in developing the written examination questions.

COMMENT: Originally, simulator training was exempted from this requirement.

Simulator contact time should not be factored into the criteria for who can write written test questions. The contact time limit should be for the classroom instruction not including the GFES.

6. ES-201 D 2.c: Now clearly requires signing the security agreement before gaining knowledge of the exam.

COMMENT: Once the security agreement is signed, it does not allow for any training related activities with the applicants until the exam is completed. The security agreement needs to be revised to allow training related activities (i.e.

supervisor review and approval of training materials).

7. ES-201 Attachment 2 first paragraph: Adds flexibility to provide limited reference material. The fifth paragraph calls for sending everything unless otherwise directed.

These two conflict.

COMMENT: The facility should not be required to send any reference material to the regional office. Reference material should only be provided if requested to support exam validity.

8. ES-201 Attachment 2 1.a last paragraph: Adds that the material sent to the NRC be "complete, comprehensive and of sufficient detail to support the development of accurate and valid examinations without being redundant."

COMMENT: As in the previous comment, reference material should not be provided for facility written exams. This paragraph should be deleted.

2

9. ES-201 Attachment 2 15.b: Increases the material required on malfunctions available on the simulator. It specifically requires a: "list of all malfunctions with identification numbers and cause and effect information, including a concise description of the expected range of results that will occur upon initiation and an indication of which annunciators will be actuated as a result of the malfunction."

COMMENT: This is available to the NRC during the exam validation week at the site. Exam validation eliminates the need to send this documentation.

10. ES-202 C.2.b fourth paragraph: Requires significant control manipulations ... at least five are required on the facility for which the license is sought.

COMMENT: The use of simulators for these manipulations should be allowed. The present requirement could place an economic impact on utilities where full power operation could be limited in order for trainees to manipulate the controls. In addition, maneuvering the plant for training purposes can result in unnecessary plant transients and could impact plant safety. The industry has invested heavily in fidelity simulation facilities. These facilities accurately reflect the plant conditions and provide the controlled training environment necessary for effective training. A large variety of complex control manipulations can be conducted on these simulators that are not allowable on the actual plant.

11. ES-301 B first paragraph: The lead in statement requires the applicant to demonstrate an understanding of and the ability to perform the actions necessary to accomplish a representative sampling from the 13 items identified in 10 CFR 55.45(a).

COMMENT: The problem here is that this is the only place that this requirement exists. The skyscraper model of ES-401 should be modified to ensure that this requirement is met when selecting test items for inclusion on the exam. Currently, if the exam is developed in accordance with ES-401, this requirement probably won't be met.

12. ES-301 B.2: The applicant's knowledge and abilities relative to each system are evaluated by administering JPMs and specific follow-up questions.

COMMENT: Recommend that the prescripted questions for the JPMs be eliminated.

They have little discriminatory value and this testing medium is already accomplished via the written exam and Category A, "Administrative Topics."

13. ES-301 E.2.a: The chief examiner shall independently review each operating test for content, wording, operational validity, and level of difficulty.

COMMENT: This requirement is entirely subjective and results in the "moving target" acceptance criteria. This is currently being interpreted differently in the different NRC regions. Clear guidance needs to be established so that the facility writers can hit the mark the first time and not require substantive changes to the exam.

3

14. ES-301, Form ES-301-4: The target quantitative attributes at the bottom of the QA checklist list requirements for scenarios and scenario sets.

COMMENT: This is not described anywhere in the text. The only way you would know this is if you used the QA checklist. Normally this is done after the scenarios are developed.

15. ES-303 0.2.c first bullet under the fifth paragraph: Allows for failure of an applicant who made errors with serious safety consequences for the plant or the public, even if the failure can not be justified on the basis of the competency grades.

COMMENT: There should be no case in which an applicant makes an error of significant safety consequences that can not be justified by competency grades.

This item should be deleted from the standard or the grading tool should be modified.

4

DOCKETED US RC /v 2..J./ 1frJ "97 OCT 25 P2 :l 2 L-97-269 Secretary U.S. Nuclear Regulatory Commission Washington DC 20555 [ '"'"-T r ! ' ** * .I Attention: Rulemaking and Adjudications Staff C6 LF ~ ~1- Lf 2-6

Subject:

Proposed Rule Initial Licensed Operator Examination Requirements Request for Public Comment On August 7, 1997 (62 FR 42426), the NRC requested public comment on the proposed rulemaking which proposes to amend its regulations to require all nuclear power facility licensees to prepare, proctor, and grade the written examinations and prepare the operating tests that NRC currently uses to evaluate the competence of individuals applying for operator licenses at those plants. These comments are submitted on behalf of the Florida Power and Light Company (FPL), a licensed operator of two nuclear power plant units in Dade County, Florida and two nuclear power plant units in St. Lucie County, Florida.

The Nuclear Energy Institute (NEI) is issuing comments on the subject proposed rulemaking. FPL endorses the NEI comments and recommendations. FPL especially agrees that this process should continue to be voluntary.

FPL appreciates the opportunity to comment.

Very truly yours,

~."0 ?o_~

H. N. Paduano Manager Licensing and Special Programs Acknowledged by card ..................................

an FPL Group company

• '11l::J

" ,J *s *n

DOCKET NUMBER PR r S PROPOSED RULE..:..:;.;;;....;;.>.,_,...,___

DOCKETED October 21. 1997 ((;,2- Fl< 42'-l 26) USNRC Secretary U.S. Nuclear Regulatory Commission "97 OCT 22 PS :48 ATTN: Docketing and Service Branch Washington. D.C. 20555 OFFIC~ ": * .,.,. , .*t

'..:.r* , ' , :::

• ii,

pv

,. , I ,'""'\ , , I Ru! [ \A t ,.. 'r

- l\ i j~~

• • ,,1 \ 1._;., / I

, ,j.,.....,

__ .I I am providing comments on the "Proposed Rule for &9~~p]c R;1~-~n:Y~f t-ial Reactor Operator License Examinations." My initial response to this process when it was first proposed in 1995 was that I was opposed to the concept.

After two years of participating in the pilot process my opinion has not changed. I am still opposed to the concept of the utility developing the initial examination used for licensure of their candidates. I feel that this process change will be a disaster of undue proportions. It will be about five years into this process before the NRC realizes that this methodology was flawed.

Before this process started the NRC and its contractors administered examinations to licensees* candidates as unbiased and impartial participants.

Our independence and impartiality have helped with maintaining the NRC's charter of protecting the health and safety of the public. I do not believe this new method will allow me to provide the same impartial and objective view. Allowing the Part 50 licensees the option of developing examinations will dilute and circumvent this impartial process. I have observed in a number of pilot examinations attempts to dilute and decrease the difficulty of the examinations.

In the discussion section of Federal Register Notice (FRN)62FR42426, it states that "The pilot program demonstrated that the revised process. using licensee developed examination. can be both effective and efficient." It has been my experience that this process is neither effective nor efficient. The amount of time that it takes me to perform a review of an inadequate product has been extensive. During the pilot process examiners were required to provide feedback about the process using a pilot questionnaire. The pilot questionnaire asks "How did the overall quality and difficulty of the examinations compare with recently administered NRC-developed examinations?"

This question on the surface appears to be a valid question. The question was answered that the examination was comparable. However, that was only after extensive modification by the Chief Examiner. It has been my experience that, in the majority of cases. the overall examinations submitted were SUBSTANDARD when compared to the last NRC generated examination. Of course the examinations under this new process would be comparable because the NRC examiners ensured they would be. In reviewing the pilot questionnaire data from October 1. 1995 through April 5. 1996. examiners have reported 12 of the 22 exams submitted were inadequate in quality, requiring extensive examiner effort too correct.

The FRN further states that "With training and experience. it is expected that the industry would gain proficiency in preparing the examinations." I am not sure how this statement has any credence. The process. as it stands. has been going on for about two years. How much time does the industry need to gain proficiency? Granted some of the licensees have not had that much experience in development of examinations used by the NRC but they do run training

_- [8[1:111 Acknowledged by card ..................................

~ .- . ._;:~ ,'  : I.

• A?1 f :t: '* . 1* :.'..! :: ., . '..,:)

1:1 ~-1s c~;--Jou.\*~m:-:1'G* * =s:,, *,:.---;:r:;-:n1::1 NOIS,..,it":r*:08 ;,1101:,nn :-i::iti !:!'J:fi:::r:*J *s*n

departments and examine their employees throughout the year. This leads to question the quality of the examinations provided during the annual requalification examinations. NRC examiners and their contractors are provided extensive training in order to learn how to develop valid examinations in accordance with the criteria delineated in NUREG-1021, "Operator Licensing Examiner Standards." We the examiners. have been training the utility "training" personnel in the concepts of developing valid examinations. Teaching the utility under this method has been through trial and error. During one examination when I was the chief examiner. the utility was reluctant to change any part of the examination. This evolution was time consuming and resulted in a great deal of conflict between the NRC examiners and that utility's training staff. As a result. NRC-Utility management meetings have taken place to resolve communications problems associated with this particular examination. Another obstacle that may be encountered which could hinder this process is that utilities may assign "new" personnel to test development on successive examinations. This may essentially void the effective training that was provided to the previous examination developers.

Licensee instructors who develop and evaluate training effectiveness are supposed to follow INPO document 88-12. Guidelines for Instructor Training and Qualification. This document requires that training personnel to understand test item construction. It has been my experience that most training personnel lack the skill to develop valid test items. To my knowledge there has been no provision to determine the qualifications of licensee personnel developing examinations or to provide them training on how to develop an NRC style examination. It is also my understanding that the NRC has no intention of inspecting in this area.

The FRN further states that "The fact that NRC examiners will be administering all of the operating tests without contractor assistance is expected to improve the NRC staff's focus on operator performance and its core of experience because every applicant will be directly observed by an NRC employee. Before beginning the transition process. contractor examiners administered about half of the operating tests and collected the observations that formed the basis for NRC's licensing actions. The contractor's efforts focused primarily on task completion. so any broader insights and experience that might have been gained while giving the examinations was of little benefit to the NRC." This statement is incorrect. Prior to the Pilot process. the examination teams usually consisted of at least one NRC examiner with two contractor examiners depending on the number of candidates that were examined. The NRC examiner acted as the Chief. In order to appease certain parties. HOLB changed the examination process to a prescripted task-based examination using Job Performance Measures (JPMs). This was done because there was some question as to the efficacy of allowing non-NRC personnel to make licensing decisions. In my opinion, the contract examiners maintained consistency between the regions. A contractor from one of the three contractors could visit all four regions. Since the contractors consistently wrote most of the written examinations they were able to apply knowledge deficiencies found in one region and apply it to those they subsequently visited. They provided more than just a body to administer an examination.

They were not as the FRN states " . . . of little benefit to the NRC." It is amazing to me that if this was the feeling of NRC management. why did we spend 3 to 4 million dollars a year using their services?

It is my opinion that the licensee ' s training department has a vested interest in obtaining licensees for their students they have trained to be operators at their utility. The licensee will do anything to develop the easiest possible examination in order to see if the NRC will accept it. There are many examples that the utilities are not testing at the comprehension or analysis level as is required by NUREG-1021. They have nothing to loose if the product is not discriminatory. The worst possible action we would or could take would be not to give the examination on the scheduled date. and the least would be to write an unsatisfactory examination report . In any event. the NRC examiners would be required to FIX the substandard product . My colleagues and I have taken painstaking efforts to revise examinations in order to meet the scheduled deadline .

Allowing the licensee the opportunity to write the examination. the licensee may not test in areas that are appropriate. As an NRC examiner I researched facility materials . operating procedures. administration procedures. and emergency operating procedures to find plausible pitfalls that may cause the candidate to possibly take a wrong path . With the licensee developing the examinations we do not find those problem areas being tested as we would have in the past . The NRC examiners will not be as familiar with the procedures and will not find problems in these areas . Examiner knowledge and proficiency will also degrade. because a reviewer does not have to have the same knowledge base to review as they do to write an examination. It is more difficult to fix those questions and simulator examinations as time goes on.

In discussions with licensee training personnel I have heard most of them express their opposition and dislike of this new process. They do not feel that the process saves them money, in fact. in most cases it has been more costly to the utility then having the NRC generate an examination . In these discussions they describe that they were caught in the middle of what their management wants (new licensed operators) and what the NRC desires (a test that determines minimal competent operators). They feel like they are in a compromising no-win position.

At a number of sites. security of the examination process has been a big concern to all NRC examiners. The utility personnel do not think on the same level when dealing with security. On a previous examination a proctor inadvertently provided operator candidates information about the examination that was about to be administered. The NRC examiners had to stop the examination process. evaluate this occurrence. and develop a spare scenario in order to complete the examinations . Many examples of a loss of examination security have been identified and documented in examination reports . This process clearly increases the threat to examination security.

In closing, I believe the proposed rule does not justify nor warrant change.

In light of the conflict of interest. degradation of examiner proficiency, and the cost to the licensee. we need to reevaluate our beliefs and expectations concerning this change.

Richard S. Baldwin 632 Kingsgate Ridge Stone Mountain. GA 30088-1822 L__

October 22, 1997 NOTE TO: Emile Julian Chief, Docketing and Services Branch FROM: Carol Gallagher RES, ORA ~~

SUBJECT:

DOCKETING OF COMMENT ON PROPOSED RULEMAKING Attached for docketing is a comment letter related to the Proposed Rulemaking on Initial Licensed Operator Examination Requirements, Amendments to 10 CFR 55. This letter was received on our interactive rulemaking website on October 21 , 1997 . The commenter ' s address is 632 Kingsgate Ridge, Stone Mountain, GA 30088-1822. Please send a copy of the docketed comment to Harry Tovmassian (mail stop T9-F31) for his records.

Attachment:

As stated cc w/o attachment :

H. Tovmassian

I DOCKETED US NRC NUCLEAR ENERGY INSTITUTE "97 OCT 20 All :22

_Ralph E. Beedle October 20, 1997 Secretary nocKEr NUMBER PRoPosEo RULE PR ~s- -=,

J U.S. Nuclear Regulatory Commission Washington, DC 20555 (6 2 FR Lj

J_LJ 26)

ATTN: Rulemakings and Adjudictations Staff

SUBJECT:

Proposed Rule for Initial Licensed Operator Examination Requirements -- 62 Federal Register 42426 -- August 7, 1997 PROJECT NUMBER: 689

• These comments are submitted on behalf of the nuclear power industry by the Nuclear Energy Institute (NEI) 1 in response to the Federal Register notice concerning proposed rulemaking on Initial Licensed Operator Examination Requirements (62 Federal Register 42426 of August 7, 1997).

We believe that the shift to licensee prepared initial licensed operator examinations has improved the examination process and should be continued. The industry would prefer to continue the voluntary process that has worked well for the past year. A voluntary process would allow flexibility for a few licensees with small training staffs. Requiring that all licensees prepare the examination package is preferable to the previous practice of using contractor prepared examinations.

Currently, licensees voluntarily prepare 70 percent of initial licensed operator examination packages, using the criteria of Interim Revision 8 to NUREG-1021.

Both the industry and NRC staff agree that the licensee prepared, NRC reviewed and approved examinations meet the high standards that have been established.

1 NEI is the organization responsible for establishing unified nuclear industry policy on matters affecting the nuclear energy industry, including regulatory aspects of generic operational and technical issues. NEI's Members include all utilities licensed to operate commercial nuclear power plants in the United States, nuclear plant designers, major architect/engineering firms, fuel fabrication facilities, materials licensees, and other organizations and individuals involved in the nuclear energy issue.

'GCT"I t,-18t ..

Acknowledged by card ..............,,,,,m..........,.

PH <JNf 2 02 7)9 . 8089 FAX 202 1'8 .':J l b98

U.S. MUC!..'.:AR RE,J:J!..tTOfW COMMISSION RULEiHK i 1(:S & AD 'UD'vATIOt 'S STAFF

Secretary October 20, 1997 Page 2

• The industry believes that recent licensee prepared examination packages are of better quality than those prepared prior to the pilot program. Prior to the pilot program, licensee technical reviews resulted in many corrections to examination packages.

The Commission asked whether there are common portions of the operator exams that could be more efficiently prepared by the NRC. The industry believes that all common material is already included in the Generic Fundamentals Examination discussed in ES-205. All elements required for current licensed operator examinations are best prepared as part of a site specific examination package.

The Commission also asked for views on whether licensee prepared examinations were less costly than NRC contractor prepared examinations. We believe that in the future, with clearly established and stable criteria, an industry prepared examination package will be less expensive than contractor prepared examinations under the same criteria.

Realistic cost comparisons have been difficult. Some facilities found significant savings using the pilot process when compared to previous contractor prepared examinations. Other facilities found that a higher level of effort was required,

• particularly when expectations or requirements shifted while the facility was actively preparing the examination package. Administrative requirements added during and after the pilot program have increased the examination preparation effort. The industry finds that preparing an examination takes about 50 percent more effort than estimated at the start of the pilot program. With the revised criteria on Revision 8 to NUREG 1021, future contractor prepared examinations would also involve more effort, resulting in higher licensee fees.

Relative cost of the two processes should not be the only factor considered in deciding whether to continue licensee prepared examination packages. Preparing quality examinations with higher cognitive level questions requires detailed plant knowledge that can be best provided by the licensee. Under the revised process the NRC staff can also concentrate on directly evaluating each candidate, without reliance on third party observers.

Specific comments on Interim Revision 8 to NUREG 1021, Operator Licensing Examination Standards for Power Reactors, are provided in the enclosure.

The industry's most significant concern with the standards continues to be the personnel restrictions of ES-201, Section D, that preclude use of most instructors who have trained a candidate. These restrictions have the most impact on facilities with small training staffs. It was our understanding that the original intent was to preclude exam preparation by those few individuals with primary responsibility for

Secretary October 20, 1997 Page 3

• conducting the entire license training class. The clarification provided in Interim Revision 8 to NUREG 1021 has significantly expanded personnel restrictions. We continue to strongly endorse the criteria used in ES-601 for requalification examinations as the standard for initial licensed operator examinations.

We appreciate the opportunity to comment on this proposed rule. Should you have further questions, please contact Jim Davis of the NEI staff at 202-739-8105.

Sincerely, Ralph E. Beedle JWD/rs Enclosure c: Mr. Robert Gallo, NRR/DISP/PSIB

Enclosure Specific Recommendations on Interim Revision 8 to NUREG-1021

• Operator Licensing Examination Standards for Power Reactors The following comments and recommendations are made on Interim Revision 8 to NUREG-1021:

1. ES-201 C.1.a.

We recommend changing the second paragraph to read:

The facility licensee should respond in writing to the NRC's annual administrative letter soliciting estimated operator licensing needs and

,wtify its IVRC regional olfiee if its examination requirements change significantly from those stated in its response. Facility licensees are strongly encouraged to schedule their examinations with their NRC regional office before commencing an initial license training class. The NRC regional office should be notified if examination requirements change from those previously scheduled.

Rationale: Per NRC Administrative Letter 97-02, scheduling has become a regional function.

2. ES-201 C.2.c The time line for submission and review of examination packages "leaves the licensee little time to respond to the Chief Examiners concerns. In a few cases reviews were delayed by limitations on regional staff resources. Some in the industry have recommended submitting the examination package 60 days instead of 30 days before the examination. Most believe the current time line is optimum if it is rigorously adhered to. This section should add discussion of completion dates for review and feedback. Additionally, the facility should be allowed to propose earlier submission and review dates if necessary.

3. ES-201 D.2.a We recommend making the restrictions on facility employees the same as those in the requalification process. Specifically, replace the second paragraph with:

'To ensure examination security, each facility representative who acquires knowledge of the content of the NRC initial examination before it is administered will be subject to the security restrictions described _below from

the time he or she first acquires the specific knowledge until the examination

• exit meeting. "

Rationale: The detailed process used to develop and review the examination package will preclude an instructor from unconsciously biasing the examination. The first step, preparation and approval of a sample plan, determines the specific areas to be examined. When combined with the detailed criteria for preparing the written and operating test, an objective examination package is assured. The personnel restrictions of this section have impacted the efficient use of training staffs, particularly at single unit facilities. The industry continues to believe that the personnel restrictions applied to the requalification examination are also fully adequate for the initial operator licensing examination.

4. ES-201 Attachment 2 We recommend changing the last paragraph to read:

For a licensee prepared examination, reference material will be submitted as agreed with the regional office. For NRC prepared examinations, unless otherwise instructed by the NRC regional office, the facility licensee is expected to provide the following reference materials for each NRC initial licensing examination:

Rational: ES-201 C.l.d indicates that the amount of reference material submitted to the regional office is based on the level of facility participation in exam preparation. In ES-401, examination authors are encouraged to provide specific explanations for each question. Reducing the reference material submitted is an important aspect of licensee prepared examinations.

ES-201 Attachment 2 should clearly reflect the need to specify the reference material actually needed for review of a licensee prepared examination.

5. Form ES-201-3 Security Agreement Change the third sentence to read:

I understand that I am not to participate in any instruction ~ evaluation, eF other trairii,ig related aetivitiea involving those applicants scheduled to be administered these licensing examinations from this date until completion of examination administration.

Rational: ES-201 C.l.e was revised to allow a supervisory review of the examination package and to participate in pre-NRC examination review 2

boards. This would constitute "other training related activities," conflicting

• with the security agreement as written.

6. ES-301 B This background section requires a sampling from the 13 items identified in 10 CFR 55.45(a). It is not clear whether the operating test must include a sample from each of these areas. The detailed operating test dcv~lopmcnt process does not contain any checks to verify that each area has been covered.

The background discussion should be clarified to indicate that sampling process does not have to include items from each of the 13 areas or specific checks added to the detailed procedure that all areas are covered.

7. ES-301 B.2 We recommend deleting the requirement for prcscriptcd questions. They provide little discriminatory value and topics arc adequately covered in the written examination and in the Category A, Administrative Topics.

8. ES-301 D.1.a We recommend deleting the third paragraph:

Operating tests *.vritten by the facility licensee may not repeat test items (simulator scenarios or JPl\fo) from the applicants' audit test given at or near the end of the license training class. Simulator events that are similar to events that were tested on the audit examination arc permitted provided the actions required to mitigate the transient arc significantly different from those required during the audit examination.

Rational: The requirement to have different actions to mitigate an event would preclude the use of some accident types if they had been used on the audit examination. For example, in a Steam Generator Tube Rupture plant, emergency procedures dictate a mitigation strategy that cannot be made "significantly different". The chance of duplication of audit examination scenario types is the same for both licensee prepared and NRC prepared examinations.

3

9. From ES-301-4 We recommend changing Qualitative Attribute 11 to read:

Every operator will be evaluated using at least one new or significantly modified scenario. All other licensee bank scenarios have been modified altered in accordance with Section d.4 of ES-301.

Rationale: This wording better reflects the requirements of ES-301 D.4.

10. ES-302 D.3.i The printed document distributed to facilities is missing part of the first sentence in this section. The electronic version contained the missing material.

11. ES-303 D.2.c We recommend deleting the second paragraph under the first bullet item:

If the applicant made (an) error(s) with serious safet~/ consequences for the plant or the puhlie, the e:£aminer ean recommend a failure even if the failure ean not he justified on the hasis of the competency grades. In such eireumstarwes, the re-giorb must ohtain written eoneurrenee from the Chief, OLB, hefore completing the licensing action.

Rationale: There should be no case in which an applicant makes an error of significant safety consequences that cannot be justified by the competency grades.

12. ES-401 B As with re qualification, use of these site-specific task lists should be allowed in development of content-valid licensing examinations.

Rationale: The last paragraph in this section refers to NUREG-1122 and NUREG-1123 as the source of K&As for examination outline development.

Some facilities have developed plant-specific operator task lists. These task lists include importance and difficulty ratings.

4

13. ES-402 D.4.d With the changes to require 50 percent of the written examination be at a higher comprehension and analysis level, the time limit for completing the examination should be changed to five hours .

• 5

--=-- ENTERGY DOCKETED USNRC Entergy Operations, Inc.

PO Bo> 31995 Jackso, 1. MS 39286-19%

Tc:I 60 1 3t'i8 50()[J 0)

'97 OCT 17 All :46 OFFICE Or SECRE*i; r~ y RULEl\*1AKlt* G"J 1\i* *O ADJUDICATIONS STAFF October 14, 1997 OOCl(ET NUMBEA'PR ~

PR POSED RULE.-=-=-=......::,.....,. .J _..,,.

Secretary U. S. Nuclear Regulatory Commission ( 62 F Jt Lj 2 J./ 1-6)

Washington, DC 20555-0001 Attention: Rulemakings and Adjudications Staff

Subject:

Comments on the Proposed Changes to 10 CFR Part 55 Regarding Initial Licensed Operator Examination Requirements CN RO-97 /00020 Gentlemen:

Entergy Operations, Inc. appreciates the opportunity to comment on the subject proposed changes to 10 CFR Part 55, as documented in the Federal Register, Volume 62, Number 152, dated August 7, 1997. Entergy supports the comments provided by the Nuclear Energy Institute (NEI).

In addition to NEl's comments, Entergy has a concern regarding personnel restrictions specified in NUREG-1021, "Operator Licensing Examination Standards for Power Reactors," Interim Revision 8. Section D, "Personnel Restrictions," of ES-201, "Initial Operator Licensing Examination Process,"

precludes the use of most instructors who have trained a candidate. These restrictions place an undue burden on the licensee since it would appear that the instructors in question would not even be allowed to train students in generic fundamentals. It is our understanding that the original intent of the restrictions was to preclude exams prepared by those few individuals with primary responsibility for conducting the entire license training class. The clarification provided in Interim Revision 8 to NUREG-1021 significantly expands this original intent. We believe the security criteria for requalification examinations found in ES-601, "Conducting NRC Requalification Examinations," are also adequate for initial licensed operator examinations. Therefore, we recommend ES-201 be revised to reflect the criteria specified in ES-601.

1rtrrs*1 fB7 Acknowledged by card ..................................

l' '~.: * ... , .., ...

... ... M ' 0 0 L~/../~*J*.\: :.

Comments on Proposed Changes to 10 CFR Part 55 October 14, 1997 CN RO-97 /00020 Page 2 of 2 Again, thank you for the opportunity to provide our comments on this proposed rule.

Sincerely, JGD/SJB/GHD/baa cc: Mr. C. M. Dugger (W-GSB-300)

Mr. J. J. Hagan (G-ESC3-VPO)

Mr. C. R. Hutchinson (N-GSB)

Mr. J. R. McGaha (R-GSB-40)

Mr. J. W. Yelverton (M-ECH-65)

Mr. Jack N. Donohew, Project Manager (GGNS)

Office of Nuclear Reactor Regulation U.S. Nuclear Regulatory Commission Mail Stop 13-H-3 Washington, DC 20555 Mr. George Kalman, Project Manager (ANO)

Office of Nuclear Reactor Regulation U.S. Nuclear Regulatory Commission Mail Stop 13-H-3 Washington, DC 20555 Mr. Chandu P. Patel, Project Manager (W-3)

Office of Nuclear Reactor Regulation U.S. Nuclear Regulatory Commission Mail Stop 13-H-3 Washington, DC 20555 Mr. David L. Wigginton, Project Manager (RBS)

Office of Nuclear Reactor Regulation U. S. Nuclear Regulatory Commission Mail Stop 13-H-3 Washington, DC 20555

Jim Edgar Thomas W. Ortciger 0)

Governor Director October 1, 1997 Secretary DOCKETED U.S. Nuclear Regulatory Commission OCT 1 0 1997 Washington, D.C. 20555 RULEMAKINGS AND -~

ADJUDICATIONS STAFF ,-...,,

SECY-NRO Attn: Rulemakings and Adjudications Staff

Dear Sir or Madam:

The Illinois Department of Nuclear Safety (IDNS) hereby submits its comments concerning the proposed rulemaking addressing initial licensed operator examination requirements. IONS participates with the Nuclear Regulatory Commission (NRC), Region III, under memoranda of understanding in Illinois licensee inspection programs, and appreciates this opportunity to comment.

The NRC licensed operators at nuclear plants have historically been relied upon to be a second line of defense after the automatic design features of the reactor protection systems to mitigate accident sequences. Because the licensee initially hires and trains operator license candidates, the licensing process is the check and balance that the regulator uses to ensure that qualified persons are controlling nuclear power plants. We believe that having the licensee formulate and administer the final written examination in the initial licensing process diminishes this check and balance system. This check and balance is more critical in initial licensing examinations than requalification examinations, since they establish an individual's basic qualification. We do not believe the same rigor needs to apply to the requalification program.

The written operator licensing examination is a vital screening method to ensure the quality of licensed operators. Written examinations are important and reveal more about the intellectual capabilities of candidates than simulator operating and walk-around examinations.

With operating procedures numbering in the hundreds, it is important to know that licensed operators are capable of being aware of, understanding and following complex written material. The written examination is a means of assuring this capability. The quality and consistency of examination questions can be maintained under the proposed rulemaking because NRC staff can change and approve them before use. But, we believe that the NRC should compile the examination questions and proctor the examination. As recent events tJ Acknowledged y c r ..................................

@ recyclable

T'... * - .. . . * .* ,*-.r-

- * ~J ' * 'I, ** ,_,;,,,

1)os:rnar'~ J: ~ I ~/ / J1__._..1-J_ __

Cop,os R:: .... **.- * {

Addi Cp:"'.:: f *.

• s~,ec::-l o;,t*' .,_;ti-:,'. §1~

ll J h~ J-J p () it!

---

--** -*~ - ..-- -----

Secretary Page 2 October 1, 1997 have shown, it is difficult and disruptive to remove licensed operators from shifts once individuals are licensed.

Licensees administer their own training programs for license candidates. The Institute of Nuclear Power Operations (INPO) accreditation programs are effective in ensuring the program quality. However, having NRC compile and administer a final examination to candidates helps keep the training program instructors and administrators alert, and is a check to monitor the effectiveness of the INPO accreditation program. Such NRC involvement should prevent the training program from "teaching the examination" to ensure success of the candidates over the term of the program, which could be an outcome of allowing the licensee

- to compile as well as administer the written examination, In addition, we are concerned about security of the examinations. Licensees identified two recent instances of examinations being compromised in Region III. Even when no malfeasance is intended, the security of examination questions during the administrative process of formulation and approval is difficult to secure. Culpability and intent are hard to prove if examination questions are found in a copy machine, as recently happened at one plant. IDNS Resident Inspector staff recently discussed how inspection efforts could identify problems in this area and concluded that regulatory inspection programs are not likely to identify instances like this.

In addition, the examination is one indicator of how efficient the training program is in training candidates. Therefore the licensee training departments have a vested interest in a high success rate.

There are not sufficient checks or balances to prevent abuses if the licensee is involved in all aspects of the written examination formulation and administration. We believe the proposed rule changes are asking too much of the regulated community, and will open NRC to serious criticism if abuse is detected. IDNS believes the NRC should continue to provide the check and balance in this system by formulating the initial written examinations as well as administering the test.

0 .'

Thomas W . Ortcige Director TWO:sld

DOCKETED USNRC "97 OCT 10 P2 :42 OFFICE OF SECHE- iARY RULEM,-l.K/r*JC.S AH)

ADJUOICi:..TIO *~, .3TAFF

UNITED STATES NUCLEAR REGULATORY COMMISSION DOCKETED REGION II USNRC ATLANTA FEDERAL CENTER 61 FORSYTH STREET. SW. SUITE 23T85 ATLANTA. GEORGIA 30303 "97 SEP -8 P4 :32 DOCKET NUMBER PROPOSED RULE Pff 55

( ~~Ff<t/:J.'1:;.1,) August 11, 1997 MEMORANDUM TO : Robert Gallo, Chief Operator Licensing Branch Office of Nuclear Reactor Regula *o FROM: Johns P. Jaudon, Director Division of Reactor Safety

SUBJECT:

COMMENTS ON SECY-97-079 The purpose of this memorandum is to forward some comments on SECY-97-079 received from a member of our licensing staff.

The enclosed comments do not, in the view of the individual who wrote them, represent a differing professional view or opinion. Therefore, we recommend that you consider them as comments made by a member of the public.

Enclosure:

as stated cc w/encl : L. Reyes, RII L. Spessard, NRR

~einer, RII SEP 1 2 1997

• clfflowfadged by card -

U.S. NUCLE R REGULATORY COMMISSION RULEWK!NGS &ADJUDICATIONS STAFF OFFICE OF THE SECRETARY OF THE COMMISSION Document Statistics C/ _1____

Postmark Date _ __.;q:...i.-:i.__,_-.-

Copies Received _ _~ - - - - -

Add'I Cop:es Reported =---'-- -:--- - -

Special Distribution a..g

{QV!r\a G-a. l I be r-1 'ED 1? 1J):S_ 4 :sria "1

UNITED STATES NUCLEAR REGULATORY COMMISSION REGION II ATLANTA FEDERAL CENTER 61 FORSYTH STREET, SW, SUITE 23T85 ATLANTA, GEORGIA 30303 July 30, 1997 MEMORANDUM TO: Luis A. Reyes, Region II Regional Admin~or FROM: Paul M. Steiner, Reactor Engineer, DRS ~JC{J ~

SUBJECT:

EXAMINER FEEDBACK TO STAFF REQUIREMENTS -

SECY-97-079 - PROPOSED RULE - INITIAL LICENSED OPERATOR EXAMINATION REQUIREMENTS I am writing in response to the commissioners request for information as outlined in SECY-97~079 regarding the proposed rule change that would require licensees to generate licensed operator initial examinations. For the past year, we have been allowing the facility licensees to generate their own initial examinations under pilot guidance. Region II examiners have been aggressively enacting the pilot program and learning valuable lessons from it. Almost all of the Region II licensees have already worked with us in generating an initial examination.

As with most areas we regulate, we have come across good performers and poor performers.

My concern is that we have compromised our ability to administer an examination that will independently and objectively evaluate operator competency. This would result in an increased risk of an incompetent operator being placed at the controls of a licensee facility.

The purpose of the letter is to be informative. To date, the communications between our headquarters branch of operator licensing and the regional examiners, have been centered on our experiences in implementing the pilot process, not on whether or not we are doing the right thing. I feel strongly opposed to the direction we are taking with regards to the proposed rule change. I have talked with the headquarters branch, and I have volunteered to generate a survey which would poll all NRC examiners about their experiences with the pilot process. The results of the survey will then be discussed at our annual examiner's conference. I also intend to poll the licensee training managers prior to our Region II training manager's conference.

In SECY-97-079, the commission requests that the staff provide additional discussion in the Federal Register notice (FRN) explaining the safety benefits of this proposed rule making, indicating, if appropriate, that the NRC focus on operator performance will improve through more direct (i.e., NRC staff replacing contractor personnel) involvement in s~mulator and walbhrough examinations. I do not believe this to be the case. On the contrary, I believe that the simulator and wa!J....-through examinations have degraded significantly, and are now only at minimal acceptable standards. The depth of simulator scenarios \Vas created by examiners spending months working through EOP's, and AOP's. We developed exam material that ensured an operator could adhere to the facility license under a full range of accident conditions.

This no longer occurs. The training staff pulls a scenario from their bank, which they may have run previously on the candidates, and adds the NRC required instrument and component failures.

This has two negative effects on exam quality. First, the scope of the exam is narrowed to the procedures the licensee believes are both important, and heavily trained on. This is negative in the sense that we are not able to determine if the candidates are prepared to work with all licensee procedures. Second, the NRC loses that valuable review of licensee procedures. The number of procedure discrepancies found during a pilot style exam is lower than that of an NRC generated exam, because the facility has already worked over the bank scenario procedures.

Also, the NRC staff loses the knowledge gained from this intense procedure review. This can be *a problem when the candidate strays off the correct path. Previously, the examiners were exposed to most of the EOP's and AO P's and could figure out where the candidate was going and whether or not that path was acceptable. The emphasis of the examiner's time has shifted from researching licensee material and generating a quality examination, to that of ensuring the licensee has met our guidelines for content. We are regressing into an organization that is primarily focused on enforcing administrative guidance.

• We have r_ecently been faced with a licensee training manager who openly stated (not a direct

~o quotation) that he intended to make the examination as easy as possible ensure all of his candidates passed. He went on to mention that he already thought all o.f the operators were competent and why would he ever generate an exam that would risk failing them. As an example, a question the licensee proposed on the Senior Reactor Operator (SRO) administrative section was, "What does S.T.A.R. stand for?". I suspect that the site janitors know this answer.

It is not even on the scale, relative to the level of knowledge that we require of an SRO candidate. The disturbing part, is that we had to debate with the licensee to have the question removed from "our" examination. It is this type of continual debate over the two month review period that results in a degraded product.

The root cause of our problem is that we have created a conflict of interest with the training managers. They feel we have put them in a no-win situation. If they generate an exam that results in a failure, there management will evaluate them as poor trainers . If they generate an exam that is too simplistic, the NRC will write a negative report. The weak performers have chosen to write the simplistic exam.

The SECY also asks the staff to include in the FRN a discussion of the importance that the NRC places on ensuring adequate security of examination materials. I believe that the NRC places a very important belief on maintaining security, but we have no way of ensuring it. The licensees have always had an opportunity to compromise an examination. Recently, a licensee encountered a willful compromise of exam material. During a non-pilot examination, a copy of an NRC vnitten examination was found in the licensee's copier. We rely heavily on the licensee's integrity and on a security agreement signature stating they will not compromise the exam. In the past, our primary means of ensuring security was to limit the q.mount of time the licensee was exposed to examination materials, usually around three weeks. With the examination being on-site for a much longer period, the potential for exposure increases proportionately. Under the new rule, the exam security process will start around a year and half prior to the exam .. During this time, the licensee will have to separate the exam Vffiter and the

exam material from interaction with the potential candidates. After administering a simulator examination at a Region II licensee, a training member staff informed the examiners that he found a copy of the scenario in the copy machine. This resulted in the examiners switching to a backup scenario, which had to be validated on the spot, resulting in a degraded exam and undue stress placed on the candidates. Clearly the pilot process increases the threat to examination security.

The last item the SECY asks the staff to provide, is additional legal analysis regarding the basis for determining that the backfit rule does not apply to this rulemaking, particularly with regard to modification of or addition to "the procedures or organizations required to design, construct, or operate a facility." The licensees initially proposed that they generate the examinations to improve quality and cost savings. Feedback from the licensees indicates they realize the pilot program and proposed rule change is a mistake. They did not understand the level of difficulty of writing an NRC examination, nor the impact it would have on their staff. A typical licensee generates one initial license class per year. This does not provide them with sufficient

• opportunities to train their staff on how to generate an NRC style examination. This results in the licensee starting from ground zero every year. The typical NRC examiner participates in 12 or more initial examinations per year and becomes very skilled and effictent at generating material. Although the intent of this program was to save contractor do.llars, the licensee is now hiring the NRC contractors, investing an average of seven hundred man-hours per examination, and are still struggling with the NRC over exam quality. The licensee has been advised to create procedures for exam security. Some licensees have created separate rooms, purchased computers, copiers, modems, and software all in support of maintaining examination security.

We have placed an additional financial burden on the licensee, with a potential decrease in safety. I find it difficult to see how the backfit rule does not apply .

With this learned knowledge of an inherent conflict of interest, degradation of examiner knowledge, increased threat to exam security, and a potential backfit issue, we need to take a step back and reevaluate our beliefs about the examination process. Our initial feeling was that having the licensee generate the examination would improve the quality of the exam in both technical and content areas, all while saving contractor dollars . The poor performing licensee's have demonstrated that they are unable/unwilling to produce both a technically accurate and high quality product. I do not believe that the pilot program has demonstrated that the commissions concerns as outlined in SECY-97-079 are not warranted.

CC: J. Jaudons, RlI DRS Division Director C. Casto, RII DRS Deputy Division Director T. Peebles, RII Operator Licensing Branch Chief R. Gallo, Chief Operator Licensing Branch G. Meyer, RI Operator Licensing Branch Chief M. Leach, RIII Operator Licensing Branch Chief J. Pellet, RIV Operator Licensing Branch Chief

August 14, 1997 oocKET NUMBER PR  !':"" DOCKETED

([)

PROPOSED RULE...:..,;;.-;._;;.... 5 0--==-- US RC Secretary ( (pp_ FR. '-I~ ti ;;to)

U.S. Nuclear Regulatory Commission "97 AUG 21 A10 :37 ATTN: Docketing and Service Branch Washington, D.C. 20555 OFFI CE OF SE GRE TARY OOCKETI ~G SERVICE BRANCH I am providing comments on the "Proposed Rule for Utility Role in Initial Reactor Operator License Examinations." I have twenty-three (23) years experience related to commercial nuclear operations, and I am opposed to the concept and viability of the proposed rule.

The NRC, and their contractors, are specifically trained to develop license examinations that meet the NUREG-1021 criteria. Giving this task to the utility requires that they train a staff of people to develop, TA"lf;e,-.:1

- - . .. .., " ._ .... _ -.:,,=.r-:h,

'l-t"\rl '~- *') thA ....... ..., ,:,Ol"l"llr.,l',1 _ ..... , .. '-" ,:>,vam;nat;,,...,.,c

_..,,- ..... ..... -) nf"*ho .... ~ .................. _. ** .., . Tliar.o 't').01"\.'t"\lo

. . _ **-"'"' t""""'t' ...... u"-ro 'llC'n

.. - ........ ..,,. re("+-"M,....tod

& -.,11,,& &...,._....., -fro""'

.&..a. &&&

1"'\0ril"'\rrr'l:!1,...

.t' ............ _..i. ............ 0 any training on the initial license candidates. Implementation of this rule would increase the cost of operating the nuclear facility and may decrease the examination quality.

The utility has a vested interest in obtaining licenses for their operators. They spend a lot of time and money on training. There is a possibility of the utilities developing examinations that technically meet the standards, but do not challenge the operator beyond what may have been presented in an inadequate training program. It is possible to "teach to the exam" or "exam only on what has been taught."

Independent examiners do not have this conflict of interest, since they do not work and socialize with the candidates.

The NRC budget is cited as a basis for the proposed rule change. This seems to be a weak reason, since the utilities pay for the examinations and the money just needs to be transferred between accounts in the Treasury.

A major problem created by the pilot program has been the loss of qualified NRC certified examiners.

Examiners have transferred to other positions that have a perceived long term job security. This problem will be compounded if the NRC is reduced to just administering examinations. The small examiner staff will spend most of their time traveling to give examinations and will have less time at

,uau::: with tl11;;ir farnllie8. Thts always causes i.t:nsion, job dissatisfaction, and high turnover.

I believe that the proposed rule was an ill-conceived plan to solve a perceived budget problem at the NRC. A majority of the Commissioners and initial "champions" of the process are no longer involved in the process. They started the process and have left the mess for those remaining. The people directly involved in the examination process were reluctant to tell their boss ( or their Regulator) that the proposed rule was not a good idea. I recommend NOT approving the proposed rule.

KenEri~ ~

5700 W. 25th Ave Kennewick WA 99338-1905 509-736-2386

J 'S NUCLEAR REGULATORY COMMiss,o DOCKETING & SERVICE SECTK)N OFFICE OF THE SECRETAfW OF THE COMMISSI01 Oocuments ostmark Dallt ~ _ 'i /J 6 / 'I Z I

Opiis Recel~ I d'I Copies Reproduced t-r---=

ecial Dislribution_1ovma ffioa.

a //a51-::I hrr 1 ?>h R.J -i> tbs:

DOCKETED USNRC

[7590-01-PJ NUCLEAR REGULATORY COMMISSION "97 AUG -5 P3 :16 10 CFR Pa rt 55 OFFI CE OF SEC RETA RY RIN 3150-AF62 DOCKE Tl~iG *: .3!:RVICF.

t ti.h (..H Initial Licensed Operator Examination Requirements AGENCY: Nuclear Regulatory Commission. DOCKET NU BERp PROPOSED RULE 5~

( ~RFl<t.f~'/~t,,)

ACTION: Proposed rule.

SUMMARY

The Nuclear Regulatory Commission (NRC) is proposing to amend its regulations to require all nuclear power facility licensees to prepare.

proctor. and grade the written examinations and prepare the operating tests that the NRC currently uses to evaluate the competence of individuals applying for operator licenses at those plants . The proposed amendment would require the licensee to submit each examination and test for the NRC's review and approval and would preserve the NRC's authority to prepare the examinations and tests. as necessary. if it loses confidence in a licensee's ability to prepare these examinations acceptably . In addition. the NRC would periodically invoke this authority in order to maintain the proficiency of its own license examiners.

t),~ ;J./1 19'17 DATES: Submit comments by [ Im;ert the detei 7§ days after f)Ybl i cati oA i M the

-F~der~l ~egister] . Comments received after this date will be considered if it is practical to do so. but the Commission is able to assure consideration only for comments received on or before this date.

ADDRESSES : Comments may be sent to : Secretary. U.S. Nuclear Regulatory Commission. Washington. DC 20555 . Attn : Rulemakings and Adjudications Staff.

Hand deliver comments to 11545 Rockville Pike. Rockville. Maryland . between 7:30 am and 4:15 pm on Federal workdays . For information on submitting comments el ectroni call y. see the discussion under Electronic Access in the Supplementary Information section.

Single copies of this proposed rulemaking may be obtained by written request or telefax ((301) 415-2260) from Harry S. Tovmassian. Office of Nuclear Regulatory Research. U.S. Nuclear Regulatory Corr111"1ission. Washingtor. DC 20555 . Certain documents related to this rulemaking. including comments received. may be examined at the NRC Public Document Room. 2120 L Street NW (Lower Level). Washington. DC. These same documents may also be viewed and downloaded electronically via the Electronic Bulletin Board established by NRC for this rulemaking as indicated in the Supplementary Information section.

FOR FURTHER INFORMATION CONTACT : Harry S. Tovmassian. Office of Nuclear Regulatory Research. U.S. Nuclear Regulatory Commission. Washington. DC 20555.

telephone (301) 415-6231 : e-mail hst@nrc.gov .

SUPPLEMENTARY INFORMATION :

Background

Section 107 of the Atomic Energy Act (AEA) of 1954. as amended. requires the NRC to determine the qualifications of individuals applying for an operator license. to prescribe uniform conditions for licensing such 2

individuals. and to issue licenses as appropriate . Pursuant to the AEA.

10 CFR Part 55 requires applicants for operator licensees to pass an examination that satisfies the basic content requirements specified in the regulation. Although neither the AEA nor Part 55 specifies who must prepare.

proctor. or grade these examinations. the NRC has traditionally performed those tasks itself or through its contract examiners. In accordance with 10 CFR 170(i). NRC staff and contractual costs are recovered from facility licensees who receive examination services . The NRC and its contract examiners have used the -JUi dance in NUREG-1021. "Operator Licensing Examination Standards for Power Reactors." to prepare the initial operator licensing examinations. This document has been revised as experience has been acquired in preparing these examinations. The current version is designated Interim Revision 8. 1 The intended modifications to 10 CFR Part 55 would allow facility licensees to have greater participation in the initial operator licensing process and enable the NRC to eliminate contractor assistance in this area .

Between $3 million and $4 million in contractor support for the preparation and administration of the initial operator licensing examinations and for support of requalification program inspections would be eliminated .

On April 18. 1995 . the Commission approved the NRC staff's proposal to initiate a transition process to revise the operator licensing program and directed the NRC staff to carefully consider experience from pilot examinations before fully implementing the changes . On August 15. 1995. the 1

Copies are available for inspection or copying for a fee from the NRC Public Document Room at 2120 L Street NW . . Washington. DC 20555: the PDR's mailing address is Mail Stop LL-6 : telephone (202) 634-3273 . fax (202) 634-3343. Interim Revision 8 is also available for downloading from the Internet at "http:/ /Wwvv . nrc . gov."

3

NRC staff issued Generic Letter (GU 95-06. "Changes in the Operator Licensing Program." 2 outlining the revised examination development process and soliciting volunteers to participate in pilot examinations to evaluate and refine the methodology.

Between October 1. 1995. and April 5. 1996. the NRC staff reviewed and approved 22 operator licensing examinations. including both the written examinations and the operating tests. prepared by facility licensees as part of a pilot program. These examinations were prepared using the guidance in Revision 7 (Supplement 1) of NUREG-1021 and the additional guidance in GL 95 -

06.2 These examinations were used to test 146 reactor operator (RO) and senior reactor operator (SRO) applicants.

The results of the pilot examinations were discussed in SECY-96-123.

"Proposed Changes to the NRC Operator Licensing Program." dated June 10. 1996.

Based on the results of the pilot program. the staff recommended that the Commission approve the implementation of the new examination process on a voluntary basis until rulemaking could be completed to require all power reactor facility licensees to prepare the entire initial examination for reactor operators and senior reactor operators and to proctor the written portion of the examination. On July 23. 1996. the Commission authorized the staff to continue the pilot examination process on a voluntary basis and requested the staff to develop a detailed rulemaking plan to justify the changes that may be necessary to 10 CFR Part 55 . The Commission also directed the staff to address a number of additional items (e.g .. pros. cons. and 2Copies are available for inspection or copying for a fee from the NRC Public Document Room at 2120 L Street NW .. Washington. DC 20555: the PDR's mailing address is Mail Stop LL-6.: telephone (202) 634-3273: fax (202) 634-3343.

4

vulnerabilities) regarding the revised examination process to facilitate a Commission decision on whether to implement the revised process on an industry-wide basis.

On September 25. 1996. the staff forwarded the requested rulemaking plan and a response to the additional items to the Commission in SECY-96-206.

"Rulemaking Plan For Amendments to 10 CFR Part 55 to Change Licensed Operator Examination Requirements." On December 17. 1996. the Commission directed the staff to proceed with the proposed rulemaking.

With Commission approval. the staff resumed conducting pilot-style examinations on August 19. 1996. and by the end of December 1996 had reviewed.

approved. and administered 12 additional examinations that were developed by facility licensees based on the guidance in GL 95-06. This raised the total number of examinations completed using the pilot process to 34 and the number of applicants tested to 84 ROs and 144 SROs.

Discussion The pilot program demonstrated that the revised process. using licensee developed examinations. can be both effective and efficient. Comments from the NRC staff and industry personnel who participated in the pilot examinations were generally favorable . The quality of the licensee-developed examinations (as modified by the NRC) was generally comparable to the examinations prepared by the NRC staff or its contractors. All of the licensee-developed examinations required some modifications subsequent to NRC review: however. several of these examinations required significant rework.

indicating that some licensees did not fully understand the criteria for 5

preparing examinations which meet NRC standards. With training and experience. it is expected that the industry would gain proficiency in preparing the examinations. The monitoring and assessment of this voluntary pilot program has demonstrated that facility licensee developed examinations.

as modified by the NRC. are comparable in terms of their quality to those prepared by the NRC and its contract examiners under the existing process:

therefore. the safe operation of the facility in question is in no way compromised. The fact that the pass/fail results on the 34 pilot examinations administered to the 84 ROs and 144 SROs through the end of December 1996 were comparable to the power reactor licensing examination results during Fiscal Year 1995. when all the examinations were prepared by the NRC or its contractors. supports this conclusion . The provisions of the proposed rule in

§ 55.40(a)(2). which require NRC staff review and approval of facility licensee developed tests and examinations. should facilitate the monitoring of the quality of the submittals and the modification of those which do not meet NRC standards.

The fact that NRC examiners will be administering all of the operating tests without contractor assistance is expected to improve the NRC staff's focus on operator performance and ~ts core of experience because every applicant will be directly observed by an NRC employee. Before beginning the transition process. contract examiners administered about half of the operating tests and collected the observations that formed the basis for the NRC's licensing actions. The contractors* efforts focused primarily on task completion. so any broader insights and experience that might have been gained while giving the examinations was of little benefit to the NRC.

6

The Commission has assessed the pros and cons associated with the revised examination process. as discussed in SECY-96-206. and considered the measures that the NRC staff has taken to mitigate the vulnerabilities. The Commission acknowledges that the revised examination process increases the risk of lapses in examination quality (including level of difficulty),

consistency, and security and wishes to emphasize the NRC's resolve to maintain the existing standards of performance in each of these areas.

With regard to examination security. in particular. applicants.

licensees (opera+8rs). and facility licersees are reminded that lG CFR 55.49 prohibits their engagement in any activity that compromises the integrity (security) of any application. test. or examination required by 10 CFR Part 55 and that examination will need to be proctored in accordance with 10 CFR 55.40 . These provisions require facility licensees to maintain proper examination security. The Commission expects that licensees will meet the security provisions in ES-201 and ES-402 of NUREG-1021 or similar NRC-approved standards. Consistent with NUREG-1021. facility employees with specific knowledge of any NRC examination before it is given may not communicate the examination contents to unauthorized individuals and may not participate in any further instruction of the students scheduled to take the examination.

Before they are given access to the examination. the facility employees are expected to sign a statement acknowledging their understanding of the restrictions and the potential consequences of noncompliance and sign a post-examination statement certifying that they did not knowingly compromise the examination. In addition to the restrictions on personnel. NUREG-1021 also discusses a number of physical security precautions. including protecting and mailing the examination materials and simulator considerations. The guidance 7

also cautions NRC examiners to be attentive to examination security measures and requires them to review the security expectations with the facility licensee at the time the examination arrangements are confirmed.

The Commission considers a violation of 10 CFR 55.49 for compromising an examination has occurred when (1) a failure to control the integrity of an examination occurs such that there is a potential for an applicant to have an unauthorized advantage in the examination process or (2) an applicant obtains an unauthorized advantage. Both facility licensees and applicants for examinations may be subject to enforcement action for violations of 1~ CFR

e 55. 49 commer;surate with the nature and ser ousness of the compromise.

I As part of the final rulemaking in this matter. the Commission intends to modify its "General Statement of Policy and Procedures for NRC Enforcement Actions" (Enforcement Policy). NUREG-1600. Security compromises will normally be considered at least at Severity Level IV. A violation where it was likely that an applicant obtained unauthorized access to examination material will be considered a significant regulatory concern and categorized at least at Severity Level III. The NRC intends to utilize its full enforcement authority including, as warranted. civil penalties and orders against persons found to have been involved in willful compromises of examinations in violation of 10 CFR 55.49. This will include use of the rule on Deliberate Misconduct (10 CFR 50.5). In addition. cases involving willful violations will be referred to the Department of Justice.

8

Availability of Guidance Document for License Examination Preparation Although 10 CFR Part 55 does not specify who will prepare . administer .

and grade the written examinations and operating tests for reactor operator and senior reactor operator licenses. the NRC or its contract examiners have traditionally performed these tasks . As a consequence of performing the tasks associated with preparing and administering the initial licensing examinations. the NRC has developed a substantial body of guidance . which has been published in var~ous versions of NUREG-1021 to aic both NRC and it~

contract examiners. The latest version of NUREG-1021 (Interim Revision 8) incorporates the pilot examination criteria in GL 95-06 . lessons learned during the pilot examinations. and a number of refinements prompted by the comments submitted in response to the Federal Register notice dated February 22. 1996 (61 FR 6869). which solicited public comments on the proposed NUREG changes . A copy of Interim Revision 8 of NUREG-1021 has been mailed to each facility licensee . Copies may be inspected and/or copied for a fee at the NRC ' s Public Document Room . 2120 L Street NW (Lower Level).

Washington. DC . NUREG-1021 is also electronically available for downloading from the Internet at "http://WwW .nrc.gov . " All interest2d parties are invited to comment on Interim Revision 8 of NUREG-1021 in addition to the proposed rule . These public comments will be addressed. and Revision 8 will be published as a final NUREG document .

The NRC plans to prepare. administer. and grade initial operator licensing examinations at least four times per year . using NUREG-1021 as guidance. Licensees would also be expected to use the guidance contained in NUREG-1021 to prepare the licensing examinations . The NRC staff would review 9

and approve any deviations from this guidance. The NRC will not approve any deviation that would compromise its statutory responsibility of prescribing uniform conditions for the operator licensing examinations. Examples of unacceptable deviations include. but are not limited to. the use of essay questions in place of multiple choice questions and the administration of open book examinations.

Proposed Rule This proposed regulation would add a new section. § 55 .40.

"Implementation." to Subpart E of 10 CFR Part 55 which would require power reactor facility licensees to prepare the written examinations and operating tests. to submit them to the NRC for review and approval. and to proctor and grade the written examinations. These requirements would be contained in

§§ 55 .40(a)(l) , (2). and (3). respectively .

Each power reactor facility licensee would be required to prepare and submit the proposed examinations (including the written examination. the walk-

- through. and the dynamic simulator tests) to the NRC consistent with the guidance contained in NUREG-1021 . The NRC staff would review the entire examination and direct whatever changes are necessary to ensure that adequate levels of quality. difficulty. and consistency are maintained. After the NRC staff reviews and approves an examination. the facility licensee would proctor and grade the written portion consistent with the guidance in NUREG-1021. The NRC staff would continue to independently administer and grade the operating tests. review and approve the written examination results. and make the final 10

licensing decisions. The facility licensee would not conduct parallel operator evaluations during the dynamic simulator or the walk-through tests.

Pursuant to proposed requirements in§ 55.40(b). the NRC staff would maintain the authority to prepare the examinations and tests and to proctor and grade the site-specific written examinations. This proposed rule would allow NRC to maintain its staff capability to perform these activities. Also.

if the NRC has reason to question a licensee*s ability to prepare an acceptable examination. § 55.40 (b) provides the NRC authority to prepare and administer the examinat7Jns and tests.

Paragraph (c) of§ 55.40 reasserts that the NRC would continue to prepare and administer the written examinations and operating tests at non-power reactor facilities. The NRC has taken this position because the non-power reactor community does not have an accreditation process for training and qualification or the resources to prepare the examinations. However. the process will be implemented using only NRC examiners. thereby allowing the elimination of all routine contract assistance in that area.

Electronic Access Comments may be submitted electronically, in either ASCII text or WordPerfect format (version 5.1 or later). by calling the NRC Electronic Bulletin Board (BBS) on FedWorld or connecting to the NRC interactive rulemaking web site. "Rulemaking Forum." The bulletin board may be accessed using a personal computer. a modem. and one of the commonly available communications software packages. or directly via Internet. Background 11

documents on the rulemaking are also available. as practical. for downloading and viewing on the bulletin board.

If using a personal computer and modem. the NRC rulemaking subsystem on FedWorld can be accessed directly by dialing the toll free number (800) 303-9672. Communication software indicators should be set as follows: parity to none. data bits to 8. and stop bits to 1 (N.8.1). Using ANSI or VT-100 terminal emulation. the NRC rulemaking subsystem can then be accessed by selecting the Rules Menu option from the "NRC Main Menu.

II II 11 Users will find the FedWorld Online User*s Guides" particularly helpful.

11 Many NRC subsystems and data bases also have a "Help/Information Center" option that is tailored to the particular subsystem.

The NRC subsystem on FedWorld can also be accessed by a direct dial phone number for the main FedWorld BBS. (703) 321-3339. or by using Telnet via Internet: fedworld.gov. If using (703) 321-3339 to contact FedWorld. the NRC subsystem will be accessed from the main FedWorld menu by selecting the "Regulatory. Government Administration and State Systems." then selecting "Regulatory Information Mall." At that point. a menu will be displayed that has an option "U.S. Nuclear Regulatory Commission" that will take you to the NRC Online main menu. The NRC Online area also can be accessed directly by typing "/go nrc" at a FedWorld command line. If you access NRC from FedWorld's main menu. you may return to FedWorld by selecting the "Return to FedWorld" option from the NRC Online Main Menu. However. if you access NRC at FedWorld by using NRC's toll-free number. you will have full access to all NRC systems. but you will not have access to the main FedWorld system.

If you contact FedWorld using Telnet. you will see the NRC area and menus. including the Rules Menu. Although you will be able to download 12

documents and leave messages. you will not be able to write comments or upload files (comments) . If you contact FedWorld using FTP. all files can be accessed and downloaded but uploads are not allowed: all you will see is a list of files without descriptions (normal Gopher look) . An index file listing all files within a subdirectory. with descriptions. is available.

There is a 15-minute time limit for FTP access .

Although FedWorld also can be accessed through the World Wide Web. like FTP. that mode only provides access for downloading files and does not display

,_ the NRC Rules Menu .

You may also access the NRC's interactive rulemaking web site through the NRC home page (http ://w-ww .nrc .gov). This site provides the same access as the FedWorld bulletin board. including the facility to upload comments as files (any format). if your web browser supports that function.

For more information on NRC bulletin boards call Mr. Arthur Davis .

Systems Integration and Development Branch. NRC. Washington. DC 20555-0001 .

telephone (301) 415-5780: e-mail AXD3@nrc.gov. For information about the interactive rulemaking site. contact Ms Carol Gallagher. (301) 415-5905 :

- e-mail CAG@nrc.gov.

Environmental Impact : Categorical Exclusion The NRC has determined that this proposed rule is the type of action described as a categorical exclusion in 10 CFR 51 .22(c)(l). Therefore.

neither an environmental impact statement nor an environmental assessment has been prepared for this proposed regulation.

13

Paperwork Reduction Act Statement This proposed rule amends information collection requirements that are subject to the Paperwork Reduction Act of 1995 (44 U.S.C . 3501 et seq). This rule has been submitted to the Office of Management and Budget for review and approval of the information collection requirements .

The public reporting burden for this collection of information is estimated to average 500 hours0.00579 days <br />0.139 hours <br />8.267196e-4 weeks <br />1.9025e-4 months <br /> per response. including the time for reviewing instructions . searching exist~~g data sources. gathering and maint~ining the data needed . and completing and reviewing the collection of information Ci .e . .

preparing the examinations). The U. S. Nuclear Regulatory Commission is seeking public comment on the potential impact of the collection of information contained in the proposed rule and on the following issues :

1. Is the proposed collection of information necessary for the proper performance of the functions of the NRC. including whether the information will have practical utility?

2. Is the estimate of burden accurate?

3. Is there a way to enhance the quality . utility . and clarity of the information to be collected?

4. How can the burden of the collection of information be minimized.

including the use of automated collection techniques?

Send comments on any aspect of this proposed collection of information .

including suggestions for reducing the burden . to the Information and Records Management Branch (T -6F-33). U.S. Nuclear Regulatory Commission. Washington.

14

DC 20555-0001. or by Internet electronic mail at bjsl@nrc .gov: and to the Desk Officer. Office of Information and Regulatory Affairs. NEOB-10202. (3150-00187 and 3150-0101). Office of Management and Budget. Washington. DC 20503.

Comments to 0MB on the collections of information or on the above issues should be submitted by [insert date 30 days after publication in the Federal Register) . Comments received after this date will be considered if it is practical to do so. but assurance of consideration cannot be given to comments received after t~ :s date.

Public Protection Notification The NRC may not conduct or sponsor. and a person is not required to respond to. a collection of information unless it displays a currently valid 0MB control number.

Regulatory Analysis The Commission has prepared a draft regulatory analysis on this proposed regulation. The analysis examines the costs and benefits of the alternatives considered by the Commission . The draft analysis is available for inspection in the NRC Public Document Room. 2120 L Street NW (Lower Level). Washington.

DC. Single copies of the analysis may be obtained from Harry S. Tovmassian at (301) 415-6231.

The Commiss ion requests public comment on the draft regulatory analysis and the following specific questions .

15

1. Are there portions of the operator exams that are common to all licensees. and would therefore be more efficiently developed by the NRC?

2. Is the conclusion in the regulatory analysis correct that it would be less costly for each licensee to prepare their own initial operator examinations to be reviewed. revised . and administered by the NRC. than to have one NRC contractor prepare these exams for all licensed operators with the costs to be reimbursed by licensee fees.

Comments on the draft analysis may be submitted to the NRC as indicated under the ADDRESStS heading .

Regulatory Flexibility Certification In accordance with the Regulatory Flexibility Act of 1980. (5 U.S.C .

605(b)). the Commission certifies that this rule will not. if promulgated.

have a significant economic impact on a substantial number of small entities .

This proposed rule affects only the licensing and operation of nuclear power plants. The companies that own these plants do not fall within the scope of the definition of "sma 11 entities" set forth in the Regulatory Fl exi bil i ty Act .

or the Small Business Size Standards set out in regulations issued by the Small Business Administration at 13 CFR Part 121.

Backfit Analysis The pertinent part of 10 CFR 50 .109 (a)(l) defines backfitting as "the modification of or addition to . .. the procedures or organization required to

... operate a facility: any of which may result from a new or amended 16

provision in the Commission rules or the imposition of a regulatory staff position interpreting the Commission rules that is either new or different from a previously applicable staff position .... " Although Part 55 addresses the qualifications and requirements for operators* licenses and changes are not per se subject to the backfit rule in Part 50. changes to these requirements could be included within the backfit definition of "procedures or organization required to ... operate a facility." However. in this case. the proposed shift of responsibility from the NRC staff (or its contractors) to the facility licensee for developing and administering the initial written examination for the operator license exam would not constitute a "modification of the procedures required to operate a facility" within the scope of the backfit rule: therefore. no backfit analysis needs to be prepared.

The proposed rule does not affect the basic procedures for operator license qualification. i.e .. the required training programs. the required testing. the content and format of the exams. the grading of the exams. or the basis for issuing an operator license. The shift in responsibility for

- preparing the initial exam does not affect the content or format of the exam.

The proposed rule is designed to ensure that the format. content. and quality of the initial written examination will not be modified. The proposed rule requires the NRC to provide oversight of facility licensees* development and administration of initial written examinations. The NRC would also retain its discretion to determine whether to administer the initial written examination itself. as well as continuing to determine whether to grant or deny an application for an RO or SRO license and to consider candidates* appeals.

17

The licensee *s organizational structure required to operate the facility will not be modified . All reactor licensees have a training component as part of their organizational structure . and the proposed rule does not alter that organizational structure. Although . the proposed rule could have an "effect" on the licensee's organization . it does not require any modification to the organizational structure .

Finally. the proposed rule does not impose any new costs on licensees since the NRC's costs LJ develop examinat icns ere preser.~ly recovered ir the fee base . These costs are basically the same as the costs that will be incurred by licensees to develop the examinations under the proposed rule .

List of Subjects in 10 CFR Part 55 Criminal penalties. Manpower training programs . Nuclear power plant s and reactors. Reporting and recordkeeping requirements.

For the reasons set out in the preamble and under the authority of the Atomic Energy Act of 1954. as amended: the Energy Reorgar1ization Act of 1974.

as amended: and 5 U.S .C. 553: the NRC proposes to adopt the following amendments to 10 CFR Part 55 .

18

PART 55--OPERATOR'S LICENSES

1. The authority citation for Part 55 continues to read as follows:

AUTHORITY: Secs. 107 . 161 . 182. 68 Stat. 939. 948. 953 . as amended.

sec . 234. 83 Stat. 444. as amended (42 U.S.C. 2137. 2201. 2232. 2282): secs . 201. as amended. 202. 88 Stat. 1242. as amended. 1244 (42 U.S .C. 5841 . 5842).

Sections 55.41. 55.43. 55 .45. and 55.59 also issued under sec. 306. Pub.

L.97-425. 96 Stat. 2262 (42 U.S.C . 10226). Section 55.61 also issued under secs . 186. 187. 68 Stat . 955 (42 U.S.C . 2236. 2237) .

2. In§ 55 .8 paragraph (b) is revised to read as follows :

§ 55.8 Information Collection Requirements; 0MB Approval .

(b) The approved information collection requirements contained in this part appear in§§ 55 .31. 55 .40. 55 .45. 55 .53 . and 55 .59 .

3. A new§ 55.40 is added to read as follows:

§ 55.40 Implementation.

(a) Power reactor facility licensees shall --

(1) Prepare the required site-specific written examinations and operating tests:

(2) Submit the written examinations and operating tests to the Commission for review and approval : and (3) Proctor and grade the NRC-approved site-specific written examinations.

19

(b) In lieu of requiring a specific power reactor facility licensee to prepare the examinations and tests or to proctor and grade the site-specific written examinations. the Commission may elect to perform those tasks.

(c) The Commission will prepare 3nd administer the written examinations and operating tests at non-power reactor facilities.

Dated at Rockville. Maryland. this 3/ :,_!'day of .J7" . 1997.

For the Nuclear Regulatory Commission.

20

____________________________ - - - .